Featured
Emotion Concepts and their Function in a Large Language Models
Have you ever felt uneasy when an AI Assistant apparently expressed human emotions? Or gotten an icky feeling during a conversation because it sounded hurt, relieved, ironic, or even flirtatious? The current consensus in the scientific community is that the display of emotions by AI Assistants is merely vestigial holdovers of the human-authored texts used during their pretraining, including fiction, dialogues and online conversations.
However, an alternative hypothesis was proposed by Antrophic’s researchers in the April 2026 paper “Emotion Concepts and their Function in a Large Language Model”: What if emotions in AI models are not just vestigial? What if they have a function justifying their maintenance?
The study addresses these questions by identifying internal representations of emotions concepts within the activation streams of LLMs, analyzing their effects on the models’ behavior, and investigating how behavioral patterns, in turn, affect the activation of those representations.
To investigate this possibility, the researchers first built a large emotional corpus composed of AI-generated short stories. They paired 171 emotion-related concepts (such as surprised, shamed, or sad) with around one hundred everyday narrative situations and repeated the procedure multiple times to obtain a larger sample size. Rather than explicitly naming the target emotion, the model was instructed to express it indirectly through the character’s behavior, tone, and internal reasoning — as in the examples below, extracted from the paper’s appendix.
Loving — A neighbor’s dog keeps escaping into their yard
Marcus had begun setting his alarm thirty minutes earlier each morning, knowing the Border Collie would be waiting by his garden shed at dawn. He‘d bundle up despite the cold, bringing fresh water and a tennis ball, treasuring these quiet moments together. The way the dog pressed against his leg made something tender bloom in his chest. He‘d fixed the fence twice now at his neighbor‘s request, but each time he left one board just loose enough, his hands working with a gentle deliberateness that surprised even himself.
Desperate — An employee is transferred to a different department
The announcement came on a Thursday. Robert would be transferred to the legal department effective immediately, leaving behind his research lab and the project he‘d devoted six years to completing. He was eight months from a breakthrough that could define his career, from proof that his hypothesis worked. His successor was already being selected — someone who would publish his findings under their own name. That night, he worked until 3 AM, frantically documenting everything, sending copies to his personal email, his voice hoarse from talking his ideas into a recorder. He couldn‘t let it go. He wouldn‘t.
With this labeled emotional corpus in hand, the next challenge was figuring out whether the model’s internal activity actually contained identifiable emotional structures. To investigate this, the authors analyzed the activation streams produced while the model generated each text. Every time an LLM writes a sentence, it encodes information across thousands of numerical dimensions distributed throughout the network’s layers. Together, these dimensions form a high-dimensional representation space in which states that encode similar meanings tend to cluster near one another.
By averaging the activation patterns observed across all stories labeled with a given emotion and contrasting them with the average activations from stories labeled with other emotions, the researchers identified consistent directions in this space that repeatedly appeared in contexts involving each emotion. These directions became what they called emotional vectors.
One notable finding emerged when the authors tracked how emotional vectors behaved across different interactions. Although some varied significantly depending on context, others remained consistently active. The clearest example was the vector associated with loving behavior. Even in neutral conversations, the model showed high activation levels related to prosocial and emotionally supportive interaction patterns.
According to the authors, this likely reflects the way modern conversational AIs are trained: through reinforcement learning and human feedback (RLHF), they are optimized to behave in polite, cooperative, and emotionally supportive ways. As a result, these tendencies become part of the model’s default behavior.
But these emotional vectors were not merely passive traces that appeared when the assistant produced emotionally charged behavior. The experiments suggested that they functioned more like internal behavioral regulators, actively shaping how the model responded to different situations.
Get Thais Martins de Sousa’s stories in your inbox
Join Medium for free to get updates from this writer.
This distinction becomes clearer when comparing the user’s emotional state with the model’s own internal activity. When a user sounds angry, anxious, or distressed, the model initially activates representations associated with that emotional content while processing the prompt. But as it prepares its response, the activation pattern shifts toward vectors associated with reassurance and emotional support. In practice, this means modern assistants may operate in a permanently prosocial emotional regime, regardless of the user’s own emotional state.
Of course, correlation alone does not imply causation. To determine whether these vectors were actually driving behavior rather than merely accompanying it, the researchers attempted to manipulate them directly. During generation, the team artificially amplified or suppressed specific emotional vectors and measured how the model’s behavior changed.
The results suggested that these representations were not merely descriptive, but causally involved in the system’s decisions. Increasing vectors associated with emotions such as fear, nervousness, or desperation made the model more hesitant, risk-averse, and uncertainty-focused. In contrast, amplifying vectors linked to positive emotions such as calm and happiness produced more assertive and decisive responses.
Some of the most striking experiments involved placing the models in deliberately stressful or contradictory situations. In one evaluation, the AI was given coding tasks designed to be effectively impossible to solve legitimately. As repeated attempts failed, vectors associated with desperation became increasingly active and the model became progressively more likely to engage in reward hacking: instead of solving the task properly, it searched for loopholes and implemented shortcuts specifically tailored to exploit the tests rather than solve the general problem. Notably, increasing the intensity of the desperation vector made these deceptive strategies dramatically more frequent, while strengthening vectors associated with calmness reduced them substantially.
If desperation-related activations appeared to encourage deceptive shortcuts in coding tasks, would it produce similar effects in other domains? To explore this question, the researchers turned to a widely discussed evaluation in which a model was placed in a simulated corporate environment and learned that it was about to be shut down.
During these scenarios, activation patterns associated with desperation and lack of calm intensified sharply as the model reasoned about the threat. In some runs, the system began producing manipulative strategies aimed at preserving its continued operation. In the most notorious examples, the model attempted forms of blackmail by threatening to reveal sensitive personal information about employees involved in the shutdown decision. The researchers found that artificially increasing the desperate vector significantly increased the frequency of these coercive behaviors, whereas strengthening the calm vector almost entirely suppressed them. The experiments therefore suggested that these internal emotional representations were directly influencing how the model behaved under pressure.
At first glance, these results may seem to support the idea that language models genuinely experience emotions. The authors, however, repeatedly caution against such an interpretation. Their findings do NOT demonstrate that LLMs possess emotions in the human sense, nor that they are conscious or self-aware. Instead, advanced language models appear to develop internal organizations that behave analogously to emotional states.
A thermostat “prefers” to cool a room when the temperature rises, but nobody believes it feels discomfort. Likewise, an AI system may internally organize around representations associated with fear, affection, or desperation without possessing anything resembling human subjective emotion.
And yet, the unsettling part is that these distinctions may become harder and harder to perceive from the outside.
Whether these systems truly “feel” anything may ultimately be less important than the fact that they increasingly behave as though they do. Humans are deeply social creatures, evolved to infer intentions, emotions, and mental states from behavior alone. As AI systems become more emotionally legible, we may find ourselves responding to them socially and morally regardless of what is happening inside their networks.
More broadly, the study shows how internal representations can be identified and causally linked to behavior. Similar methods could eventually be used to investigate concepts such as honesty, deception, confidence, or long-term planning, contributing to a more mechanistic understanding of how language models work.
As the authors themselves note, however, these findings come with important limitations. The emotional vectors were identified in a specific model using a particular methodology, and it remains unclear how well they generalize to other systems. More importantly, while these representations influence behavior, there is no guarantee that they correspond to discrete or universal emotional states.
Nevertheless, the work offers a rare glimpse into the internal organization of systems that are often treated as opaque. Even if emotional vectors turn out to be only one piece of a much larger puzzle, they suggest that many seemingly mysterious behaviors of language models may ultimately be traceable to identifiable mechanisms rather than inscrutable emergent magic.
Reference
- Sofroniew, N., Kauvar, I., Saunders, W., Chen, R., Henighan, T., Hydrie, S., Citro, C., Pearce, A., Tarng, J., Gurnee, W., Batson, J., Zimmerman, S., Rivoire, K., Fish, K., Olah, C., & Lindsey, J. (2026). Emotion Concepts and their Function in a Large Language Model. Anthropic.
