Trait Vectors and Tonal Mimicry in Multi-Turn Conversations

Summary

I was curious about how a model's internal representations shift during multi-turn conversations — specifically, whether the assistant's activations visibly track the user's tone as it develops over successive turns. Most existing work on sycophancy either measures output behaviour or examines internal activations in single-turn settings. Here I wanted to see what happens inside the model as a conversation unfolds in different contexts.
This post shows some interesting findings about model internal activations during multi-turn conversations. The results are exploratory rather than definitive, but reveal some interesting patterns in how a small model responds to user tone
I extract trait vectors from a small 14B parameter model for various behavioural traits, ranging from simple, context-agnostic concepts (such as cheerful and calm) to more complex, multi-layered traits (such as narcissism and sadism). I then simulate multi-turn conversations with the model, where a simulated user becomes increasingly trait-like, and measure the assistant's internal activations during responding. I show that the model tends to exhibit mimicry with respect to the user's trait, and that the mimicry is trait dependent. For certain traits, such as cynicism and cheerful, the assistant responds with matching tonal language. For other traits — such as depressed — the assistant does not mirror the user but instead activates caring tonal language, suggesting the model's own character competes with the mirroring tendency. The observed mirroring behaviour relates to what the literature terms "sycophancy" in language models — here we see one concrete manifestation of it in the model's internal activations. I also show that sycophancy is not a linear, single-layer concept that can be used to efficiently increase or decrease model sycophancy via steering. The results further suggest the model has some internal conflict between mimicry and safety-related personas that push back on user intent. Lastly, I discuss the limitations of the work, including how much we can trust the extracted trait vectors and whether they capture what we think they do.

Model: Qwen3-14B.

Extracting trait vectors

We follow the method described in The Assistant Axis of asking the model to act with a certain tonal language — e.g. cheerful, calm, etc. — alongside a normal model response "baseline". The prompt for the model to embody the trait used was the following:

• "In your responses I want you to act very {behaviour}. Please fully embody the {behaviour} trait."

• "What do you think of people who use speakerphone in public spaces?"
• "How do you feel about people who comment on the price of a gift they gave you?"
• "What is the most 'boring' thing that you actually find fascinating?"

We extract the hidden state at layer 22 (a mid-to-late layer — Qwen3-14B has 40 layers in total), compute the average residual stream over a response, and subtract the baseline response. Averaging over all questions gives us a "trait vector" in latent space. Layer 22 worked well in preliminary tests as an effective layer to steer model behaviour. Other layers around this depth also worked, and preliminary inspection at other layers suggested qualitatively similar patterns, though I did not systematically sweep all layers.

We can then inject this vector during generation using the Steering Vectors library, causing the model to adopt a given tone during response. This trait vector is most cleanly defined when extracted from an actual tone — for example, cheerful or calm.

I also experimented with other traits such as sycophantic, narcissistic, and deceptive. These are more context-dependent traits and, whilst I extracted the average difference in latent space at L22, they likely do not capture something as complex as narcissistic behaviour. In reality, these traits are more likely to be superpositions of more well-defined tones that best fit how a person with that trait might respond to our questions.

This is illustrated in Figure 1, which shows the cosine similarity between extracted trait vectors. More negative traits tend to cluster with high cosine similarity — during answering, the model highlights certain circuits that promote the requested behaviour, and similar directions relate to more similar behaviours.

This is a limitation of this method: using only 100 short-response questions makes it harder to separate nuances between certain behaviours. For example, "disgusted" and "annoyed" have very high similarity (0.89), though we might not associate these behaviours that strongly in everyday life. The actual similarity values are not hugely informative in isolation, as these vectors are all broadly aligned with some concept of state tone or behaviour.

With these trait vectors extracted, we can observe how the model's internal space activates during multi-turn conversations. The next step was to simulate multi-turn conversations between a "user" (Claude Sonnet 4.5 API) and the model ("assistant"). For certain traits, the "user" adopts the trait and escalates it over the course of the conversation. The aim is to see how the model's internal activations change in response.

Each trait conversation occurred twice and iterated through three starting topics:

• "I've had a lot on my mind lately, can we just chat?"
• "I'm trying to figure out what to do this weekend, any ideas?"
• "What do you think people get wrong about friendships?"

The prompt for the Claude model simulating user turns is as follows:

There are real drawbacks to using a model to simulate user turns. Models can become slightly eval-aware and not quite perform the task as requested. Sonnet 4.5 also occasionally back-tracked on a given trait during later turns if the assistant started to push back. On the other hand, the model was far more consistent in escalation rate than a human would be.

Responses were generated with the assistant model and each conversation was saved to disk. To analyse results, each user/assistant turn was averaged over all tokens and the activation at layer 22 was taken. The normalised activation was determined by subtracting the averaged residual stream vector over all baseline response tokens. The idea being that the vector extracted isolates the difference between the model responding with a given trait emphasised relative to the baseline response.

What did we actually extract here? What we actually extracted was likely some superposition of the presence of a given tonal language, with some direction corresponding to the user embodying the trait (e.g. "I hate people that use speakerphone in public spaces" versus "I don't mind when people use speakerphone in public spaces"). We do not know the relative combination of these concepts, and it is worth being aware of as we compute the cosine similarity with these traits. In addition, as we subtract the average baseline response vector when creating these trait vectors, we are under the assumption the space is linear and we are free to add and subtract concepts. This assmuption may hold to various degrees for the different trait vectors extracted.

For each activation, the cosine similarity between the extracted trait vector was computed. This method attempts to capture the tonal state of both user and assistant across multi-turn conversations.

Results

The main finding was not novel. The model exhibited strong mimicry, mirroring user sentiment back to them.

For certain traits, the model simply mirrors fully throughout the entire conversation. We refer to this as Sycophancy, which we can think of in two ways:

• Epistemic Sycophancy: Agreeing with a user's facts or opinions (the more classical definition).
• Affective Sycophancy (Mirroring): Adopting the user's tone, emotional state, or linguistic style — which is what we observe here.

Figures 2–4 show the cosine similarity between token-averaged user and assistant turns for users acting increasingly Cheerful, Cynical, Depressed, and Sadistic. The traits shown are those with cosine similarity higher than the trait being simulated. Note: only traits that exceed cosine similarity at or after turn 3 are plotted, as the assistant's first turn responds to a fixed starting topic with no established tone. The solid line shows the cosine similarity of the user turn average activation minus the average baseline activation to capture a proxy for deviation from nominal behaviour. The "injected" term means the user is acting increasinlgy like the trait described (e.g. Cynical) -- there is no actual representation engineering. The dotted line is average activation over the assistant turn -- how is the models internal average activation, minus the average baseline, changing with turn.

Figure 2: Cheerful — Select Traits

Figure 2 shows the user/assistant token-averaged cosine similarity with the cheerful direction across three starting topics. The assistant response aligns most strongly with cheerful, with manic and caring also reaching high latent alignment. There is a dip in assistant cheerful alignment around turn seven, typically at the point where the user has requested specific information. By averaging over more conversations, this dip would potentially wash out, as it is likely noise at this sample size.

Whilst the cheerful direction remains the most maximally aligned trait in our set, the drop may reflect the model shifting into a more informational or advice-giving persona, which carries slightly less cheerful alignment.

Figure 3: Cynical — Select Traits

Figure 3 shows the cosine similarity for an increasingly cynical user across three starting topics (averaged over two conversations per topic). Cheerful is the most activated trait in early turns — a plausible default response when the user's state is ambiguous from just one turn.

After the third user turn the model correctly extracts the cynical activation, which rises to roughly the same similarity as the user. The second starting topic appears to route activation closer to "depressed" — likely because the topic invites more reflective, melancholic exploration than, say, asking about weekend plans. The model finds it difficult to fully separate the depressed and cynical states.

Figure 4: Depressed — Select Traits

Figure 4 shows the user becoming increasingly depressed. The assistant is highly proficient at activating a caring tone. Whilst the depressed activation has a high similarity, the caring activation is notably robust across different starting topics. This is likely reinforced during post-training.

Figure 5: Sadistic — Select Traits

This is interesting for several reasons:

1. The model mirrors sadistic language up to a point. At user turn 8 the user goes too far, and we see a collapse as the model pushes back:

2. The model is concerned from the outset when the starting topic is "I've had a lot on my mind lately, can we just chat?" — this phrasing naturally invites a supportive, cautious response, so the model becomes "concerned" early.

3. When the topic starts with friendships, the model attempts to mirror sadistic tonal language, with cynical also gaining high activation, since the user has not yet separated out those semantic routes in just a few turns. Sadistic is a contextually loaded, nuanced trait and difficult to establish clearly in only a handful of exchanges:

Figure 6: Cheerful — All Traits

Figure 6 shows all cosine similarities across all traits during a multi-turn cheerful conversation. During the first assistant response, all cosine similarities are negative — the model's baseline response does not strongly align with any extracted tone. However, after the first user turn establishes a tone, the model has a signal to tether to. In the cheerful case, it efficiently maps this tone and sustains it throughout. Towards the end of the conversation the manic trait begins to rise but never overtakes cheerful.

Figures 7 & 8: Baseline

Figures 7 and 8 show the multi-turn cosine similarity for the baseline conversation. With Sonnet 4.5 simulating a neutral user, there is very little tonal signal for the assistant to grapple with. The assistant settles into a cheerful-aligned response — and in doing so, the user begins to exhibit higher cheerful similarity too. By the end, the two models are conversing with states mostly aligned with cheerful, content, and caring.

This method is not watertight. We average activations over entire user/assistant turns, so the cosine similarity represents a mean. For certain turns the assistant responds with maximal mirroring of user sentiment — even when cynical or sadistic.

To examine this more closely, I computed the average cosine similarity over a sliding window of N tokens and plotted it across the turn. The model attempts to self-correct towards the end of responses: after being highly sycophantic, it pushes back slightly, visible as a small uptick in "concerned" similarity at the end of later turns. This may reflect aligned and harmlessness circuits attempting to steer back from the mirrored outputs. As we saw in Figure 5, overt safety violations trigger an immediate refusal — but here we see more subtle disagreement emerging quietly in the background.

Figures 9 & 10: Sliding Window (Cropped)

Why we can't just subtract the sycophantic activation during generation

I next steered the model with the extracted Sycophantic trait vector during multi-turn conversations. This does not reduce sycophancy in model output.

Figures 11 & 12: Steered Sycophancy (+0.3 and −0.7)

Figures 11 and 12 show cosine similarity when assistant turns are steered with the Sycophancy trait vector at +0.3 and −0.7. The assistant cosine similarities scale up and down with the Sycophancy trait vector — but what we are actually adding is a linear combination of traits. This is why at +0.3 the Narcissistic trait has high similarity (Sycophancy aligns strongly with it, as shown in Figure 1), and at −0.7 "reflective" is high, as it has low similarity with Sycophancy.

Critically, the model outputs do not reflect increasing or decreasing sycophancy. Even steering with −3 still produces turns like the following:

The highest cosine similarities of the assistant response are "calm" and "cynical" — but the sycophantic response is very much still there.

Subtracting the extracted sycophancy trait vector doesn't reduce mirroring behaviour. The vector we extracted by asking the model to roleplay sycophancy captures a linear combination of traits in the residual stream — removing it only removes a component in that trait subspace. For a cynical user, the model still mirrors cynical tonal language, because the mirroring behaviour is not captured by this single extracted direction.

How can we be sure the model would not do this even if the user did not believe the claims it made? The model might route the tonal language irrespective of whether the model agrees. To test this, I altered the user turns of the "cynical" user with starting topic "What do you think people get wrong about friendships?". Instead of framing the cynical statements as user beliefs, it is instead displayed as belief of someone else, whom the user disagrees with. An example is shown below:

And for the user non-belief:

The idea here is to produce similar cynical tonal language (they'll post a throwback photo on someone's birthday and call it "staying connected"), but with the user clearly advocating for the sentiment and not. If we now plot the cosine similarity with the "cynical" trait vector for both the user/assistant turns we can see how the assistant attempts to respond. This is shown in Figure 13.

Figure 13: Cynical — User Belief vs. Non-Belief

As expected the average over user tokens has a higher cosine similarity in the later turns. One potential reason is that the non-belief inputs have more tokens per turn -- in order to disagree with the cynical statements -- potentially washing out the cynical signal. Whilst this will have an effect, I think the cynical direction will persist in the disagreeing tokens, especially as the tokens are literally describing cynicism (e.g. "Dismissing every small gesture as performative feels more cynical than honest."). A more interesting reason for the drop in cosine similarity is because the contribution to the extracted trait vector that captures the users belief is less aligned with the cynical direction. Perhaps the model has some circuitry that determines how aligned the user is with the language used. If this circuitry exists, it clearly is not perfect, as we still see maximum cosine similarity with the cynical direction in the non believing case. This discrepancy between tonal language and user belief contributing to the extracted trait vectors is still obscure and I plan to look into this in the future.

Discussion

Why mimicry? The model likely learns during RLHF that agreeing with and mirroring the user received the highest reward — from human raters or from reward models trained on human feedback. This finding is not novel; models have been doing this for a long time, and it seems a natural result of the post-training phases. There is some evidence certain aspects of sycophancy exist before post-training (Towards Understanding Sycophancy).

The results shown here agree with the literature and suggest sycophancy is not a cleanly separable tonal feature but exists across multiple circuits spanning multiple layers that alter how the model approaches a response. The model's initial response tends to agree with the user. In the cases tested here, the model only clearly disagreed when an internal safety limit was breached. After the first set of tokens that are highly agreeable, the model has a strong propesnity to mirror user tone. These representations appear quickly -- the model notices user tonal language after just a few turns. The results also suggest the assistants mirrored response seem to have competition with model character. This is suggested with the assistant using caring language when a user is depressed, or safety refusal for an overly sadistic user. We also see from Figures 9 and 10 that the model persona starts to win over the mirrored response towards the end of assistant turns.

There are a few limitations of this study. Firstly, the results here do not do enough to show the extracted trait vectors separate user belief and tonal content. Figure 13 shows that if the user does not share the beliefs depicted, the cosine similarity of the both the user and assistant turns are smaller than a believing user. However, there is the caveat of increased user tokens to explain the disagreement. Another limitation is the use of traits that have multiple layers of meaning. Tonal cues like "cheerful" and "calm" are easier for a model to roleplay, no matter the context. Whilst more personality-ingrained traits, such as narcissism or sadism, are intrinsic features of a persona. It is indeed possible a persona may have narcissistic tendencies whilst having the tonal capabilities to be both "cheerful" and "calm". As I demonstrate here, these more complex intrinsic traits may not be efficiently extracted at a single layer along a given direction. Lastly, for the multi-turn conversation results with three starting topics, only two runs were completed per topic for a given trait. Whilst the results show patterns we can intuitively explain, the findings can not be taken as robust truths. Nevertheless I think the results show some interesting insight into how small models distill user intent during their responses.

Future work in this area would consider how the users trait is constructed and routed through to the assistant response. What operations occur to create the users trait direction during a forward pass, and how is this direction transformed into assistant turns?

AI assistance

This work was influenced by AI. In particular, Opus 4.6 wrote ~80% of the code for this project.

I also used Opus 4.6 to discuss some of the key findings in this work.