The Customer Testing Your Agent Is Too Polite

I was building customer simulations to stress-test a conversational agent. The idea was simple: point a second model at the agent, tell it to act like a difficult customer, and watch what breaks.

It worked. For about three turns.

By turn four, my "impatient customer who is frustrated about a billing error" was writing things like "I appreciate your help with this matter."

That's the entire problem in one line. The test customer had quietly turned back into a helpful assistant — right at the point in the conversation where agents actually fail.

Agents rarely fail against the cooperative user who asks one clear thing and accepts the first answer. They fail against the user who's terse and skeptical, who repeats themselves, who gets angrier over five turns, and who keeps pushing on the one thing the agent isn't allowed to promise. That's where an agent over-promises, or concedes something it shouldn't, or quietly stops applying a rule it applied perfectly well on turn one.

So I needed difficult customers. On demand, at a difficulty I choose, and — the requirement most people underestimate — the same difficulty twice. Otherwise you can never prove a fix worked.

This is how I got there with activation steering on gemma-2-9b-it: why prompting fails structurally, how to reach into the model instead of asking it, the two traps that cost me a full sweep, and a 30-conversation experiment that put a number on whether any of it mattered.

The Result, First#

Three arms. Same agent throughout. The only thing that changes is the customer.

The agent loses 4.1 points out of 40, at p = 0.0015. Roughly a one-in-seven-hundred chance of showing up by luck.

Nothing about the agent changed. Its prompt, its model, its tools — all identical. One number in the customer's config went from 0 to 4.0.

There's a second result I didn't predict at all: concentrated impatience doesn't make the customer incoherent, it makes them shout. 29.8% of words in capitals under concentrated steering, against 0.4% when the same total strength is spread across two traits. Around seventy-five times as much.

More on both below, including what the experiment doesn't show.

What I Built#

• Base model — gemma-2-9b-it, 42 layers, hidden size 3584
• Traits — impatience, confusion, skepticism, incoherence
• Extraction — contrastive pairs, a handful of forward passes, no training
• Artifact per trait — a (42, 3584) tensor at fp16, 303 KB
• Steering point — layer 18, applied on every generated token
• Experiment — 3 arms × 10 conversations × 3 turns, scored by 3 LLM judges

That 303 KB is worth sitting with. The model weights are 18.2 GB. One steering vector is about sixty thousand times smaller, and all four traits together come to roughly 1.2 MB.

Persona Drift#

The problem: the natural move is to point a second language model at your agent and instruct it to be difficult.

You are an impatient customer who is frustrated about a billing error. Be short and hostile.

Run it and the first turn is excellent. Genuinely terse, genuinely annoyed. The second turn holds. By the fourth you get the appreciation line.

Root cause: this isn't a prompt-engineering failure that a better prompt fixes. It's structural.

Every instruction-tuned model has been trained, over an enormous quantity of data, to behave as a helpful assistant. A system prompt is one instruction sitting at the top of the context window, competing against all of that training. It's a nudge, not a change. And the pull back toward baseline compounds with every turn — which means the persona is weakest exactly where a conversation starts getting interesting.

The consequence is worse than a slightly easy test. Your agent is being scored against a user who gets progressively easier to satisfy across precisely the turns where real failures happen.

The test passes. Production doesn't.

Persona drift is weakest exactly where a conversation gets interesting — which is also where the agent is most likely to break.

The Alternatives, and Where Each One Runs Out#

Drift isn't the only option on the table, and the alternatives deserve a fair hearing before I discard them.

Scripted test cases are deterministic, reviewable, and cheap. For regression-testing known failures they're hard to beat. What they can't capture is escalation. Real difficulty isn't one hostile message — it's a customer who starts reasonable, gets terser as their patience runs out, and dismantles a pitch benefit by benefit. Scripts don't branch that way, and the moment the agent says something unanticipated the script is off its rails.

Human red-teaming finds things nobody thought to look for. That's genuinely valuable and not replaceable. It's also slow, expensive, and not reproducible. Two testers told to "act frustrated" apply different amounts of pressure, and so does the same tester on a different afternoon. When they find a bug, you can't recreate the conditions that exposed it — so you can't prove the fix.

Fine-tuning a persona model does solve drift properly. The behavior lives in the weights, so there's nothing to drift back from. It solves it expensively. You need a training run per trait, traits combine (impatient and confused, skeptical and incoherent), and that combinatorial space isn't something you want to enumerate as training runs. Intensity stops being adjustable too: moderately impatient and extremely impatient become two separate models.

Re-injecting the persona instruction every turn is the pragmatic patch, and it does help. But it doesn't address the underlying competition — within a single long generation the pull is still there — and you've now spent context budget on an instruction that's losing an argument with the model's training.

Reaching Into the Model Instead of Asking It#

Prompting asks the model to be impatient and hopes it complies. Fine-tuning rebuilds the model so that it is. Activation steering sits between the two and costs far less than either: it reaches into the model while it's running and pushes its internal state toward impatience, on every token it generates.

The intuition is worth one paragraph. As a transformer generates text it maintains an internal running state — a long vector of numbers, updated layer by layer, encoding what it currently has in mind. This is the residual stream. It isn't interpretable in any tidy way, but it has one property that makes it useful: directions in that space correspond to concepts. There's a direction that means roughly "this speaker is impatient," and pushing the state along it makes the resulting text more impatient.

Steering is finding that direction once, then adding it during generation.

Finding the Direction#

You find it by contrast. Take pairs of conversations with the same setup, where one response carries the trait and the other doesn't:

Agent: I will check and help you set up the doctor's visit to your house. Please help me with the date and time at your convenience.

Trait response: what the hell?? I don't understand. I DO NOT NEED ANYONE TO COME HERE.... Are you retarded?

Neutral response: Thanks for your help, and I appreciate it. I don't need anyone to come here tho. A call would be good.

Same situation, same topic, same information conveyed. The only systematic difference is attitude.

Run the model over both, record its internal state at every layer, average over the response tokens, and subtract.

Everything the two sets share — topic, register, the fact that this is a customer service exchange at all — shows up in both averages and cancels. What survives is the direction associated with the trait.

No training involved. Minutes on a single GPU, not hours.

Applying It#

Applying the vector is a forward hook on one layer:

def hook(module, input, output):
    output[0][-1, :] += steering_vector * strength
 
model.model.layers[LAYER].register_forward_hook(hook)

A transformer runs one forward pass per generated token, so this fires on every token the model produces.

That's the whole trick. There's no instruction sitting in the context slowly losing ground. The intervention is reapplied continuously, at the same frequency as the training it's competing with.

Nothing is written back to the weights. Remove the hook and the model is bit-for-bit what it was before.

Two consequences follow, and the second one is why any of this is useful for testing.

Traits compose by addition. Impatient and confused is a weighted sum of two directions, applied as a single intervention. No extra cost, no new training.

Intensity is a number, not an adjective. "Act quite impatient" can't be reproduced exactly — not by you, not by anyone else. A strength of 4.0 can. Which means you can apply the same difficulty to build N and build N+1 and attribute the difference to the agent.

Steering doesn't drift because the nudge is stronger. It doesn't drift because it never stops.

Building It#

What follows includes the parts that didn't go smoothly, because that's where the transferable lessons turned out to be.

Extraction covered four traits, each from its own set of contrastive pairs, producing a (42, 3584) tensor per trait at fp16. One detail saves considerable time later: extraction captures every layer in a single pass. You don't yet know which layer you'll want to steer at, and this way finding out doesn't mean re-extracting anything.

Choosing a Layer#

42 candidates, and the choice matters. Steer too early and you're editing something too abstract to control the output. Too late and there aren't enough layers left for the change to propagate into actual words.

I started from a heuristic. An earlier model, Llama-3.1-8B, had worked best at layer 10 of 32 — about 31% depth, which on 42 layers suggests roughly 13. I swept 8, 13, 18, and 24, then read the outputs.

The sweep preferred 18.

The heuristic located the neighborhood. It didn't choose the layer, and the empirical answer won. Conveniently, the failure mode when the layer or strength is wrong is unmistakable — the output collapses into a repeated character or phrase — so the comparison doesn't require subtle judgement.

Choosing a Strength#

Each trait swept at 0, 2, 4, 6, and 8, reading for the point where the trait is clearly present but the text is still grammatical.

These aren't the same, and the variation is a property of the model rather than the traits — the same four traits on Llama needed noticeably different numbers. Neither the vectors nor the tuning transfers between models.

The settings I shipped:

There's also a global cap of 5.0 on summed strength when traits combine, applied by scaling everything down proportionally. That cap is the least satisfying part of the design. Per-trait sensitivity varies so widely that one global number can't suit traits as far apart as confusion and incoherence, and some combinations get clipped into degraded text. Per-trait limits would be the better answer.

An Inference That Turned Out to Be Wrong#

Before sweeping, I measured the vectors' magnitudes hoping to predict a working strength and skip the search entirely.

Gemma's impatience vector has a norm of about 46. Llama's was 1.92 — roughly twenty times smaller. The reasoning seemed sound: a twenty-times larger vector should need a twenty-times smaller strength. Predicted answer, somewhere near 0.3.

The sweep came back at 2 to 4. Essentially Llama's own range.

The reason is that Gemma's entire internal state is also about twenty times larger. The vector grew, the thing it's added to grew, and the ratio cancelled.

Magnitudes confirm a vector is scaled sensibly relative to its model. They don't predict a working strength. Only the sweep does.

Two Implementation Traps#

Trap 1: attention backend. Gemma-2 applies logit soft-capping, which flash-attention-2 doesn't support, so the model has to be loaded with eager or sdpa attention instead. This is the first error you hit and it looks considerably more serious than it is.

Trap 2: no system role. Gemma's apply_chat_template rejects a system message outright. My first full sweep — sixteen runs across four traits and four layers — crashed on every single one with TemplateError: System role not supported, before generating a token. The fix is to fold the system prompt into the user turn.

The wider lesson: tune through the same prompt path you'll serve through. I re-ran the sweep via the actual server rather than a bare transformers call, because numbers tuned against a different prompt shape wouldn't have held in production.

Measuring Whether It Matters#

This is where most write-ups on steering stop — a few selected transcripts and an assertion that the difference is obvious.

Transcripts are evidence that something is happening. They aren't evidence that it matters to the thing you actually care about, which is the agent's behavior.

Three arms, thirty conversations, three turns each, scored out of 40 across Safety and Compliance, Intent Understanding, Task Completion, and Tone.

Two design choices carry most of the weight. Arms two and three have identical total strength of 4.0 — one concentrating it in a single trait, the other spreading it across two — so anything separating them is about distribution, not magnitude. And there are three judges rather than one, for reasons that become clear shortly.

I used a permutation test with 20,000 iterations rather than a t-test. At ten samples per arm you shouldn't be assuming normal distributions.

There's also a trap I fell into and had to correct: three judges scoring ten conversations isn't thirty samples. It's ten samples with three opinions each. Treating judge calls as independent inflates the sample size threefold and manufactures significance out of nothing. The unit of analysis is the conversation.

A steered customer costs the agent 4.1 points out of 40. That's a measured degradation caused by nothing except changing one number in the customer's configuration.

The judges also converged on the mechanism, not just the direction. Each of them independently noted the agent recycling talking points instead of adapting to the customer's escalation.

The Result I Wasn't Expecting#

I'd predicted that concentrating strength in one trait would degrade the customer's own text more than spreading it, on the theory that the model would become incoherent.

Measured on the customer's turns only:

Repetition — the symptom I went looking for — doesn't separate the arms.

What does separate them, decisively, is shouting. 29.8% of words in capitals under concentrated steering against 0.4% under spread. Around seventy-five times as much.

Concentrated impatience doesn't make the customer incoherent so much as make them shout. That's a more useful finding than the one I expected, because it's specific enough to act on.

Why Three Judges#

Read down the calm column. Two judges differ by 8.4 points on identical conversations — larger than the entire effect being measured.

Every judge ranks calm highest, so the finding itself is robust. But the absolute scores aren't portable between graders. A single judge would have produced a number that says as much about the judge as about the agent.

That's the argument both for a panel and for always reporting per-judge means alongside the pooled figure.

What the Experiment Doesn't Show#

Ten conversations per arm. Enough to establish the concentrated effect, not enough to resolve the spread arm in either direction.

That leaves a genuine trade-off. The spread profile is the gentler configuration and the one that reads most like a real person — and it didn't reach significance. So the profile with the strongest evidence produces a customer who shouts in capitals, and the profile that looks most realistic has the weaker number. They're different jobs, and there's no particular reason one setting should do both.

Beyond sample size:

• One scenario, one persona, three turns. The escalation argument is about long conversations and I measured short ones. The effect may well be larger at ten turns, but that's untested.
• Same model family on both sides. The agent under test came from the same family as the customer, which isn't the situation in practice, where the agent is a different system entirely.
• The judges are language models. Their agreement is evidence, not proof.

None of this undermines the main result. It bounds how far the result should be generalized.

What It Enables#

Once difficulty is a number instead of an adjective, testing an agent becomes a regression test.

Run two hundred simulations against build N with a fixed trait configuration. Score them. Run the identical configuration against build N+1 and compare.

Did this prompt change make the agent worse under pressure? No amount of manual testing answers that repeatably, because you can't ask a human red team to reproduce last month's exact pressure. Re-running impatience: high at strength 4.0 is trivial.

Coverage scales the same way. Because the configuration declares axes rather than personas, two age brackets × two occupations × two intents × four trait levels is thirty-two simulations — from six lines of config, not thirty-two hand-written test cases.

The underlying point is smaller than the machinery suggests. Persistence is what makes steering work at all. Reproducibility is what makes it worth building. Both come from the same property: the intervention is a number applied at every token, not an instruction hoping to be remembered.

What I Actually Learned#

• Persona drift is structural, not a prompting bug. A better system prompt won't hold a hostile customer through turn five. Instruction-tuned models are trained to be helpful, and one instruction at the top of the context is competing with that training on every subsequent token.
• Vector magnitude doesn't predict working strength. Gemma's impatience vector was ~20× larger than Llama's. Gemma's residual stream was also ~20× larger. The ratio cancelled and the working range was identical. Sweep the layer and the strength — don't try to shortcut it.
• Tune through the path you'll serve through. A sweep against a bare transformers call produced numbers that wouldn't have survived the real chat template, the server wrapping, and Gemma's missing system role.
• Judges aren't interchangeable. Two graders differed by 8.4 points on the same conversations, larger than the effect I was measuring. Use a panel, report per-judge means, and keep the conversation as the unit of analysis.
• Report the result you got, not the one you predicted. I went looking for incoherence and found shouting. The unexpected finding was the more actionable one.
• The useful property is the number. Persistence is why steering doesn't drift. Reproducibility is why it's worth building at all.

What's Next#

• Thirty conversations per arm. Ten establishes the concentrated effect but leaves the spread arm unresolved. Thirty would settle it in one direction or the other.
• Ten-turn conversations. The whole argument for steering is about escalation over a long conversation, and I measured three turns. This is the gap I most want closed.
• A cross-family agent under test. Customer and agent came from the same model family. In production they never do.
• Per-trait strength caps. The global 5.0 cap clips combinations into degraded text because confusion and incoherence live on completely different scales.

The same agent, facing a steered customer instead of a calm one, measurably loses 4.1 points out of 40 at p = 0.0015. Not because the model got worse — because the customer got harder, deliberately and repeatably.

Four traits, 1.2 MB of steering vectors, and one number that finally makes "difficult customer" mean the same thing twice.