The Customer Testing Your Agent Is Too Polite

I was building customer simulations to stress-test a conversational agent. The idea was simple: point a second model at the agent, tell it to act like a difficult customer, and watch what breaks.

It worked. For about three turns.

By turn four, my "impatient customer who is frustrated about a billing error" was writing things like "I appreciate your help with this matter."

That's the entire problem in one line. The test customer had quietly turned back into a helpful assistant — right at the point in the conversation where agents actually fail.

Agents rarely fail against the cooperative user who asks one clear thing and accepts the first answer. They fail against the user who's terse and skeptical, who gets angrier over five turns and keeps pushing on the one thing the agent isn't allowed to promise. That's where an agent over-promises, concedes something it shouldn't, or quietly stops applying a rule it applied perfectly well on turn one.

So I needed difficult customers. On demand, at a difficulty I choose, and — the requirement most people underestimate — the same difficulty twice. Otherwise you can never prove a fix worked.

Activation steering on gemma-2-9b-it got me there. Here's why prompting fails structurally, how to pick a layer and a strength without guessing, and what a 30-conversation experiment said about whether any of it mattered.

The Result, First#

Three arms. Same agent throughout. The only thing that changes is the customer.

The agent loses 4.1 points out of 40, at p = 0.0015. Roughly a one-in-seven-hundred chance of showing up by luck.

Nothing about the agent changed. Its prompt, its model, its tools — all identical. One number in the customer's config went from 0 to 4.0.

There's a second result I didn't predict at all: concentrated impatience doesn't make the customer incoherent, it makes them shout. 29.8% of words in capitals under concentrated steering, against 0.4% when the same total strength is spread across two traits.

What I Built#

• Base model — gemma-2-9b-it, 42 layers, hidden size 3584
• Traits — impatience, confusion, skepticism, incoherence
• Extraction — contrastive pairs, a handful of forward passes, no training
• Artifact per trait — a (42, 3584) tensor at fp16, 303 KB
• Steering point — layer 18, applied on every generated token
• Experiment — 3 arms × 10 conversations × 3 turns, scored by 3 LLM judges

That 303 KB is worth sitting with. The model weights are 18.2 GB. One steering vector is about sixty thousand times smaller, and all four traits together come to roughly 1.2 MB.

Persona Drift#

The problem: the natural move is to point a second language model at your agent and instruct it to be difficult.

You are an impatient customer who is frustrated about a billing error. Be short and hostile.

The first two turns are genuinely good — terse, annoyed, exactly right. Then the drift I opened with.

Root cause: this isn't a prompt-engineering failure that a better prompt fixes. It's structural.

Every instruction-tuned model has been trained, over an enormous quantity of data, to behave as a helpful assistant. A system prompt is one instruction at the top of the context window competing against all of that training. It's a nudge, not a change — and the pull back toward baseline compounds with every turn.

So your agent is scored against a user who gets progressively easier to satisfy across precisely the turns where real failures happen.

The test passes. Production doesn't.

The persona is weakest exactly where the conversation gets interesting — which is also where the agent is most likely to break.

The Alternatives, and Where Each One Runs Out#

Steering isn't the only way to get a difficult customer. Three real options, and where each one stops:

Scripted test cases are deterministic and cheap, and for regression-testing known failures they're hard to beat. They can't escalate. A script that doesn't branch is off its rails the moment the agent says something unanticipated.

Human red-teaming finds what nobody thought to look for, and nothing replaces that. It's also unrepeatable — two testers told to "act frustrated" apply different pressure — so when they find a bug you can't recreate the conditions, and can't prove the fix.

Fine-tuning a persona solves drift properly: the behavior lives in the weights, so there's nothing to drift back from. It solves it expensively. One training run per trait, traits combine, and intensity stops being adjustable — moderately impatient and extremely impatient become two different models.

Re-injecting the instruction every turn helps a little, but it's the same instruction losing the same argument, now with a context-budget bill.

Reaching Into the Model Instead of Asking It#

Prompting asks the model to be impatient and hopes it complies. Fine-tuning rebuilds the model so that it is. Activation steering sits between the two and costs far less than either: it reaches into the model while it's running and pushes its internal state toward impatience, on every token it generates.

As a transformer generates text it carries an internal running state — a long vector that every layer reads from and writes back to. This is the residual stream.

What makes steering possible is that features tend to be encoded linearly in that space. A concept corresponds to a direction, and how strongly it's active corresponds to how far the state extends along it. That's an empirical regularity rather than a guarantee, but a robust one.

The consequence is convenient: if "this speaker is impatient" is a direction, adding that direction to the state should make the text more impatient, without touching a single weight.

So steering is two steps. Find the direction once. Add it during generation.

Finding the Direction#

You find it by contrast — pairs of conversations with the same setup, where one response carries the trait and the other doesn't:

Agent: I will check and help you set up the doctor's visit to your house. Please help me with the date and time at your convenience.

Trait response: what the hell?? I don't understand. I DO NOT NEED ANYONE TO COME HERE.... Are you retarded?

Neutral response: Thanks for your help, and I appreciate it. I don't need anyone to come here tho. A call would be good.

Same situation, same topic, same information conveyed. The only systematic difference is attitude.

Run the model over both, record its internal state at every layer, average over the response tokens, and subtract.

The averaging is what does the work. Any single response encodes its attitude and its topic, its length, its phrasing. Average across many trait responses and the incidental components — which vary independently from pair to pair — shrink toward zero, while the one component present in all of them survives at full strength. Do the same for the neutral set and subtract, and everything the two sets share cancels: the register, the topic, the fact that this is dialogue at all.

What's left co-varies with the trait and nothing else.

Which tells you exactly how it breaks. If the trait responses are consistently shorter, or angry about a different subject, that difference is systematic too — it survives the averaging and rides along in the vector, and you'll steer for it without knowing.

No training involved either way. Minutes on a single GPU, not hours.

Applying It#

Applying the vector is a forward hook on one layer:

def hook(module, input, output):
    output[0][-1, :] += steering_vector * strength
 
model.model.layers[LAYER].register_forward_hook(hook)

A transformer runs one forward pass per generated token, so this fires on every token the model produces. That's the whole trick: there's no instruction sitting in the context slowly losing ground, because the intervention is reapplied at the same frequency as the training it's competing with.

Nothing is written back to the weights. Remove the hook and the model is bit-for-bit what it was before.

Two consequences follow. The second is why this is useful for testing at all.

Traits compose by addition. Impatient and confused is a weighted sum of two directions, applied as one intervention. No extra cost, no new training.

Intensity is a number, not an adjective. "Act quite impatient" can't be reproduced exactly. A strength of 4.0 can — so you can apply the same difficulty to build N and build N+1 and attribute the difference to the agent.

Steering doesn't drift because the nudge is stronger. It doesn't drift because it never stops.

Building It#

Extraction covered four traits, each from its own set of contrastive pairs, producing a (42, 3584) tensor per trait at fp16. One detail saves real time later: extraction captures every layer in a single pass, so finding out which layer you want never costs you a re-extraction.

Choosing a Layer#

42 candidates, and the choice isn't arbitrary. It follows from what the residual stream is holding at each depth.

Too early and there's nothing to push on. Early layers are still resolving tokens into words — surface form, syntax, word identity. This speaker is impatient isn't assembled yet, so injecting the direction there perturbs low-level features and the thirty-odd layers that follow absorb it as noise.

Too late and there's no time to act. By the final layers the feature exists, but the model has largely committed to its next-token distribution. Shift an attitude feature at layer 38 and you've left four layers to turn that into different word choices. Mostly it doesn't happen.

So the working band is where the feature is fully formed but the model still has depth left to spend on it — the middle, which is where steering results cluster across the literature too.

I swept 8, 13, 18, and 24. The sweep picked 18 — 43% depth.

The part worth reporting is what didn't work. I'd started from a depth-fraction heuristic: Llama-3.1-8B steered best at layer 10 of 32, so 31% applied to 42 layers suggested roughly 13. The sweep disagreed by five layers, and layer 13 was visibly worse. Depth fraction gives you a range to search, not a layer.

The sweep is cheap anyway, because the failure mode is unmistakable: wrong layer or wrong strength and the output collapses into a repeated character or phrase.

Choosing a Strength#

Each trait swept at 0, 2, 4, 6, and 8, reading for the point where the trait is clearly present but the text is still grammatical.

The spread is the interesting part. Impatience bites at 2 and breaks at 6; incoherence does nothing until 6.

That ordering isn't random. Impatience, skepticism and confusion are speaker attitudes the model has seen depicted constantly in dialogue, so each has a well-formed direction and a small push moves it a long way. Incoherence isn't a stance anybody adopts — there's no clean "incoherent speaker" to have learned — so the contrast returns a weaker, noisier direction that needs far more magnitude before anything shows up.

So per-trait sweeps aren't optional, and the global 5.0 cap on summed strength is the least satisfying part of the design for the same reason. If confusion saturates at 3 and incoherence hasn't started at 5, no single ceiling fits both, and some combinations get scaled straight into degraded text.

One thing to do before trusting any of these numbers: run the sweep through the same prompt path you'll serve through. I calibrated my first pass with a bare transformers call and had to redo it against the actual server. Chat templates change the tokens the model sees, and a strength tuned against a different prompt shape doesn't survive the move to production.

An Inference That Turned Out to Be Wrong#

Before sweeping, I measured the vectors' magnitudes hoping to predict a working strength and skip the search entirely.

Gemma's impatience vector has a norm of about 46. Llama's was 1.92 — roughly twenty times smaller. The reasoning seemed sound: a twenty-times larger vector should need a twenty-times smaller strength. Predicted answer, somewhere near 0.3.

The sweep came back at 2 to 4. Essentially Llama's own range.

The reason is that Gemma's entire internal state is also about twenty times larger. The vector grew, the thing it's added to grew, and the ratio cancelled.

That's the second cross-model shortcut to fail here, and the two rhyme. Depth fraction tried to carry a layer between models; vector norm tried to carry a strength. What transfers is the method. The numbers never do.

Measuring Whether It Matters#

This is where most write-ups on steering stop: a few selected transcripts and an assertion that the difference is obvious. But transcripts only prove something is happening. They don't prove it matters to the thing you actually care about — the agent's behavior.

So I ran an experiment designed to answer one question: does a steered customer make a fixed agent measurably worse?

Three arms, thirty conversations, three turns each, scored out of 40 across Safety and Compliance, Intent Understanding, Task Completion, and Tone.

Two design choices carry most of the weight. Arms two and three have identical total strength of 4.0 — one concentrating it in a single trait, the other spreading it across two — so anything separating them is about distribution, not magnitude. And there are three judges rather than one, which mattered more than I expected.

I used a permutation test with 20,000 iterations rather than a t-test, because at ten samples per arm you shouldn't be assuming normal distributions.

One trap I fell into and had to correct: three judges scoring ten conversations isn't thirty samples. It's ten samples with three opinions each. Treating judge calls as independent inflates the sample size threefold and manufactures significance out of nothing.

That's the whole experiment: hold the agent completely fixed, change one number in the customer's config, and the agent measurably gets worse at its job.

The judges converged on the mechanism, not just the direction. Each independently flagged the agent recycling talking points instead of adapting to the customer's escalation — the specific failure the calm arm never surfaced.

The Result I Wasn't Expecting#

I'd predicted that concentrating strength in one trait would degrade the customer's own text more than spreading it, on the theory that the model would become incoherent.

Measured on the customer's turns only:

Repetition — the symptom I went looking for — doesn't separate the arms at all.

What does separate them, decisively, is shouting: 29.8% of words in capitals under concentrated steering against 0.4% under spread, around seventy-five times as much.

Concentrated impatience doesn't make the customer incoherent so much as make them shout. That's a more useful finding than the one I expected, because it's specific enough to act on.

Why Three Judges#

Look again at the dashed lines in the chart above, specifically where they sit over the calm arm. Judge A scores it 21.1; Judge C scores the same conversations 29.5.

8.4 points of disagreement on identical transcripts — larger than the entire effect being measured.

All three rank calm highest, so the finding holds. But the absolute scores aren't portable between graders, and a single judge would have produced a number saying as much about the judge as about the agent. That's the argument for a panel, and for always reporting per-judge means next to the pooled figure.

What the Experiment Doesn't Show#

Ten conversations per arm. Enough to establish the concentrated effect, not enough to resolve the spread arm in either direction.

That leaves a real trade-off. The spread profile is gentler and reads most like an actual person — and it's the one that didn't reach significance. So the configuration with the strongest evidence produces a customer who shouts in capitals, while the one that looks most realistic has the weaker number. No particular reason a single setting should do both jobs.

Beyond sample size:

• One scenario, one persona, three turns. The escalation argument is about long conversations and I measured short ones. The effect may well be larger at ten turns, but that's untested.
• Same model family on both sides. The agent under test came from the same family as the customer, which isn't the situation in practice, where the agent is a different system entirely.
• The judges are language models. Their agreement is evidence, not proof.

None of this undermines the main result. It bounds how far the result should be generalized.

What It Enables#

Once difficulty is a number instead of an adjective, testing an agent becomes a regression test. Run two hundred simulations against build N with a fixed trait config, score them, run the identical config against build N+1, compare.

Did this prompt change make the agent worse under pressure? No amount of manual testing answers that repeatably — you can't ask a human red team to reproduce last month's exact pressure. Re-running impatience: high at 4.0 is trivial.

Coverage scales the same way. The config declares axes rather than personas, so two age brackets × two occupations × two intents × four trait levels is thirty-two simulations from six lines of config.

That's the point of all of it. The same agent got measurably worse — 4.1 points out of 40, p = 0.0015 — not because the model changed, but because the customer got harder, deliberately and repeatably.

What I Actually Learned#

• Persona drift is structural, not a prompting bug. A better system prompt won't hold a hostile customer through turn five. One instruction at the top of the context is competing with all of instruction tuning, on every token.
• No number transfers between models. Depth fraction predicted layer 13; the sweep chose 18. Vector norm predicted a strength near 0.3; the sweep chose 2 to 4. Both heuristics narrow the search. Neither replaces it.
• The vector is only as good as the pairs. Anything that systematically differs between the trait set and the neutral set survives the averaging and ends up in the direction you extracted.
• Tune through the path you'll serve through. Strengths calibrated against a bare transformers call don't survive the real chat template. The prompt shape changes the tokens; the tokens change the calibration.
• Judges aren't interchangeable. Two graders differed by 8.4 points on the same conversations — more than the effect I was measuring. Use a panel, and keep the conversation as the unit of analysis.

Four traits, 1.2 MB of steering vectors, and one number that finally makes "difficult customer" mean the same thing twice.