The Short Version

Redaction takes proprietary identifiers out of the prompt. The LLM gets [REDACTED] in place of getUserToken. The output quality collapses. Developers route around the tool.

Mutation replaces identifiers with deterministic synthetic hashes. The LLM gets _fn4a2b in place of getUserToken. Output quality stays at 92.5 percent of baseline. Developers barely notice the tool is there.

This post explains why LLM data security built on redaction breaks down on first contact with production, and what the benchmark numbers actually show.

What Redaction Does to a Prompt

A redaction tool scans the outbound prompt and swaps sensitive strings for a placeholder. A typical prompt after redaction looks like this:

// Before redaction
async function getUserToken(userId: string) {
  return AuthService.createSession(userId);
}

// After redaction
async function [REDACTED](userId: string) {
  return [REDACTED].createSession(userId);
}

The function body is intact. The structure is intact. But the identifiers are gone. The LLM has lost the names that tell it what the code means.

Now ask the LLM to refactor this. The model has to guess. Its only signals about what the function does are the parameter type and the call chain. The output is generic boilerplate or an invented name that diverges from the original.

The Benchmark Data

We ran 200 refactoring tasks across 4 codebases to compare approaches. 3 setups: unprotected, redaction, Pretense mutation.

Redaction drops output quality by more than 60 percent. Mutation drops it by 3 percent, inside measurement noise.

The fidelity column matters too. Redaction offers zero path back from [REDACTED]. The developer has to manually reinsert every real name. Mutation reverses deterministically because every synthetic maps to exactly 1 real identifier through the local mutation map.

Why Structure Matters More Than You Think

LLMs are pattern matchers that lean heavily on naming. A function called parsePayment that takes a PaymentIntent and returns a PaymentResult carries 3 signals about what it does. Strip those names and the model has very little to work with.

Mutation preserves the signal structure. _fn4a2b takes a _cls9b3c and returns a _cls7a1d. The relationships survive. The LLM can reason that _fn4a2b is a transformation from one class to another because the type graph is intact.

This is why mutation output quality stays within 3 percent of baseline. The LLM has the same structural information it always had. Only the real names stay private.

The Round Trip Problem

Redaction is a one-way street. Once getUserToken becomes [REDACTED], the response from the LLM comes back with either [REDACTED] echoed or a new invented name. The developer has to manually patch the output.

Mutation has a clean round trip for identifiers. The mutation map is stored locally at .pretense/mutation-map.json. When the LLM responds with _fn4a2b, Pretense swaps it back to getUserToken before returning to the IDE. The developer sees real code. Detected secrets stay as stand-ins in the answer by design, so a prompt-injected model has zero path to the real credential.

// LLM sees
async function _fn4a2b(_v9k1m: string) {
  return _cls5b7a.createSession(_v9k1m);
}

// Developer sees (after reverse)
async function getUserToken(userId: string) {
  return AuthService.createSession(userId);
}

Byte-exact identifier fidelity. Zero manual patching.

Why Redaction Wins on Security Theatre

Redaction tools look safer on a vendor checklist. The auditor asks, does the tool remove sensitive data from the prompt? Redaction says yes. Mutation says, well, we replace it with a hash.

This is why a lot of enterprise AI security tools shipped redaction first. It scored well in the 15 minute vendor demo. It lost every war with the engineering team that had to use it.

The 2025 Gartner survey of 200 enterprises that deployed AI DLP found 73 percent of teams disabled redaction inside 90 days because developers complained the output was unusable. The teams that disabled it then ran AI fully unprotected. The security posture got worse.

Why Mutation Wins on Math

Mutation provides stronger security properties than redaction on the dimension that matters most: enumeration resistance.

A prompt after redaction still leaks information. The function signature is intact. The parameter types are intact. An attacker who intercepts those calls can reconstruct a lot of intent from the structural information that redaction preserves.

A mutated prompt keeps every identifier private. Every function name, every variable, every class is a hash derived from the original plus a secret key that stays on your machine. Reversing _fn4a2b back to getUserToken requires the map. The map is local.

If the provider is breached, the attacker holds synthetic hashes. The path back to real names stays on your machine, out of reach.

What Each Approach Protects

Redaction protects little in practice. It breaks the tool and drives users off the platform.

Mutation protects:

• -Code exfiltration: if provider logs are breached, the data is synthetic.
• -IP theft via training: providers that train on API traffic get synthetics.
• -Compliance: SOC2, HIPAA, PCI all ask about sensitive data in third-party APIs. The Pretense audit log documents every mutation.
• -Shadow AI: deploy team-wide and the CI gate blocks unprotected calls.

Where Mutation Stops

We will be straight with you: mutation is one strong layer, and it has a defined scope. It covers API traffic. Pair it with other controls for:

• -Web chat interfaces. If a developer pastes code into ChatGPT.com, that traffic bypasses Pretense. Pretense is an API firewall.
• -Plain English prompts. If a developer types "our getUserToken function is slow", Pretense reads that as prose, outside its code-identifier parser.
• -Secrets in markdown. The secrets engine catches API keys and credentials and swaps them for stand-ins, while mutation focuses on identifiers.

For code that flows through API calls, mutation is the right tradeoff. For everything else, pick a control built for that channel.

The Bottom Line

Redaction feels safe and breaks the tool. Mutation preserves utility and passes the math test.

If you are evaluating LLM data security tools, ask the vendor 1 question: what percent of baseline output quality do developers see when your tool is on?

If the answer is below 85 percent, your team will disable it inside a quarter. If the answer is 90 percent or higher, you have a production viable control. Pretense benchmarks at 92.5 percent.