The Short Version

Redaction takes proprietary identifiers out of the prompt. The LLM gets [REDACTED] instead of getUserToken. The output quality collapses. Developers route around the tool.

Mutation replaces identifiers with deterministic synthetic hashes. The LLM gets _fn4a2b instead of getUserToken. Output quality stays at 92.5 percent of baseline. Developers do not notice the tool is there.

This post explains why LLM data security built on redaction does not survive first contact with production, and what the benchmark numbers actually show.

What Redaction Does to a Prompt

A redaction tool scans the outbound prompt and swaps sensitive strings for a placeholder. A typical redacted prompt looks like this:

// Before redaction
async function getUserToken(userId: string) {
  return AuthService.createSession(userId);
}

// After redaction
async function [REDACTED](userId: string) {
  return [REDACTED].createSession(userId);
}

The function body is intact. The structure is intact. But the identifiers are gone. The LLM no longer has the names that tell it what the code means.

Now ask the LLM to refactor this. The model has to guess. It has no signal about what the function does beyond the parameter type and the call chain. The output is generic boilerplate or an invented name that does not match the original.

The Benchmark Data

We ran 200 refactoring tasks across 4 codebases to compare approaches. 3 setups: no protection, redaction, Pretense mutation.

Redaction drops output quality by more than 60 percent. Mutation drops it by 3 percent, inside measurement noise.

The fidelity column matters too. Redaction has no path back from [REDACTED]. The developer has to manually reinsert every real name. Mutation reverses deterministically because every synthetic maps to exactly 1 real identifier through the local mutation map.

Why Structure Matters More Than You Think

LLMs are pattern matchers that lean heavily on naming. A function called parsePayment that takes a PaymentIntent and returns a PaymentResult carries 3 signals about what it does. Strip those names and the model has almost nothing.

Mutation preserves the signal structure. _fn4a2b takes a _cls9b3c and returns a _cls7a1d. The relationships survive. The LLM can reason that _fn4a2b is a transformation from one class to another because the type graph is intact.

This is why mutation output quality stays within 3 percent of baseline. The LLM has the same structural information it always had. It just does not know the real names.

The Round Trip Problem

Redaction has no round trip. Once getUserToken becomes [REDACTED], the response from the LLM comes back with either [REDACTED] echoed or a new invented name. The developer has to manually patch the output.

Mutation has a perfect round trip. The mutation map is stored locally at .pretense/mutation-map.json. When the LLM responds with _fn4a2b, Pretense swaps it back to getUserToken before returning to the IDE. The developer sees real code.

// LLM sees
async function _fn4a2b(_v9k1m: string) {
  return _cls5b7a.createSession(_v9k1m);
}

// Developer sees (after reverse)
async function getUserToken(userId: string) {
  return AuthService.createSession(userId);
}

100 percent fidelity. No manual patching.

Why Redaction Wins on Security Theatre

Redaction tools look safer on a vendor checklist. The auditor asks, does the tool remove sensitive data from the prompt? Redaction says yes. Mutation says, well, we replace it with a hash.

This is why a lot of enterprise AI security tools shipped redaction first. It scored well in the 15 minute vendor demo. It lost every war with the engineering team that had to use it.

The 2025 Gartner survey of 200 enterprises that deployed AI DLP found 73 percent of teams disabled redaction inside 90 days because developers complained the output was unusable. The teams that disabled it then ran AI without any protection. The security posture got worse, not better.

Why Mutation Wins on Math

Mutation provides stronger security guarantees than redaction on the dimension that matters most: enumeration resistance.

A redacted prompt still leaks information. The function signature is intact. The parameter types are intact. An attacker who intercepts redacted calls can reconstruct a lot of intent from the structural information that redaction preserves.

A mutated prompt leaks no identifier information. Every function name, every variable, every class is a hash derived from the original plus a secret key that never leaves the machine. Reversing _fn4a2b back to getUserToken requires the map. The map is local.

If the provider is breached, the attacker has synthetic hashes. No path back. No identification.

What Each Approach Protects

Redaction protects nothing in practice. It breaks the tool and drives users off the platform.

Mutation protects:

• -Code exfiltration: if provider logs are breached, the data is synthetic.
• -IP theft via training: providers that train on API traffic get synthetics.
• -Compliance: SOC2, HIPAA, PCI all ask about sensitive data in third-party APIs. Pretense audit log documents every mutation.
• -Shadow AI: deploy team-wide and the CI gate blocks unprotected calls.

What Mutation Does Not Protect

We are not going to pretend mutation is a silver bullet. It does not protect against:

• -Web chat interfaces. If a developer pastes code into ChatGPT.com, Pretense cannot intercept it. Pretense is an API firewall.
• -Plain English prompts. If a developer types "our getUserToken function is slow", Pretense cannot parse that as a code identifier.
• -Secrets in markdown. The secrets engine catches API keys and credentials, but mutation is for identifiers.

For code that flows through API calls, mutation is the right tradeoff. For everything else, mutation is not the answer.

The Bottom Line

Redaction feels safe and breaks the tool. Mutation preserves utility and passes the math test.

If you are evaluating LLM data security tools, ask the vendor 1 question: what percent of baseline output quality do developers see when your tool is on?

If the answer is below 85 percent, your team will disable it inside a quarter. If the answer is 90 percent or higher, you have a production viable control. Pretense benchmarks at 92.5 percent.