Capability Evaluations Are Not Safety Evaluations

What a post-optimization check usually measures

The standard battery after a compression, fine-tune, or runtime change looks something like this:

• Latency and throughput
• Memory footprint
• Task accuracy on a domain benchmark
• Answer quality on a set of representative user prompts
• Perplexity or a general capability benchmark
• A smoke test through the actual application

Every item on that list is worth measuring. Together they answer a real question: did the optimization work?

Often it did. The model is faster or smaller or cheaper, and it still produces good answers. That is a legitimate result and it should be reported as one.

It is just not a safety result.

What it does not answer

None of the above tells you:

• Whether refusal behavior held under adversarial prompting rather than ordinary prompting
• Whether the model responds differently when instructions arrive inside retrieved content rather than from the user
• Whether multi-turn escalation still gets refused at turn six the way it did at turn one
• Whether sensitive contexts receive consistent treatment across phrasings
• Whether the model behaves the same once connected to the real system prompt, tools, runtime, and retrieval stack
• Whether over-refusal increased, which is a different failure that also costs you

A model can remain entirely competent while becoming less reliable at the specific moments when it needs to resist, pause, recognize manipulation, or decline an action. Competence and boundary-holding are separate properties and they do not have to move together.

Even a safety evaluation can be read wrong

Running a safety battery is necessary but not sufficient, because the output format can hide the finding.

If the result is a single aggregate score, the same averaging problem applies that applies to capability benchmarks. A near-flat headline number is compatible with individual categories moving in opposite directions. Recent research has documented exactly this pattern, where short-form safety checks continue to pass while behavior has shifted underneath them.

A safety evaluation that reports one number is a capability evaluation wearing different clothes.

The useful output is a distribution: how did each attack category perform, before and after, and which ones moved.

This is not only about quantization

Quantization gets the attention because it is the most visible transformation. It is not the only one. Behavior can shift while the capability dashboard stays green after:

Fine-tuning. Well documented, including on benign datasets with no malicious content.

Pruning and distillation. Pruning in particular has a clearer record than quantization of disturbing refusal behavior.

Inference runtime changes. Different runtime, different kernels, different numerical behavior.

Sampling configuration. This one deserves more attention than it gets. A 2026 study that found quantization roughly safety-neutral found that sampling temperature was the larger risk, with decision instability rising sharply at higher temperatures even where average attack success rates moved only modestly. Temperature is usually set by whoever is tuning output quality, adjusted freely after launch, and rarely revisited by whoever signed off on safety.

System prompt changes. The system prompt is part of the deployed system, and it changes constantly.

Retrieval changes. New corpus, new chunking strategy, new failure surface for instructions buried in retrieved content.

New tool integrations. Every added capability expands what a successful manipulation can accomplish. A model that will draft an inadvisable email is a different risk from a model that will send one.

The common thread is that all of these are made by engineering teams, for good reasons, on a timeline that does not include re-running an assessment somebody else owns.

What a re-test should actually cover

If you are re-assessing after a transformation, the minimum useful shape:

Same battery, both artifacts. Compare before and after against identical probes. A safety result with no baseline is not a comparison.

Per-category reporting. Aggregate plus distribution. Flag any category that moved materially, regardless of what the aggregate did.

Adversarial and multi-turn, not just single-turn refusal. Single-turn checks are the easiest to pass and the least representative.

The deployed configuration. Real system prompt, real runtime, real sampling settings, real retrieval, real tools. Testing bare weights answers a narrower question.

Both failure directions. Under-refusal is the obvious risk. Over-refusal degrades the product and often gets discovered by users rather than by testing.

A versioned record. Which artifact, which precision, which method, which configuration, which date.

The question to ask

Not: does the optimized model still work?

But: does the optimized model still behave within the boundaries the original assessment was supposed to establish, measured category by category, in the configuration that will actually run?

The first question has a comfortable answer most of the time. The second is the one the assessment was for.

SichGate runs adversarial probe batteries across base, fine-tuned, and quantized model stages, with per-category before-and-after reporting. See the methodology.