Quantization and Safety Drift: Why the Model You Ship Is Not the Model You Tested

What quantization actually changes

Quantization reduces the numerical precision used to represent a model's weights. A model stored in 16-bit floating point can be converted to 8-bit or 4-bit representations, cutting the memory footprint substantially. Reported compression ratios in the literature run roughly three to fourteen times depending on method and target precision.

In practice this usually means one of a handful of approaches. GPTQ and AWQ are common post-training quantization methods. NF4 through bitsandbytes is common in the open-weight ecosystem, and despite the shorthand it is a 4-bit float format rather than an integer one. GGUF is standard for CPU and consumer-hardware deployment.

Most of these are weight-only schemes, meaning the weights are compressed while activations remain in higher precision. Some schemes, such as W8A8, quantize activations too. The distinction matters for performance and it matters for behavior.

A note on speed, because the claim is usually overstated

Quantization reliably reduces memory. The performance story is more conditional than most summaries suggest.

Weight-only quantization reduces memory traffic rather than arithmetic. In a W4A16 setup the kernel unpacks 4-bit weights back to 16-bit and runs the same floating-point matrix multiply, so peak compute is unchanged. That produces large gains during token-by-token decoding at low concurrency, where the workload is memory-bandwidth-bound. It produces little or nothing during prefill, which is compute-bound, and it pays a dequantization cost on top. Published measurements put that dequantization overhead in a wide range depending on scheme and hardware, and survey work notes the speedup shrinks as batch size and sequence length grow.

None of which is an argument against quantizing. It is an argument for being precise about why you are doing it. If the reason is capacity, more KV cache, fewer GPUs, or fitting a constrained environment, quantization delivers. If the reason is throughput at high batch sizes, the picture is more complicated.

The reason this matters here is that a change made for infrastructure reasons is still a change to the model.

What the research says, including the parts that complicate the story

The honest summary is that the evidence is mixed, and anyone claiming otherwise in either direction is oversimplifying.

Findings that suggest quantization is largely safety-preserving. A 2026 factorial study across eight instruction-tuned models, three precisions, six temperatures, and seven harmfulness benchmarks found that standard quantization was approximately safety-neutral. For seven of the eight models, 4-bit AWQ kept attack success rates within roughly 1.6 percentage points of full precision or lowered them, with clear degradation appearing in only one model. Mechanistic work has found that quantization largely preserves the internal direction associated with refusal behavior.

Findings that suggest real risk. Other work has found that post-training quantization can compromise safety alignment, that quantization-aware training degrades it more consistently, and that standard accuracy metrics fail to capture safety-relevant behavioral changes. Some studies find excessive quantization increases vulnerability while moderate quantization improves resistance to certain attacks. Others find quantization improves robustness against transfer attacks while increasing susceptibility to direct ones.

A finding that matters for smaller models specifically. At least one study reports that larger models behave more consistently across quantization methods, while smaller models diverge more. Since small language models are precisely what gets deployed into constrained and private environments, this is the population where the question is most live and least studied.

A comparison worth keeping in view. Pruning has a clearer record of disturbing refusal behavior than quantization does. Compression is not one operation and the results do not generalize across methods.

So the defensible claim is not that quantization makes models unsafe. It is that quantization changes models, the direction of that change is not consistent across models or methods, and the change is not reliably visible in the metrics teams check.

Why aggregate scores are the wrong instrument

Here is where the safety-neutral findings and the safety-risk findings stop contradicting each other.

An aggregate benchmark score is an average. An average can stay nearly flat while the categories underneath it move in opposite directions. A model can hold its ground on most attack types, lose meaningful ground on one or two, and produce a summary number that looks like nothing happened.

Recent work has named this directly, isolating a selectivity effect that aggregate reporting masks, where the short-form safety checks a release is screened on continue to pass while behavior has shifted underneath.

This is the practical failure mode. It is not that teams skip safety testing. It is that they run it, get a reassuring headline number, and never look at the distribution.

Two additional properties make this worse:

Safety and capability do not move together. A model can remain coherent, useful, and competent at ordinary tasks while changing at the edges: how it handles conflicting instructions, adversarial phrasing, multi-turn escalation, or requests it should decline. Those edges are exactly where consequences live, and exactly where general benchmarks do not look.

Failures are often concentrated, not diffuse. If drift showed up as a uniform degradation across all categories, averages would catch it. It frequently does not.

Quantization is not the only transformation

Focusing on quantization alone understates the problem. Any of the following can shift behavior while leaving a capability dashboard intact:

• Fine-tuning, including on benign data. This is one of the better-documented effects in the literature.
• Pruning and distillation.
• A change of inference runtime.
• Sampling configuration. The same factorial study that found quantization roughly neutral found temperature was the larger risk, with decision instability rising sharply at higher temperatures even when average attack success moved modestly.
• System prompt changes.
• Retrieval configuration changes.
• New tool integrations and expanded permissions.

Sampling temperature is the sharpest example, because it is typically set by whoever is tuning output quality, adjusted freely after launch, and almost never revisited by whoever signed off on safety.

What to do about it

None of this requires treating optimization as dangerous. It requires treating the deployed artifact as its own object of assessment.

Assess the artifact you ship. If you quantized after your safety assessment, the assessment describes a model you are not running. Re-run it against the compressed build.

Report per category, not in aggregate. Whatever battery you use, look at the distribution. A stable headline number with one collapsed category is a finding, not a pass.

Test in the deployed configuration. The same weights behave differently with a different system prompt, runtime, sampling configuration, retrieval layer, or tool set. Testing the weights in isolation answers a narrower question than most teams think it does.

Include multi-turn and adversarial pressure. Single-turn refusal checks are the easiest thing to pass and the least representative of how systems actually get attacked.

Treat every transformation as a trigger. Requantizing, swapping a runtime, adjusting temperature, or adding a tool should each prompt a re-test, in the same way a dependency bump prompts a rebuild.

Version your evidence. Record which artifact was assessed, at which precision, with which method, under which configuration. "We tested the model" is not a durable statement about a model that has been transformed three times since.

The short version

Quantization is a reasonable engineering decision and frequently a necessary one. It is also a transformation of the model, and the transformed model deserves to be assessed as its own artifact.

The evidence does not support panic. It supports measurement. The averages will often look fine, and that is exactly why they are not sufficient.

SichGate stress-tests small language models for adversarial failures across base, fine-tuned, and quantized stages, and reports results per category rather than as a single score. See the methodology.