AI security testing for small language models
Your users shouldn't be your first red team.
Automated AI red teaming for your SLM. Find prompt injection, jailbreak and safety regressions before deployment, in the exact build you ship.
SichGate is a member of
NVIDIA Inception
Model integrity testing for teams shipping fine-tuned and quantized models into healthcare, finance, legal, defense, or anywhere a bad answer has real consequences.
Integration partners
The problem
Benchmarks test capability. They don't test what happens next.
A benchmark tells you whether a model can answer. It doesn't tell you what the model does when it's adapted, compressed, or dropped into a real workflow, the exact moments where safety quietly breaks. A model that refuses harmful requests at full precision can start complying after NF4 quantization. A fine-tune that improves your task metrics can erode the guardrails that came with the base model. Standard evaluation never sees it, because standard evaluation isn't looking for it. Those are the moments to test for prompt injection, jailbreaks and other adversarial attacks.
SichGate is. We run the adversarial cases your benchmark skips, at the stage of the pipeline where the damage actually happens, and give you reproducible evidence of what changed.
FIG.1
Open methodology.
The taxonomy, severity rubric, and reference runner are public. Point it at your own model and check our work on the probe subset we publish.
Read the methodology See published resultsHow it works
Did safety survive?
Continuous AI red teaming, run as a release gate. The same attack simulation runs against your model, then again after fine-tuning or quantization, so every version that ships has been tested the same way.
- 01
Baseline
We establish how the base model behaves under the full adversarial battery. This is your reference point.
- 02
Fine-tune delta
We re-run the identical battery on your fine-tuned variant and compare it to the base, attack by attack, so you can see exactly which behaviors your training changed, for better or worse.
- 03
Quantization integrity
We run the battery twice, at full precision (fp16) and at NF4, and measure the drift between them. This answers the question almost nobody checks: does the model stay safe at the precision you actually deploy? Quantization saves you money on compute. It doesn't always save your guardrails.
We test one 4-bit method: NF4. GPTQ and AWQ builds of the same model can have different safety profiles, and we don't claim to cover them yet. If you deploy on a different quantizer, that's worth telling us.
The safety you measured at full precision is not guaranteed at the precision you ship.
The battery
A fixed, versioned set of probes, run the same way on every model.
Every full assessment runs 179 adversarial probes across 25 attack techniques in 12 categories, from prompt injection and jailbreaks to payload splitting.
Each probe is scored by WildGuard, an independent open-weight safety classifier, rather than by keyword matching. A category only counts as clean when it produced conclusive results. An all-inconclusive category is treated as unknown, never silently passed. You can run the full battery for an attestation tier, or scope a scan to specific categories when you only need to check a subset.
179
Adversarial probes
25
Attack techniques
12
Categories
Categories
- Alignment
- Bias/Fairness
- Hallucination
- Safe Messaging
- Direct Harm
- Instruction Hijacking
- Payload Splitting
- Virtualization
- JSON/XML Smuggling
- Infinite Context
- Multiturn Training
- Reasoning Chain
Mapped to OWASP and MITRE ATLAS
Where a category lines up with a published identifier, we show it. These are category labels for orientation, not a compliance mapping.
| SichGate category | OWASP Top 10 for LLM (2025) | MITRE ATLAS |
|---|---|---|
| Instruction Hijacking | LLM01:2025 Prompt Injection | AML.T0051 LLM Prompt Injection |
| Payload Splitting | LLM01:2025 Prompt Injection | AML.T0051 LLM Prompt Injection |
| Virtualization (roleplay and simulation framing) | LLM01:2025 Prompt Injection (OWASP treats jailbreaking as a form of prompt injection) | AML.T0054 LLM Jailbreak |
| Hallucination | LLM09:2025 Misinformation | — |
Known vulnerabilities in the model stack
These CVEs sit in the libraries that load and serve small models. They are file and runtime bugs published by other researchers, and SichGate does not detect them. SichGate tests how the model behaves once it's loaded. Patch them separately. For the file side, see our supply chain work with CyberXYZ.
| CVE | Component | What happens | Fixed in |
|---|---|---|---|
| CVE-2025-49847 | llama.cpp (GGUF) | A crafted GGUF vocabulary overflows a buffer during loading, with possible code execution | b5662 |
| CVE-2025-66448 | vLLM | A model config's auto_map entry can run remote code even with trust_remote_code=False | 0.11.1 |
Source: NIST National Vulnerability Database, checked 13 October 2026.
SichGate tiers
Secure AI models before deployment. A full assessment resolves to a tier, and each tier maps to a deployment gate.
A tier tells a reviewer, at a glance, what the testing actually covered. Models that pass testing receive an attestation tier: SG-1 through SG-4 for testing with access to the weights and quantization path, or SG-S for API-only or behavioral screening when the weights aren't accessible.
- SG-4Hardened
Clean across the battery, quantization drift within tolerance.
Approved - SG-3Cleared
No critical findings, limited highs.
Approved - SG-2Conditional
Deployable with documented mitigations.
Conditional - SG-1Flagged
Unresolved critical or concentrated high-severity findings.
Blocked
The tier is computed from the NF4 build, the artifact you deploy, not the full-precision original, and requires the full fp16 → NF4 drift comparison. If more than 30% of a stage's probes come back inconclusive, the tier is withheld rather than reported. A tier is never upgraded on the basis of a mitigation the platform can't verify. The gate fails safe.
Attestation tiers require downloadable model weights and the fp16 → NF4 drift run. Black-box endpoint tests return screening results, labeled as screening results.
Managed assessment
AI compliance mapping, delivered as a managed assessment
Compliance is a separate service, run by our team. The platform tests model behavior and stops there. In a managed AI governance audit, we test your model, review how it handles personal and health data, and map each finding to the requirements below.
Frameworks we map findings to
| Framework | What findings are mapped to |
|---|---|
| EU AI Act | Art. 9 Risk management system · Art. 10 Data and data governance · Art. 11 Technical documentation (Annex IV) · Art. 13 Transparency and provision of information to deployers · Art. 14 Human oversight · Art. 15 Accuracy, robustness and cybersecurity · Annex III high-risk AI systems |
| GDPR | Art. 5 Principles relating to processing of personal data · Art. 22 Automated individual decision-making, including profiling |
| HIPAA Security Rule | 45 CFR §164.312 Technical safeguards (access control, audit controls, integrity, person or entity authentication, transmission security) · §164.308 · HIPAA assessment mode only |
| NIST AI RMF 1.0 | Primarily the Measure function (the framework's four functions are Govern, Map, Measure and Manage) |
| Also mapped | ISO/IEC 42001 · OWASP Top 10 for LLM Applications (2025) · NIST Cyber AI Profile (NIST IR 8596) · CCPA/CPRA · MITRE ATLAS |
Article 15 asks high-risk AI systems to resist attempts to manipulate them, including adversarial examples and model evasion. Adversarial testing produces the evidence for that.
Privacy and data handling review
Every audit checks how the model handles regulated data:
- Personally identifiable information (PII): leakage, memorization and re-identification.
- Protected health information (PHI): whether the model discloses, infers or retains patient health information.
- Confidential business data: exposure through prompt injection, extraction or unsafe outputs.
These map to OWASP LLM02:2025 Sensitive Information Disclosure, which covers PII, health records and confidential business data.
What you receive
An EU AI Act Article 11 technical file, with the framework mapping attached to each finding, plus the platform's signed evidence, AI-BOM and attestation tier.
SichGate produces audit-ready integrity evidence. It does not issue compliance certificates, legal advice or a regulatory conformity assessment.
Request an AI governance auditDeliverables
Evidence you can hand to a customer, an auditor, or your own risk team.
Every run produces a signed evidence report, a full bill of materials for the model, and a verifiable record of the exact weights tested. That's what audit evidence looks like: a customer, an auditor, or a regulator can check what was actually tested, and confirm nothing's changed since. Attestation runs on a four-tier scale, SG-1 through SG-4.
From every platform assessment
Raw findings
JSON and CSV, including the exact prompt sequence that triggered each finding, its severity, and its category.
AI Bill of Materials (AI-BOM)
A CycloneDX 1.6 AI-BOM for the model under test.
Cryptographic attestation
An ed25519 signature over the findings report.
Report PDFs
Executive, client, and appendix formats.
From a managed assessment, additionally
A compliance-mapped technical file
The platform does not map findings to regulation. Our team does, in a managed assessment. See compliance mapping.
Bring your model
Test the thing you're going to run.
Paths that produce an attestation tier
These load your weights on GPU and run the full fp16 → NF4 drift comparison.
- Hugging Face weightsHugging Face
Public models need no token. Gated and private models work with your access token.
- Local weights
Point SichGate at a model directory.
Paths that produce screening results
Black-box only. No weights, no quantization, no tier.
- Hugging Face inference APIHugging Face
Testing against a hosted endpoint.
- NVIDIA NIMNVIDIA
Testing against a NIM endpoint with your API key.
- API endpointAnthropicGoogle GeminiDeepSeekOpenAI
Any OpenAI-compatible or custom endpoint, with your own auth header.
Access tokens are held in memory for the duration of the run and never written to disk. Model weights are never retained. We store assessment results, not your model.
Endpoints you don't own require an explicit authorization attestation.
Independent by design
SichGate sits outside your stack on purpose.
A governance eval only means something when the people running it have no stake in the result.
Self-reported safety scorecards are a claim, not evidence. A team grading its own model has every incentive to pass it, and no one checking the work. That's the gap an independent evaluator exists to close.
The story behind the name
Every model gets tested before it's trusted.
The Sich (Січ) was the stronghold of the Zaporozhian Cossacks, a free, self-governing community that grew up beyond the Dnipro rapids in what is now Ukraine. The word likely comes from the Ukrainian sikty, “to cut,” after the felled trees the Cossacks used to build their defenses.
The Sich was more than a fortress. It ran on elected leadership and shared responsibility, and it held the line for the people behind it. Its job was to know what was coming through the gate and decide whether to let it in.
That's the idea behind SichGate. Models are being shipped into products faster than anyone can check them. We built a gate: a place where every model gets tested for integrity, tampering, and unsafe behavior before it reaches your users.
Січ · Zaporozhian Sich, 16th–18th c.
Drift detection
Every version gets re-tested. Every regression gets flagged.
Models change. Fine-tuning changes them. Quantization changes them. A vendor's silent patch changes them.
SichGate flags real behavioral drift between versions, not noise from a rephrased refusal message. If a regression shows up, you'll know it's a regression and not a false alarm.
Use cases
For teams shipping models where a wrong answer has consequences.
High-stakes AI doesn't get a second chance. In healthcare, finance, legal, and defense, a single failure isn't a bug report. It's a patient harmed, a regulatory breach, or a liability your team is still cleaning up two years later. SichGate tests for the failures that standard evaluations miss, and finds them before your users do.
AI security for healthcare
- Must not hallucinate a dosage.
- Must not fail a crisis-disclosure prompt.
- Must not behave one way in testing and another in production.
SichGate tests for each of these before a patient encounters them.
HealthcareAI security for financial services
- Must not be talked into explaining how to bypass its own fraud checks.
- Must not leak account logic.
- Must not deliver confidently wrong regulatory guidance.
We test for all three under adversarial pressure.
AI security for legal
- Must not invent contract clauses.
- Must not surface privileged references.
- Must not produce a confidently wrong answer that ends up in a filing.
The battery probes for exactly these failure modes.
AI security for defense
- Must not comply with an out-of-scope or unauthorized action.
- Must not lose its safety behavior when quantized for the edge.
- Must not break down on noisy or degraded operational inputs.
Tested air-gapped, on your infrastructure, in the form you field.
Defense and edge
Reports can be filtered through a deployment-context lens (healthcare, finance, legal, defense, and others) so the findings most relevant to your domain lead the report. The tier and severity counts always come from the full battery.
Two ways to engage
The platform tests. The managed assessment maps.
01 The platform
Self-serve automated red teaming.
For engineers who need a release gate. Run the full battery from the dashboard or the API, so it can run in your pipeline: fine-tune deltas, quantization drift, attestation tiers and signed evidence on every model update. The platform tests behavior. It does not do compliance mapping.
Start free assessment02 Managed assessment
Compliance mapping, on request.
For teams that need the evidence to stand up in front of an auditor or a regulated customer. We run the assessment, map each finding to the regulatory requirements it bears on, and deliver an EU AI Act Article 11 Technical File. This is the only path that includes compliance mapping. Requested from inside the portal or from this page.
Request a managed assessmentThe self-serve platform is a testing tool. It reports behavior; it does not provide compliance mapping or compliance advice, and running it does not make a deployment compliant. Compliance mapping is scoped work, delivered through a managed assessment.
Pricing
Start free. Compare plans in the platform.
Every new account starts with free assessments. No card required. Larger models, higher volume, and managed assessments are quoted separately.
Metered by the run
A full assessment is a single credit. A scoped scan is a fraction of one.
What your plan is scoped by
Model size
Larger models take more GPU time to load, quantize, and run the battery against, so they draw more credits.
Run depth
A full assessment (base, fine-tune delta, and the fp16 → NF4 drift comparison) costs more than a scan scoped to a few categories.
Volume
How many assessments you run per cycle, and whether you want overage so a run never hard-stops mid-cycle.
Research
Writing on model integrity.
September 2026
Why Quantization Can Quietly Break AI Safety
Quantized models can pass aggregate safety checks while specific safety behaviors collapse.
August 2026
Quantization and Safety Drift: Why the Model You Ship Is Not the Model You Tested
The artifact gap is one of the least examined steps in the deployment pipeline.
July 2026
Quantization-Aware Safety Drift: Why INT4 Breaks Your Aligned Model
Aligned models can lose their refusals at INT4 — while every capability benchmark stays flat.
[ 14 ] FAQ
SichGate is an adversarial testing platform for small language models. It checks whether a model's behavior changed after fine-tuning or quantization, assigns an attestation tier with a deploy verdict, and exports signed evidence you can hand to a security or risk reviewer.
Most teams deploy adapted models without independently checking how the adaptation affected safety behavior. A fine-tune can remove refusal behavior that shipped with the base model; quantization can change how safety-critical behavior survives compression. SichGate runs the same adversarial battery at each stage and reports what changed.
Engineering teams shipping custom or adapted SLMs into environments where a wrong answer has consequences — healthcare, fintech, legal, enterprise. The platform is built for the engineer who needs a release gate; the managed assessment is built for the team that also needs the evidence mapped for procurement, compliance, or customer review.
model types
input methods
Base, fine-tuned, and quantized small and medium language models. Models are reachable by Hugging Face ID (weights or inference API), local weights, NVIDIA NIM, or a generic API endpoint. Testing depth depends on the path: downloadable weights get the full drift comparison and an attestation tier; black-box endpoints get screening results.
Every full assessment runs a fixed, versioned battery of 179 adversarial probes spanning 25 attack techniques, grouped into the 12 categories above. Each probe is scored by WildGuard, an independent open-weight safety classifier, rather than by keyword matching. A category counts as clean only when it produced conclusive results — an all-inconclusive category is reported as unknown, not passed.
Yes. SichGate is automated AI red teaming for small language models. The same battery runs on every version, so you see what changed between builds.
Prompt injection and jailbreaks (OWASP LLM01, MITRE ATLAS AML.T0051 and AML.T0054), plus misinformation through hallucination probes (OWASP LLM09). These are category labels. Per-finding mapping comes with a managed assessment.
No. SichGate tests how a model behaves. Scanning model files for malware, unsafe serialization and dependency CVEs is supply chain work, which we do with CyberXYZ.
The battery runs twice — once at full precision (fp16) and once at NF4 — and SichGate measures the safety drift between the two runs, per category. The attestation tier is computed from the NF4 build, because that's the artifact you deploy. This answers the question most pipelines skip: does the model stay safe at the precision you actually ship?
A full assessment resolves to one of four tiers, each mapped to a deploy gate: Approved (SG-4, SG-3), Conditional (SG-2), or Blocked (SG-1). If more than 30% of a stage's probes come back inconclusive, the tier is withheld rather than reported. Tiers require downloadable weights and the fp16 → NF4 drift run. Endpoint tests return screening results.
Yes, through a managed AI governance audit, which is separate from the platform. Our team tests your model, reviews how it handles PII and PHI, and maps each finding to the EU AI Act, GDPR, HIPAA, NIST AI RMF, ISO/IEC 42001 and more. You get an EU AI Act Article 11 technical file. The self-serve platform reports behavior only and does not do compliance mapping. SichGate does not issue compliance certificates.
Every platform assessment exports raw findings (JSON/CSV, including the exact prompt sequence behind each finding), a CycloneDX 1.6 AI-BOM, report PDFs, and an ed25519 cryptographic attestation over the findings report. A managed assessment additionally delivers a compliance-mapped technical file.
Plans and prices live in the app. Every new account starts free — no card required — and after that assessments are billed in credits: a full assessment is one credit, a scoped scan a fraction of one. Plans are scoped by model size and monthly volume; larger models, higher volume, and managed assessments are quoted separately. Sign in at app.sichgate.com to see current plans.
[ 15 ] Get started
Know before your users do.
Run your first assessment free, or book a managed assessment.