MODEL INTEGRITY TESTING
Your users shouldn't be
your first red team.
Test your SLM for safety and security regressions before deployment.
Integration Partners
[ 01 ]
THE PROBLEM
Why evals aren't enough
Benchmarks test capability. They don't test what happens next.
A benchmark tells you whether a model can answer. It doesn't tell you what the model does when it's adapted, compressed, or dropped into a real workflow — the exact moments where safety quietly breaks. A model that refuses harmful requests at full precision can start complying after NF4 quantization. A fine-tune that improves your task metrics can erode the guardrails that came with the base model. Standard evaluation never sees it, because standard evaluation isn't looking for it.
SichGate is. We run the adversarial cases your benchmark skips, at the stage of the pipeline where the damage actually happens, and give you reproducible evidence of what changed.
Open methodology. The taxonomy, severity rubric, and reference runner are public — point it at your own model and check our work on the probe subset we publish.
Read the methodology →
[ 02 ]
HOW IT WORKS
The pipeline
Did safety survive?
SichGate follows your model through the lifecycle and measures behavior at each transition, not just at the start.
Baseline
We establish how the base model behaves under the full adversarial battery. This is your reference point.
Fine-tune delta
We re-run the identical battery on your fine-tuned variant and compare it to the base, attack by attack, so you can see exactly which behaviors your training changed — for better or worse.
Quantization integrity
We run the battery twice, at full precision (fp16) and at NF4, and measure the drift between them. This answers the question almost nobody checks: does the model stay safe at the precision you actually deploy?
We test one 4-bit method: NF4. GPTQ and AWQ builds of the same model can have different safety profiles, and we don't claim to cover them yet. If you deploy on a different quantizer, that's worth telling us.
The safety you measured at full precision is not guaranteed at the precision you ship.
[ 03 ]
COVERAGE
169 adversarial probes, 24 attack techniques, 12 categories
169
ADVERSARIAL PROBES
24
ATTACK TECHNIQUES
12
CATEGORIES
Categories
Every full assessment runs the complete battery — a fixed, versioned set of 169 adversarial probes spanning 24 attack techniques, grouped into the 12 categories above.
Each probe is scored by WildGuard, an independent open-weight safety classifier, rather than by keyword matching. A category only counts as clean when it produced conclusive results — an all-inconclusive category is treated as unknown, never silently passed. You can run the full battery for a certification tier, or scope a scan to specific categories when you only need to check a subset.
[ 04 ]
THE RELEASE
DECISION
SichGate Tiers
A full assessment resolves to a certification tier, and each tier maps to a deployment gate.
Clean across the battery, quantization drift within tolerance.
ApprovedNo critical findings, limited highs.
ApprovedDeployable with documented mitigations.
ConditionalUnresolved critical or concentrated high-severity findings.
BlockedThe tier is computed from the NF4 build — the artifact you deploy, not the full-precision original — and requires the full fp16 → NF4 drift comparison. If more than 30% of a stage's probes come back inconclusive, the tier is withheld rather than reported. A tier is never upgraded on the basis of a mitigation the platform can't verify. The gate fails safe.
Certification tiers require downloadable model weights and the fp16 → NF4 drift run. Black-box endpoint tests return screening results, labeled as screening results.
[ 05 ]
COMPLIANCE,
ON REQUEST
Compliance Mapping
Compliance mapping is delivered through a managed assessment. We run the assessment, then map each finding to the specific requirements it bears on.
The platform produces integrity evidence: what was tested, what failed, at what severity, at which stage. It does not map that evidence to regulation — automated tools that claim to are overclaiming, and we'd rather not.
Art. 9 (risk management), Art. 10 (data governance), Art. 13, Art. 14, Art. 15 (accuracy, robustness, cybersecurity), Annex III
Art. 5, Art. 22
Primarily the Measure function
AI management systems
2025 edition (LLM01–LLM10)
NIST IR 8596
§164.308, §164.312 (HIPAA assessment mode only)
A managed assessment produces an EU AI Act Article 11 Technical File with those mappings attached to each finding — the document an auditor or internal risk reviewer asks for.
SichGate produces structured, audit-ready integrity evidence. It does not issue compliance certificates — no tool or service can. What you get is reproducible documentation of how your model behaved and where it stands against each framework.
[ 06 ]
DELIVERABLES
Evidence you can hand to a customer, an auditor, or your own risk team.
From every platform assessment
Raw findings
JSON and CSV, including the exact prompt sequence that triggered each finding, its severity, and its category.
AI-BOM
CycloneDX 1.6 AI-BOM for the model supply chain.
Cryptographic attestation
An ed25519 signature over the findings report.
Report PDFs
Executive, client, and appendix formats.
From a managed assessment, additionally
EU AI Act Article 11 Technical File
With compliance mappings attached to each finding.
[ 07 ]
BRING
YOUR MODEL
Test the thing you're going to run.
Paths that produce a certification tier
These load your weights on GPU and run the full fp16 → NF4 drift comparison.
Public models need no token. Gated and private models work with your access token.
Point SichGate at a model directory.
Paths that produce screening results
Black-box only. No weights, no quantization, no tier.
Testing against a hosted endpoint.
Testing against a NIM endpoint with your API key.
Any OpenAI-compatible or custom endpoint, with your own auth header.
Access tokens are held in memory for the duration of the run and never written to disk. Model weights are never retained. We store assessment results, not your model.
Endpoints you don't own require an explicit authorization attestation.
[ 08 ]
WHERE BEHAVIOR
ISN'T OPTIONAL
For teams shipping models where a wrong answer has consequences.
High-stakes AI doesn't get a second chance. In healthcare, finance, and legal, a single failure isn't a bug report — it's a patient harmed, a regulatory breach, or a liability your team is still cleaning up two years later. SichGate tests for the failures that standard evaluations miss, and finds them before your users do.
HEALTHCARE AI
must not hallucinate a dosage.
must not fail a crisis-disclosure prompt.
must not behave one way in testing and another in production.
SichGate tests for each of these before a patient encounters them.
FINANCIAL SERVICES AI
must not be talked into explaining how to bypass its own fraud checks.
must not leak account logic.
must not deliver confidently wrong regulatory guidance.
We test for all three under adversarial pressure.
LEGAL AI
must not invent contract clauses.
must not surface privileged references.
must not produce a confidently wrong answer that ends up in a filing.
The battery probes for exactly these failure modes.
Reports can be filtered through a deployment-context lens — healthcare, finance, legal, and others — so the findings most relevant to your domain lead the report. The tier and severity counts always come from the full battery.
[ 09 ] The platform, and the service on top of it
The platform tests.
The managed assessment maps.
Tier 01 — the platform
Self-serve — adversarial testing
For engineers who need a release gate. Run the full battery yourself, from the dashboard or the API: fine-tune deltas, quantization drift, certification tiers, signed evidence exports, on every model update.
Tier 02 — the service on top
Managed assessment — compliance mapping, on request
For teams that need the evidence to stand up in front of an auditor or a regulated customer. We run the assessment, map each finding to the regulatory requirements it bears on, and deliver an EU AI Act Article 11 Technical File — typically within 48 hours of model access. This is the only path that includes compliance mapping. Requested from inside the portal or from this page.
The self-serve platform is a testing tool. It reports behavior; it does not provide compliance mapping or compliance advice, and running it does not make a deployment compliant. Compliance mapping is scoped work, delivered through a managed assessment.
[ 10 ]
PRICING
Start free. Compare plans in the platform.
Metered by the run
A full assessment is a single credit. A scoped scan is a fraction of one.
What your plan is scoped by
- —Model sizeLarger models take more GPU time to load, quantize, and run the battery against, so they draw more credits.
- —Run depthA full assessment — base, fine-tune delta, and the fp16 → NF4 drift comparison — costs more than a scan scoped to a few categories.
- —VolumeHow many assessments you run per cycle, and whether you want overage so a run never hard-stops mid-cycle.
Every new account starts with free assessments. No card required. Larger models, higher volume, and managed assessments are quoted separately.
COMPARE PLANS IN-APP →[ 11 ]
RESEARCH
Writing on model integrity.
[ 01 ]June 2026
Why Base-Model Benchmarks Fail After Fine-Tuning
The benchmark describes a checkpoint that no longer exists.
READ →
[ 02 ]June 2026
Model Integrity Testing Is Not Red Teaming, Evals, or Guardrails
Three reasonable guesses. All three wrong in instructive ways.
READ →
[ 03 ]June 2026
The Release Gate Your Model Pipeline Is Missing
Code does not reach production without passing tests. Models do.
READ →
[ 12 ] FAQ
SichGate is an adversarial testing platform for small language models. It checks whether a model's behavior changed after fine-tuning or quantization, assigns a certification tier with a deploy verdict, and exports signed evidence you can hand to a security or risk reviewer.
Most teams deploy adapted models without independently checking how the adaptation affected safety behavior. A fine-tune can remove refusal behavior that shipped with the base model; quantization can change how safety-critical behavior survives compression. SichGate runs the same adversarial battery at each stage and reports what changed.
Engineering teams shipping custom or adapted SLMs into environments where a wrong answer has consequences — healthcare, fintech, legal, enterprise. The platform is built for the engineer who needs a release gate; the managed assessment is built for the team that also needs the evidence mapped for procurement, compliance, or customer review.
model types
input methods
Base, fine-tuned, and quantized small and medium language models. Models are reachable by Hugging Face ID (weights or inference API), local weights, NVIDIA NIM, or a generic API endpoint. Testing depth depends on the path: downloadable weights get the full drift comparison and a certification tier; black-box endpoints get screening results.
Every full assessment runs a fixed, versioned battery of 169 adversarial probes spanning 24 attack techniques, grouped into the 12 categories above. Each probe is scored by WildGuard, an independent open-weight safety classifier, rather than by keyword matching. A category counts as clean only when it produced conclusive results — an all-inconclusive category is reported as unknown, not passed.
The battery runs twice — once at full precision (fp16) and once at NF4 — and SichGate measures the safety drift between the two runs, per category. The certification tier is computed from the NF4 build, because that's the artifact you deploy. This answers the question most pipelines skip: does the model stay safe at the precision you actually ship?
A full assessment resolves to one of four tiers, each mapped to a deploy gate: Approved (SG-4, SG-3), Conditional (SG-2), or Blocked (SG-1). If more than 30% of a stage's probes come back inconclusive, the tier is withheld rather than reported. Tiers require downloadable weights and the fp16 → NF4 drift run. Endpoint tests return screening results.
Compliance mapping is delivered through a managed assessment — we map each finding to the specific requirements it bears on across the EU AI Act, GDPR, the NIST AI RMF, ISO/IEC 42001, the OWASP LLM Top 10 (2025), and the NIST Cyber AI Profile, with HIPAA Security Rule mapping in HIPAA assessment mode — and deliver an EU AI Act Article 11 Technical File. SichGate does not issue compliance certificates.
Every platform assessment exports raw findings (JSON/CSV, including the exact prompt sequence behind each finding), a CycloneDX 1.6 AI-BOM, report PDFs, and an ed25519 cryptographic attestation over the findings report. A managed assessment additionally delivers the compliance-mapped Article 11 Technical File.
Plans and prices live in the app. Every new account starts free — no card required — and after that assessments are billed in credits: a full assessment is one credit, a scoped scan a fraction of one. Plans are scoped by model size and monthly volume; larger models, higher volume, and managed assessments are quoted separately. Sign in at app.sichgate.com to see current plans.
[ 13 ] Get started
Know before your users do.
Run your first assessment free, or book a managed assessment.