What Assay produces
A proof pack is a self-contained evidence bundle for a single AI system run. Five files. One Ed25519 signature. Independently verifiable.
| File | What it contains |
|---|---|
receipt_pack.jsonl |
Structured evidence records, usually one per LLM call or governance event. |
pack_manifest.json |
SHA-256 hashes of every file included in the pack. |
pack_signature.sig |
Ed25519 signature over the manifest. |
verify_report.json |
Machine-readable verification result. |
verify_transcript.md |
Human-readable verification narrative. |
What receipts record
Included by default: model name, provider, timestamp, prompt hash, response hash, token counts, and call site identifier.
Not included by default: prompt text, response text, API keys, or PII. Full content capture is opt-in and off by default.
What verification checks
Running assay verify-pack ./proof_pack_*/ answers two different questions: whether the evidence was tampered with, and whether the declared governance checks passed.
| Result | Exit code | Meaning |
|---|---|---|
| Integrity PASS / Claims PASS | 0 |
Evidence is authentic and behavior meets declared standards. |
| Integrity PASS / Claims FAIL | 1 |
Evidence is authentic but standards were violated. This is honest failure, not tampering. |
| Integrity FAIL | 2 |
Evidence changed after signing or the pack is structurally broken. |
| Bad input | 3 |
Invalid arguments or missing files. This is not a trust verdict. |
Why the split matters: a system that can prove it failed honestly is more trustworthy than a system that only ever claims to pass.
What an auditor review looks like
uvx --from assay-ai assay verify-pack ./proof_pack_*/ uvx --from assay-ai assay explain ./proof_pack_*/
The reviewer does not need access to the original server, database, or API keys. The pack is designed to travel.
assay explain summarizes what happened, the integrity result, claim results, what the pack proves, and the current trust boundaries.
What Assay proves — and what it does not
What it proves
- Evidence files have not been modified since signing
- Declared call sites emitted receipts under a completeness contract
- Declared governance checks passed or failed against authentic evidence
What it does not prove
- That receipts perfectly represent runtime reality if the operator fabricates a run
- That every possible action was captured outside the instrumented or contracted surface
- That timestamps correspond to real-world time without an external anchor
- That signer authority is established beyond structural cryptographic validity at the base tier
How to strengthen guarantees
Raise the trust tier
- Transparency log: external append-only witness such as assay-ledger
- CI-held org keys: separates the signing key from the individual developer
- External timestamping: third-party proof of “before this date”
Operational boundary
- Assay is a checkpoint governance evidence tool
- It produces tamper-detectable signed evidence at discrete evaluation or run boundaries
- It supplements continuous operational logging; it does not replace it
Framework alignment
| Framework | How Assay helps |
|---|---|
| SOC 2 (CC7.2) | Supports monitoring and anomaly review with portable, tamper-evident evidence artifacts. |
| ISO 42001 | Supports AI management system documentation and audit evidence conversations. |
| EU AI Act (Articles 12 & 19) | Supports traceability and evidence handoff; does not replace operational lifetime logging. |
| NIST AI RMF | Supports governance, accountability, and documentation practices with independently reviewable artifacts. |
CI integration
# 1) Generate pack during test run assay run -c receipt_completeness -- python test_app.py # 2) Verify integrity + claims against locked policy assay verify-pack ./proof_pack_*/ --lock assay.lock --require-claim-pass # 3) Enforce operational gates assay diff ./baseline_pack/ ./proof_pack_*/ --gate-cost-pct 25 --gate-errors 0 --gate-strict
Exit code 0 means merge allowed. Exit code 1 means claims or gates failed honestly. Exit code 2 signals tampering or lock drift. Exit code 3 is bad input.
Engineering handoff checklist
assay scan . assay scan . --report assay run -- python app.py assay doctor
This runbook distinguishes “no supported call sites,” “not instrumented,” and “run command wiring issue” including the required -- separator.
Need the protocol view behind this page?
Decision Escrow describes the four-phase protocol model behind Assay: preflight permits, execution with evidence, settlement, and reputation update. It includes the trust-tier ladder and the path from runtime evidence toward consequence gating.