VAL 04 — Audit Completeness Validation¶

1. Purpose and Claims¶

This validation proves that the autonomy platform’s audit subsystem is complete relative to the set of currently implemented security-relevant actions. Completeness is defined across three dimensions:

#	Claim
VAL04-C1	Every active lab phase that performs a security-relevant action writes at least one record to the retained file-backed audit store (store is non-empty; all 6 categories are populated)
VAL04-C2	Every audit record returned from the retained store is schema-complete: it contains all 6 mandatory fields (`event_name`, `category`, `action`, `outcome`, `source`, `timestamp`)
VAL04-C3	The full retained store can be queried end-to-end within a 2000 ms wall-clock latency bound, and the query itself must succeed
VAL04-C4	All 25 of the 25 defined wired event types appear in the retained store by the end of the lab run, proving complete wired-surface coverage for the current lab contract

What “completeness” does and does not mean here:

This validation enumerates the 25 event types that are wired (emitted by implemented code paths). Six additional event types are defined in audit/events.go but marked deferred — they correspond to features not yet implemented (rollout.gate.approved, rollout.recovered, rollout.stuck.detected, auth.login.succeeded, auth.login.failed, relay.deadletter.inspected). VAL04 explicitly excludes deferred events; their absence from the store is expected and correct.

2. Scope¶

Covered¶

Retained store population across all 6 audit categories: rollout, ha, cert, relay, auth, rollback
Schema validation for mandatory fields on every record returned in each category query
Full-store query latency measured against a 2000 ms practical bound
Event-type coverage: 25/25 wired event types present in retained store

Not covered (known gaps)¶

Deferred events (rollout.gate.approved, rollout.recovered, rollout.stuck.detected, auth.login.succeeded, auth.login.failed, relay.deadletter.inspected): not emitted; excluded by design
Database audit path: the PGAuditEmitter path (DB-backed audit, PR-29-followup-d) writes to a PostgreSQL audit_events table; VAL04 validates only the file-backed retained store; DB-path completeness requires a live PostgreSQL instance outside this lab
Per-field value correctness: VAL04 confirms field presence, not semantic correctness of values (e.g., actor identity, resource format)
Archive path: records rotated to $AUTONOMY_AUDIT_ARCHIVE_DIR are not separately counted; the lab runs within a single day so no rotation occurs
Concurrent write safety: all lab phases run sequentially; concurrent-write correctness is tested in audit/ unit tests, not here

3. Wired Event-Type Inventory¶

The following 25 event types are implemented and expected to appear in the retained store by the end of a full lab run. The emitting lab phase is listed for traceability.

Category	Event type	Emitting lab phase
rollout	`rollout.plan.created`	rollout lab (plan create)
rollout	`rollout.plan.published`	rollout lab (plan publish)
rollout	`rollout.plan.cancelled`	rollout lab (plan cancel)
ha	`ha.failover.triggered`	HA lab (failover trigger)
ha	`ha.failover.completed`	HA lab (failover complete)
ha	`ha.backup.created`	pgstore backup lab
ha	`ha.backup.restored`	pgstore restore lab
ha	`ha.quorum.lost`	quorum lab (loss detection)
ha	`ha.quorum.restored`	quorum lab (restored)
ha	`ha.split_brain.detected`	split-brain lab (detect)
ha	`ha.split_brain.recovered`	split-brain lab (promote-leader / manual-reconcile)
relay	`relay.deadletter.retried`	relay lab (retry)
relay	`relay.deadletter.purged`	relay lab (purge)
relay	`relay.bandwidth.configured`	relay lab (bandwidth set)
cert	`cert.issued`	cert lab (phase 1–4, node issuance)
cert	`cert.rotated`	cert lab (phase 8, VAL01 cert rotate)
cert	`cert.revoked`	cert lab (phase 6–7, CRL revocation)
cert	`cert.crl.synced`	cert lab (phase 6, sync-crl)
cert	`cert.revocation.rejected`	cert lab (phase 7, revoked-client handshake)
rollback	`rollback.preview.requested`	rollback lab (preview)
rollback	`rollback.executed`	rollback lab (execute)
auth	`auth.role.assigned`	RBAC lab (role assign)
auth	`auth.access.denied`	RBAC enforcement lab + VAL03 (DENY checks)
auth	`auth.break_glass.used`	RBAC lab (break-glass path)
auth	`auth.bootstrap.access`	RBAC lab (bootstrap mode)

Deferred event types (excluded from VAL04):

Event type	Reason deferred
`rollout.gate.approved`	gate approval CLI not yet wired
`rollout.recovered`	recover CLI not yet wired
`rollout.stuck.detected`	detection is server-side, not yet emitted
`auth.login.succeeded`	login flow not yet wired
`auth.login.failed`	login flow not yet wired
`relay.deadletter.inspected`	read-only, deferred from emission

4. Harness¶

VAL04 is implemented as run_audit_completeness_val04_lab() in scripts/labs/run_cli_audit_lab.sh. It runs after all prior lab phases (rollout, cert, HA, quorum, split-brain, RBAC, relay, rollback, VAL01, VAL02, VAL03) so the retained store already contains records from every category.

No additional setup is required. The function reads from $AUTONOMY_AUDIT_DIR (set to $RETAINED_EVIDENCE_DIR/store at the top of the lab script) and from the per-category JSON files it writes itself.

Evidence directory: $EVIDENCE_DIR/val04/

5. Exact Scenarios¶

VAL04-01 — Retained Store Non-Empty¶

Purpose: Confirm that the retained store contains at least one JSONL file after all prior lab phases have run.

Action:

find "$AUTONOMY_AUDIT_DIR" -maxdepth 1 -name "*.jsonl" | wc -l

Evidence file: val04/val04-store-inventory.txt

Pass criterion: store_jsonl_files > 0.

VAL04-02 — All 6 Categories Return ≥1 Record¶

Purpose: Confirm that every audit category is populated in the retained store — no category is silently empty.

Action: For each category in {rollout, ha, cert, relay, auth, rollback}:

AUTONOMY_RBAC_ENFORCEMENT=0 \
  autonomy audit query \
    --audit-dir "$AUTONOMY_AUDIT_DIR" \
    --category <cat> \
    --limit 0 \
    --output json \
  > val04-category-<cat>.json

Evidence files: val04/val04-category-{rollout,ha,cert,relay,auth,rollback}.json, val04/val04-category-summary.txt

Pass criterion: Every category file contains at least one "event_name" key (count > 0 for all six).

VAL04-03 through VAL04-08 — Schema Completeness per Category¶

Purpose: Prove that records in each category contain all 6 mandatory audit fields.

Mandatory fields: event_name, category, action, outcome, source, timestamp

Action: For each category, parse the per-category JSON output produced by VAL04-02 and require every returned record to contain all 6 mandatory fields.

Check	Category	Evidence file
VAL04-03	rollout	`val04/val04-category-rollout.json`
VAL04-04	ha	`val04/val04-category-ha.json`
VAL04-05	cert	`val04/val04-category-cert.json`
VAL04-06	relay	`val04/val04-category-relay.json`
VAL04-07	auth	`val04/val04-category-auth.json`
VAL04-08	rollback	`val04/val04-category-rollback.json`

Evidence file (schema check log): val04/val04-schema-check.txt

Pass criterion per check: Every record in the category output contains all 6 mandatory fields. Any missing field is logged as MISSING field <field> in category <cat>.

VAL04-09 — Full-Store Query Latency ≤ 2000 ms¶

Purpose: Confirm that a full retained-store query with --limit 0 completes within 2 seconds at the lab’s corpus size. This provides an operability SLA floor — actual elapsed time is typically under 100 ms.

Action:

t_start=$(date +%s%3N)
AUTONOMY_RBAC_ENFORCEMENT=0 \
  autonomy audit query \
    --audit-dir "$AUTONOMY_AUDIT_DIR" \
    --limit 0 \
    --output json \
  > val04-coverage-all.json
t_end=$(date +%s%3N)
elapsed_ms=$(( t_end - t_start ))

Evidence files:

val04/val04-latency.txt — query_ok, query_elapsed_ms, bound_ms=2000, pass=true/false
val04/val04-coverage-all.json — full retained store JSON (also used by VAL04-10)

Pass criterion: query_ok=true and query_elapsed_ms ≤ 2000.

VAL04-10 — Event-Type Coverage 25/25¶

Purpose: Confirm that the retained store shows complete wired-surface coverage for the current lab contract: all 25 wired event types appear at least once. The formerly conditional paths (ha.failover.triggered, ha.failover.completed, auth.break_glass.used) are now intentionally exercised by the lab and are therefore part of the strict pass contract.

Action: For each of the 25 expected event types, check whether "<event_type>" appears in val04-coverage-all.json.

Evidence files:

val04/val04-coverage-report.txt — per-event PRESENT/ABSENT lines, final found/expected/threshold/pass
val04/val04-coverage-all.json — full retained store (produced by VAL04-09)

Pass criterion: found = 25.

6. Evidence Files¶

All files are written to $EVIDENCE_DIR/val04/.

File	Produced by	Contains
`val04-store-inventory.txt`	`find … wc -l`	`store_dir`, `store_jsonl_files` count
`val04-category-rollout.json`	`audit query --category rollout`	JSON array of rollout records
`val04-category-ha.json`	`audit query --category ha`	JSON array of ha records
`val04-category-cert.json`	`audit query --category cert`	JSON array of cert records
`val04-category-relay.json`	`audit query --category relay`	JSON array of relay records
`val04-category-auth.json`	`audit query --category auth`	JSON array of auth records
`val04-category-rollback.json`	`audit query --category rollback`	JSON array of rollback records
`val04-category-summary.txt`	category loop	`category=<cat> count=<N>` lines
`val04-schema-check.txt`	schema loop	Per-category PASS/FAIL with MISSING lines
`val04-latency.txt`	`date +%s%3N` bracketing full query	`query_ok`, `query_elapsed_ms`, `bound_ms=2000`, `pass=true/false`
`val04-coverage-all.json`	`audit query --limit 0 --output json`	Full retained store JSON array
`val04-coverage-report.txt`	coverage loop	PRESENT/ABSENT per event, final `found/expected/threshold/pass`
`val04-report.txt`	composite report	10-check PASS/FAIL + summary line
`val04-report.json`	composite report	Machine-readable JSON with all check statuses

7. Pass/Fail Criteria¶

Check ID	Name	File	Pass condition
VAL04-01	store_nonempty	`val04-store-inventory.txt`	`store_jsonl_files` > 0
VAL04-02	all_categories_present	`val04-category-summary.txt`	All 6 categories have count > 0
VAL04-03	schema_rollout	`val04-category-rollout.json`	All 6 mandatory fields present
VAL04-04	schema_ha	`val04-category-ha.json`	All 6 mandatory fields present
VAL04-05	schema_cert	`val04-category-cert.json`	All 6 mandatory fields present
VAL04-06	schema_relay	`val04-category-relay.json`	All 6 mandatory fields present
VAL04-07	schema_auth	`val04-category-auth.json`	All 6 mandatory fields present
VAL04-08	schema_rollback	`val04-category-rollback.json`	All 6 mandatory fields present
VAL04-09	query_latency	`val04-latency.txt`	`query_ok=true` and `query_elapsed_ms` ≤ 2000
VAL04-10	event_coverage	`val04-coverage-report.txt`	`found` = 25

Overall pass: all 10 checks pass and val04-report.txt reports pass=10 fail=0 total=10.

Failure handling:

VAL04-01 fails: the retained store directory is empty — check that AUTONOMY_AUDIT_DIR is exported and that prior lab phases ran successfully before VAL04
VAL04-02 fails for a category: that category’s lab phase may have failed or been skipped; check the corresponding section of the lab output
VAL04-03..08 fails (schema): a code change may have removed a mandatory field from emitRecord() or the category-specific emit call; inspect the missing field in val04-schema-check.txt and cross-reference audit/record.go
VAL04-09 fails: either the full-store query itself failed (query_ok=false) or it exceeded the latency bound; inspect val04-latency.stderr for query errors before treating the result as a pure performance issue
VAL04-10 fails (found < 25): check val04-coverage-report.txt for ABSENT lines; cross-reference whether the emitting lab phase ran and succeeded; any ABSENT wired event is now a true validation failure because the lab is expected to exercise all 25 wired event types deterministically

8. Report Template¶

# VAL 04 — Audit Completeness Validation Report
timestamp: 2026-03-20T10:00:00Z

## Results
VAL04-01 store_nonempty:          PASS
VAL04-02 all_categories_present:  PASS
VAL04-03 schema_rollout:          PASS
VAL04-04 schema_ha:               PASS
VAL04-05 schema_cert:             PASS
VAL04-06 schema_relay:            PASS
VAL04-07 schema_auth:             PASS
VAL04-08 schema_rollback:         PASS
VAL04-09 query_latency:           PASS  (elapsed=42ms  bound=2000ms)
VAL04-10 event_coverage:          PASS  (found=25/25  threshold=25)

## Summary
pass=10  fail=0  total=10

The runner also prints VAL 04: pass=10 fail=0 total=10 (report: val04-report.txt) to stdout so CI log scanners can grep for VAL 04: pass= without parsing the report file.

9. How to Run¶

VAL04 executes automatically as the final validation slice when the full lab is run:

export GOROOT=/home/ubuntu/.local/go1.25.7
export PATH="$GOROOT/bin:$PATH"
export GOTOOLCHAIN=local

bash scripts/labs/run_cli_audit_lab.sh

To inspect results after a run:

# Quick pass/fail
cat evidence/pr17-cli-audit-local-2026-03-17/val04/val04-report.txt

# Coverage breakdown (which event types fired)
cat evidence/pr17-cli-audit-local-2026-03-17/val04/val04-coverage-report.txt

# Schema check detail
cat evidence/pr17-cli-audit-local-2026-03-17/val04/val04-schema-check.txt

# Latency measurement
cat evidence/pr17-cli-audit-local-2026-03-17/val04/val04-latency.txt

# Inspect a specific category's records
jq '.[0] | {event_name, category, action, outcome, source, timestamp}' \
  evidence/pr17-cli-audit-local-2026-03-17/val04/val04-category-cert.json

# Count records per category
jq length \
  evidence/pr17-cli-audit-local-2026-03-17/val04/val04-category-rollout.json

# Check machine-readable report
jq '{pass_count, fail_count, coverage_found, latency_ms}' \
  evidence/pr17-cli-audit-local-2026-03-17/val04/val04-report.json