VAL 06 — Support Bundle Validation¶

1. Purpose and Claims¶

This validation proves that the autonomy support-bundle generate command produces complete, correctly redacted archives under both normal and degraded conditions.

#	Claim
VAL06-C1	Bundle generation succeeds (exit 0, non-empty archive) within 30 seconds under normal lab conditions
VAL06-C2	The archive contains the three always-present core files (`manifest.json`, `system_info.json`, `build_info.json`) and all 6 collectors are recorded in `manifest.json` regardless of their individual outcome
VAL06-C3	Secrets are redacted: `identity.fleet_salt` is replaced with `<REDACTED>` and the postgres URL password is replaced with `REDACTED`; neither original value appears in `config_redacted.yaml`
VAL06-C4	The bundle degrades gracefully: when an optional collector (`ha_status`) cannot reach the control plane, the bundle still exits 0 and `manifest.json` records `ha_status` with status `"failed"`, not a fatal error

What “validation” covers here:

VAL06 validates the CLI surface and archive structure against the four claims above. It does not test bundle ingestion by a support ticketing system, bundle decryption, or multi-bundle diffing. Per-field value correctness within system_info.json (exact hostname, CPU count, etc.) is not asserted — only field presence.

2. Scope¶

Covered¶

Archive creation: exit code, file non-empty, timing ≤ 30 s
Core file presence: manifest.json, system_info.json, build_info.json
Collector inventory: all 6 expected collector names present in manifest
system_info.json field presence: os, arch, go_version, hostname, collected_at
audit_recent.json population: ≥ 1 audit record from the retained store
Secret redaction: fleet_salt placeholder, postgres password placeholder
Archive purity: no PEM block (-----BEGIN) in any archive entry
Degraded mode: ha_status collector failure with non-fatal bundle exit

Not covered (known gaps)¶

ha_status content validation: the JSON structure of ha_status.json is validated only by the HA surface tests; VAL06 treats it as opaque
logs collector content: log lines included from --log-file are not parsed; VAL06 confirms presence in the archive via manifest only
Concurrent bundle generation: only one bundle is generated at a time
Archive size: no upper bound is enforced on archive size in this validation
RBAC guard: support-bundle generate is intentionally unguarded (VAL03 confirms this); VAL06 bypasses RBAC with AUTONOMY_RBAC_ENFORCEMENT=0
config_redacted.yaml value correctness: only the two known secret fields are checked; all other YAML values are preserved as-is and not validated here

3. Bundle Architecture¶

autonomy support-bundle generate collects data from up to 6 collectors and writes a single .tar.gz archive with this layout:

<bundle-dir>/
  manifest.json         — always written last; records all 6 collectors and their status
  system_info.json      — os, arch, go_version, num_cpu, hostname, collected_at
  build_info.json       — cli_version, go_version, main_path, build_settings
  config_redacted.yaml  — effective config with fleet_salt and postgres_url password replaced
  ha_status.json        — GET /v1/ha/status snapshot (requires --orchestrator-url)
  audit_recent.json     — 50 most recent audit records (requires --audit-dir)
  logs/
    autonomy.log        — tailed log lines (requires --log-file)

<bundle-dir> is derived from the output filename: val06-bundle.tar.gz → val06-bundle.

Collector status values:

Status	Meaning
`ok`	Collector ran and wrote its file to the archive
`failed`	Collector ran but returned an error; file is not in the archive
`skipped`	Required flag (`--orchestrator-url`, `--audit-dir`, `--log-file`) was not provided

Secret redaction (line-oriented, no full YAML parse required):

Field	Replacement
`fleet_salt: <value>`	`fleet_salt: <REDACTED>`
`postgres_url: postgres://user:<password>@…`	password component replaced with `REDACTED`
Certificate file paths	preserved as-is (paths are not secrets; contents not collected)

Errors from individual collectors are non-fatal: the archive is always written with all available data, and manifest.json records the error message for each failed collector.

4. Harness¶

VAL06 is implemented as run_support_bundle_val06_lab() in scripts/labs/run_cli_audit_lab.sh. It runs after run_otel_val05_lab so the retained audit store already contains records from all prior phases (needed for the audit_recent_populated check).

Setup performed by the function:

Creates $WORK_DIR/val06/autonomy.yaml with a known fleet_salt value (deadbeef…) and a known postgres password (val06-secret-pass) for deterministic redaction verification.
Generates a normal bundle (val06-bundle.tar.gz) with --config-file, --audit-dir, and --log-file but without --orchestrator-url, while explicitly clearing AUTONOMY_ORCHESTRATOR_URL, so ha_status is skipped (not failed) in the normal run.
Extracts manifest.json, system_info.json, config_redacted.yaml, and audit_recent.json from the archive for content inspection.
Generates a degraded bundle (val06-bundle-degraded.tar.gz) with --orchestrator-url http://127.0.0.1:19999 (no server listening) to exercise the graceful-failure path.

Evidence directory: $EVIDENCE_DIR/val06/

5. Exact Scenarios¶

VAL06-01 — Archive Created¶

Purpose: Confirm that support-bundle generate exits 0 and writes a non-empty .tar.gz file.

Action:

AUTONOMY_RBAC_ENFORCEMENT=0 \
  autonomy support-bundle generate \
    --output val06-bundle.tar.gz \
    --config-file autonomy.yaml \
    --audit-dir "$AUTONOMY_AUDIT_DIR" \
    --log-file server-rbac-enforced.log \
    --log-lines 50

Evidence file: val06/val06-timing.txt

Pass criterion: Command exits 0 and val06-bundle.tar.gz is non-empty.

VAL06-02 — Generation Time ≤ 30 s¶

Purpose: Confirm that bundle generation completes within a practical wall- clock bound at lab corpus sizes.

Action: date +%s%3N brackets the generate invocation from VAL06-01.

Evidence file: val06/val06-timing.txt — generate_ok, elapsed_s, bound_s=30, pass_timing

Pass criterion: elapsed_s ≤ 30 and generate_ok=true.

VAL06-03 — Core Files Present in Archive¶

Purpose: Confirm that manifest.json, system_info.json, and build_info.json appear in the archive listing for every bundle. These three files are always collected; their absence indicates a fatal archive-write failure.

Action:

tar -tzf val06-bundle.tar.gz | grep -c '<filename>'

Evidence file: val06/val06-core-files.txt — PRESENT/ABSENT per file

Pass criterion: All three files report PRESENT.

VAL06-04 — Manifest Records All 6 Collectors¶

Purpose: Confirm that manifest.json contains entries for all 6 collector names, regardless of their status value. A missing entry means the collector was silently dropped rather than recorded as skipped or failed.

Expected collector names: system_info, build_info, config, ha_status, audit_recent, logs

Action: For each collector name, test:

jq -e --arg n "<name>" '.collectors[] | select(.name == $n)' manifest.json

Evidence file: val06/val06-manifest-check.txt — collector=<name> status=<status> lines

Pass criterion: All 6 collector names are present in manifest.json.

VAL06-05 — system_info.json Has Required Fields¶

Purpose: Confirm that the system information collector populates all 5 required fields.

Required fields: os, arch, go_version, hostname, collected_at

Action: For each field:

jq -e --arg f "<field>" 'has($f)' system_info.json

Evidence file: val06/val06-sysinfo-check.txt — PRESENT/ABSENT per field

Pass criterion: All 5 fields are present.

VAL06-06 — audit_recent.json Is Populated¶

Purpose: Confirm that the audit_recent collector returns at least one record from the retained audit store, proving end-to-end connectivity between bundle generation and the file-backed audit path.

Action:

jq 'length' audit_recent.json

Evidence file: val06/val06-audit-check.txt — audit_recent_count, pass

Pass criterion: audit_recent_count > 0.

VAL06-07 — fleet_salt Is Redacted¶

Purpose: Confirm that the known fleet_salt value written into the test config (deadbeef…) is replaced with <REDACTED> in config_redacted.yaml, and that the original value is absent.

Action:

grep 'fleet_salt: <REDACTED>' config_redacted.yaml     # must match
grep 'deadbeef' config_redacted.yaml                   # must not match

Evidence file: val06/val06-redaction-salt.txt — fleet_salt_placeholder, fleet_salt_actual_absent, pass

Pass criterion: fleet_salt: <REDACTED> present AND deadbeef absent.

VAL06-08 — Postgres Password Is Redacted¶

Purpose: Confirm that the known postgres password (val06-secret-pass) embedded in the test config URL is replaced with REDACTED in config_redacted.yaml, and that the original password is absent.

Action:

grep 'REDACTED' config_redacted.yaml          # must match
grep 'val06-secret-pass' config_redacted.yaml # must not match

Evidence file: val06/val06-redaction-pg.txt — pg_redacted_present, pg_secret_absent, pass

Pass criterion: REDACTED present AND val06-secret-pass absent.

VAL06-09 — Archive Contains No Private Key Material¶

Purpose: Confirm that no PEM block (-----BEGIN …) appears anywhere in the extracted archive contents. The bundle collects only JSON, YAML, and plain-text log data; certificate paths in the config are preserved as path strings, not file contents.

Action: Extract archive to a temp directory and search all files:

grep -r -- '-----BEGIN' <extract-dir>

Evidence file: val06/val06-privkey-check.txt — privkey_hits, pass

Pass criterion: privkey_hits = 0.

VAL06-10 — Degraded Mode Exits 0¶

Purpose: Confirm that the bundle command exits 0 even when the ha_status collector fails to contact the control plane, and that manifest.json accurately records the failure.

Action:

AUTONOMY_RBAC_ENFORCEMENT=0 \
  autonomy support-bundle generate \
    --output val06-bundle-degraded.tar.gz \
    --config-file autonomy.yaml \
    --audit-dir "$AUTONOMY_AUDIT_DIR" \
    --orchestrator-url http://127.0.0.1:19999

Port 19999 has no server listening, so collectHTTPSnapshot returns an error. The collect() wrapper marks the collector as "failed" and continues.

Evidence file: val06/val06-degraded-check.txt — bundle_exit_ok, ha_status_status, pass

Pass criterion: bundle_exit_ok=true AND ha_status_status=failed.

6. Evidence Files¶

All files are written to $EVIDENCE_DIR/val06/.

File	Produced by	Contains
`val06-bundle.tar.gz`	`support-bundle generate` (normal)	Full archive under inspection
`val06-generate-stdout.txt`	normal generate stdout	Reserved; normally empty because generate progress is emitted on stderr
`val06-generate.log`	normal generate stderr	`generating…`, per-collector progress, warnings, and `bundle written:`
`val06-timing.txt`	`date +%s%3N` bracketing	`generate_ok`, `elapsed_s`, `bound_s=30`, `pass_timing`
`val06-contents.txt`	`tar -tzf`	Archive file listing
`val06-manifest.json`	`tar -xOf … manifest.json`	Full manifest with collector statuses
`val06-system-info.json`	`tar -xOf … system_info.json`	OS/runtime fields
`val06-config-redacted.yaml`	`tar -xOf … config_redacted.yaml`	Redacted YAML
`val06-audit-recent.json`	`tar -xOf … audit_recent.json`	50 most recent audit records
`val06-core-files.txt`	core file loop	`PRESENT`/`ABSENT` per required file
`val06-manifest-check.txt`	collector loop	`collector=<name> status=<status>` per collector
`val06-sysinfo-check.txt`	field loop	`PRESENT`/`ABSENT` per required field
`val06-audit-check.txt`	`jq length`	`audit_recent_count`, `pass`
`val06-redaction-salt.txt`	grep checks	`fleet_salt_placeholder`, `fleet_salt_actual_absent`, `pass`
`val06-redaction-pg.txt`	grep checks	`pg_redacted_present`, `pg_secret_absent`, `pass`
`val06-privkey-check.txt`	`grep -r -- '-----BEGIN'`	`privkey_hits`, `pass`
`val06-bundle-degraded.tar.gz`	`support-bundle generate` (degraded)	Degraded archive
`val06-degraded-stdout.txt`	degraded generate stdout	Reserved; normally empty because generate progress is emitted on stderr
`val06-degraded.log`	degraded generate stderr	`generating…`, per-collector progress, and warning lines for failed `ha_status`
`val06-degraded-manifest.json`	`tar -xOf … manifest.json` (degraded)	Manifest with `ha_status: failed`
`val06-degraded-check.txt`	degraded mode checks	`bundle_exit_ok`, `ha_status_status`, `pass`
`val06-report.txt`	composite report	10-check PASS/FAIL + summary line
`val06-report.json`	composite report	Machine-readable JSON with all check statuses

7. Pass/Fail Criteria¶

Check ID	Name	File	Pass condition
VAL06-01	archive_created	`val06-timing.txt`	`generate_ok=true` and archive is non-empty
VAL06-02	generation_time	`val06-timing.txt`	`elapsed_s` ≤ 30
VAL06-03	core_files	`val06-core-files.txt`	All 3 required files report `PRESENT`
VAL06-04	manifest_collectors	`val06-manifest-check.txt`	All 6 collector names present in manifest
VAL06-05	system_info_fields	`val06-sysinfo-check.txt`	All 5 required fields report `PRESENT`
VAL06-06	audit_recent_populated	`val06-audit-check.txt`	`audit_recent_count` > 0
VAL06-07	fleet_salt_redacted	`val06-redaction-salt.txt`	Placeholder present AND original value absent
VAL06-08	postgres_pw_redacted	`val06-redaction-pg.txt`	`REDACTED` present AND original password absent
VAL06-09	no_private_keys	`val06-privkey-check.txt`	`privkey_hits` = 0
VAL06-10	degraded_mode	`val06-degraded-check.txt`	`bundle_exit_ok=true` AND `ha_status_status=failed`

Overall pass: all 10 checks pass and val06-report.txt reports pass=10 fail=0 total=10.

Failure handling:

VAL06-01 fails: inspect val06-generate.log for the generation error; the most common causes are a missing audit dir or write permission issue on the output path
VAL06-02 fails: the generation time exceeded 30 s; check whether the audit store has grown unusually large or the --audit-dir path is on a slow filesystem
VAL06-03 fails: a required file is absent from the archive; inspect val06-manifest.json to determine which collector failed at the archive- write level (distinct from a collector-level failure recorded as status: "failed")
VAL06-04 fails: a collector name is absent from manifest.json; this indicates a code-level removal of a collector; cross-reference support_bundle.go to confirm
VAL06-05 fails: a required system_info field is absent; inspect collectSystemInfo() in support_bundle.go
VAL06-06 fails: the audit_recent collector returned an empty array; check that prior lab phases populated $AUTONOMY_AUDIT_DIR before VAL06 ran
VAL06-07 or VAL06-08 fails (redaction): inspect val06-config-redacted.yaml directly and cross-reference redactConfigBytes() and redactPostgresURL() in support_bundle.go; a failure here means a secret was included in plain text in the bundle
VAL06-09 fails (privkey_hits > 0): inspect which file contains -----BEGIN by re-running grep -r -- '-----BEGIN' on the extracted archive; a code change may have accidentally included PEM material such as a cert or key file’s contents
VAL06-10 fails: either the bundle exited non-zero when ha_status failed (regression in the non-fatal collector pattern), or ha_status did not appear as "failed" in the manifest; inspect val06-degraded.log and val06-degraded-manifest.json

8. Report Template¶

# VAL 06 — Support Bundle Validation Report
timestamp: 2026-03-20T10:00:00Z

## Results
VAL06-01 archive_created:         PASS
VAL06-02 generation_time:         PASS  (elapsed=0s  bound=30s)
VAL06-03 core_files:              PASS
VAL06-04 manifest_collectors:     PASS
VAL06-05 system_info_fields:      PASS
VAL06-06 audit_recent_populated:  PASS
VAL06-07 fleet_salt_redacted:     PASS
VAL06-08 postgres_pw_redacted:    PASS
VAL06-09 no_private_keys:         PASS
VAL06-10 degraded_mode:           PASS

## Summary
pass=10  fail=0  total=10

The runner also prints VAL 06: pass=10 fail=0 total=10 (report: val06-report.txt) to stdout so CI log scanners can grep for VAL 06: pass= without parsing the report file.

9. How to Run¶

VAL06 executes automatically as the final validation slice when the full lab is run:

export GOROOT=/home/ubuntu/.local/go1.25.7
export PATH="$GOROOT/bin:$PATH"
export GOTOOLCHAIN=local

bash scripts/labs/run_cli_audit_lab.sh

To inspect results after a run:

# Quick pass/fail
cat evidence/pr17-cli-audit-local-2026-03-17/val06/val06-report.txt

# Collector status breakdown
cat evidence/pr17-cli-audit-local-2026-03-17/val06/val06-manifest-check.txt

# Redaction verification
cat evidence/pr17-cli-audit-local-2026-03-17/val06/val06-redaction-salt.txt
cat evidence/pr17-cli-audit-local-2026-03-17/val06/val06-redaction-pg.txt

# system_info field presence
cat evidence/pr17-cli-audit-local-2026-03-17/val06/val06-sysinfo-check.txt

# Degraded mode result
cat evidence/pr17-cli-audit-local-2026-03-17/val06/val06-degraded-check.txt

# Machine-readable report
jq '{pass_count, fail_count, elapsed_s}' \
  evidence/pr17-cli-audit-local-2026-03-17/val06/val06-report.json

# Inspect the archive listing
cat evidence/pr17-cli-audit-local-2026-03-17/val06/val06-contents.txt

# Check one extracted field
jq '{os, arch, go_version, hostname}' \
  evidence/pr17-cli-audit-local-2026-03-17/val06/val06-system-info.json