VAL27 — Relay Proof Report Generator

Audience: engineering leads, product managers, and external reviewers who need a consolidated, evidence-backed relay reliability assessment.

VAL27 is a report generator, not a test runner. It auto-discovers the latest evidence directories from five completed relay validation runs (VAL19–VAL23) and optionally incorporates VAL24 soak results, producing a single relay proof report with explicit readiness conclusions and clearly labelled beta claims.

1. Scope

VAL27 consolidates evidence from:

Slice

Name

What it proves

VAL19

Relay Impairment Validation

Outage, bandwidth, latency, and cutoff handling via TCP impairment proxy

VAL20

Relay Throughput Benchmark

segs/sec and bytes/sec across 5 load tiers (warmup → constrained 1 Mbps)

VAL21

Queue Depth + Overflow

Deep drain (200 segments), LRU eviction accounting, relay-status accuracy

VAL22

Deadletter Workflow

Group-R recovery rate = 1.000, Group-U re-deadletter, BoltDB retention after SIGTERM

VAL23

Bandwidth Management

Unlimited / rate-only / quota-only / hot-reload; counter accuracy; unit-test daily reset

VAL24

30-Day Soak (optional)

Gate D: rounds ≥ 1,440, clean_delivery_rate ≥ 0.990, retry_recovery_rate ≥ 0.990, loss = 0

VAL24 is optional. Its absence does not block Design Partner readiness — it is the Gate for the GA claim only.

Branch rule: coverage by existing runner

Existing asset

Coverage

run_soak_val24_report.sh

VAL24 30-day soak only — per-round aggregation, no cross-VAL proof

run_edge_deadletter_lab.sh

Point-in-time deadletter lab; not the same domain as VAL19–VAL23

run_cli_audit_lab.sh

Runs only audit/rollout/HA slices; does not include VAL19–VAL23

New aggregator required. No existing script reads and combines VAL19–VAL23 (+ optional VAL24) into a single relay proof artifact.

Architectural note: standalone evidence directories

VAL19–VAL23 runners are not slices of run_cli_audit_lab.sh. Each produces its own dated evidence directory under evidence/. VAL27 auto-discovers the latest matching directory for each VAL using Python glob sorted by mtime.

Out of scope

  • Multi-peer relay (all tests use a single accumulating peer)

  • BoltDB crash-consistency on power failure (graceful SIGTERM only)

  • Concurrent edged processes sharing relay state

  • WAN-latency impairment or inter-DC topology

  • Production-representative hardware benchmarking

  • 24-hour live daily-quota reset (validated by unit test only — see §4)

2. Evidence Structure

VAL27 auto-discovers evidence under EVIDENCE_ROOT (default: evidence/):

VAL

Directory pattern

Report file

VAL19

val19-relay-impairment-YYYY-MM-DD/

val19-report.json

VAL20

val20-relay-throughput-local-YYYY-MM-DD/

val20-baseline.json

VAL21

val21-relay-overflow-local-YYYY-MM-DD/

val21-summary.json

VAL22

val22-relay-deadletter-local-YYYY-MM-DD/

val22-summary.json

VAL23

val23-relay-bandwidth-local-YYYY-MM-DD/

val23-summary.json

VAL24 soak

val24-relay-soak-YYYY-MM-DD/

reports/final.json → checkpoint-14d.json → checkpoint-7d.json (most complete found)

The most recently modified directory for each pattern is used. If a directory is missing or its report does not match the expected schema, the slice is reported as MISSING in the coverage table without aborting.

For the core relay readiness verdict, VAL27 also requires VAL19–VAL23 to come from one coherent validation campaign. The generator checks the mtimes of the five discovered core report files and requires them to fall within a single 7-day window before issuing a Design Partner or GA readiness conclusion. VAL24 is excluded from that window because it is intentionally a long-running soak.

3. Beta-Marked Claims

The following relay claims have known limitations that must be resolved before the GA or Public Production claim is made. They are labelled ★ BETA in the report output and recorded in the JSON artifact under beta_claims.

Claim

Beta reason

Resolution path

Throughput figures (VAL20)

Single-host local runs; production hardware differs

Rerun on production-representative hardware; set hard thresholds

Bandwidth daily reset (VAL23 S-E)

Validated by injected-clock unit test only

Run a live 24-hour lab OR accept unit-test coverage as approved exception

Impairment throughput (VAL19)

Informational proxy stats; no hard threshold

Define and measure SLA-grade throughput targets under impairment

Relay 30-day soak (VAL24)

Beta only until Gate D passes

Run and complete VAL24 soak

4. Metric Definitions and Targets

Network Scenarios (VAL19)

Scenario

Expected outcome

Threshold

Outage (TCP refused)

All segments → DEADLETTER within ~10 s (max_retry=3)

Deterministic (no threshold)

Bandwidth 1 Mbps

Segments delivered (delivery > throttled path)

Informational ★ BETA

Bandwidth 10 Mbps

Segments delivered faster than 1 Mbps

Informational ★ BETA

Latency 200 ms

Delivery confirmed; elapsed observable

dial_timeout=5 s / ack_timeout=5 s

Latency 500 ms

Delivery confirmed; elapsed observable

dial_timeout=5 s / ack_timeout=5 s

Cutoff (TCP RST mid-stream)

Relay retry path triggered → delivery on clean proxy

No explicit ms threshold

Throughput (VAL20)

Tier

Load

Target

T1 warmup

1 × 64 B

zero loss [target]

T2 baseline

10 × 64 B

zero loss [target]; sps ≥ T1 sps (sanity)

T3 scheduler

100 × 64 B

zero loss [target]

T4 large

10 × 128 KB

zero loss [target]

T5 constrained

10 × 128 KB, 1 Mbps proxy

zero loss [target]; constrained_bps > 0

No hard segs/sec or bytes/sec floor is set in VAL20. The targets are zero-loss correctness and queue-monotonicity. Throughput figures are informational ★ BETA until recalibrated on production hardware.

Message-Loss Accounting (VAL21)

Invariant

Check

S-A: 200 × 64 B all delivered (no ceiling)

sa.delivered == 200

S-B: delivered + deadletter == 10 (LRU eviction accounting)

sb.delivered + sb.deadletter == sb.accounted == 10

S-C: relay-status accuracy

sc.delivered == 10 within 0.4 s

Deadletter/Retry (VAL22)

Metric

Target

Group R (clean proxy) retry rate

= 1.000 [target]

Group U (outage proxy) re-deadletter

= 4 (all re-deadletter after max_retry=3 exhausted)

BoltDB retention across SIGTERM+restart

All entries survive [target]

Aggregate retry rate

50% (intentional — Group U always fails under outage)

Bandwidth Management (VAL23)

Scenario

Expected delivered

Counter check

S-A unlimited

4/4

unlimited = true

S-B rate-only (64 B/s)

5/5

throttle_count ≥ 4, quota_drop_count = 0

S-C quota-only (128 B daily)

2/6 (first 2 exhaust quota)

quota_drop_count ≥ 4

S-D hot-reload (rate 8→0)

6/6 (all after unlimited applied live)

—

S-E daily reset

Unit test PASS

TestBandwidthLimiter_DailyQuota_ResetsAfter24h ★ BETA

VAL24 Soak Gate D (GA gate)

Criterion

Target

Total rounds

≥ 1,440

Clean delivery rate

≥ 0.990

Retry recovery rate

≥ 0.990 (only evaluated when total_retried > 0)

Silent loss

= 0

5. Readiness Level Definitions

Relay Design Partner Ready

Criteria (all must hold):

  1. VAL19 all 10 checks pass

  2. VAL20 all 10 checks pass

  3. VAL21 all 10 checks pass

  4. VAL22 all 10 checks pass

  5. VAL23 all 10 checks pass

  6. Zero silent loss proven by explicit accounting: VAL21 sa.delivered == 200 and sb.delivered + sb.deadletter == sb.accounted == 10, plus VAL22 group_r_delivered == 4 and group_u_redeadlettered == 4

  7. VAL20 T2 baseline zero-loss

  8. VAL22 Group R retry rate = 1.000

  9. VAL23 all bandwidth scenarios pass

  10. VAL19–VAL23 evidence files fall within one 7-day campaign window

VAL24 soak is NOT required for Design Partner readiness.

Relay GA Ready

Design Partner criteria PLUS:

  1. VAL24 Gate D pass: rounds ≥ 1,440, clean_delivery_rate ≥ 0.990, retry_recovery_rate ≥ 0.990, loss = 0

  2. Multi-peer relay validation (at least basic 2-peer delivery + isolation)

  3. VAL20 throughput recalibrated on production-representative hardware

  4. VAL23 S-E daily reset validated with live 24-hour run or approved exception

Relay Public Production Claim

GA criteria PLUS:

  1. BoltDB crash-consistency on power failure

  2. Multi-peer relay soaks with message isolation proof

  3. Production-grade observability: queue depth alerting, deadletter paging, bandwidth quota depletion notification

6. 10-Check Matrix

ID

When

Description

Pass criterion

VAL27-01

Setup

VAL19 impairment report found and all 10 checks pass

s19 == "PASS"

VAL27-02

Setup

VAL20 throughput report found and all 10 checks pass

s20 == "PASS"

VAL27-03

Setup

VAL21 overflow report found and all 10 checks pass

s21 == "PASS"

VAL27-04

Setup

VAL22 deadletter workflow report found and all 10 checks pass

s22 == "PASS"

VAL27-05

Setup

VAL23 bandwidth report found and all 10 checks pass

s23 == "PASS"

VAL27-06

Metric

Zero message loss: VAL21 accounting exact + VAL22 zero silent loss

sa.delivered == 200 AND sb.delivered + sb.deadletter == sb.accounted == 10 AND group_r_delivered == 4 AND group_u_redeadlettered == 4

VAL27-07

Metric

Throughput T2 baseline (10×64B) zero loss

val20 T2 tier zero_loss == true

VAL27-08

Metric

Deadletter retry Group R rate = 1.000

group_r_retry_rate >= 1.0

VAL27-09

Metric

All bandwidth scenarios deliver correct segments

val23.fails == 0

VAL27-10

Summary

Relay design partner readiness — all above pass and evidence coherent

VAL27-01..09 all PASS + VAL19–VAL23 mtimes within one 7-day campaign window

VAL27-SOAK

GA gate

VAL24 30-day soak Gate D (not counted in PASS/FAIL total)

soak_complete AND gate_d.overall == true

7. Run the Report

Prerequisites

Run each standalone relay validation runner (Docker not required for VAL19–VAL23):

export GOROOT=/home/ubuntu/.local/go1.25.7
export PATH="$GOROOT/bin:$PATH"
export GOTOOLCHAIN=local

# Run each relay validation (each creates its own dated evidence dir)
bash scripts/labs/run_relay_impairment_val19_lab.sh
bash scripts/labs/run_relay_throughput_val20_lab.sh
bash scripts/labs/run_relay_overflow_val21_lab.sh
bash scripts/labs/run_relay_deadletter_val22_lab.sh
bash scripts/labs/run_relay_bandwidth_val23_lab.sh

# VAL24 soak (optional, needed for GA claim):
bash scripts/labs/run_soak_val24_setup.sh evidence/val24-relay-soak-$(date +%F)
# ... wait 30 days for Gate D ...
bash scripts/labs/run_soak_val24_report.sh evidence/val24-relay-soak-YYYY-MM-DD final

Generate the proof report

bash scripts/labs/run_relay_proof_report_val27.sh evidence/

Output files

File

Contents

stdout

Human-readable relay proof report

evidence/val27/val27-proof-report.txt

Same content as stdout

evidence/val27/val27-proof-report.json

Machine-readable JSON with beta_claims array

8. Tooling

File

Role

scripts/labs/run_relay_proof_report_val27.sh

VAL27 relay proof report generator

scripts/labs/run_relay_impairment_val19_lab.sh

VAL19 — source of impairment evidence

scripts/labs/run_relay_throughput_val20_lab.sh

VAL20 — source of throughput evidence

scripts/labs/run_relay_overflow_val21_lab.sh

VAL21 — source of overflow evidence

scripts/labs/run_relay_deadletter_val22_lab.sh

VAL22 — source of deadletter evidence

scripts/labs/run_relay_bandwidth_val23_lab.sh

VAL23 — source of bandwidth evidence

scripts/labs/run_soak_val24_setup.sh

VAL24 soak setup (optional GA gate)

scripts/labs/run_soak_val24_report.sh

VAL24 soak report (inputs to VAL27)

docs/tutorials/relay-soak-validation.md

VAL24 formal plan