Deterministic Relay Layer

Scope

Edge implements deterministic local content relay with bounded local resources.

Implemented surfaces:

  • inbound offer/segment accept and local persistence

  • outbound relay executor and persistent relay ledger

  • bounded retries with operator-managed deadletter recovery

  • startup reconciliation for abandoned/missing relay records

  • local RPC observability for status/index/storage/quota/retry/assurance

Not implemented:

  • consensus/quorum/leader-election/shared-state replication

  • fleet convergence/orchestration semantics

Outbound Relay Lifecycle

edge/relay/ledger.go defines delivery states for (segment_id, peer_id):

  • StateScheduled

  • StateInflight

  • StateAcked (terminal)

  • StateFailed

  • StateDeadletter (terminal for automatic execution; operator-recoverable)

Transition summary:

  1. Scheduler inserts/keeps records in Scheduled.

  2. Worker CAS-claims Scheduled -> Inflight (TryTransitionInflight).

  3. On ACK: Inflight -> Acked (TransitionAcked).

  4. On failure: Inflight -> Failed|Deadletter (TransitionFailed with retry budget).

  5. Startup recovery:

    • InflightToFailed marks abandoned in-flight records.

    • PruneMissing force-deadletters non-terminal records whose segment is gone.

    • RescheduleReady moves retry-ready failed records back to scheduled.

Transport and ACK Contract

Outbound relay worker flow (edge/relay/executor.go:executeRelay):

  1. Resolve peer address from static known_peers list.

  2. Transport.Connect (mTLS).

  3. Read segment from local store.

  4. SendOffer then SendSegment.

  5. Wait for RecvAck before marking ACKed.

ACK frame details:

  • wire type msgAck (0x04)

  • payload contains segment_id

  • sender treats missing/timeout/mismatch ACK as failed attempt

Receiver-side flow (edge/cmd/edged/main.go:handleRelayConn):

  1. RecvOffer

  2. quota check (AllowSegment)

  3. RecvSegment

  4. store.Write (atomic local commit path)

  5. SendAck

Success Condition and Eviction Gate

Configured at relay.success_condition:

  • one_peer

  • all_peers

Validation rejects other values before startup.

Executor success evaluation (checkSuccessCondition):

  • updates index relayed marker when condition is met

  • emits success log with condition and ack count

  • if evict_on_relay=true, code marks TODO for eviction gating integration

Bounded statement: evict_on_relay flag is parsed and passed into executor, but actual eviction behavior for this flag is not implemented in current code path.

Determinism and Retry Semantics

Determinism sources:

  • pending scan order is deterministic (NextAttemptAfter, SegmentID, PeerID)

  • scheduler comparator uses deterministic tie-break (lex_segment_id)

  • no PRNG in relay path (enforced by prohibited-symbol scan)

Retry semantics:

  • attempt count is monotonic per pair

  • retry failure transitions to deadletter at max_retries

  • deadletter halts automatic execution until an operator retries or purges it

Evidence

Code:

  • edge/relay/ledger.go

  • edge/relay/boltledger.go

  • edge/relay/executor.go

  • edge/relay/recovery.go

  • edge/cmd/edged/main.go (runDaemon, handleRelayConn)

  • edge/transport/wire.go

  • edge/config/validate.go (validateRelay)

Tests:

  • edge/relay/executor_test.go

  • edge/relay/recovery_test.go

  • edge/relay/e2e_test.go

  • edge/transport/wire_test.go

  • edge/cmd/edged/main_integration_test.go

Verification commands:

GOWORK=off go test ./edge/relay -run 'TestExecutor|TestReconcile|TestE2E' -v
GOWORK=off go test ./edge/transport -run 'TestWire|TestTCPTLS' -v
GOWORK=off go test ./edge/cmd/edged -run OfferToStoreAndRPC -v

See Also