Multi-Modal Policy Authoring

Note

Status — Phase 1 design proposal (epic #747) This page documents the planned Phase 1 target architecture. The substrate it references — the policy/ir/ package, the policy_frontend fields on policy.BundleManifest and bundle.Manifest, the widened runtime.Decision fields, the --with-explainability CLI flag — does not exist in the repository yet. Each is scheduled for a specific child PR (C02–C07) tracked in the epic. This page is the design contract those PRs implement against; treat any field name or test name as a target to land, not a present-day fact.

What does exist today: the unchanged policy/evaluator.go runtime, the Rego authoring path, the existing policy.BundleManifest (without policy_frontend), and the existing 3-value runtime.Outcome enum (Allow/Deny/Defer). The Phase 1 plan adds on top — it does not move or remove anything that exists.

Multi-Modal Policy is a planned set of authoring frontends that will sit above the existing OPA/Rego policy runtime. Each frontend compiles into a canonical intermediate representation (IR); the IR compiles to Rego; the Rego bundle is evaluated by the existing policy/evaluator.go without behavioral change. The runtime remains the sole policy authority (INV-MM-3); the fail-closed evaluator semantics (INV-MM-2) are preserved verbatim.

[ Rego (existing) | YAML DSL | JSON Rules | NL (P3) | Graph (P3) ]
                              ↓
                              IR
                              ↓
                       Rego (Phase 1 backend)
                              ↓
                  policy.NewEvaluator (unchanged)
                              ↓
                         Decision

The IR is the contract between authoring modality and runtime evaluation. Its shape determines what each frontend can express and what the Rego backend must preserve. The IR is the most important single decision of this epic; the taxonomy page catalogs the Phase 1 shape.

Why a new authoring layer, not a new runtime

policy/evaluator.go is the right correctness story for production governance: undefined results, non-bool values, and evaluation errors all resolve to Deny; only data.autonomy.allow == true produces Allow. The fail-closed contract is load-bearing for fleet operators reasoning about worst-case behavior.

It’s not the right authoring story for the audiences whose evaluation matters most during RFI/RFP phases:

Audience

Authors in

Today’s friction

Security / GRC / compliance teams

YAML, approval matrices

Must learn Rego just to write effect: deny when network=public

Robotics mission operators

Approval chains, workflow graphs

Predicate logic doesn’t model gated transitions naturally

Integration teams

Machine-generated JSON from CMDBs, ticketing, classification systems

Hand-writing Rego adapters for each upstream source

Acquisition evaluators

Whatever they can read in 48 hours

Rego is not that

Adding a new evaluator engine would put authority on the adapter side, which INV-MM-3 forbids. Adding new frontends that all reduce to Rego lets the runtime stay the sole authority while the authoring surface expands.

Where it sits in the stack

                                         ↑   operator-facing authoring
   YAML DSL    JSON Rules    NL  Graph
       │           │          │    │     ─── frontends ───
       └───────┬───┴─────┬────┘    │
               ↓         ↓         ↓
                       IR                ─── canonical AST ───
                        │
                        ↓
            IR → Rego compiler           ─── Phase 1 backend ───
                        │
                        ↓
                   policy/                ─── unchanged runtime ───
                   evaluator.go
                        │
                        ↓
                   runtime.Decision        ─── widened with explainability ───

The frontends are loose-coupled — adding YAML doesn’t touch JSON code; adding the future NL frontend doesn’t touch YAML. The IR is the only shared substrate. The Rego backend is the only path to the runtime in Phase 1; future backends (CEL, WASM, native) can plug into the IR without re-doing the frontend story (INV-MM-7).

Core architectural commitments

  1. Single intermediate representation. Every non-Rego frontend produces IR; the Rego backend reads IR. New frontends or backends edit the IR boundary, not each other. (INV-MM-7)

  2. Deterministic compile. Same IR program ⇒ same Rego bytes, independent of clock, randomness, or filesystem ordering. Tests pin this with a property check. (INV-MM-4)

  3. Existing fail-closed contract preserved. The runtime evaluator is unchanged. Every Rego bundle the IR backend emits is evaluated the same way as a hand-written one; undefined / non-bool / eval-error still resolve to Deny. (INV-MM-2)

  4. Frontend provenance on both manifest layers (planned — C04 lands). The inner policy.BundleManifest will gain a policy_frontend field so the runtime and autonomy policy inspect|load|eval know who authored the bundle. The outer bundle.Manifest schema bumps to v1.3 with a matching policy_frontend field so OCI consumers + bundle inspect|verify can read it without unpacking. bundle.Manifest.Validate() will cross-check the two; mismatch must fail verify (INV-MM-5).

  5. Explainability widening is end-to-end (planned — C05+C06 land). The widened runtime.Decision gains MatchedRule, EvaluationTrace, RemediationHint. The evaluator populates them; the WAL decision event schema carries them; autonomy wal inspect --kind decision surfaces them. Widening any one of these without the others would mean the IR knows the rule origin but the operator never sees it (INV-MM-6).

  6. Outcome backwards compatibility. Phase 1 ships no new runtime.Outcome values — Allow, Deny, Defer stay the closed set. The Phase 2 additions (audit_only, require_approval, rate_limited, sandboxed) are tracked separately under Phased scope. A pure-Rego policy that never references the new outcomes continues to produce Allow/Deny/Defer exactly as today, byte-identical WAL events except for the optional explainability fields. (INV-MM-1)

Phased scope

This epic is sequenced in three phases. Phase 1 is the load-bearing substrate this page documents; Phases 2 and 3 are roadmap.

Phase 1 — YAML DSL → Rego compiler + explainability (this epic)

Phase 1 is sliced into seven child PRs against epic/747-policy-multi-modal-authoring. Each row lands the substrate the next depends on; C01 (this PR) is docs-only.

Child PR

Scope

C01 (this PR)

Architecture + IR taxonomy + INV-MM-* invariants

C02

policy/ir/ package — Go types + ir.schema.json + property tests pinning deterministic compile (see taxonomy page for the planned shape)

C03

YAML DSL parser → IR + IR → Rego backend; property test that generated Rego evaluates identically to a hand-written equivalent

C04

policy.BundleManifest.policy_frontend field + bundle.Manifest schema v1.3 + cross-check at verify

C05

runtime.Decision widening (MatchedRule, EvaluationTrace, RemediationHint) + policy.Eval populates them

C06

telemetry/events.go WAL decision event widening + autonomy wal inspect + audit query --with-explainability

C07

End-to-end tutorial under docs/tutorials/ exercising the YAML → IR → Rego → runtime → WAL path

Phase 2 — Context-aware inputs + new outcomes (separate epic)

  • Optional context buckets on runtime.Action (or successor runtime.EvalRequest): identity, environment, session, ticket_state, classification.

  • Pluggable context-provider interface.

  • Four new outcomes:

    • audit_only — executes, flags in WAL, emits a Prometheus signal.

    • require_approval — blocks pending out-of-band human approval; new runtime.ApprovalStore interface; autonomy policy approve <token>.

    • rate_limited — per-identity / per-tool-kind token bucket.

    • sandboxed — runtime contract; tool-side semantics defined per-tool.

  • Bundle manifest schema v1.4.

Phase 3 — NL frontend + policy marketplace (speculative)

  • Natural-language policy generation (assisted draft → reviewed by Rego-fluent owner → IR → Rego).

  • Compliance packs (curated IR bundles for common regulatory frames).

  • Policy registry / marketplace.

Backward compatibility

Phase 1 is required to be additive — every child PR must preserve the following properties:

  • Existing Rego bundles MUST load unchanged. The inner BundleManifest.policy_frontend field (planned, C04) MUST default to "rego" on a legacy bundle; absence MUST NOT be an error.

  • Existing Rego policies that don’t read the new context buckets MUST see no new input. Phase 1 ships no context-bucket plumbing; that’s Phase 2.

  • WAL consumers that don’t recognize the new explainability fields (planned, C06) MUST be able to ignore them. The decision event schema’s existing fields are unchanged.

  • Existing autonomy policy build, autonomy policy push|sign|verify, autonomy bundle inspect|verify commands MUST continue to work on legacy bundles. After C04, verify MUST accept both v1.2 (no policy_frontend) and v1.3 (with it).

Interaction with ContractRefine (epic #1020)

ContractRefine ships two preflight-related surfaces today that this epic MUST NOT regress or accidentally override:

  1. policy.BundleManifest.RequiresPreflight (C07 #1027 landed it; spec at docs/architecture/contract-refine.md) — a manifest-layer boolean that gates /v1/tool on a verified X-Autonomy-Preflight-Id binding before the policy layer runs.

  2. input.preflight — when the /v1/tool request carries a resolved PreflightDecision, the runtime adapter populates input.preflight so Rego rules can gate on input.preflight.outcome and walk input.preflight.contract_results[_].contract. The full shape is documented in contract-refine.md § Integration surface.

The Phase 1 plan interacts with both as follows. The contract is INV-MM-8: frontend choice MUST NOT affect either surface.

RequiresPreflight is frontend-orthogonal

policy.BundleManifest.RequiresPreflight and the planned policy.BundleManifest.policy_frontend field (C04) are independent manifest fields. They live side by side; neither field’s value affects how the runtime reads the other. Specifically:

  • A YAML-authored bundle (policy_frontend: yaml-dsl) MAY declare requires_preflight: true and the runtime gates /v1/tool identically to a Rego-authored bundle (policy_frontend: rego) that declares the same. The binding-required check is in runtime/server.go and reads only bundle.Manifest.RequiresPreflight — never policy_frontend.

  • policy_frontend MUST NOT become behaviorally significant for any preflight-sensitive bundle. Adapter-side branching on policy_frontend is forbidden by INV-MM-3; the runtime treats every signed-and-verified Rego bundle the same way at evaluation time.

  • The acceptance test plan for C04 explicitly covers a YAML-authored bundle declaring requires_preflight: true and asserts the runtime binding gate fires under enforce mode — the same way C08’s tests cover the Rego-authored case today.

input.preflight participates in the IR’s input contract verbatim

The IR’s predicate model reads from input — the exact same input the existing Rego evaluator sees today. That includes:

  • input.kind and input.params (existing C07 substrate; the policy evaluator’s Eval method has set these for every call since pre-#1020).

  • input.preflight when ContractRefine has resolved a binding (the C07 plumbing — runtime.Action.Preflight rides into the policy call; the evaluator surfaces it as input.preflight). Absent binding ⇒ input.preflight is absent and Rego rules evaluate not input.preflight to true (the documented “absence ≠ true” contract).

The IR’s leaf predicate field is a dotted path into input — the same shape Rego authors write today. A YAML rule expressing the canonical “only allow if preflight passed” gate compiles into the same Rego predicate a hand-written input.preflight.outcome == "pass" guard would produce; the IR taxonomy spells this out in § How frontends compile into the IR.

A YAML/JSON frontend therefore expresses exactly the gating behavior the existing Rego path can express against input.preflight. The C03 property test pinning IR → Rego equivalence covers preflight-bound predicates as part of the standard fixture set; if a future IR predicate addition affects how input.preflight.* paths lower, the property test catches it before the PR merges.

What this is NOT

Multi-Modal Policy is deliberately narrow. Operators reading the architecture sometimes ask whether the layer also does the following — it does not:

  • Not a replacement for Rego. Rego stays the canonical advanced authoring surface. The IR backend emits Rego; the runtime evaluator reads Rego.

  • Not a new evaluator engine. No CEL runtime, no WASM runtime, no native predicate evaluator in Phase 1. The runtime stays policy/evaluator.go.

  • Not adapter-side enforcement. All decisions flow through the runtime. Adapters MUST NOT short-circuit on the frontend used to author the bundle. (INV-MM-3)

  • Not a translation cache. Every autonomy policy build recompiles the IR → Rego deterministically. The Rego bytes go into the signed bundle; the IR doesn’t ride the wire.

  • Not unsigned hot-reload. Every emitted bundle goes through the existing signing path. The frontend is a build-time concern; the runtime sees only signed, verified Rego.