Two failure modes that disqualify raw LLMs in law

• Fabrication / over-assertion — invented citations; discretionary standards (“extreme hardship”, “well-founded fear”) answered as if settled.
• Premature completion — partial analysis returned as final.
• Scaling the model fixes neither: a larger black box is still unauditable. In law, an answer no one can check is unusable.

Approach — a deterministic verifier, not a bigger model

• The model proposes; the deterministic compiler disposes. Correctness is decided by an oracle, not by another LLM.
• One firewall, in code: bright-line rule → verified; open-textured judgment → escalated, never auto-asserted.

Architecture — three axes around one arbiter

Axis 1 — the verifiability compiler

• Input: a typed Application-IR — facts, cited rules, claimed verdict. Each gate emits Diagnostic{ code, severity, location, hint }.
• Severity is the firewall: error = rule, deterministically wrong → block + repair; warning = judgment → escalate.
• No LLM in the verdict. Each rule carries its Akoma Ntoso source URI — rule ↔ text isomorphism.

The rule/judgment firewall, concretely

“≥ 913 days present?” → the compiler recomputes. “extreme hardship?” → it refuses and escalates. The line between decision-support and unauthorized practice, in code.

Knowledge bases — the authority corpus

Four segregated bases, each a distinct retrieval target with its own access policy:

• Statutes & regulations — INA (8 U.S.C.), 8 C.F.R.; EU Directives/Regulations (LTR 2003/109/EC, Blue Card 2021/1883, Schengen Reg. 2016/399). Held as structured Akoma Ntoso norms.
• Agency guidance — USCIS Policy Manual; EUR-Lex consolidated texts.
• The rulebook — the compiler’s executable norms: conditions, deontic operator, in-force dates, superiority, source URI.
• Matter documents — per-matter evidence (the client file), never mixed across matters.

RAG — two grounding paths, firewalled

RAG implementation — the data plane

• Capability-scoped — a worker holds one matter’s plane plus the shared authority bases; no raw filesystem, DB, or network.
• Per-matter segregation enforced in code (and, in production, by the microVM boundary).
• Audit — every retrieve logged: which base, the query, the returned norm/document ids.
• Provenance — each fact → its evidence document; each rule → its Akoma Ntoso source. The whole verdict is reconstructable.
• Live today: a deterministic lexical core on GPU. Phase 1 swaps in the BM25 + embeddings hybrid over the real corpora.

Three agent roles — a dialectic the compiler grounds

• Proposer — thesis. The LLM agent (Qwen) reads the matter, retrieves via RAG, and emits the Application-IR.
• Critic — antithesis. A different model family (Nemotron / Gemma / Olmo) adversarially refutes it and mines weaknesses — independence keeps it honest.
• Compiler — synthesis. The deterministic arbiter resolves the dialectic against the law, not against another opinion — a check, not a vote (the metadox: no tower of LLMs judging LLMs).

A harness that improves itself — anchored, not gamed

• The agent mines its own failures and proposes edits to its own harness; a held-out gate accepts only real gains.
• A co-evolving critic (a different model family) attacks weak answers — but is anchored to the compiler as fixed ground truth, so it cannot reward-hack the evaluator.
• Live: across epochs the critic earned trust on the compiler anchor, 0.47 → 0.65.

Long-horizon matters — the model reads, the kernel decides

Stopping is compiler-anchored: it cannot finish until every rule prong is verified and every judgment prong is escalated.

Axis 2 — security & isolation

• Security as perimeter, not bolt-on: LAN / mTLS, default-deny egress, a secrets vault, per-matter privilege segregation, hash-chained audit.
• microVM isolation (gVisor → Firecracker): the agent’s action-runtime is sandboxed; the verifier runs outside the boundary — a hard, hypervisor-enforced wall, per matter.
• A boundary monitor zeroes-and-excludes any trajectory that touches the verifier or another matter.

This runs — end-to-end, on a GPU

On a Qwen model on a V100:

• EU long-term residence — the model read three matter documents, extracted “six years three months” → 6.25, gap-found across three prongs → GRANTABLE, compiler-verified, every fact provenance-traced.
• Asylum — it established the timely-filing rule, then escalated the merits (“well-founded fear”) to a human. The firewall held against a live model.
• Every component is test-covered and green.

Architectural rollout

What we are careful about

• All thresholds and citations are SOURCE-TO-VERIFY — a qualified lawyer signs off before anything is authoritative.
• Self-improvement guarantees are epoch-local; the critic is only as good as the compiler’s coverage.
• Research / decision-support, not legal advice — the firewall exists precisely to hold that line.

In one line

The model proposes; the compiler disposes. Every verdict is checkable, cited, and reconstructable.

Verifiability compiler · rule/judgment firewall · authority + evidence RAG · self-improving harness — live.