AI Proposes, Rules Dispose

Six articles ago this series opened with forty-seven people.

An 834 enrollment file from a mid-size employer. Three hundred and twelve members. The file passed X12 syntax validation and the parser read every record without throwing. Forty-seven of those records failed to map, and the reason was almost boring: the employer's new HR system set coverage effective dates to the first of the month and termination dates to the last day of the same month. Run that through a system expecting a half-open interval and you get a coverage span of zero days. Forty-seven people who, walking into a clinic that week, would have shown up as uninsured.

Part 6 is the part that was supposed to catch them.

It does. The rule is eleven lines of YAML and it has a name — coverage_span_ordered. It ships in the base rule set, it runs on every coverage record that reaches it, and it writes a row to the audit trail every single time it passes.

It also cannot fire. Not once, not on the ingest path, not ever. I found that out by trying to break my own system, and it turned out to be the most interesting thing in this part.

The Gate Knows No Healthcare

The obvious way to build a validation layer is to write validation code. A function per rule, a module per family, a registry that runs them. It works, and it's usually what gets built — the direct path is faster to start, and time is the constraint that always feels binding. What that trade buys is hours now against hours later, at a rate nobody sets deliberately.

The problem shows up the first Tuesday a payer revises a companion guide.

Under that design, the response to "this payer now requires a plan code in a different element" is a code change, a review, a test, a release, and a deploy. The rule is correct in about forty minutes and live in about four days. Multiply by the number of payers a real platform serves and the validation layer becomes the slowest-moving part of a system whose entire job is absorbing other people's variance.

So the gate layer in this platform contains no healthcare knowledge. It's an interpreter. Everything it enforces lives in metadata:

- name: coverage_span_ordered
  family: schema
  severity: error
  applies_to: coverage
  when: coverage.valid_to is not null
  assert: coverage.valid_to > coverage.valid_from
  message: >
    Coverage ends before it begins:
    {coverage.valid_from} to {coverage.valid_to}
  description: >
    Half-open [start, end): valid_to is exclusive, so
    equal dates are also invalid — that span covers zero
    days and is almost always a mapper bug rather than a
    zero-day enrollment.

That's the forty-seven records, written down. valid_to > valid_from, with a description explaining why equal dates fail too — which is the exact shape the employer's HR system produced.

Twenty-five rules ship in the base set across five families. Schema asks whether a record is coherent on its own terms. Compliance asks what CMS and the standard require. Business carries domain logic that's neither. Referential asks whether the things a record points at exist. Disposition acts on what the AI proposed.

The engine reading them knows about none of that. It knows how to evaluate an expression against a context and write down what happened.

I should say plainly that this pattern has a long history, and that some of that history is mine.

The industry got around to naming most of this later. What I had in 2004 was a table, a reader, and the slow realisation that the table had become the more valuable half.

That's the shape I recognised in this part, twenty-two years on, and it's why src/gates/metadata.py exists as a thing separate from the rules. It describes the canonical model — every field path, its type, whether it's nullable, which enum members are legal — derived from the Pydantic models rather than restated beside them. The rule publisher reads it to reject coverage.valid_to > coverage.first_name before that rule can ever run. The API serves it, so "what does this system know about a coverage record" is a request rather than a code review. And the field-mapping agent from Part 5 should be reading it too — its prompt currently describes the canonical model in a generated string, which is the same idea reached by a different road.

Three consumers, one description. In 2004 that took me by surprise. This time I built the registry first.

Which leaves a question I keep turning over. Those systems — the validation engine, the metadata-driven repeaters, the alerting, the reporting — solved problems that have not gone anywhere, on a stack that has. A metadata layer in 2026 gets type introspection, real schemas, streaming, and a model that can read a description and propose what to do with it. Some of what took me months in ArrayList and casting is a weekend now. Some of it is harder, because the expectations moved.

I'd like to find out which is which. That may well be the next series.

Rules That Live in a Table

Metadata in a YAML file is configuration. Metadata in a versioned store is something else, and the difference is worth the extra machinery.

The YAML is a seed. make seed-rules publishes it, and from that moment the engine reads rule_definitions in Postgres. A published rule set is immutable and content-hashed. Publishing and activating are separate acts, which is the part that took me longest to be sure about and turned out to matter most.

CREATE UNIQUE INDEX idx_ruleset_versions_active
    ON ruleset_versions((activated_at IS NOT NULL))
    WHERE activated_at IS NOT NULL AND deactivated_at IS NULL;

That index enforces exactly one active version. It's there because of a failure mode I want to be blunt about: a gate layer with zero active rule sets and a gate layer that permits everything are indistinguishable from the outside. Both produce clean files. One of them is broken. The database refuses to let "which rules are live" have two answers, and the code refuses to process a file when it has none.

Every row in gate_results names the rule and the ruleset version that produced it. Passes included — and storing the passes is the whole point. An audit trail that records only failures can't distinguish "we checked this and it was fine" from "we never checked." The second is what you're being asked to rule out.

Versioning buys one operation that justifies the rest of it:

POST /gates/rulesets/{version}/dry-run

Give it a past file and a candidate version. It loads the entities that file produced, evaluates them under those rules, and returns the diff against what was recorded at ingest. This rule change would have blocked forty members last month.

That question is answerable before activation rather than after. It's also what makes "change rules without a deploy" a defensible position rather than a reckless one — the speed comes with a way to see what the speed is about to do.

Moving rules out of code makes what a system enforces legible. It doesn't lower the bar for changing them.

The Expression Language That Can't Do Anything

assert: coverage.valid_to > coverage.valid_from is a string in a database that decides whether someone's enrollment processes. Any sentence containing "a string in a database" and "gets evaluated" should make you nervous.

Python offers an obvious answer and it's the wrong one. eval() on rule text — even validated rule text, even from a table only engineers write to — puts arbitrary code execution one bad review away.

So expressions are parsed with ast.parse in eval mode, walked against an allowlist of node types, and compiled to a tree of closures. Comparisons, boolean operators, negation, constants, attribute access rooted at a declared name, and calls to registered predicates. Everything else is rejected by default, at load time, with the rule name and the offset.

I tested that boundary the way it deserves — twenty hostile expressions, the greatest hits of Python sandbox escapes:

'__import__("os").system("echo pwned")'
'coverage.__class__.__mro__'
'coverage.__init__.__globals__'
'[x for x in coverage.plan_id]'
'(lambda: 1)()'
'getattr(coverage, "plan_id")'
'(y := coverage.plan_id)'
'eval("1+1")'

Twenty blocked, zero compiled. Nothing reached an evaluator, because there is no evaluator to reach — a rejected expression never becomes a callable at all.

The vocabulary is closed in the other direction too. The only functions a rule can call are the fourteen predicates registered in predicates.py, each one typed, tested, and under mypy strict like the rest of the codebase:

  assert: not spans_overlap(coverage, related.existing_coverage)

spans_overlap is real interval arithmetic — half-open bounds, open-ended spans on either side or both. It has its own test suite, including the case that matters most:

The rule file says when to apply that. It cannot reimplement it, and it cannot invent a new one. When a rule needs something the vocabulary lacks, the answer is a new predicate with a test — a code change, deliberately, because that's the boundary where a code change belongs.

There's no arithmetic in the expression language either. coverage.valid_to - coverage.valid_from > 365 looks harmless and means writing a date library inside a config format. days_covered(coverage, as_of) > 365 is a predicate someone can read.

Two Gates, Not One

The pipeline has gates in two places, and I went back and forth on this longer than I'd like to admit.

Three families run before the canonical write, against mapped-but-uncommitted entities. A record failing an error-severity rule is dropped from the write set. Warnings are recorded and the record goes in anyway.

Referential rules run after the commit. Their questions — does this subscriber exist, does this span collide with one already on file, is this identifier claimed by someone else — are questions about the database. Answering them against a half-applied write set means reimplementing transaction isolation in application code, and I have never seen that go well.

The cost of the split is honesty about what each phase can do. The post-write gate records but never deletes. A referential failure produces a finding, not a rollback. Deleting committed canonical data on a rule's say-so is a larger risk than the finding is usually worth, and the finding now sits in gate_results where a human can act on it deliberately.

Blocking is per member, matching the parser's discipline from Part 4. One bad record costs that record. A file where some members pass and some fail returns 200 with status: "partial", the good ones written and the bad ones named. A file where every member fails returns 200 with status: "rejected" — the upload worked, the parse worked, and we have a complete explanation of why nothing was written. A 4xx there would be telling the sender their request was malformed, which would be false.

The Audit Commits First

This one looks backwards written down, and I want to defend it properly.

Gate results are written in their own transaction, before the canonical write they're gating.

Put them inside the canonical transaction and they're lost whenever that transaction rolls back — which is precisely the file someone will ask about. "We refused twelve members and then the write failed" becomes an empty table and a status column reading failed.

Writing first means gate_results can hold a passed row for an entity that was never written. That reads like a bug and it's accurate: the gate did pass it, and something downstream failed. raw_files.status says so.

The durable artifact is the judgement, not the row. A record can be rewritten from the source file. The reasoning about it cannot.

The Measurement That Came Back Empty

Part 5 ended with a promise. The agent produces proposals carrying a confidence score, nothing thresholds on it yet, and the gate layer would supply the judgement — confidence bands, auto-approvals, the human reviewer demoted to exception handler.

That isn't what shipped, and the reason is the part worth reading.

Self-reported confidence from a language model is not a calibrated probability. A 0.9 is not a claim of being right nine times in ten. It's a number the model produced because the prompt asked for one. Before thresholding on it, somebody has to check whether it means anything — and the honest way to check is to run the agent against pairs where the answer is already known.

So this part built the harness. Sixty-three labelled synthetic pairs, roughly a third clear merges, a third clear non-merges, and a third the genuinely hard middle:

- id: twins_same_address
  truth: different_people
  difficulty: hard
  incoming:  {first_name: Marcus, last_name: Reyes,
              date_of_birth: 2011-09-02, ssn_last_four: "8830"}
  candidate: {first_name: Marco,  last_name: Reyes,
              date_of_birth: 2011-09-02, ssn_last_four: "8831"}

The harness runs, the scoring works, the report renders, and all of it is tested end to end against a stubbed resolver. What it has never done is run for real. make calibrate needs an ANTHROPIC_API_KEY, and the environment this part was built in doesn't have one.

So there is no reliability curve. And with no curve, there is no auto-approve band — disposition.yaml ships with the auto-approve rule commented out and three auto-reject rules live.

The design detail I'm happiest with is how the absence is enforced. Auto-approval requires at least one auto_approve_* rule to exist and all of them to pass. With none present, nothing can be approved unattended — by construction, rather than by everyone remembering. The safe state is the one you get when the evidence is missing.

Writing an article that says "we built the measuring instrument and haven't taken the measurement" is less satisfying than publishing a curve. But a part that could only have concluded one thing was never an experiment, and shipping a threshold I hadn't earned would have been the actual failure.

There's a coda, and it arrived after the part had shipped.

Someone asked what the platform is for if it can't auto-approve — auto-adjudication rate being the number systems like this actually get bought on. Going to answer that properly, I went looking at what auto-approval would need, and found something that had been sitting in plain sight since Part 3. unlink_persons existed in queries.py. It was reachable from no route. A merge could be created through the API and undone only by someone writing SQL against production.

So the real reason to withhold auto-approve was never the missing curve. A decision you cannot reverse through the API is a decision nothing should make unattended, and that holds however well calibrated the thing proposing it turns out to be. The calibration argument was true, and it was second.

The endpoint exists now — POST /identity/links/{id}/unlink, answering 404 for a link that never existed and 409 for one already reversed, because a steward who reverses a link somebody else already reversed has to be able to tell that from a mistyped id.

What it cannot do is undo the consequence. A claim adjudicated against a bad link before anyone noticed survives the unlink cleanly. The schema's reversibility makes the record recoverable and the harm permanent, which is the conservative principle arriving by a road I hadn't expected to walk down.

The Rule That Can Never Fire

Which brings me back to the forty-seven.

I built a file to prove the gate worked. I took a good 834 fixture, reversed one member's coverage span so it ended before it began — the zero-day shape from the opening story — and posted it.

{
  "status": "failed",
  "errors": [
    "member 1: record: valid_to must be after valid_from (half-open interval [from, to))"
  ],
  "gate": { "evaluated": 0, "passed": 0, "blocked_members": 0 }
}

The record was caught. coverage_span_ordered had nothing to do with it, and the gate evaluated zero rules.

The canonical Coverage model carries a Pydantic validator that refuses to construct an object with a reversed span. The mapper never produced a Coverage. The gate never received one. The rule I wrote for the forty-seven people this series opened with sits in the active rule set, correct and untriggerable, shadowed by a validator written four parts earlier.

Two readings of that, and choosing between them turned out to matter less than the third thing it exposed.

One: it's dead code that reads as complete — the exact failure Part 5's retrospective called out, where the span helpers existed with zero call sites and every agent call ran untraced.

Two: it's defence in depth. The rule fires for every coverage that does reach it, and coverages arrive by more than one path — a future CSV mapper, a backfill, a reprocessing job, any writer that constructs entities differently. On the ordinary path it writes a passed row every time, which is what turns "prove you checked" into a question with an answer. A rule that never fires because the data is always clean is indistinguishable from a rule that never fires because it's broken — unless it records its passes.

I went with the second. Then I went looking at the rest of the base rule set and found the same thing almost everywhere.

Coverage's validators and migration 002's CHECK constraints already refuse an inverted span, an unknown maintenance code, a future date of birth. Most of the base rules are standing behind a type system that got there first. Which raises the obvious question — what is the gate layer actually for, if the model already refuses the bad records?

The answer is in the two rules that do block, and both of them are scoped to a payer:

A NOT NULL column says a group number is always required. A Pydantic validator says the same thing. Neither of them can say required for this payer, under this version of their companion guide, and optional for everyone else — and that sentence is most of what a companion guide contains.

Constraints express what is universally true. Rules express what is conditionally required. The forty-seven were a universal truth, and a validator was always going to be enough for them.

That reframes the layer, and it reframes my own build plan. I wrote coverage_span_ordered as the flagship example — the rule that answers the story this series opened with. It's the least load-bearing rule in the file. The value is concentrated in the payer-scoped rules I treated as a footnote, an override example bolted onto the end of the seed set to demonstrate the scoping mechanism.

The mechanism was the point all along. I just had the examples the wrong way round.

What the Gate Found in Its Own House

Building the gate exposed two problems that predated it.

The second was quieter and I nearly shipped it. Every identifier rule in the compliance family was guarded on id_type — MBI format for MBIs, HICN structure for HICNs, and so on. Sensible, and the consequence is that an ordinary file of plain subscriber IDs matched no guard and produced no verdicts at all. Not a pass, not a failure. Silence. The audit trail for a normal file said nothing about its identifiers, which reads identically to not having checked.

The fix was a rule with no guard — identifier_authority_known — that every identifier reaches. Now the common case produces a row.

What the Pipeline Holds

An 834 arrives. It's detected, tokenized, parsed into intermediate models, and mapped to canonical entities. Three rule families judge those entities before anything is written, and a record that fails an error-severity rule doesn't get written. Everything that survives commits in one transaction with its lineage. Referential rules then check it against what was already on file. With ?resolve=true, the agent proposes identity links, and disposition rules reject what a deterministic fact contradicts and queue the rest for a human.

Every one of those decisions — pass and fail alike — is a row naming the rule and the version of the rule set that produced it. Six months from now, "why was this member rejected in March" resolves to the rule text that ran in March, rather than whatever the rule says today.

The gate answers a narrower question than the agent, and the narrowness is the point. The agent asks whether two records look like the same person. The gate asks whether the evidence is strong enough to act on that belief — and right now, for identity, its answer is that a human decides. That will change when somebody runs make calibrate and the numbers earn it.

Part 7 steps out of the code entirely. Six parts of this series have described machinery for handling a file, and never explained what the file is, why it looks the way it does, or how forty-seven people end up holding a coverage span that covers nothing. That piece assumes you have never seen an 834, and it is the one I should probably have written first.

Forty-seven people lost their coverage to a rule nobody had written down. The rule exists now. It has never fired, and it proves every day that it ran.