The Second Format

Twelve parts of this series have been about files arriving.

An 834 lands in a directory. The platform reads it, judges it against rules that were versioned before the file existed, writes what survives, records what it did, and hands a person whatever it could not settle. Every claim it makes about a human being came from outside. Its whole job is to be careful with somebody else's assertion.

This part is the first time the platform says something back.

A 270 is an eligibility inquiry — a provider asking a payer whether this patient is covered, on this date, for this service. A 271 is the answer. It goes out in about twenty seconds, to a front desk with a patient standing at it, and in 96 percent of cases the question and the answer are both machine-to-machine.

That is a different kind of exposure than anything in the twelve parts before it. It is also exactly why this was the right format to add second.

An Abstraction With a Single Implementation Is an Assertion

detector.py has recognised six file formats since Part 4. src/ingest/parsers/ held one implementation and five one-line placeholders right up to this part. Provider, Encounter, Claim and Payment are still the stubs Part 3 declared them.

Part 1's layer table promised an ingest layer that accepts any format. Part 2 then made four structural claims the entire architecture rests on, and every one of them has had exactly one witness since:

One transaction has been modelled. Everything the series has said about format independence has been a prediction, held up by an abstraction with a single implementation behind it — which is a sentence rather than a system.

So this part added a second format and gave each claim one of three verdicts. Held, meaning the second format needed no change. Leaked, meaning it needed one, and here is what it cost. Never tested, meaning the format that would settle it is still not built, and saying so.

Every structural claim this architecture rests on had exactly one witness, and the witness wrote the architecture.

The honest reason 270/271 was chosen over HL7v2 ADT, which is the better story and the worse test:

ADT would prove the write path twice and add a stub entity to do it. The inquiry pushes on the seam in the one direction nothing has pushed on it, and needs no new canonical table to do so.

Where This Pays for Itself: The Four-Cent Answer

The eligibility inquiry is the highest-volume administrative transaction in American healthcare, and almost nobody outside the industry has heard of it.

CAQH measures it annually. In 2023, medical plans and providers conducted 31.5 billion eligibility and benefit verifications — 51 percent of all medical administrative volume, more than claims, remittances and prior authorisations put together. Spending on this one transaction came to $44 billion, which is 53 percent of total medical administrative spend.

Here is what one of them costs, by how it is done:

A phone call costs a health plan a hundred times what answering the same question in X12 costs it. Which is why 96 percent of medical eligibility verifications now run fully electronic on the 270/271 standard, four percent through portals, and the fully manual share rounds to zero. The average time saved per transaction, on the provider's side, is twelve minutes.

Twelve minutes is a person. Four cents is the absence of one.

Part 12 was written from the chair of a person working a queue, and the argument there was that the queue is where a platform's hardest cases go to be judged by the most expensive resource in the building. This transaction has no queue and no chair. The answer is composed, sent and acted on inside the time it takes to find a parking space.

That is the standard the rest of this part gets measured against, and the measurement is in the last third.

The Verdicts

The seam held, including in a case it was never designed for. The 270 parser subclasses BaseParser with no change to it and emits models that have no canonical counterpart at all — because a 270 asserts nothing about anybody. The 834 needed that boundary to separate reading from mapping. The 270 needed it to separate reading from answering, and the same boundary served both. A test counts the rows in persons, person_identifiers and coverages before and after a seven-question inquiry against live Postgres, and the counts are identical.

The crack is underneath the seam rather than in it. x12_common.py described itself in its own docstring as shared by every X12 parser — 834, 270, 837, 835 — and said one of its functions existed so those parsers could reuse the module unchanged. The 270 needed it changed on its first day. X12 5010 declares a fourth delimiter in ISA11, the repetition separator, and an inquiry uses it: EQ*30^1^33 asks about three service types in one element. The 834 never repeats an element, so in ten parts the tokenizer had never read ISA11 at all.

Two additive changes, an afternoon's work, nothing the 834 does altered. The finding is the docstring: "shared" meant "used once". Discovering that with the 837, which repeats elements constantly, would have cost the same afternoon plus everything built on the assumption in between.

The branch test held, and decided a real question in one sentence. The person code was the first new value since Part 6 that needed a home. Nothing in the platform reads the dependent's suffix apart from the contract it qualifies — so it earns no column. It is stored as a composite identifier value, SUB002-03, and no migration was needed for its shape. The build plan had reserved migration 009 for it, and the shape never asked.

Format independence of the gate: never tested, and the verdict stays that way. A 270 writes no canonical entity, so there is nothing for the gate to judge, and the read path correctly never calls it. The claim is about what happens when a second asserting format produces entities and the same rules judge them without knowing the file. Passing the gate untouched is evidence of nothing. HL7v2 ADT would settle it — an A04 asserts a Person from a PID segment — and that is now the strongest argument for building it.

A verdict of "never tested" is worth more here than a verdict of "held" would have been, because "held" was available for free and would have been untrue.

The Dependent Nobody Could Ask About

Part 3 made a promise, in a sentence I wrote with some confidence at the time: subscriber ID plus person code handles the 834/270 dependent case cleanly, including twins, because the person code is how the standard distinguishes dependents on one contract.

IdentifierType.PERSON_CODE has been sitting in enums.py since that part, to serve exactly that case.

Nothing has ever written one.

Three decisions made that true, and each one was right on its own.

Part 11 counted the consequence without ever naming it. 839 of 1,847 re-arrivals carried no identifier of their own, which is why they sat unlabelled in the estimator's negative class, and the carried-forward item blamed the sampler.

The cause was a mechanism that was designed, named, documented in a published article, and never populated. The sixth of those in this series.

What makes it different from the other five is that this one was found by reading the build plan before writing any code, rather than by running something and watching it fail. So this part ships the fix instead of the confession.

The fix, and the two things the standard corrected

What gets stored is the contract's id qualified by the suffix — SUB002-03 — which points at exactly one person on exactly one contract. The 834 parser reads the suffix, the mapper writes the composite value, and tier 1 resolves on it.

The build plan said the person code comes from the INS segment. 005010X220A1 has no element for a person code — INS17 is the multiple-birth sequence number, which is a different thing entirely. The platform reads REF*ZZ in Loop 2000, which is a companion-guide convention rather than a standard element, and both the parser and the model now say so in those words.

The second correction is more pointed. The plan — and Part 12's closing paragraph, which promised this part would answer "the dependent who arrives with no identifier of their own" — assumed a dependent carrying a person code would be asked about in the 2000D dependent loop. The 5010 search rules send a patient who has an identifier of their own at the subscriber level, with that identifier. X12's own published dependent example carries no identifier in the dependent loop at all.

So a dependent with a person code is asked about as a subscriber and answered as themselves, and a dependent arriving in 2000D — named under somebody else's contract, with no number of their own — is declined. The platform tells the asker the contract exists and asks for an identifier.

Also corrected: the comment at resolver.py:61 had been wrong about its own reasoning for ten parts. That list is the tier-2 hard veto, not resolution. Being absent from it never stopped a person code from resolving. What stopped it was that nothing wrote one.

What the fix moved

The enrollment corpus was regenerated with one segment added per dependent for one of the three synthetic payers:

443 of the 839 now have a triple to repeat. The 396 that remain all belong to the one synthetic payer that sends nothing of its own for a dependent — which no amount of code can fix, and which is the honest residue of Part 12's prediction that all 839 were the same person.

And the end-to-end check: 150 of 150 corpus dependents asked for by their card number were answered as themselves rather than as their father.

Six Files

The measure the build plan asked for, set before the work started: how many files outside the parser, mapper and transaction packages have to change to add a format to a platform that claims to be format-independent?

Six. Two of them are repairs to things the second format exposed rather than costs of the format itself — a parser constructed by name, and a database read living inside a request handler. One is a migration whose second half exists only because of the trap the first half creates.

Part 1 claimed a platform, and a short list is what that claim predicted. The list is not empty, and every entry on it is a place where the first format had quietly been mistaken for the general case.

The sharpest one: parse_file had been constructing X12_834Parser() by name for ten parts and labelling its trace span x12_834 whatever it was handed. The stage described throughout this series as the pipeline's parse step was the 834's parse step.

The Pipeline Ran Backwards

The write path is parse, map, gate, write, resolve, and every stage returns the thing the next one writes. An inquiry is parse, resolve, read, respond. It must create no person, write no coverage, and has no entity for the gate to judge.

The build plan expected the new path to sit as a sibling beside run_pipeline, sharing detection, the file row, tokenizing and the parsed lineage row. Two of those four things turned out not to exist.

There is no run_pipeline. The write path's sequencing lives in the route handler and, separately, in the Dagster asset graph; stages.py is six stages and no conductor. And nothing has ever written a parsed lineage row — Transformation.PARSED has been in the vocabulary since migration 001 and in the constraint since 007, while lineage rows are only ever written for canonical entities, and parsing produces none.

That is the eighth mechanism in this series designed, named and never called. It is left unbuilt deliberately: nothing reads a parse-level lineage row, and writing one to satisfy a vocabulary would be committing the exact error the vocabulary already is.

What the two paths genuinely share is file registration, storage and detection, the contribution opened before the format is known, and the parse stage — after it learned to take a format. What they do not share is the gate, the canonical write, the terminal status, and the audit record.

That last one raised a question twelve parts of audit machinery had never had to answer.

1,136 Questions With Known Answers

This is where the four-cent arithmetic earns its place.

The enrollment corpus has a ground-truth manifest — it is generated and seeded, so every person in it has a knowable right answer. So 1,136 inquiries were generated against it across eleven classes, each one carrying the response the manifest says it should receive, then run through the real ingest endpoint and judged on the 271 that came back, read with the platform's own tokenizer because that is what a trading partner would have in hand.

940 agreed. 196 did not.

Four classes, all of them wrong in every single instance, which is the shape of a defect rather than a tolerance. Every one of the 196 is listed individually in the report, with the person, the number asked, the date, the expected answer and what the 271 actually said.

The one that matters most

A child's date of birth, sent with the father's number, is answered about the father. Fifty times out of fifty.

This is the commonest mistake made at a front desk anywhere in American healthcare, for an entirely understandable reason: the insurance card shows one number for the whole family. So the receptionist types the number on the card and the name and date of birth of the child in front of them.

The right answer is a rejection — patient birth date does not match the patient on the database — which is how both a state uniform companion guide and the federal Medicare system refuse it.

The platform returned active coverage, under the father's name, to a question about his daughter.

The cause is that the read path verifies nothing the inquiry says about the patient. It resolves the number and stops. The response does carry the subscriber's name as held on file, so a careful human reader could spot the mismatch — and at four cents a transaction, 31.5 billion times a year, the careful reader is the thing that was automated away.

The platform answered a question about a child with a fact about her father, and it was right about the fact.

This is the build plan's own finding — a 270 for a child resolves to the father — arriving by a route the person code does not close, because in this case the provider never sent a person code and never would. Fixing it means checking the demographics in the request against the resolved person, which is a new decision about when a date of birth may veto an identifier. It is carried forward rather than slipped in quietly after the measurement was taken.

The one that contradicts its own docstring

A member whose payer re-sent them with a later effective date is unreachable on any date before the re-issue. Forty-three out of forty-three.

Part 11's fix for re-issues closes the older assertion at the new one's start and marks its status superseded. The docstring on that function says the row's start date is kept deliberately, so that "what did this identifier mean in March" stays answerable. But resolution reads only active rows, so the closed window is invisible and March resolves to nobody. Closing the window was sufficient on its own; the status change double-closed it.

A one-line fix, in Part 11's write path, found by a part that was not looking for it.

And the two that are the same mistake from different ends

A date before coverage began returns "nobody holds this number", when the truth is that a known member was not yet enrolled. Sixty of sixty. The cause is that the 834 mapper dates an identifier from the member's first coverage begin, so on an earlier date the number did not exist yet. Every step defensible; the composition tells a front desk wrong number when the honest answer is not yet. The same endpoint has been returning 404 for this since Part 11 — the 271 made it visible rather than introducing it.

And a re-issued member asked about after the re-issue gets the right answer with the wrong dates: covered, but eligible from April when they have been covered since January. Two person rows that nothing links, because tier 1 declines re-issues on purpose and tier 2 is still switched off. Part 11 called these "nobody's match" and Part 12 put them in the queue's identifier lane — and a 271 is where an unworked queue item becomes a wrong date on a provider's screen.

What agreed, and why it counts

The 940 carry their own weight. 150 of 150 dependents asked for by card number answered as themselves — the fix this part shipped, working end to end. 150 of 150 dependents named without an identifier declined properly, none falling through to demographic matching, none answered about the subscriber. 120 of 120 father-and-son halves, each asked by his own number, got his own coverage dates — the same pairs a confident model merged at 0.93, kept apart here because the answer rests on identifiers and deterministic links alone.

The Standard Had Already Thought About It

Part 11 reasoned its way to three distinctions by hand: one status for a number nobody holds, another for a number two people hold, and a third for a known member with no active coverage. X12 has encoded all three since the 1990s, and the mapping is clean. The endpoint's reasoning was right for reasons considerably older than the endpoint.

Which is the pleasant version. The less pleasant version is that the build plan's own table of codes had one of them wrong — it used a rejection code that the 5010 guide prohibits in that position. X12's published interpretation answers a payer asking exactly that question, and the federal Medicare eligibility system made the same change when it moved to 5010. Corrected in the writer, in the roadmap, and cited line by line, because a wrong code in a 271 is a wrong answer that validates.

The 271s were then run through an independent open-source X12 validator, in its own virtualenv because it pins an old Pydantic. A covered response and a not-covered response validate exactly as written. Every rejection response is refused — on one element, because the validator checks the rejection code in the patient loops against the code list for a different loop. Swap that single element for a code from its list and each one passes.

That is a defect in the validator. Both it and a second one — it refuses any file carrying more than one subscriber, which the guide permits — are pinned by tests that assert the wrong behaviour from both sides, rather than worked around. A correct code was not changed to please a tool.

What a Four-Cent Answer Costs When It Is Wrong

The build is clean by every gate this project has: 1,934 tests passing against live Postgres, strict type checking across 213 source files, no lint. Six files to add a format to a platform that promised format independence. The person code written at last, and 150 of 150 children answered as themselves.

And 196 wrong answers out of 1,136, every one of them traceable to a decision that was correct when it was made.

That is the finding, and it took a second format to produce it. Each of those four classes is a composition of individually defensible choices, made in different parts, months apart, by someone who checked each one against the standard at the time. The 834 never asks about a date, so dating an identifier from the enrollment was free. Closing a superseded window and marking it superseded both sounded like the same precaution. Refusing to give a child the father's identifier was straightforwardly correct.

A format that only asserts can be wrong quietly for years. A format that asks finds out in twenty seconds.

Twelve parts built a write path and judged it by whether the data that went in came out intact. It always did. The inquiry judged the same data by whether it could answer a question about it, and the answer was no, four different ways, in the single highest-volume conversation in American healthcare — the one that runs 31.5 billion times a year, costs a plan four cents, and has nobody reading it.

Part 12 ended on what the platform owes the person working its queue. This part ends on what it owes a receptionist who will never see a queue, never file a ticket, and simply tell a patient what the screen said.

Part 14 is the retrospective: what this series got wrong, what the real numbers were against the planned ones, and the cost of the eight mechanisms that were designed, named, published in an article, and never once called. Two of those eight were found in the last two parts, both by reading rather than by running, which is either an argument for writing the plan first or an admission about the ten parts before it.

Probably both.

This is Part 13 of Building the Platform, a build-in-public series on a healthcare enrollment data platform. The corpus, the people in it and the payers are synthetic; the generator is seeded and reproducible, and the ground-truth manifest is the only reason any of the numbers above can be checked. Transaction volumes and per-transaction costs are from the 2024 CAQH Index, which reports 2023 data.