The Rules That No Longer Exist

There is one genuinely good sentence buried in the pile of AI agent guides published this year, and it is this:

Recurring failures become harness features.

It is good advice. Your agent keeps choking on a malformed API response, so you stop pleading with it in the prompt and write a validator. It keeps asking for a field it could have fetched itself, so you build a tool that fetches the field. It runs away with itself on long tasks, so you give it a step budget. Every one of those moves is correct, and almost every guide published in 2026 says some version of it.

Then they stop, because the advice is about a Tuesday. You have a bad day, you harden the harness, you ship.

Nobody writes about year five.

What a harness turns into

Follow that advice for eighteen months and look at what you have. A few hundred small decisions about the world, each one encoded somewhere in the runtime: this response shape is legal and that one isn't, this action needs a human and that one doesn't, this is the escalation threshold, this is what a valid submission looks like, these are the fields that must be present before we are allowed to act.

Every one of those was true on the day you wrote it.

That is a rule base. And we have built rule bases before — thousands of them, across every regulated industry there is. We know exactly what happens to them over twenty years, because some of them are now old enough to autopsy.

I spent most of August reading one.

Two digits

Picture the end of a clinic day. A physician has seen a patient, examined them, worked out what is wrong and what to do about it, and written all of it down. One task remains before they can go home, and it has nothing to do with medicine.

They have to choose a number. 99213, or 99214.

Choose low, consistently, across a career, and you have quietly given away a meaningful part of your income. Choose high without the documentation to carry it, and the word for that is fraud. Somewhere in between is the code you actually earned, and the only person positioned to work out which one that is happens to be the person who has just spent nine hours thinking about anything else.

Nobody goes to medical school for this. It is a small tax of dread, paid daily, at the end of an already long day.

In 2006, my team was building an ambulatory EHR, and we decided to stop asking.

The idea

Our answer was audacious, and the rest of this depends on it, so let me put it plainly: never ask the physician.

The visit has already been documented. Every box ticked, every finding recorded, every system reviewed — all the evidence the code depends on is sitting in the chart before anybody thinks about billing. So read the chart and derive the code. Leveling as a by-product of documentation, rather than a separate act performed by a tired person at six in the evening.

That is a harness in the sense above, built two decades before the word arrived.

It is also the kind of thing that stops being visible once it works. The schema is wide enough that you can walk it for a long time without ever meeting the sixteen tables that hold the whole idea.

What was inside

The rules were in the database. Not written in code and deployed — stored as rows, and editable by anyone with the right screen.

Twelve thousand seven hundred and twenty-one stored predicates. Six thousand three hundred and forty-seven conditions. Three thousand nine hundred and ninety-one named condition sets, each a small boolean expression over what the physician had touched on screen.

And a vocabulary. Two hundred and six tokens, nine characters each, every one belonging to a body system:

Every one of those descriptions is lifted, word for word, from the CMS 1997 Documentation Guidelines. The twenty body systems are the 1997 list. The eleven exam types are the ten single-organ-system exams of 1997 plus General Multi-System.

Somebody sat down with a federal document and typed all two hundred and six of them into a table, by hand, getting the wording exactly right — because in this system the wording is the rule.

That was the easy half.

Sixty conversations

The bullets are national. What counts as having documented one is entirely local.

One ophthalmologist records a slit lamp exam in a dedicated field with four discrete values. The man down the hall types the same finding into a free-text box at the end of his note. Both performed the exam. Only one of them did it somewhere a query can see.

Nothing about that problem has gone away. Any system built in 2026 to read clinical documentation faces the identical question, and it is still the part that consumes the calendar.

The string in the middle

The engine walks a provider's rules, evaluates each condition set against the visit, and every set that holds concatenates its token onto a growing string:

VSTAmbNewHPXCplPrsHPXHpi=04HPXThrDocHPXRos=10HPXPstHisHPXKinHis
HPXLifHisEXMGenMltOPHPupResCPXFluRtnCPXImmPrvCPXDx0340CPXDx4011CPXi25000

A visit-type prefix, the history elements with their counts, the exam type, each satisfied exam bullet, and the medical-complexity markers carrying diagnosis codes. The whole visit, compressed into something a machine can reason over and a human can still read at a glance.

Then the final step, and this is the part I keep returning to:

using CodingLibrary; // Referenced class for the licensed coding COM component.
 
int result = CodingLibrary.SetEvalManagementData(ICDList, EMVisitData);
if (result == 2)
{
    EMCode       = CodingLibrary.EMCode;
    EMHistory    = CodingLibrary.EMHistory;
    EMExam       = CodingLibrary.EMExam;
    EMComplexity = CodingLibrary.EMComplexity;
}

We did not write the part that turns tokens into 99214. We licensed it, from somebody whose entire job was keeping pace with CMS.

Our code answered what did this physician document, which is local, idiosyncratic and ours to know. A bought component answered what does the federal government say that levels to, which changes without asking you and carries audit liability when you get it wrong. The token string is the seam between the two, and it sits in exactly the right place.

Score it against the 2026 checklist

Take the list of principles every agent harness guide agreed on this year and hold the 2006 system up against it.

It passes. Comfortably.

It also worked. This was never a demo:

A 99213/99214-dominated curve with a thin tail on either side is what an ambulatory practice actually bills. Somebody built a rule-driven decision engine with a clean intermediate representation and a swappable reasoning back end, in 2006, and it produced clinically plausible output on real visits for years.

The part nobody plans for

In 2021, CMS revised how office visits are leveled. History and examination were removed from the calculation entirely. The 2023 update extended that to the remaining visit families.

Physicians still document history. They still document the exam. Those things no longer determine the code. Level is now medical decision making, or total time on the date of service.

Every one of those two hundred and six bullets, every one of the eleven exam types, the entire mapping layer, the counting logic, the history levels, the exam levels — all of it void for the purpose it was built to serve. Twelve thousand predicates that faithfully encode a rule which no longer exists.

The engine still runs. It still produces a token string. It still hands that string to the component. It answers, perfectly, a question that nobody is asking any more.

And it was not alone in there. Once I started looking, the whole system told the same story:

Every regulatory rule that system encoded has been replaced at least once. Not one of them was modelled with an effective date.

That last sentence is the whole essay. Every one of those changes arrived on a published schedule, announced years in advance, with a date on which the old rule stopped and the new one started. The information was public. The system had nowhere to put it.

Why this is about your agent

Open your repository and look at what you are actually shipping.

Your system prompt is a rule base with no effective date. Somewhere in it is a line about which requests to refuse, a line about the pricing tiers, a threshold above which something escalates to a human. Those were facts in March. Some of them are still facts. You have no way to tell which, and no way to ask.

Your tool descriptions are an API contract written in prose, versioned by nothing, read by something that changes underneath you.

Your permission matrix encodes a risk appetite from one specific quarter — the quarter in which somebody senior was nervous, or the quarter after the incident.

Your eval suite is a photograph of the world on the day it was written. When it passes, what it tells you is that the agent still behaves the way it behaved in March. Whether March was right is a separate question, and the suite has no opinion on it.

We had it easier in 2006. Our rules moved while the code stood still. Your rules move — and so does the thing reading them.

That is the genuinely new problem. The engine reading your prompt gets replaced roughly twice a year, and the replacement reads every ambiguous sentence you wrote slightly differently. You are maintaining a contract whose counterparty is swapped out on a schedule you do not control, against rules that change on a schedule you also do not control, with no record of which version of either produced any given decision.

Four ways it goes wrong

Nearly every architectural failure in that system reduced to one of four mistakes. All four are alive and well in agent codebases today.

Customisation by copy instead of overlay. In the old system, letting a physician tailor their documentation meant handing them a private copy of the entire tree. Eight hundred and fourteen physicians, eight hundred and fourteen forks, and a corrected baseline that reaches none of them. Those 3,991 condition sets are really about two hundred rules copied sixty times, once per provider.

The instinct behind it was sound — hand a clinician an empty authoring tool and they will never finish, so ship them something complete and let them bend it. The mechanism underneath was the problem. Nobody needed to tell those physicians to stop tailoring their own screens; what the system never gave them was a way to say this is the shared model, and this is my restriction of it, so every act of tailoring became an act of copying.

You are one forked system prompt per tenant away from the same position.

No shared model for a repeated shape. Thirteen copies of the patient matcher, in eight different versions, each of them deciding whether two records describe the same human being. Sixteen tables carrying an identical clinical shape, each with its own query code and the same undocumented magic number. Eleven work queues, each with its own schema and screens. Twenty-seven lab integration projects, one per partner per direction.

Count the retry logic in your agent fleet. Count the response validators. Count the places that parse a date.

No intermediate representation. Where that system built one, the code is still readable twenty years later — the E&M engine is comprehensible precisely because the token string sits in the middle of it. Where it built none, you get 5,952 lines of procedural XML generation in a single file, and a billing routine of 46,659 characters with two dated backups of itself sitting beside it in the schema.

An agent that goes from natural language straight to a side effect, with nothing inspectable in between, is the second kind.

Failure has nowhere to go. This one is the worst, and it is the most common.

catch (Exception e)
{
    string x = e.ToString();
}

That block is why 167 visits carry no billing code at all. The component failed, the exception was caught, a string was assigned to a variable nobody reads, and the visit quietly went unlevelled. Somewhere downstream a human either noticed and coded it by hand, or did not.

Elsewhere in the same system: 346,691 scheduling import errors sitting in a log file, and 44,637 claim errors in a table. Eleven queues for work, and not one queue for failure. Every one of those numbers is work a human being needs to do, and none of it reaches a person by design.

Your agent's graceful fallback is that catch block, wearing better clothes. So is the confident answer it produces when a tool returns nothing.

What to build instead

So build the dimension everybody skips, which is time. The 2006 engine was good engineering by any standard — layered, data-driven, honest about its own boundaries. It was simply built as though 1997 would hold.

The seventh one is the piece of judgement in that old system that has aged best. A COM component is not how anyone consumes anything in 2026 — but deciding what your harness owns and what it rents from a model provider is the same call those engineers made, on the same seam, for exactly the same reason.

What I keep thinking about

The people who built that engine were not careless. They read a federal guideline, understood it properly, and built a machine-readable transcription of it with a clean architecture and a sensible vendor boundary. It holds up better than most of what ships today.

Fifteen years later a committee changed its mind, and twelve thousand carefully-written rules became an elaborate answer to a dead question.

The engineering holds up. What undid it was one unmodelled dimension inside a system that was otherwise modelled with real care.

Every agent harness being written this year is laying down the same kind of sediment: validators, thresholds, refusals, permission rows, eval cases. Almost none of them are writing down the one thing that decides how long any of it stays true.

The rules will move. They always do. The only real question is whether your system finds out from you, or from an auditor.

Part 2, When the Wrong Answer Is a Person, takes the other half of this: what changes in harness design when the failure mode is a human being rather than a bad answer.