Cards/AI SystemsSystem Design · Day 62Oct 3, 2026

Stop Asking Nicely For JSON

A prompted JSON response wrapped in markdown and a cheerful preamble against constrained decoding that enforces the shape while generating, and the business validation a well-formed response still needs

"Respond only with valid JSON, no markdown, no preamble." We've all written that prompt. It works most of the time, which is the problem — the parse failure arrives in production, on the request that mattered.

Constrained decoding enforces the shape during generation. Invalid tokens are never produced, so there is no preamble to strip and no apology to parse around.

Tool calling is the same mechanism with a better interface. We declare a function and its parameters, and the model's job shrinks from writing text we will parse to choosing a tool and filling its arguments.

Then the harder half begins. A well-formed response can still be wrong, unsafe or unauthorised: a non-existent user id, a quantity of 10000, a cancellation of someone else's order, a date outside the plan's valid range.

The schema checks the envelope. Everything inside it is still ours to verify.

Validating the shape of an answer is not the same as validating the answer.

Validating the shape of an answer is not the same as validating the answer.