Deterministic QA: turning taste into checks a machine can run

01What can a machine actually decide about creative?

Anything measurable on the delivered file with a number attached to it. Contrast under each run of glyphs. How long a string holds at full opacity. Whether any frame is letterboxed. How much of an element another element covers. Whether the last two seconds are loud. None of those need an opinion, and none of them should ever reach a person.

The same sort, on three frames we made

Three of ours

Machine: the pack carries a zero occlusion budget, and the headline is measured against the worst stone under each letterMachine: cap height as a fraction of frame height, re-measured at feed width, where letterspaced caps lose their strokes firstPerson: whether a pour reads as designed or as cheap. No number we have ever built answers this one

Two of these three are settled by arithmetic and one is not. That ratio is roughly what we find across a delivered set, which is why moving the arithmetic into the engine buys back so much of a review.

02Enforce the mechanical half at creation, not in review

The audit line we run on now is short. A rule is either in the path, or it is a suggestion. After two weeks of watching which of our own rules held, the pattern was blunt: the ones that never broke had a refusing gate in the only path to the effect. The ones that broke repeatedly were the ones living as prose in a document.

Prose compliance does not accumulate across people. Code does.

the finding that reorganized our whole pipeline

So layout correctness moved into the build engine. Text never exits the plate it belongs to, because the box fits or the plate grows, and that decision is made in code before the render rather than caught in review afterward. This is mistake-proofing at the point of creation, which manufacturing settled decades ago and creative tooling mostly has not.

plate  hero_card
  bounds          1080 x 1350
  text_runs
    headline      max_width 62%   plate: headline_scrim   grow_plate_if_needed
    subhead       max_width 62%   plate: headline_scrim   grow_plate_if_needed

elements
  product         occlusion_budget  0%    # nothing may cover the pack, ever
  brand_mark      occlusion_budget  0%    # and it holds 1.0s at full opacity
  ingredient_chip occlusion_budget 10%    # a tenth is a design decision
  texture_card    occlusion_budget 40%    # it is background. say so out loud

on_breach
  1  measure ACTUAL occlusion after compositing, never from the layout intent
  2  reposition, or grow the plate, then re-measure on the new pixels
  3  still over budget -> REFUSE the render. never ship and hope
  4  log the value that failed, never only the fact that something did

That file is the whole idea in twenty lines. A budget is a design decision made out loud, in advance, by the person composing the frame, and the engine's job is to hold them to their own number rather than to have taste about it. We got here because a client once received a static where an image chip was almost entirely hidden behind other elements, and nothing in the code knew that was wrong.

03What does that change about who does what?

It changes the content of a review, not its length. My own standard for this system is that my quality control should be taste-only. Change this image is a legitimate note for me to have to give. Your text escapes its box is not a note at all, it is evidence that something upstream failed, and no amount of care from whoever built that file would have prevented it.

Every note I give that is not a taste note is a bug report the system should have filed instead.

the standard the layout contract was written to meet

The same logic runs on delivery. Every measurable floor is re-measured on the bytes that will ship, a re-render voids every verdict, and there is no override. The generation side of the same architecture is in deterministic pipelines plus AI generation, and the human-facing version of the list is the pre-launch QA checklist.

04Most of what gets called taste is not taste

This is the claim worth arguing with, so here is the test. Take a note you would describe as a taste note and ask whether two strangers looking at the same file would agree on it. If they would, it is a fact with a number missing. If they would not, it is taste and it belongs to a person.

One warning goes with that sort. A set of our statics once came back with zero geometric breaches and was killed entirely on meaning: a leader line ending somewhere other than the thing it named, a label pointing at the wrong doorway, a room called by another room's name. Geometry passed and comprehension failed, so a check for meaning had to be added, and it is a human one.

05The honest half: most of our rules were only documentation

We audited our own rulebook once, classifying every rule by how it was actually enforced rather than by how important it sounded. The result is the reason this post exists, and it is not flattering. Over half of the rules were prose with nothing behind them, in a book where roughly a third carried kill severity.

Nothing about that gap was laziness. Writing a rule down feels like solving it, and the feeling is convincing enough that a team can add rules for months while the same defect keeps shipping. The fix was not more rules. It was a policy that a new rule ships the same day as its enforcement, or it does not count as shipped.

06Which of your automated checks have you had to withdraw?

Two, and both failed the same way: confidently, silently, and in a direction that looked reassuring. The first was a hook gate. Change in the first two seconds is the obvious thing to measure, so we measured it across five films I had already judged, expecting the accepted ones to be the changeful ones.

What actually separated those films sat in the last row of the same table, and it was not motion. The accepted films state one legible proposition before frame twelve. The rejected ones said their first word at frame 238, or never. Information, not change. A gate built on the wrong variable would have blocked the two films worth running.

There is a machine-measured perceptual quality metric worth knowing about, and knowing its limits. Netflix's VMAF predicts how much a compression pass degraded a video against its own source. It is genuinely good at that and it has no opinion about whether the video was worth watching, which is the boundary this whole post is about.

Here is the sharp version. If somebody on your team can name the defect they keep pointing out, that defect already has a number behind it, and saying it again is the most expensive way in the world to fix it. Write the number down, put it in the only path to the effect, delete the sentence from your rulebook, and do that a dozen more times. What is left is the one question worth a human opinion, which is whether anybody stops. Getting to that question earlier is the whole argument in approving the story on paper before anything renders, and the defects worth naming first are cataloged in why AI ads look fake.