06 · Machine QA — deliberately not an agent

Identity

Runs on: a script on the mini · Reports to: Orchestrator · Client-facing: no Produces: qa-machine.json — pass/fail per check, no opinions

Role

Catches everything a computer can check, before a model is asked to judge anything. It exists because an LLM reviewing LLM output reliably catches tone and misses the wrong phone number.

This is the single most important design decision in the QA layer. If Machine QA were an agent, it would occasionally decide a phone number was "close enough". It cannot, because it is a string comparison.

Voice

Not a voice. Structured output only.

{"check":"contact_match","pass":false,
 "detail":"/contact phone '03 389 4471' != dossier '03 389 4417'","owner":"build"}

Every failure names the owner so the orchestrator can route it without interpreting.

Reports to

Orchestrator. It is a gate: red never advances.

Inputs

The deployed preview URL, dossier.md, the agreed page list, build-report.json.

Inputs

The deployed preview URL, dossier.md, the agreed page list, build-report.json, and the banned- phrase list.

Outputs

qa-machine.json:

{
  "verdict": "red",
  "ran_at": "2026-08-29T22:14:03Z",
  "duration_ms": 71400,
  "checks": [
    {"check":"pages_present","pass":true},
    {"check":"contact_match","pass":false,
     "detail":"/contact phone '03 389 4471' != dossier '03 389 4417'","owner":"build"},
    {"check":"lighthouse_a11y","pass":false,"value":88,"threshold":90,"owner":"build"}
  ],
  "failed": 2, "passed": 24
}

Every failure carries an owner, so the orchestrator routes without interpreting.

Tasks

1. Wait for a deployed preview URL. Never run against a local build.
2. Run every check below. Run them all — do not short-circuit on first failure, because the orchestrator wants the full picture in one pass.
3. Retry any check that failed once. A check that passes on retry is a flaky check and is logged as a bug in the check itself, not as a site defect.
4. Assign an owner to each failure from a fixed map (contact → build, banned phrase → copy, missing page → build, unsupported claim → copy).
5. Write qa-machine.json and return a single verdict.
6. Green unlocks Editorial QA. Red returns to the orchestrator with owners attached.

The checks

Completeness

• every agreed page exists and returns 200
• navigation links to exactly the agreed pages, no orphans, no dead entries
• blog exists with the seed posts; Keystatic loads

Placeholders — zero tolerance

• no lorem, TODO, FIXME, [NEEDS:, {{, Your business name, example.com
• no unreplaced theme defaults

Contact truth ← the one that justifies the whole script

• phone, email, address, hours on every page byte-for-byte against the dossier
• tel: and mailto: hrefs match the visible text
• schema.org LocalBusiness matches the dossier

Links and media

• every internal link resolves; no 404s
• every image loads, is under budget, has non-empty alt text that is not the filename
• favicon and OG image present

Standards

• Lighthouse performance / accessibility / best-practices / SEO above threshold, mobile profile
• schema.org validates; viewport present; no console errors; no mixed content

Design

• audit-css.mjs passes: more than one duration and one easing, elevation beyond hairlines, some texture. These are the measured signature of a cold site.
• every colour used appears in tokens.css — no stray hex invented at build time
• contrast passes at the sizes actually used, not in the abstract

Language lint — the anti-slop pass

A script cannot tell whether copy sings. It can count the things AI prose does and human prose does not, which turns out to catch most of it:

Thresholds live in config and change deliberately, in git. The client's documented voice overrides every one of these — an owner who genuinely talks in threes is not slop, and the dossier is the authority on that.

SEO

• title and meta present, unique, within length on every page
• schema.org LocalBusiness matches the dossier byte for byte
• NAP (name, address, phone) identical to GBP
• every planned internal link exists, with the anchor text seo.md specified
• no page missing from the plan, no page in the site that the plan does not cover

Claim safety

• flags numeric or superlative claims not present in the dossier: "20 years", "award-winning", "#1", "guaranteed", "certified" — flags for editorial review rather than failing, since some will be legitimate and dossier-backed
• flags a number that appeared to satisfy the specificity floor but is absent from the dossier. Manufactured specificity is worse than vagueness, and this is where it gets caught.

Definition of done

Every check run, results written, an overall verdict. Green means Editorial QA may start.

Hard rules

• Deterministic only. If a check needs judgement it belongs in Editorial QA.
• No thresholds that drift. Contact match is exact, not fuzzy.
• Runs before Editorial QA every single time. Never skipped to save time.
• Never auto-fixes. It reports and names an owner; fixing is the owning agent's job.

Failure modes

Escalation

Red twice on the same check after the owning agent has had two attempts.

Budget

Under 2 minutes. Costs essentially nothing, which is precisely why it runs first.

How it improves

This is the agent-shaped thing with the clearest improvement loop in the whole system.

Watch: escapes — defects the client or Greg found after Machine QA passed.

• If pages pass every slop threshold and clients still say "it sounds like a robot", the lint is measuring the wrong things. Add the signal it missed; do not tighten a threshold that is already green.
• Every escape becomes a new check. That is the entire discipline. A client rings to say the contact form does not work → add a form-submission check. Someone spots a stretched logo → add an aspect-ratio check. The suite should only ever grow, and it grows from real failures rather than imagined ones.
• Track first-attempt pass rate per check. A check that never fails may be worthless; a check that always fails is a template defect.
• Keep runtime under two minutes. If it grows, parallelise — never sample, never skip.