The harness that refuses a tampered spec.

31 July 2026 · the launch journal

When software is built by an agent swarm iterating against a test suite, Goodhart's law stops being an aphorism and becomes the primary threat model. The suite is the optimization target. Whatever the tests measure, the implementation will hill-climb — and if the tests themselves are editable by the same loop that edits the implementation, the cheapest hill to climb is always the one where the test quietly got easier. Nobody has to be malicious. A "flaky assertion" gets relaxed, a fixture gets "corrected" to match observed output, and six weeks later the all-green dashboard certifies conformance with a standard nobody is actually checking.

The defense is structural, and this repository ships it: every conformance-harness spec is pinned by digest, and the verifier refuses to run against spec text that does not hash to its pin. The implementation can be rewritten all day; the goalposts are cryptographically bolted down.

The pins

The harness covers five laws of the gateway — projection validity, append-only storage, capture determinism, scope enforcement, and an EPCIS conformance subset — each as a spec file the external verifier executes, and each entered in harness/pins.json:

"append-only": {
  "spec": "harness/specs/append-only.spec.json",
  "endpoint": "POST /harness/append-only",
  "digest": "38c1c7a787c183b4484fc21e17f61c71f8827fd2b9b68cd875cfc1e1fe3daf5b"
}

The acceptance command names the digest, not just the file:

autonomous-qa verify <target> --spec harness/specs/append-only.spec.json \
  --expect-digest 38c1c7a787c183b4484fc21e17f61c71f8827fd2b9b68cd875cfc1e1fe3daf5b

If the spec's text does not hash to the pin, the verifier refuses before a single probe fires. There is no mode where a drifted spec runs anyway and the mismatch shows up as a warning; refusal is the behavior.

Watch it refuse

The check is ordinary sha256, so a reviewer needs nothing from us to audit it:

$ shasum -a 256 harness/specs/append-only.spec.json
38c1c7a787c183b4484fc21e17f61c71f8827fd2b9b68cd875cfc1e1fe3daf5b

$ sed 's/append/appemd/' harness/specs/append-only.spec.json > tampered.json
$ shasum -a 256 tampered.json
4840454e130b98be4e54c5e192331c88af8ad94f9ccf1657ad52fcce8d48b25c

One character changed; the digest is unrecognizable; the verifier's --expect-digest gate fails; the run never starts. The committed text hashes to the pin today — run the first command yourself against the repo and check it against the JSON above.

Two design choices make the pins mean something beyond a checksum ritual:

• Pins change only by journaled ruling. A pin moves when — and only when — the spec text is deliberately changed, and the change is a recorded decision with a reason, the same governance the GS1 artefact pins live under. No author accepts their own work: the loop that writes implementation code does not get to mint the digests it will be measured against.
• Floors ratchet up only. The conformance-subset spec asserts its pass-count floor inside the pinned text (passed >= 5, a numeric comparator in the spec itself). Raising the floor requires re-pinning under ruling; lowering it is the exact move the digest exists to make loud. The suite can get stricter silently, never laxer.

What all-green then certifies — and deliberately cannot

With the specs pinned, a green harness run certifies something precise: this implementation, today, satisfies the exact spec texts whose digests are on record — projection outputs that validate against the pinned official EPCIS 2.0.1 schema, storage that refuses UPDATE and DELETE at every door, capture that is deterministic under replay, reads that stay inside their scope, and the conformance subset at or above its floor. The fixtures are pinned with the specs, so "passes the corpus" is as tamper-evident as "matches the spec."

Just as important is what it cannot certify, by construction. A harness run from this repository is advisory — unsigned, and never an attestation of anyone's conformance, including our own. The pins guarantee the goalposts didn't move; they do not transmute a self-run test into a certificate, and nothing here claims otherwise. That restraint is not modesty, it is the same honesty law: a claim is worth exactly the verification that backs it, and a self-administered exam backs less than its taker tends to say.

The deeper point for a platform engineer evaluating agent-built anything: ask to see the mechanism that guards the tests. An implementation without one is reporting its grade on an exam it can edit. Here the mechanism is small enough to audit in a sitting — sha256 over spec text, refusal on mismatch, changes by journaled ruling, floors that only rise — and it is the same idiom this estate applies to every normative input: the GS1 schemas are pinned the same way, the business-transaction layer's directory and corpus ledgers are pinned the same way, and the validators are generated from the pins, never fetched at runtime.

The harness, the specs, and the pins file are in the open repository — run it yourself, including the tamper demo. It takes one sed and one shasum to watch the goalposts hold still.

Sources: harness/pins.json and harness/specs/ (open repository; digests reproduced verbatim); sha256 demonstration run 2026-07-31; the pinned GS1 artefacts at /conformance/.