Ground truth is where parsers agree.

Every conformance story on this origin starts the same way: pin the normative artifact, gate the implementation against it, refuse to run if the artifact was tampered with. The EPCIS side has that luxury — GS1 publishes the JSON schema, the JSON-LD context, the OpenAPI binding, and we pin them by sha256. The business-document side does not. X12 publishes no conformance test suite and no certification program. Its product list is the standard itself, code directories, technical reports, and licensing tiers; the free examples library is a handful of one-off scenario documents, not a systematic corpus. When the standards body sells the text and ships no oracle, "we parse X12 correctly" is an unfalsifiable claim — unless you manufacture the oracle honestly. This post is how.

The oracle stack

Three independent checks, none of which we can quietly bend toward our own implementation:

1. The parsers-agree subset. Every corpus document runs through independent open-source parsers as subprocesses, at arm's length: pyx12 (Python, BSD-style, parsing X12 since the early 2000s), stupidedi (Ruby, BSD-3-Clause, with the best labeled fixture corpus in the open), and node-x12 (JavaScript, MIT) as the generic stream oracles, plus imsweb/x12-parser (Java, BSD-style) as a verdict oracle on the transaction sets its definitions cover — with the GPL-licensed engines (bots, EDIReader) held at arm's length as subprocess tie-breakers only, never linked or vendored. Outputs are normalized to a comparison form — segment stream plus element values, down to the component split, never any parser's object model — and the subset where the capable oracles agree is ground truth. Four engines, four languages, decades of accumulated corpus exposure: our implementation must match the consensus, and the consensus is not ours to edit. The roster itself is governed: swapping an oracle in or out is a journaled spec-text ruling in the layer's ledger, not a quiet edit.

2. Byte-exact round-trip. emit(parse(x)) === x, byte-compared, for every parseable corpus document. The law carries its own honesty clause, defined up front: exactly two positions are allowed a canonical rule instead of byte-fidelity — trailing-segment-terminator handling and trailing-empty-element handling — and a document that differs only there is recorded as canonical-equal, never silently passed. The gate asserts the canonical-equal count against the recorded number; today that recorded number is zero — every parseable document in the corpus round-trips byte-exact, and the first exception will have to be recorded to pass. The same per-job honesty as the XML translator's fidelity report: the loss ledger is part of the output, not a footnote.

3. Property-based envelope fuzzing. X12 is self-describing at the byte level — the element separator is a byte position in the interchange header, the header itself is fixed-width, and that fixed-width discipline is a fuzzing goldmine. Seeded, deterministic generators probe the invariants that must hold for any input: parse-or-typed-rejection, never a crash; byte-exact round-trips on every accepted case; stable verdicts across ten-to-the-fifth generated cases.

Disagreement is governance, not tie-breaking

The differential run over the pinned corpus — permissively-licensed parser fixtures plus self-authored synthetics, every fixture carrying its license papers in the corpus ledger — is a committed artifact, not a claim. The run:

$ pnpm differential
differential: 199 fixtures, 187 full agreement, 0 queued
oracles: {"pyx12":"4.0.0","stupidedi":"1.4.5","node-x12":"1.7.1",
          "imsweb-x12-parser":"1.16 (Maven Central)"}

and the committed report it writes (corpus/agreement-report.json):

{
  "generatedAt": "2026-07-31",
  "oracles": { "pyx12": "4.0.0", "stupidedi": "1.4.5",
               "node-x12": "1.7.1",
               "imsweb-x12-parser": "1.16 (Maven Central)" },
  "fixtures": 199,
  "agreement": { "full": 187, "disputed_or_lax": 0 }
}

187 of 199 fixtures in full agreement; the other 12 are recorded under-covered — documents where fewer than two generic oracles are capable (pyx12, for instance, implements only 00401+ interchange versions) — and an under-covered fixture is never counted as agreement, and never as a win. Zero fixtures are in dispute.

When parsers do disagree, the disagreement goes to an adjudication queue and is resolved by journaled human ruling — the same governance as a pin change, with the ruling recorded beside the fixture. The queue currently holds exactly one ruling, and it is worth reading because it shows what adjudication is for: a synthetic 856 whose HL tree declares a shipment level under an order level parses cleanly in pyx12 and is refused by our grammar layer with HL_LEVEL_ORDER. The ruling: pyx12's generic reader checks HL parent-pointer validity but has no level-code order law, so this is an oracle-scope difference, not a parser disagreement — the fixture stays in the dirty corpus with its typed code pinned as expected. What never happens is the quiet alternative: our parser winning a tie because it is ours. The corpus and every verdict are pinned in the open repositories; the harness spec files themselves are digest-bound, so the verifier refuses a spec whose text no longer hashes to its pin — the swarm can hill-climb the implementation, and cannot move the goalposts.

What "all-green" then certifies — and deliberately cannot

Honest scope, stated once: differential testing certifies syntax. Agreement with independent parsers, byte-stable round-trips, and fuzz-stable envelopes tell you the wire format is handled correctly. They cannot tell you that mapping a despatch date to eventTime, or a party qualifier to owning_party, is right — semantic mapping choices are rulings, not test outcomes. That is why the document→event join is gated separately: fixture pairs (source document ⇄ expected events) adjudicated by human ruling, with a mechanical backstop the syntax side cannot offer — every generated event validates against the pinned official EPCIS 2.0.1 schema, and hashes stably under CBV §8.9. The EDIFACT track rides the same harness under the same pin governance — the licensing terrain that shapes all of this is its own post.

The pattern generalizes past EDI. Anywhere a standard ships without an official oracle, the honest sequence is the same: triangulate independent implementations, define the round-trip law with its exceptions recorded, fuzz the invariants, and journal every ruling a human makes. Ground truth you manufactured transparently beats ground truth you asserted.

Check the method against the artifacts: the corpus pins, the agreement report, the harness specs, and the adjudication record are committed beside the code — start from the conformance page and the business-transaction layer, and grade any vendor in this space, including us, against the same questions.