An ISO 8583 spec has two halves, and only one of them is code

An earlier post argued that architecture belongs in the repository rather than in a drawing: a model written declaratively can be queried, diffed, reviewed and generated from, while a diagram starts drifting the moment it is exported. The same argument applies to the semantics of a protocol, and in payments there is one protocol where the stakes are highest.

ISO 8583 dates from 1987 and still carries card authorizations worldwide. Every switch, every acquirer, every issuer host speaks a dialect of it. And a dialect is described, almost everywhere, as a byte layout and nothing more.

What is already managed as code

The wire layout has been a declarative artefact for a long time. jPOS has described packagers in XML since the beginning, and moov-io/iso8583 defines the same thing in Go, with a declarative vocabulary that can now be imported from JSON or YAML. Field 2 is n..19 with an LL prefix in ASCII. Field 55 is a composite carrying BER-TLV. That part is written down, versioned and shared.

What usually is not

Everything else about the field. Whether an authorization request may carry it at all. Whether a response must echo it or originates it. Which values are legal, and which of those values change what other fields mean. What condition makes a rule apply in the first place.

As far as we have seen, that half rarely lives in a spec. It lives in a switch statement, in a validator someone wrote for one integration, in a wiki page that was accurate two years ago, or in the memory of the engineer who certified the last dialect. It is real, it is enforced, and it is not an artefact anyone can diff.

The cost shows up in the ordinary work. A certification finds that field 39 arrives where it should not, and there is no document to point at. A second acquirer needs the same rules with three differences, so the validator is copied. Someone asks which fields carry cardholder data, and the answer is assembled by reading code.

This is not a criticism of the libraries. A packager describes structure, and it does that job well. The gap is that nothing standard sits above it.

The Swagger UI that ISO 8583 never had

The REST world settled this years ago. An OpenAPI document is not only machine input: point Swagger UI or Redoc at it and you get a reference a person can browse, and it cannot disagree with the description the code works from because it is the same file. Hand-maintained API documentation stopped being normal somewhere around 2015.

ISO 8583 never got that, and the missing half is the reason. A packager has nothing much to render: names, per-message rules, value domains and the conditions behind them are what make a document worth reading, and none of them existed as data. So the reference stayed a PDF, and the PDF stayed disconnected from the system that implements it.

fluxrig v0.10.0 puts both halves in one spec, and the same move works. The document below is generated from the reference spec that ships with it, in its public variant, which omits any field marked private. Nothing in it is maintained by hand: the per-message view is the field-keyed matrix inverted on every render, which is the only way the two stay in agreement.

A generated document is only worth something if it describes the traffic, and that is checkable rather than asserted: every message the codec touches carries codec.spec_id, codec.spec_version and codec.spec_hash, and the version printed in that page’s header is the same string. Comparing them is how you know the reference and the wire agree, which is not something a document maintained by hand can offer.

That page is a frozen copy, kept in this site’s repository rather than fetched, so what you see here is what the generator produced on the day this was written. It is also self-contained: no scripts, stylesheets or fonts are loaded from anywhere, because a protocol reference gets mailed around and opened offline.

Why that half stayed out of reach, what a spec has to carry for a document like that to be generated at all, and why the two halves have to stay separable, is the rest of this post.

Why that half stayed out of reach

There is a good reason, and it is worth stating plainly rather than treating the situation as an oversight.

The authoritative description of a dialect does not come from a repository. It comes from the scheme, as a document written to be read by people, licensed to the institutions connected to it, and revised at least twice a year. It is not machine-readable, it was never meant to be, and it is not anyone’s to redistribute.

So the same rules get re-derived from the same document, by hand, inside every institution that connects, and none of them can share the result even if they wanted to. The wire layer escaped that fate because the structure of the base standard is public enough to be written down once and reused: moov’s own spec87ascii.yaml is exactly that, and it is where the reference spec here starts from.

Which sets the boundary for anything published. The shareable asset is the format, not the content. A licensed dialect stays with whoever licensed it, and a spec format earns its keep by letting an institution hold its own rules privately while using the same tooling as everyone else. What ships in the open here is built on publicly available material and nothing more.

Two layers, kept apart on purpose

The spec format carries both halves in one document and keeps them strictly separated.

The wire layer is not restated. It names a base from the upstream library, moov:spec87ascii for instance, and declares only the differences a dialect actually has. moov’s vocabulary is consumed as it is, because a project that copies an upstream format inherits the job of tracking it forever.

The semantic layer is the part that had nowhere to live. Names and aliases, so the pipeline addresses card.pan rather than field 2. A data classification, so a field carrying cardholder data is masked in logs whether or not anyone remembered a second flag. And a per-message matrix, because a field does not mean the same thing in every message:

14:
  name: "Date, Expiration"
  messages:
    - mti: ["0100", "0200"]
      usage: mandatory
      when: "present(35) == false"
      note: "Expiration date required when Track 2 is absent."
  scope: public
  description: "Card expiration date, YYMM."
  format:
    kind: date
    layout: "YYMM"
  sensitivity: chd

The expiry is mandatory in an authorization or financial request, but only when field 35 is absent. Track 2 already carries the expiry, so a switch that demands it a second time rejects traffic that was never wrong. Every implementation has that rule. It is normally a branch in a validator, or a line in a certification checklist that someone reads once a year, and here it is a value the spec carries and a generated document can show.

The rest of the entry is doing work too. format makes comparisons on this field chronological rather than numeric, so an expiry rule means what it says. sensitivity is the data classification that masks it in logs. scope is documentation visibility, which is a different question from data handling and stays a separate axis for that reason.

Those axes are worth keeping apart. usage answers whether the field is there at all. when answers when the entry applies. On a response the same matrix carries response_value, which answers how a response’s value relates to the request’s, and that is where echo becomes the sharpest conformance check available: a trace number that arrives changed means the responder is not answering the message it was sent.

A condition is a declarative expression rather than a callback, and it is checked when the spec loads rather than when a message arrives. One that does not parse, or that reads a field the spec never declared, fails at boot instead of in production.

The property that keeps the layers honest

The separation rests on one mechanical check. Take a spec, delete every semantic key, and what remains has to be a document the upstream library loads on its own. A test does exactly that, so the day a semantic key quietly becomes load-bearing for parsing, that test fails instead of a reviewer having to catch it in a diff.

It is a small guarantee, deliberately, and what it buys is this: the wire half of a dialect keeps working without any of this. The same file, minus our additions, still feeds a moov-based program. The alternative is the ordinary way these things drift. A format grows one convenience, something starts depending on it, and eventually the only thing that can read the description of your own switch is the engine you happened to pick. An engine is a decision you can revisit. A description of your traffic that only one engine understands is not.

Rules that reach traffic, carefully

A spec that only documents is still an improvement, but the point is to apply it. The codec has three settings: off, which is the default, warn, which logs and counts every violation and lets the message through, and enforce, which fails it.

The default is off deliberately. A message accepted yesterday must not be rejected today because the code was upgraded. warn is how an operator finds out whether the spec matches the traffic, and enforce is the decision that follows. Severity also lives on the individual rule, so a new check can reach a live fleet in warning mode while the rest of the spec is enforced. Every violation is reported, not only the first, because whoever reads a rejection wants to know what to fix.

Where this lands

The layering ships in fluxrig v0.10.0, with a full reference for the format, a generated protocol document produced from the spec itself, and the enforcement modes above.

None of which is a new observation. Anyone who has spent years building switches on packagers has met the same gap over and over, in every certification and every new acquirer, and separating the physical packager from a semantic validator was the intended shape of this project long before there was anything to show. What is new is that it runs: the rules reach traffic, and the reference is generated from the same file that describes them.

There is also an open thread at moov-io where the same shape keeps surfacing from several directions at once, which is usually the sign that a gap is real rather than a preference.

If you maintain payment dialects and recognise the wiki page that was accurate two years ago, we would like to hear about it.