An ISO 8583 spec has two halves, and only one of them is code

An earlier post argued that architecture belongs in the repository rather than in a drawing. You can query a declarative model, diff it, review it and generate from it. A diagram starts to drift the moment someone exports it. The same argument applies to the semantics of a protocol, and in payments one protocol carries the highest stakes.

ISO 8583 dates from 1987 and still carries card authorizations worldwide. Every switch, every acquirer and every issuer host speaks a dialect of it. Almost everywhere, the description of a dialect covers the byte layout and nothing more.

What is already managed as code

The wire layout became a declarative artefact a long time ago. jPOS has described packagers in XML since the beginning. moov-io/iso8583 defines the same thing in Go, and its declarative vocabulary now imports from JSON or YAML. Field 2 is n..19 with an LL prefix in ASCII. Field 55 is a composite that carries BER-TLV. Teams write that part down, version it and share it.

What usually is not

Everything else about the field stays outside that document:

• whether an authorization request may carry it at all
• whether a response must echo it, or originates it
• which values are legal, and which of those change what other fields mean
• what condition makes a rule apply in the first place

As far as we have seen, that half rarely lives in a spec. It lives in a switch statement, in a validator someone wrote for one integration, in a wiki page that was accurate two years ago, or in the memory of the engineer who certified the last dialect. The rules are real, and the system enforces them. Nobody can diff them.

The cost appears in ordinary work. A certification finds that field 39 arrives where it should not, and no document settles the question. A second acquirer needs the same rules with three differences, so someone copies the validator. Someone asks which fields carry cardholder data, and the team answers by reading code.

This is not a criticism of the libraries. A packager describes structure, and it does that job well. The gap is that no standard layer sits above it.

The Swagger UI that ISO 8583 never had

The REST world settled this years ago. An OpenAPI document is more than machine input. Point Swagger UI or Redoc at it, and you have a reference a person can read. That reference cannot disagree with the description the code works from, because both read the same file. Hand-maintained API documentation stopped being normal around 2015.

ISO 8583 never had that, and the missing half is the reason. A packager holds little to render. Names, per-message rules, value domains and the conditions behind them are what make a document worth reading, and none of them existed as data. So the reference stayed a PDF, and the PDF stayed disconnected from the system that implements it.

fluxrig v0.10.0 puts both halves in one spec, and the same approach works. fluxrig generates the document below from the reference spec that ships with it. This is the public variant, which omits every field marked private. Nobody maintains any of it by hand. The generator inverts the field-keyed matrix on every render to build the per-message view, and that is the only way the two stay in agreement.

A generated document is worth something only if it describes the traffic, and you can check that rather than take it on trust. Every message the codec touches carries codec.spec_id, codec.spec_version and codec.spec_hash. The version in that page’s header is the same string. Compare them, and you know whether the reference and the wire agree. A document maintained by hand cannot offer that.

That page is a frozen copy. It lives in this site’s repository, so you see what the generator produced on the day of writing. It is also self-contained. It loads no scripts, stylesheets or fonts from anywhere, because people mail a protocol reference around and open it offline.

The rest of this post covers three things: why that half stayed out of reach, what a spec must carry before anyone can generate a document like that, and why the two halves must stay separable.

Why that half stayed out of reach

There is a good reason for it. The situation is not an oversight.

The authoritative description of a dialect does not come from a repository. It comes from the scheme. The scheme writes it for people to read, licenses it to the institutions that connect, and revises it at least twice a year. It is not machine-readable, it was never meant to be, and nobody may redistribute it.

So every institution that connects derives the same rules from the same document, by hand. None of them can share the result, even if they wanted to. The wire layer escaped that fate because the structure of the base standard is public enough for someone to write it down once and for everyone to reuse. moov’s own spec87ascii.yaml is exactly that, and the reference spec here starts from it.

That sets the boundary for anything published. The format is the shareable asset, not the content. A licensed dialect stays with whoever licensed it. A spec format earns its keep when it lets an institution keep its own rules private and still use the same tooling as everyone else. What ships in the open here uses publicly available material and nothing more.

Two layers, separate but cooperating

The spec format carries both halves in one document. The two layers stay strictly separate, and they cooperate over the same field paths. The semantic layer says nothing about bytes. It describes the fields the wire layer parses.

The wire layer restates nothing. It names a base from the upstream library, moov:spec87ascii for instance, and declares only the differences a dialect has. fluxrig consumes moov’s vocabulary as it is, because a project that copies an upstream format inherits the job of tracking it forever.

The semantic layer is the part that had nowhere to live. It carries names and aliases, so the pipeline addresses card.pan rather than field 2. It carries a data classification, so the logs mask a field that holds cardholder data even when nobody remembers a second flag. And it carries a per-message matrix, because a field does not mean the same thing in every message:

14:
  name: "Date, Expiration"
  messages:
    - mti: ["0100", "0200"]
      usage: mandatory
      when: "present(35) == false"
      note: "Expiration date required when Track 2 is absent."
  scope: public
  description: "Card expiration date, YYMM."
  format:
    kind: date
    layout: "YYMM"
  sensitivity: chd

The expiry is mandatory in an authorization or financial request, but only when field 35 is absent. Track 2 already carries the expiry. A switch that demands it a second time rejects traffic that was never wrong. Every implementation has that rule. Normally it is a branch in a validator, or a line in a certification checklist that someone reads once a year. Here it is a value the spec carries and a generated document can show.

The rest of the entry does work too. format makes comparisons on this field chronological rather than numeric, so an expiry rule means what it says. sensitivity is the data classification, and it masks the field in logs. scope is documentation visibility. That is a different question from data handling, so it stays a separate axis.

Those axes are worth keeping apart. usage answers whether the field is there at all. when answers when the entry applies. On a response, the same matrix carries response_value, which answers how a response’s value relates to the request’s. That is where echo becomes the sharpest conformance check available. A trace number that arrives changed means the responder is not answering the message it received.

A condition is a declarative expression rather than a callback, and fluxrig checks it when the spec loads rather than when a message arrives. A condition that does not parse, or that reads a field the spec never declared, fails at boot instead of in production.

The property that keeps the layers honest

The separation rests on one mechanical check. Take a spec, delete every semantic key, and the upstream library must load what remains on its own. A test does exactly that. The day a semantic key quietly becomes load-bearing for parsing, that test fails, and no reviewer has to catch it in a diff.

The guarantee is small on purpose, and it buys one thing: the wire half of a dialect keeps working without any of this. The same file, minus our additions, still feeds a moov-based program. The alternative is the ordinary way these things drift. A format grows one convenience. Something starts to depend on it. Eventually the only thing that can read the description of your own switch is the engine you happened to pick. You can revisit an engine. You cannot revisit a description of your traffic that only one engine understands.

Rules that reach traffic, carefully

A spec that only documents is still an improvement, but the point is to apply it. The codec has three settings:

• off, the default
• warn, which logs and counts every violation, and lets the message through
• enforce, which fails the message

The default is off on purpose. An upgrade must not reject today a message the system accepted yesterday. warn is how an operator learns whether the spec matches the traffic, and enforce is the decision that follows. Severity also lives on the individual rule, so a new check can reach a live fleet in warning mode while the fleet enforces the rest of the spec. The codec reports every violation, not only the first, because whoever reads a rejection wants to know what to fix.

Where this lands

The layering ships in fluxrig v0.10.0, with a full reference for the format, a generated protocol document produced from the spec itself, and the enforcement modes above.

None of this is a new observation. Anyone who spent years building switches on packagers met the same gap again and again, in every certification and every new acquirer. Separating the physical packager from a semantic validator was the intended shape of this project long before there was anything to show. What is new is that it runs. The rules reach traffic, and the generator produces the reference from the same file that describes them.

There is also an open thread at moov-io where the same shape appears from several directions at once. That is usually the sign that a gap is real rather than a preference.

If you maintain payment dialects and recognise the wiki page that was accurate two years ago, we would like to hear about it.