# Visa BASE II Clearing — Build Status & Gap Analysis **Source reviewed:** `docs/Clearing-System/Visa-related/base-ii-clearing-interchange-formats-tc-01-to-tc-49.pdf` ("BASE II Clearing Interchange Formats, TC 01 to TC 49," Visa Confidential, effective 18 October 2025, 933 pages, full text extracted directly — not summarized secondhand). Chapter 1 (file/record architecture) and the TC 33 chapter (pages 287–757, including the TC33.A Capture File sub-spec) read in full; the remaining chapters (TC 01–32, 35–49) read only far enough to build the real Transaction Code catalog (§4 below) — their per-field layouts are real spec pages this build hasn't needed yet, not guessed at. **Also checked, with a real negative result:** `docs/Clearing-System/Visa-related/base-2-clearing.pdf` — confirmed via `pdftotext`, `pdffonts`, and `pdfimages` (all return empty) to contain **zero extractable text, fonts, or images**: 101 `/FlateDecode` vector-drawing streams and nothing else. Its PDF metadata Title field is literally the big spec's filename, copied or shared from somewhere, even though the content is unrelated. Whatever an earlier pass transcribed from this file could not be re-verified against it — that transcription is exactly what turned out to be wrong (§2 below). Do not re-attempt extraction without visually rendering it first; text-layer tools cannot see what (if anything) it shows. **Compared against:** this umbrella as of `feature/visa-clearing`, principally `clearing_core` (`ClearingCore.Visa.*`), `settlement_core` (`SettlementCore.VisaOutboundPresentment`), and the `platform_web` Clearing screens — same real-spec-or-nothing standard as the Mastercard IPM and JCB Interchange builds. **Unlike Mastercard/JCB, there is no Way4 Visa interface manual in this repo** — no menu-tree document like `way4-mc-interface.txt`/`way4-jcb-interface.txt` exists for Visa. `PlatformWeb.SchemeLive.CoverageCatalog`'s `@visa_groups` is therefore built from the real BASE II **Transaction Code taxonomy** (`ClearingCore.Visa.TransactionCodeCatalog`) instead of a menu tree — a different, and necessarily less granular, construction method than Mastercard/JCB's own checklists. If a real Visa Way4 manual (or VisaNet's own operator documentation) is ever supplied, the coverage catalog should be rebuilt against it the way Mastercard/JCB's were, not left as taxonomy-derived by default. --- ## 1. Executive summary | Area | Status | |---|---| | TC33.A Capture File codec (HEDR/CP01/TRLR) | **Built** — real spec, real-sample-validated | | CP02–CP12 addenda (EMV, Lodging, Level II/III, etc.) | **Gap** — real, documented sub-schemas; captured as raw bytes, not field-decoded | | Transaction Code catalog (TC 00–92) | **Built** — full real taxonomy, deprecated codes flagged | | File Reader/Writer, envelope validation | **Built** — CTF/ASCII only | | ITF (170-byte, EBCDIC) variant | **Gap** — real, documented; no real ITF sample to validate against | | Outer CTF/ITF file envelope (TC90/91/92) | **Blocked** — layout lives in a missing companion manual | | Ingestor + reconciliation | **Built** — real ARN-vs-RRN correction landed here (§2) | | Outbound presentment (`VisaOutboundPresentment`) | **Partial** — real module, masked PAN, no UI trigger wired | | Coverage catalog | **Built**, taxonomy-derived (see above), not menu-derived | | Reports | **Partial** — 2 real (Capture Volume, Action Code breakdown); Fee/Chargeback reports blocked on TC10/TC15-17 | | BIN/IIN-equivalent (ARDEF) load | **Blocked** — glossary reference only, no field layout in this doc | | Merchant/Stop/Negative-file equivalents | **Blocked** — no real layout available (see §5) | --- ## 2. A real bug this build fixed, not introduced The `ClearingCore.Visa.*` namespace pre-existed this effort (a partial TC33 codec, already wired into `ClearingBatch.sources()`/`ClearingCore.Context`/`ClearingLive.Files`). It targeted a generic TC33 "Clearing and Settlement Advice" layout (Advice Record ID `"CAS"`) transcribed from `base-2-clearing.pdf` — the file just confirmed above to have no extractable content. Its own moduledoc already flagged the resulting mismatch: the real sample file (`454183.76.20241022.xml_tc33.txt.ascii`) carries `CP*`/`CP1`-style Application Codes, not `"CAS"`. Byte-for-byte verification against the real 933-page spec this pass confirmed the sample is a genuine **TC33.A Capture File** (`HEDR`/`CP01`–`CP12`/`TRLR`) — an entirely different, real, fully-documented record family. The codec was rewritten against CP01's real "Transaction Data"/"Additional Data" field tables. A second, related correction: the old schema's match key, `acquirer_reference_number`, assumed a 23-digit Acquirer Reference Number. That field is real — found in the spec near "Merchant Category Code"/"CRB Region Code" — but belongs to a *different* TC33 sub-message (an Exception File/CRB update record), not to CP01 Capture. CP01's real match-key field is a 12-byte **Retrieval Reference Number** (positions 51–62), an ordinary RRN like every other scheme uses. `SettlementCore.ClearingBridge` now matches Visa on `core_transactions.rrn`, the same column Mastercard's DE 37 fallback already reads, instead of the 23-digit `acquirer_reference_number` column (which remains valid infrastructure for Mastercard DE 31/JCB ARD — just not applicable to Visa CP01). --- ## 3. What's real and validated `ClearingCore.Visa.*`: `Tc33Record`/`Tc33Parser`/`Tc33Builder` (CP01 only — see §4), `TransactionCodeCatalog` (TC 00–92, deprecated codes flagged), `FileReader`/`FileWriter` (CTF/ASCII HEDR…TRLR envelope), `Ingestor`, `OutboundSubmitter`, `FileSequence` (4-digit Capture File Number ledger). `SettlementCore.VisaOutboundPresentment` (eligible-transaction selection + build + submit, masked PAN). Two real reports (`ClearingLive.VisaReports`): Capture Volume Summary (inbound/outbound) and Capture by Action Code. All validated against the real sample file (`docs/Clearing-System/Visa-related/454183.76.20241022.xml_tc33.txt.ascii`, also at `docs/settlement_recon/TC33file/`) — 1 HEDR + 159 real CP01 transactions + 1 TRLR. That file masks every literal digit `0` to `*` (a data-scrubbing artifact), which defeats the parser's exact-literal TCR-sequence-number/Application-Code matching — a spec-correct parser legitimately decodes zero transactions from the file *as shipped*. `apps/da_product_app/priv/repo/seeds/visa_demo_data_seed.exs` reverses that one substitution before seeding (confirmed safe for this file — no legitimate `*` usage found anywhere in it) and recovers all 159 real transactions; 116 store, 43 genuinely carry no RRN in the real data and surface as `clearing_exceptions` instead. --- ## 4. What's catalogued but not decoded: CP02–CP12, and TC01/02/03/10/15-17/25-27/35-37/38-49 `Visa.TransactionCodeCatalog` names every TC 00–92 code and its category, but only **TC 33's CP01** sub-message has a field-level parser. Real, documented, and genuinely present in the sample file: CP02 (EMV), CP03 (Lodging), CP04 (Level II/III), CP05 (Passenger Itinerary), CP06 (Purchasing Line Item), CP07 (Country Data), CP08 (Discretionary), CP09 (Push Payment), CP10 (Car Rental), CP12 — all captured verbatim in `Tc33Record.addenda_raw`, keyed under the transaction they belong to, but not field-decoded. Building field-level parsers for these is real, scoped, incremental work (each CP group has its own real table in the spec) — deferred, not abandoned. Beyond TC33 entirely: TC05/06/07 (original financial), TC10 (Fee Collection), TC15-17 (Dispute Financial / chargeback), TC20 (Funds Disbursement), TC25-27 (Reversal), TC35-37 (Dispute Financial Reversal), and the TC38/39/40/44-49 administrative/reporting codes all have real per-field layouts in the spec (pages not yet read for this build) but no parser here. A Fee Collection or Chargeback/Dispute Visa report needs TC10 / TC15-17 records decoded first — flagged directly in `ClearingLive.VisaReports`'s own moduledoc rather than faked with TC33 data. --- ## 5. Blocked pending material not in this repo The full BASE II Clearing documentation set is **four manuals**; this repo has only the first: 1. *BASE II Clearing Interchange Formats, TC 01 to TC 49* — in this repo, the primary source above. 2. *BASE II Clearing Interchange Formats, TC 50 to TC 92* — **missing.** Blocks: the outer CTF/ITF physical-file envelope's real TC90 (File Header)/TC91 (Batch Trailer)/TC92 (File Trailer) field layouts (only sketched conceptually in Chapter 1 of the doc we have — `Visa.FileReader`/`FileWriter` validate TC33.A's own `HEDR`/`TRLR` instead, a real but different, smaller envelope), and TC46 (Member Settlement Data, explicitly deferred by the spec itself to *"VSS User's Guide, Volume 2 Reports"*). 3. *BASE II Clearing Data Codes* — **missing.** Blocks: every enumerated code table referenced throughout (AVS response codes, POS entry modes, currency codes, MCCs, action-code detail beyond the 8 values already in `Tc33Parser`'s field docs, etc.) — decoded fields are stored as raw values today; human-readable code lookups would need this manual. 4. *BASE II Clearing Edit Package Messages Reference Guide, Release 4* — **missing.** Blocks: any real file-rejection/edit-error handling equivalent to Mastercard's own edit-error catalog. Also blocked, real concepts named only in the glossary of the one manual this repo has, with no field layout: - **ARDEF (Account Range Definition Table)** — Visa's BIN/IIN-equivalent. No loader exists; no `mix load_visa_bin_ranges` equivalent to JCB's IIN File import. - **Account Screen Authorization/Clearing File (ASAF/ASCF), Card Recovery Bulletin (CRB)** — the Stop/Negative-Data-File equivalent. Glossary/field-reference only. - **ITF (170-byte, EBCDIC)** — a real, documented CTF variant (2-byte hash-total field shifts every position by 2) with no real ITF sample in this repo to validate against; CTF/ASCII only is built. None of these are guessed at anywhere in this codebase. If any of this material becomes available, this document is the starting scope for the next phase, not a rediscovery exercise.