# Direct-REST Connectivity — Backoffice Configuration Plan

**Part of:** Acquiring Platform Transformation — spans Workstream A (`scheme_core`) and
Workstream D (Switch Direct Connectivity).
**Status:** Proposed — not started. For review before implementation.
**Companion to:** [`direct-connect-worklist.md`](direct-connect-worklist.md) (which this
partly supersedes — see §1.2), and the switch-side ADRs
`muNSwitch/../docs/adr/ADR-001` / `ADR-002`.

---

## 1. Context

### 1.1 What was built on the switch side

The switch (`muNSwitch`, a separate repository) now has a working **direct-to-network REST**
acquiring path for both Visa and Mastercard, covering Card Present and Card Not Present:

| Concern | Visa | Mastercard |
|---|---|---|
| API | VisaNet Connect Acceptance | Transaction API for Acquirers (ISO 20022 CAIN) |
| Auth | mTLS client certificate | mTLS client certificate (KMP-issued, **not yet provisioned**) |
| Idempotency key | `msgIdentfctn.correlatnId` (body) | `Customer-Context-Key` (HTTP header) |
| Network txn id | `msgIdentfctn.id` (body) | `Correlation-Id` (HTTP **response header**) |
| Lifecycle | always synchronous | 200 sync / 202 async-poll / 409 duplicate |
| Assigned identity | `msgIdentfctn.clientId` — **not yet issued** | — |

Plus a pluggable EMV 3DS layer (`DaSwitchCore.ThreeDs.Provider`) whose production vendor is
**not yet selected**, currently backed by a simulator.

Today all of this is configured in **static Elixir config files** on the switch
(`config/upstream_networks.exs`, `config/dev.exs`, `config/test.exs`) under a
`:direct_rest_networks` key. That is correct for a sandbox with one merchant and no
operators — it does not survive contact with production, where connectivity endpoints,
certificates, and 3DS registrations need to be visible, auditable, and changeable without a
deploy.

### 1.2 What this supersedes in the existing worklist

`direct-connect-worklist.md` (written when this workstream was deferred) assumes direct
connectivity means **Mastercard MIP and Visa VAP over TCP ISO 8583**, and its item 1 asks
whether the existing `NetworkConnector`/`UpstreamRouter` TCP architecture "maps directly, or
needs protocol-level adjustments for MIP/VAP".

**That question has been answered differently than the worklist anticipated.** The switch went
REST, not MIP/VAP — a separate transport alongside the existing TCP path, not an adaptation of
it. The worklist's items 1 and 2 (MIP/VAP credentials, `acquirer/mastercard_mip/` and
`acquirer/visa_vap/` adapter modules) describe a path that was **not taken**. Items 3 (live BIN
lookups) and 4 (settlement feed) are unaffected and still stand.

That worklist's core principle carries over unchanged and governs this plan too: **the existing
YSP-routed path is never modified.** Everything here is additive.

### 1.3 What exists in the backoffice today

`scheme_core` Phases 1–10 are complete — 21 master-data screens under Scheme Management,
covering schemes, countries, currencies, BIN ranges, EMV/CAPK, routing, network parameters,
interchange/fees, reason codes, calendars, certifications, bulletins, MCC rules. Every master
table follows one convention set in Phase 1:

`status` (`draft/active/suspended/inactive`) · `effective_from` · `effective_to` · `version` ·
`created_by` · `updated_by`, with every mutation writing a `SchemeCore.AuditLog` row in the
same transaction, and a NULL-safe `<=>` duplicate-active-row guard on the natural key.

**Two existing tables are adjacent to this work and neither fits it:**

- **`SchemeCore.NetworkParameter`** (`scheme_network_parameters`) is ISO 8583-shaped —
  `iso_version`, `message_spec`, `processing_codes` (DE 3), `function_codes` (DE 24),
  `network_windows`. Its Phase 4 seed values were explicitly sourced from the switch's
  *sandboxed, not-yet-live* direct-connect config. There is no field here for a base URL, a
  TLS credential, or an assigned client id, and adding them would put two unrelated
  transports in one table.
- **`SchemeCore.RoutingConfig`** (`scheme_routing_configs`) records *intent* — which network a
  scheme routes to per country and domestic/international scope — with `primary_network` /
  `secondary_network` as free text. Its own moduledoc is explicit that it does not drive the
  switch's live routing. It has no transport dimension, so it cannot currently express
  "this scheme's CNP traffic goes direct-REST while its CP traffic stays on YSP".

**Confirmed absent from the backoffice** (verified by search, matching your statement):

- No MPGS gateway configuration of any kind. The only `mpgs` reference is
  `ClearingCore.ClearingBatch`'s `"mpgs_dcf"` source enum — a *clearing file* format, marked
  "deferred, not yet built", unrelated to gateway connectivity.
- No mTLS / client-certificate configuration concept. The only certificate handling anywhere
  is `SettlementCore.Ysp.SftpClient` and `TmsAcquirerCore.KeyManagement.Hsm.Verisec`.
- No 3DS / CAVV / ECI configuration. Only an illustrative dispute-parameter seed mentions 3DS.

---

## 2. The gap, stated precisely

Five distinct things need a home, and none has one today:

1. **Where to reach each network** — base URL per scheme *per environment* (sandbox / MTF /
   production), timeouts, which transport.
2. **Which credential authenticates us** — mTLS client certificate identity and, critically,
   **its expiry**. An expired client cert fails every transaction for that network with a TLS
   handshake error that looks exactly like a network outage.
3. **Network-assigned identifiers** — Visa's `clientId`, Mastercard's acquirer/ICA
   identifiers, 3DS Requestor IDs. Currently `nil` placeholders in switch config with a
   comment saying "not yet issued".
4. **3DS configuration** — which provider, which requestor registration per network, protocol
   version, and the degrade policy when 3DS is unavailable at authorization time.
5. **Which traffic uses this path** — the routing decision between YSP, MPGS, and direct-REST,
   per scheme and per presence (CP vs CNP).

---

## 3. Design decision: where this config lives

### Option A — extend `NetworkParameter` with REST fields

**Rejected.** It is ISO 8583-shaped by design and already carries live seeded data. Adding
`base_url`/`mtls_*`/`client_id` produces a table where roughly half the columns are always NULL
depending on an implicit transport, with no validation able to tell which half. This is the same
mistake the switch side already made once and corrected: an early attempt to nest REST entries
inside the switch's `:upstream_networks` config broke `UpstreamRouter`'s startup validation,
because that validator requires TCP fields on every entry. The fix there was a separate config
key. The same reasoning applies here.

### Option B — reuse `SchemeServiceConfig`'s free-form `parameters` map

**Rejected for connectivity, recommended for service enablement.** `SchemeServiceConfig` is the
existing generic extension point (`service_key` + `parameters` map), and `ServiceCatalog`
*already declares* the relevant service keys: `authorization`, `identity_check` (Mastercard's
3DS product name), `visa_secure` (Visa's). Zero migration, existing UI.

But the UI edits `parameters` as **untyped key/value string pairs**
(`products_services.ex:693`, `zip_params/2`). For connectivity that means:

- a typo (`base_url` vs `baseUrl`) silently produces a broken config with no validation error;
- certificate expiry cannot be tracked, because there is no typed date column to run a worker
  against — and expiry monitoring is the single highest-value thing this plan delivers;
- the switch will eventually consume this over the Phase 13 lookup API, where an untyped map
  gives the consumer no contract.

### Option C — new dedicated schemas ← **recommended**

Follows the precedent of every phase in this app: each phase added its own typed, validated,
versioned, audited tables rather than overloading earlier ones. Typed columns give real
validation, a real expiry worker, and a real API contract.

**Recommendation: C for connectivity and credentials, B for 3DS service enablement flags** —
detailed in §4.

---

## 3.5 The connection model — two orthogonal axes, currently conflated

*(Added 2026-08-14 after review. This revises §4.1 below and is the most important section
of this document.)*

The acquiring side has accumulated four upstream paths. They are usually described as a flat
list, which is why they feel confusing. They are actually a **2×2**, and every quadrant is
occupied:

| | **ISO 8583 / TCP** | **REST / JSON** |
|---|---|---|
| **Aggregated** *(one connection, many schemes)* | **YSP** — external acquirer/processor, accepts any BIN | **MPGS** — Mastercard's gateway, routes internally to Visa/MC/Amex |
| **Direct** *(one connection per scheme)* | **MIP / VAP** — scaffolded, never went live | **Visa Acceptance + Mastercard CAIN** — the new work |

**Axis 1 — connection mode (aggregated vs direct)** is the one that actually changes the
routing logic. In aggregated mode the thing you connect to is *not* the scheme: you pick a
gateway, and the gateway picks the scheme. In direct mode the connection *is* the scheme.

**Axis 2 — transport** changes the adapter implementation, not the routing shape.

These are independent. MPGS is aggregated-and-REST; YSP is aggregated-and-TCP. Conflating
"REST" with "direct" is easy and wrong — MPGS disproves it.

### What the switch has today

The **existing** config already encodes almost exactly this, via `network_type` in
`config/upstream_networks.exs`:

| `network_type` | Entries | Quadrant |
|---|---|---|
| `:acquirer` | `ysp_ssl`, `ysp_plain` | aggregated + TCP |
| `:gateway` | `mastercard_mpgs_sandbox`, `mastercard_mpgs_production` | aggregated + REST |
| `:network` | `visa`, `mastercard`, `amex`, `discover` | direct + TCP |
| `:processor` / `:simulator` / `:legacy` | test + fallback entries | — |

So the taxonomy is real, it predates this work, and `UpstreamRouter` already filters and
prioritises on it.

### Where the new work does not fit — the honest gap

**The new direct-REST path sits outside that taxonomy entirely.** `:direct_rest_networks`
carries `transport: :rest`, `base_url`, `timeout`, `mtls` — and **no `network_type`**. That
was a deliberate call at the time (nesting it inside `:upstream_networks` broke
`UpstreamRouter`'s startup validation, which demands TCP fields on every entry), and it was
the right call *for shipping the sandbox*. But the consequence is that the fourth quadrant is
invisible to the classification that governs the other three.

**The deeper issue is in the new code, not the config.** `Upstream.Adapter.network/0` returns
`:visa | :mastercard`, and `Cnp.CheckoutFlow.adapter_for/1` maps scheme → adapter one-to-one:

```elixir
defp adapter_for(:visa), do: DaAcquirer.Upstream.Visa.AcceptanceAdapter
```

That is **only correct in direct mode.** It structurally cannot express "route this Visa
transaction through MPGS" or "through YSP", because it assumes scheme *is* connection. The
new code models one quadrant and hardcodes the assumption that it is the only one.

**Verdict on the alignment question: partially aligned.** The pre-existing switch architecture
anticipated this model correctly. The new REST work does not yet participate in it. This is
fixable and worth fixing now, before backoffice config is built on top of the wrong shape —
which is exactly what would have happened if §4.1 below had been implemented as originally
written.

### What this changes

The routing key is not `scheme → adapter`. But it is also **not one key** — it differs by
connection mode, and an earlier draft of this section got that wrong by putting `merchant` in
a single universal key.

**Merchant is an aggregated-mode dimension only.** It exists because a merchant is onboarded
*with* an acquirer or gateway — it has a merchant ID with YSP, a different one with MPGS,
different contracts and pricing. For a **direct** scheme connection there is exactly one Visa
connection and one Mastercard connection for the whole platform; the merchant appears *in* the
message (`Envt.Accptr` / `envt.accptr`) but plays no part in **choosing** the connection.

Resolution is therefore two-step:

```
Step 1 (merchant-level):  merchant → path = aggregated | direct
Step 2a (aggregated):     (merchant, presence) → acquirer/gateway profile
Step 2b (direct):         (scheme  , presence) → scheme profile        ← merchant NOT used
```

**Presence is the dimension that is genuinely new in both branches.** CNP needs 3DS
pass-through (CAVV/ECI); whether YSP supports that is unconfirmed. A realistic near-term state
is one merchant doing CP via YSP and CNP via direct-REST at the same time — which the current
`adapter_for/1` cannot represent at all.

---

## 4. Proposed schemas

All follow the established conventions: `@primary_key {:id, :id, autogenerate: true}`,
`@timestamps_opts [type: :naive_datetime]`, the six cross-cutting workflow columns, audit log
on every mutation via `SchemeCore.Context`, NULL-safe `<=>` duplicate-active guard, migrations
under `apps/da_product_app/priv/repo/migrations/` as `DaProductApp.Repo.Migrations.*`, and
**explicit short `name:` on every compound index** (per bug #9 in the progress log — MySQL's
64-char identifier limit already broke one migration in this app).

### 4.1 `SchemeCore.ConnectivityProfile` — `scheme_connectivity_profiles`

**Revised per §3.5.** One row = one upstream connection, in one environment — covering **all
four quadrants**, not just direct-REST. This is the change that matters: if the backoffice only
configures the new path, three of four quadrants stay in static switch config forever and the
backoffice is not actually the system of record for connectivity.

| Column | Type | Notes |
|---|---|---|
| `profile_name` | string | e.g. `"visa_acceptance_sandbox"`, `"ysp_ssl"`, `"mastercard_mpgs_production"` — **matches the switch's existing config key verbatim**, so migration is a transcription and the two sides stay traceable by eye |
| `connection_mode` | string | **`aggregated` \| `direct`** — axis 1. The column that makes the other three quadrants representable |
| `transport` | string | `rest` \| `iso8583_tcp` — axis 2 |
| `network_type` | string | `acquirer` \| `gateway` \| `network` \| `processor` \| `simulator` \| `legacy` — **mirrors the switch's existing `network_type` verbatim** rather than inventing a parallel vocabulary |
| `scheme_id` | FK, **nullable** | Set for `direct`. **NULL for `aggregated`** — YSP and MPGS are not "a scheme's connection", and forcing a scheme FK on them is precisely the modelling error §3.5 describes |
| `environment` | string | `sandbox` \| `mtf` \| `production` |
| `failover_policy` | string | `none` \| `ordered` \| `round_robin` — how to use the endpoints in §4.1a |
| `api_product` | string | `visa_acceptance` \| `mastercard_cain` \| `mpgs` \| `ysp` — which message spec applies |

> **Endpoints deliberately do not live on this row** — see §4.1a. An earlier draft put a single
> `base_url` / `host` / `port` here, which could not represent a provider that supplies a
> primary *and* a fallback endpoint.
| `assigned_client_id` | string | Visa `msgIdentfctn.clientId`; nullable — genuinely not yet issued |
| `acquirer_id` | string | Mastercard acquirer/ICA; nullable |
| `timeout_ms` | integer | default 30000 |
| `supports_cp` / `supports_cnp` | boolean | Mastercard direct CNP is currently **false** (field dictionary unconfirmed, ADR-002). YSP CNP support is **unknown** — seed NULL, not a guess |
| `supported_schemes` | map/array | **Aggregated profiles only** — which schemes this one connection fronts. Meaningless for `direct` (that's `scheme_id`) |
| `capabilities` | map | genuinely open-ended flags (capture/refund support, which the supplied Visa spec doesn't cover) |

Duplicate-active guard on `(profile_name, environment)` — not on `scheme_id`, which is NULL for
half the rows and would make the guard useless there.

**Validation the typed columns now make possible:** `direct` requires `scheme_id` and forbids
`supported_schemes`; `aggregated` requires `supported_schemes` and forbids `scheme_id`;
`transport: rest` requires `base_url`; `transport: iso8583_tcp` requires `host`+`port`. None of
that is expressible in a free-form parameters map, which is the concrete argument for Option C
over Option B in §3.

**Deliberately excluded: any credential material.** Only a reference, via §4.2.

### 4.1a `SchemeCore.ConnectionEndpoint` — `scheme_connection_endpoints`

*(Added 2026-08-14. This gap was found in review, not in design — see the note at the end of
this section.)*

**There are three distinct concepts here and an earlier draft of this plan modelled only two.**

| Concept | Scope | Example | Failure response | Status |
|---|---|---|---|---|
| **Connection pool** | N sockets, **one** endpoint | 10 concurrent connections to YSP | open another socket | ✓ existing `connection_pool_size` |
| **Endpoint failover** | **one** provider, N endpoints | YSP primary → YSP fallback | reconnect elsewhere, **same credentials, same message** | ✗ **was missing** |
| **Network failover** | **N** providers | Visa direct → YSP | different credentials, **different message format entirely** | ✓ `RoutingConfig.secondary_network` |

The middle row is what YSP actually supplies: **one connection, plus a fallback** — not a pool
of load-balanced peers, and not a different provider.

**Why conflating the middle and bottom rows is dangerous.** Failing over to a provider's own
fallback endpoint is cheap and should be automatic: same mTLS credential, same message spec, no
re-mapping, no commercial consequence. Failing over to a *different provider* re-maps the entire
message, may use a different merchant ID, and has contractual and pricing implications. If the
config can't tell them apart, you either treat a routine DR reconnect as a scary cross-provider
event, or — worse — treat a cross-provider switch as a casual reconnect.

| Column | Type | Notes |
|---|---|---|
| `connectivity_profile_id` | FK | required |
| `role` | string | `primary` \| `fallback` \| `dr` |
| `priority` | integer | order for `failover_policy: ordered` |
| `base_url` | string | REST only |
| `host` / `port` | string / integer | TCP only |
| `status` + the standard five | | per-endpoint lifecycle — lets you mark a DR endpoint `inactive` for maintenance **without touching the profile or its credentials** |

Duplicate-active guard: **one active `primary` per profile**. Fallbacks may be several.

**Credentials stay attached to the profile, not the endpoint** (§4.2) — primary and DR of the
same provider normally present the same client certificate. If a provider ever issues
per-endpoint credentials, that becomes a nullable `connection_endpoint_id` on
`NetworkCredential`, not a redesign.

**This applies to all four quadrants, not just YSP.** Visa and Mastercard publish regional REST
endpoints; MPGS has its own. The same primary/fallback shape covers them, which is the argument
for putting it on the generic profile rather than special-casing acquirers.

> **Provenance note, kept deliberately:** this was missed in the original design and caught by
> the user in review. The switch's own `:routing` config carries a comment reading *"Load
> balancing for multiple hosts per network"* — while no network entry has any field capable of
> holding multiple hosts. The intent pre-existed on both sides; the schema never did. Worth
> recording, because it is the kind of gap that a config model can hide indefinitely until the
> primary endpoint actually goes down.

### 4.2 `SchemeCore.NetworkCredential` — `scheme_network_credentials`

mTLS credential **metadata only — never key material.** This follows the precedent already set
in this codebase by `TmsAcquirerCore`'s `KeyRotationSchedule` / `KeyCeremonyLog`, whose own
description is "store lifecycle metadata only, never key material".

| Column | Type | Notes |
|---|---|---|
| `connectivity_profile_id` | FK | required |
| `credential_type` | string | `mtls_client_cert` \| `mtls_private_key` \| `ca_bundle` |
| `storage_ref` | string | Vault path / secrets-mount reference — **not** a filesystem path, and **not** the secret |
| `subject_dn`, `issuer_dn`, `serial`, `sha256_fingerprint` | string | identity for audit; populated from the cert, not typed by hand |
| `not_before`, `not_after` | date | **the point of this table** |
| `renewal_status` | string | `current` \| `expiring_soon` \| `expired` \| `renewal_in_progress` |
| `key_identifier` | string | e.g. Visa's KID (`dcba8043-…`) |

**Worker: `SchemeCore.Workers.CredentialExpiryWorker`** — a near-copy of the existing
`CertificationExpiryWorker` (daily cron, `Date.compare/2` — *not* `<`/`<=`, which compares
`Date` structs field-by-field and was a real bug caught in Phase 8), recomputing
`current`/`expiring_soon`(30d)/`expired` and never touching operator-set
`renewal_in_progress`. Fires `TmsCore.AlertsCore` notifications on transition, exactly as
Phase 9 wired certification expiry.

This is the backoffice counterpart to the switch's own `Upstream.CertMonitor`. Both are
worth having: the switch's checks the file it will actually present in a TLS handshake; this
one gives operators visibility and an alert *before* anyone is paged.

> **Real data point:** the Verisec HSM integration in `tms_acquirer_core` was blocked by an
> expired client certificate (expired 2026-01-09) — discovered only by a live handshake
> failure during testing. That is precisely the failure this table and worker exist to
> prevent.

### 4.3 `SchemeCore.ThreeDsConfig` — `scheme_three_ds_configs`

| Column | Type | Notes |
|---|---|---|
| `scheme_id` | FK | required |
| `provider` | string | `simulator` \| vendor name — pending selection per ADR-002 |
| `requestor_id` | string | Mastercard 3DS Requestor ID / Visa equivalent; **separate registration per network** |
| `protocol_version` | string | e.g. `2.2.0` |
| `challenge_policy` | string | `no_preference` \| `no_challenge` \| `challenge_requested` \| `challenge_mandated` |
| `unavailable_policy` | string | `decline` \| `proceed_unauthenticated` — the degrade decision ADR-002 flagged as needing an explicit answer |
| `attempt_eci_accepted` | boolean | whether attempted-authentication (Visa 06 / MC 01) is accepted for authorization |

**ECI values themselves are deliberately not configurable.** They are network-defined constants
(Visa 05/06/07, Mastercard 02/01/00) already encoded in the switch's `DaSwitchCore.ThreeDs.Eci`,
where being network-scoped-by-construction is what prevents the two scales from ever being
conflated. Making them editable data would reintroduce exactly the class of error that module
was written to make impossible.

**Service enablement** (is Identity Check / Visa Secure switched on for this scheme at all)
stays in `SchemeServiceConfig` under the existing `identity_check` / `visa_secure` service
keys — that is what those catalog entries are for, and no migration is needed.

### 4.4 Small extension to `Certification` (existing table)

`certification_types` is currently `l2 l3 host contactless softpos`. Direct-REST work
introduces certification tracks the enum can't express:

- `emvco_3ds` — EMVCo 3DS Server certification
- `network_cnp` — per-network CNP/3DS certification pass, separate from CP

Additive enum values only. No column changes, no data migration.

---

## 5. Routing: how traffic gets onto this path

This is the one place where an existing table genuinely needs to change, and it is worth
calling out separately rather than burying.

`RoutingConfig` today expresses *which network*, not *which transport*. With three coexisting
paths — YSP (live), MPGS (separate product), direct-REST (new) — routing needs a transport
dimension to be able to say "Visa CNP goes direct-REST; everything else stays on YSP".

**Two options, both additive and nullable:**

- **5a (minimal)** — add nullable `connectivity_profile_id` FK + `presence_scope`
  (`card_present` / `card_not_present` / `both`) to `RoutingConfig`. Existing rows keep working
  unchanged with both columns NULL, meaning "as today".
- **5b (no existing-table change)** — a new `scheme_routing_overrides` table keyed on
  `(scheme_id, presence_scope)` pointing at a profile, consulted only when present.

**Recommend 5a.** It keeps one routing concept in one table, and the change is strictly
additive with a NULL default that preserves current behaviour. 5b avoids touching an existing
table but splits routing across two places, which is the kind of split that later gets
misread. This app has widened an existing column before when real data demanded it
(`BinRange.start_bin` 6→19 digits in Phase 2a), so additive change to an existing table is not
without precedent here.

---

## 6. Phasing

| Phase | Deliverable | Blocked by |
|---|---|---|
| **1** | `ConnectivityProfile` + `NetworkCredential` schemas, context CRUD, audit, migrations | — |
| **2** | `CredentialExpiryWorker` + AlertsCore wiring (reuses Phase 8/9 pattern) | Phase 1 |
| **3** | LiveViews (`/admin/schemes/connectivity-profiles`, `/admin/schemes/network-credentials`) + `MenuProvider` entries under the existing **Network** group (`group_order 2`, alongside Network Routing / Network Parameters) | Phase 1 |
| **4** | `ThreeDsConfig` schema + LiveView; `Certification` enum extension | — (parallel with 1–3) |
| **5** | `RoutingConfig` transport/presence extension (§5a) | Phases 1, 4 |
| **6** | Seed data — real values only | Phases 1–5 |
| **—** | *Sync to the switch — explicitly out of scope, see §7* | — |

**Seeding honesty (§6).** This app has a strong, well-documented convention of seeding only
real data and shipping empty rather than fabricating (Phase 5 shipped no interchange rates;
Phase 7 shipped no calendars; the progress log tracks which seeds are illustrative). Applied
here, what is genuinely real today:

- Visa sandbox: base URL, KID, cert files, expiry dates — **all real and verifiable**
  (`SBX-2024-Prod-Inter.pem` expires 2029-09-13, confirmed by parsing the actual file)
- Visa `clientId` — **not issued.** Seed NULL, not a placeholder string.
- Mastercard: documented MTF base URL is real; **no KMP certificate exists yet.** Seed the
  profile with no credential rows rather than inventing one.
- 3DS: `provider` is genuinely undecided. Seed `simulator` for non-production environments
  only; leave production unseeded.

---

## 7. Explicit non-goals

*(The sync mechanism was originally listed here as out of scope. It has since been decided —
see §9.)*
- **MPGS gateway configuration.** Not built here. MPGS is a distinct commercial product with a
  different config shape entirely (OAuth 1.0a consumer key + signing key + merchant ID, not
  mTLS), it is a separate implementation on the switch that this work deliberately does not
  touch, and your stated direction is the direct-REST approach. **If you do want MPGS
  configuration in the backoffice, say so and it is a small additional phase** — the
  `ConnectivityProfile` shape accommodates it with an `api_product: "mpgs"` and an auth block,
  but I have not assumed it.
- **Migrating the switch off static config.** Phase 1–6 make the backoffice authoritative for
  *new* config. The switch keeps reading its own config files until the sync mechanism exists.
- **Per-merchant routing.** §5 covers per-scheme/per-presence routing. Per-merchant selection
  (which `direct-connect-worklist.md` names as the non-disruption mechanism) is a merchant-
  domain concern, not scheme master data.
- **Secrets management.** This plan stores credential *metadata and references*. Where the
  actual key material lives (Vault or equivalent) is an infrastructure decision that both this
  and the switch side are waiting on — the switch's `mtls:` config already reads paths rather
  than embedding key material, so it is a config change on that side once decided.

---

## 8. Open questions

1. ~~**MPGS in scope?**~~ **Resolved (2026-08-14): yes, in scope for *configuration*.** §3.5
   makes it unavoidable — MPGS occupies the aggregated+REST quadrant, and a
   `ConnectivityProfile` that can't represent it leaves the backoffice authoritative for only
   one of four paths. This does **not** mean touching the MPGS implementation on the switch,
   which stays a separate product and is not modified.
2. ~~**Environment modelling**~~ **Resolved: one row per environment.** Decided on four
   grounds, the first of which is current fact rather than hypothesis: Visa sandbox is `active`
   while Visa production is `draft` (no `clientId`), and Mastercard production does not exist at
   all (no KMP cert) — four independent lifecycle states a single row cannot hold. Plus:
   credentials attach per environment (so the expiry worker can distinguish a production cert
   from a sandbox one), row-level audit granularity, a meaningful duplicate-active guard, and
   push blast radius (a sandbox switch should never receive production URLs).
   **Correction applied:** `profile_name` is now environment-*agnostic* (`visa_acceptance`),
   unique on `(profile_name, environment)`; the switch's config key is derived as
   `#{profile_name}_#{environment}`. An earlier draft encoded environment twice.
3. ~~**`NetworkParameter` `transport` column?**~~ **Resolved: no.** The column would land in one
   of two bad states — either REST rows are admitted and are mostly-NULL (`iso_version`,
   DE 3 `processing_codes`, DE 24 `function_codes` have no REST equivalent), which is exactly
   the half-NULL anti-pattern rejected for this same table in §3 Option A; or REST rows are
   barred and the column only ever holds one value. Checked whether REST has message-level
   config needing a home (Mastercard `txTp`, Visa `TxAttr`, response-code maps): all are **spec
   constants, correctly hardcoded in the mappers**, not configuration. **Action instead:** update
   `NetworkParameter`'s moduledoc to state its scope ("ISO 8583 message parameters; REST
   connectivity lives in `ConnectivityProfile`"). Zero migration, no risk to Phase 4's seeded rows.
4. ~~**YSP 3DS/CNP support?**~~ **Resolved as far as it can be:** `ysp_cnp` will be modelled as
   its own profile when it exists. Until confirmed, seed YSP's `supports_cnp` as **NULL
   (unknown), not `false`** — NULL says "nobody has checked", `false` asserts a fact we don't
   have. One open sub-question remains: if YSP CNP turns out to share the *same physical
   connection* as `ysp_ssl`, the two profiles should point at a shared endpoint rather than
   implying two connections (§4.1a makes this expressible either way).

### Resolved 2026-08-14

- **A. §5a confirmed** — `RoutingConfig` gains additive, nullable `presence_scope` and
  `connectivity_profile_id`. No separate overrides table. Existing rows keep working with both
  NULL, meaning "as today".
- **B. CP ingress wiring is a confirmed gap and is now in scope** — the `ISOMsg →
  PresenceContext + Canonical.Transaction` converter will be built. See §10.2.
- **C. Reversal-on-timeout, not retry-with-idempotency.** See §10 — this is a correction to the
  approach, not a detail.

---

## 9. Sync: backoffice → switch config push

**Decided (2026-08-14):** backoffice is the system of record and **pushes** to the switch. The
switch **never calls the backoffice**. The switch serves config from **ETS** on the transaction
hot path.

### 9.1 The real question: what backs ETS

ETS is settled as the read path. Both options below load into ETS — the actual decision is what
durable store sits behind it.

| | **Option 1 — DB-backed** | **Option 2 — file-backed** |
|---|---|---|
| Push writes to | switch's own DB (new tables) | a config file on disk |
| Multi-node | shared DB + PubSub invalidation | **every node needs its own file write** |
| Container/K8s | works | **writing app config at runtime fights ephemeral/read-only filesystems** |
| Atomicity | DB transaction | file rewrite — partial-write risk |
| Audit / history | rows, queryable: *which config was live when this transaction was authorized* | none beyond the file's own mtime |
| Rollback | previous version still in the table | whatever the last file was |
| New schema | yes — migrations | none |

**Recommend Option 1.** The deciding factor is not elegance, it is that the switch is already
built to cluster (`dns_cluster`, `Phoenix.PubSub` as `DaAcquirer.PubSub`, `cachex` all present
as dependencies), and file-backed config does not survive clustering or container replacement
without a shared volume. Option 2's only real advantage — no migrations — is a one-time cost,
paid against a permanent operational constraint.

### 9.2 Read path, and the failure mode that matters most

```
transaction → ETS lookup (no DB hit, no network hit)
```

**The switch must never fail to authorize because config sync failed.** Precedence, highest
first:

1. **ETS** — hot path, always
2. **DB** — rehydrates ETS on boot
3. **Static config file** — bootstrap floor if the DB is unreachable at boot

Keeping the existing static config as the floor means a config-DB outage degrades to
last-known-good rather than taking down authorization. It also means the migration is
non-disruptive: the DB *overlays* static config rather than replacing it on day one.

### 9.3 Push flow

```
backoffice ──POST /internal/config/push──▶ switch
                                          │ validate payload
                                          │ persist (DB transaction)
                                          │ reload ETS
                                          │ PubSub broadcast → other nodes reload
              ◀────── ack {applied_version, checksum} ──────┘
```

- **Auth**: mTLS or a signed payload. This endpoint mutates authorization behaviour — it needs
  stronger protection than the checkout API.
- **Idempotency**: monotonic `config_version` on every push; replaying an already-applied
  version is a no-op ack, not a re-apply.
- **Atomic**: validate the *whole* payload before persisting anything. A partially-applied
  connectivity change is worse than a rejected one.
- **Never auto-activate blindly**: a pushed profile lands respecting its own `status`. A
  `draft` profile in the backoffice must not start taking live traffic because it was pushed.

### 9.4 Drift detection — the consequence of push-only

Push-only has one structural weakness worth stating plainly: **if a push is lost, config drifts
silently and nothing notices.** The switch cannot detect this itself, because it is forbidden
from calling the backoffice.

So reconciliation must be **backoffice-driven**. Two mechanisms, both compatible with the
constraint (backoffice → switch is allowed in both):

1. The push **ack returns the switch's applied version + checksum**; the backoffice compares
   against what it sent and flags mismatch.
2. The switch exposes a read-only `GET /internal/config/version`; the backoffice polls it on a
   schedule and re-pushes on divergence.

Recommend both — (1) catches immediate failures, (2) catches drift from a node that restarted
into stale state or joined the cluster late.

### 9.5 What this adds to the switch

New, additive — the existing TCP/YSP/MPGS paths keep reading static config until their profiles
are migrated:

- config tables + migrations in `da_acquirer`
- `DaAcquirer.Config.Registry` — ETS owner, boot loader, PubSub subscriber
- `DaAcquirer.Config.PushController` — the authenticated ingest endpoint
- resolution of `Upstream.Adapter` selection through the profile registry rather than the
  hardcoded `adapter_for/1` in `Cnp.CheckoutFlow` (§3.5)

---

## 10. Ambiguous-outcome handling: reversal, not idempotent retry

**Decided 2026-08-14.** This supersedes the "idempotency gap" framing carried in the
switch-side architecture doc, which described the fix as *deterministic, persisted correlation
IDs so a retry is safe*. That framing was wrong about the mechanism.

### 10.1 Why reversal is the right primitive

A timeout is **ambiguous**, not failed. The issuer may have approved and the response was lost,
or it may never have seen the request. Two ways to resolve that ambiguity:

| | **Idempotent retry** | **Reversal** |
|---|---|---|
| Mechanism | resend with the same key; the network dedupes and returns the original outcome | send a reversal/void to undo whatever may have applied |
| Unit of work | the **request** | the **transaction** |
| Depends on | the network implementing and honouring dedup | nothing — it is an ordinary message |
| Answers "did it apply?" | no — it hides the question | no — it makes the question irrelevant |
| Industry norm | no | **yes** |

The decisive argument is the unit of work: **not every request maps one-to-one onto a
transaction**, so a request-scoped idempotency key is the wrong granularity for a
transaction-scoped question. Reversal operates on the transaction, which is what actually needs
to be made whole. That is the "clear responsibility" this decision buys.

### 10.1a This is already the switch's own discipline

Not a new pattern — the TCP path has done exactly this since before the REST work:

- `DaAcquirer.Acquirer.ReversalOrchestrator` — "Phase 2.2: Immediate Reversal on Upstream
  Timeout", with `attempt_auto_reversal(temp_txn, "TIMEOUT")`
- `ReversalCleanupWorker`, `ConnectionLossHandler`, `PosReversal`, reversal-lifecycle statuses
  on `PosTempTransaction` (`RESPONSE_TIMEOUT`, `REVERSAL_PENDING`, `REVERSAL_SENT`, …)

The REST path adopting retry-with-idempotency would have made the two transports behave
differently under the *same* failure, for no reason. Reversal makes them consistent.

**Both networks' own APIs expect this.** Visa publishes `POST
/acs/v3/payments/authorizations/voids` — a void **without a resource id**, which exists
precisely for the case where the response carrying that id never arrived. Mastercard publishes
the `cain-reversal-requests` and `cain-financial-reversal-advices` message families.

### 10.2 What this actually requires

The requirement does not disappear — it changes shape, and gets stricter:

> **You cannot reverse a transaction you never recorded.**

So the real obligation is **durably persist before dispatch**, not "derive a deterministic id".
`ReversalOrchestrator.attempt_auto_reversal/2` takes a `PosTempTransaction` — a persisted row.
The TCP path satisfies this today. **The REST path does not**: `Cnp.CheckoutFlow` builds a
`Canonical.Transaction` in memory and dispatches without persisting, so a timeout there is
currently unrecoverable — there is nothing to reverse *from*.

Work implied, in dependency order:

1. **Persist a `PosTempTransaction` before dispatch** on the REST path (CNP now, CP once §10.3
   lands). Reuses the existing reversal lifecycle rather than a parallel table — deliberately,
   since `ReversalOrchestrator`/`ReversalCleanupWorker` already operate on it.
   - *Known friction:* `PosTempTransaction.s_tid` is required (max 8 chars) and assumes a POS
     origin. CNP has no physical terminal. Realistic resolution is a **virtual terminal id** per
     e-commerce merchant (normal industry practice), not a synthetic constant — to be confirmed
     rather than silently defaulted.
2. **REST reversal dispatch.** Visa's `void/2` is implemented, including the without-id variant.
   **Mastercard's is not** — `CainAdapter.void/2` returns `{:error, :not_supported}`; the
   `cain-reversal-requests` family is unmapped. That is now a gap with a deadline, not a
   deferral.
3. **Wire REST timeout → `ReversalOrchestrator`**, matching the TCP path's trigger points.
4. **Mastercard's 409 duplicate detection becomes a safety net, not the mechanism.** It is
   genuinely useful — it tells us a request already landed — but it is a second line of defence
   behind reversal, and it has no Visa equivalent, so it cannot be the primary design.

### 10.3 Consequence for `ConnectivityProfile` (open question C, now answered)

**No `retry_count` / `retry_enabled` column on `ConnectivityProfile`.** Retry is not the REST
path's ambiguity strategy, and offering the knob would invite exactly the double-authorization
this decision avoids. What the profile may carry instead:

- `reversal_on_timeout` (boolean, default **true**)
- `reversal_max_attempts` — retries of the *reversal*, which is safe to repeat, unlike the
  authorization

`NetworkParameter.retry_*` stays as-is for the TCP/ISO 8583 path, where it has always applied
and where the semantics are established.