# Phase Recon-6 — Hardening, Testing + Audit

**Status:** ⬜ Pending
**Duration:** Weeks 9–10
**Depends on:** Phase Recon-5
**Goal:** Production-ready. Every recon run is audited, errors are informative, edge cases are handled, and the system holds under concurrent load.

---

## 1. Audit Integration

Every recon run emits structured audit events via `mw_audit`.

```elixir
# in MwRecon.Orchestrator — already wired in Phase 3, formalised here

MwAudit.log("recon.session.started", %{
  session_id: session.id,
  tenant_id:  session.tenant_id,
  recon_type: session.recon_type,
  recon_date: Date.to_iso8601(session.recon_date),
  initiated_by: session.initiated_by_id
})

MwAudit.log("recon.session.completed", %{
  session_id:      session.id,
  tenant_id:       session.tenant_id,
  matched_count:   summary["matched"],
  unmatched_count: summary["unmatched"],
  total_count:     summary["total"],
  duration_ms:     elapsed_ms
})

MwAudit.log("recon.session.failed", %{
  session_id: session.id,
  tenant_id:  session.tenant_id,
  step:       failed_step,   # "parsing" | "matching" | "reporting"
  reason:     error_reason
})
```

These events are visible in the existing `/admin/audit` UI — add `recon.*` to the event type filter.

---

## 2. Error Handling Matrix

| Scenario | Where detected | User-facing message |
|----------|---------------|---------------------|
| File exceeds 20MB | `ReconController.upload_files/2` | "File exceeds the 20MB limit. Please export a smaller date range." |
| Wrong file format (e.g. PDF instead of XLSX) | `recon_engine_py/normaliser.py` | "Unexpected file format. Expected XLSX for Bank Card Statement." |
| Required column missing from bank file | `recon_engine_py/normaliser.py` | "Missing column: CARD NUMBER. Check that the correct file was uploaded for Bank Card Statement." |
| Empty file (0 data rows) | `MwRecon.Orchestrator.step_parse/1` | "The uploaded file contains no data rows." |
| Duplicate active job for same tenant | `MwRecon.Orchestrator.start_run/1` | "A reconciliation is already running for this hospital. Please wait for it to complete." |
| CloudI service timeout | `MwRecon.EngineClient.run_match/1` | "Reconciliation timed out. The files may be too large — try a single day at a time." |
| HIS date filter returns 0 rows | `recon_engine_py/matchers/his_bank_card.py` | "No HIS transactions found for the selected date. Check the reconciliation date." |

---

## 3. Input Validation in the Matching Engine

### `normaliser.py` — column validation

```python
REQUIRED_COLUMNS = {
    "bank_card":   ["CARD NUMBER", "TERMINAL NUMBER", "DOMESTIC AMT", "APPROV CODE"],
    "bank_upi":    ["PAYER VPA", "TERMINAL_NO", "Transaction Amount", "Txn ref no.(RRN)"],
    "his":         ["Doc Date", "payment mode", "Amount", "Card No/Cheque", "Unit/Location"],
    "momentspay":  ["card_num", "terminal_id", "total_amount", "approval_code"],
    "amex":        ["Amount", "Approval No"],
}

def validate_required_columns(df: pd.DataFrame, file_role: str,
                               custom_mappings: dict = None) -> list:
    """
    Returns a list of missing column names (empty list = all present).
    Checks both canonical names and all known aliases.
    """
    needed = REQUIRED_COLUMNS.get(file_role, [])
    missing = []
    for col in needed:
        aliases = DEFAULT_COLUMN_MAPPING.get(col, [col])
        if not any(a in df.columns for a in aliases + [col]):
            missing.append(col)
    return missing
```

Engine returns immediately with an error if any required columns are missing:

```python
missing = validate_required_columns(bank_df, "bank_card", config.get("column_mappings"))
if missing:
    return {"status": "error", "missing_columns": missing,
            "reason": f"Missing required columns: {', '.join(missing)}"}
```

---

## 4. Rate Limiting — 1 Active Job Per Tenant

Already implemented in `MwRecon.Orchestrator.active_job_exists?/1` (Phase 3).
Formalise as a named function and add telemetry:

```elixir
defp guard_single_active_job(tenant_id) do
  if active_job_exists?(tenant_id) do
    :telemetry.execute([:mw_recon, :job, :rejected], %{count: 1}, %{tenant_id: tenant_id})
    {:error, "A reconciliation is already running for this hospital"}
  else
    :ok
  end
end
```

---

## 5. File Size Enforcement

Already in `ReconController`. Add the same check in `ReconWizardLive` (client-side, via JS hook) and in the Python engine as a guard:

```python
# In main.py, before dispatching to matcher:
MAX_BYTES = 20 * 1024 * 1024
for role, b64 in payload.get("files", {}).items():
    size = len(base64.b64decode(b64))
    if size > MAX_BYTES:
        return json.dumps({"status": "error",
                           "reason": f"File '{role}' exceeds 20MB limit ({size // 1024 // 1024}MB)"})
```

---

## 6. ExUnit Integration Tests

```
apps/mw_recon/test/
├── test_helper.exs
├── fixtures/
│   ├── sahyadri_bank_card.xlsx        # extracted from reference scripts
│   ├── sahyadri_momentspay.csv
│   ├── sahyadri_his_export.xlsx
│   └── sahyadri_bank_upi.xlsx
├── mw_recon/
│   ├── orchestrator_test.exs
│   ├── engine_client_test.exs         # mocked CloudI calls
│   └── recon_controller_test.exs
```

### Sample test (`orchestrator_test.exs`)

```elixir
defmodule MwRecon.OrchestratorTest do
  use InfraRepo.DataCase
  import Mox

  alias MwRecon.Orchestrator
  alias InfraRepo.ReconSessions

  setup :verify_on_exit!

  describe "start_run/1" do
    test "rejects if a job is already active for the tenant" do
      tenant = insert_tenant()
      _active = insert_session(tenant, status: "matching")
      session = insert_session(tenant, status: "pending")
      assert {:error, message} = Orchestrator.start_run(session)
      assert message =~ "already running"
    end

    test "transitions through all status steps" do
      tenant  = insert_tenant()
      session = insert_session(tenant, recon_type: "bank_card_vs_momentspay")
      _files  = insert_run_files(session)

      # Mock CloudI calls
      MwRecon.MockEngineClient
      |> expect(:run_match, fn _ -> {:ok, %{"status" => "ok", "matched" => [], "unmatched" => [],
                                             "summary" => %{"total" => 0, "matched" => 0, "unmatched" => 0}}} end)

      MwRecon.MockReportClient
      |> expect(:generate, fn _ -> {:ok, Base.encode64("fake xlsx bytes")} end)

      {:ok, _job_id} = Orchestrator.start_run(session)
      :timer.sleep(500)   # let async task complete

      updated = ReconSessions.get_session!(session.id)
      assert updated.status == "completed"
    end
  end
end
```

---

## 7. Python Unit Tests (`services/recon_engine_py/tests/`)

```python
# test_card.py
import pytest
import pandas as pd
from matchers.card import CardMatcher

BANK_ROW = {
    "CARD NUMBER": "4111111111111234",
    "TERMINAL NUMBER": "T001",
    "DOMESTIC AMT": "500",
    "INTNL AMT": "0",
    "APPROV CODE": "123456"
}
MP_ROW = {
    "card_num": "4111111111111234",
    "terminal_id": "T001",
    "total_amount": "500",
    "approval_code": "123456",
    "processing_id": "P001",
    "transaction_id": "TX001",
    "email": "test@test.com"
}

def make_payload(bank_rows, mp_rows):
    import io, base64, openpyxl
    # Build XLSX for bank
    wb = openpyxl.Workbook()
    ws = wb.active
    ws.append(list(bank_rows[0].keys()))
    for row in bank_rows:
        ws.append(list(row.values()))
    buf = io.BytesIO(); wb.save(buf)
    bank_b64 = base64.b64encode(buf.getvalue()).decode()
    # Build CSV for momentspay
    mp_df = pd.DataFrame(mp_rows)
    mp_b64 = base64.b64encode(mp_df.to_csv(index=False).encode()).decode()
    return {"recon_type": "bank_card_vs_momentspay", "config": {}, "files": {"bank_card": bank_b64, "momentspay": mp_b64}}

def test_4key_match():
    result = CardMatcher(make_payload([BANK_ROW], [MP_ROW])).run()
    assert len(result["matched"])   == 1
    assert len(result["unmatched"]) == 0

def test_3key_fallback():
    mp_no_approval = {**MP_ROW, "approval_code": "999999"}   # different approval code
    result = CardMatcher(make_payload([BANK_ROW], [mp_no_approval])).run()
    # Should match on 3 keys (card last4 + terminal + amount)
    assert len(result["matched"]) == 1

def test_no_match():
    mp_different = {**MP_ROW, "total_amount": "9999", "approval_code": "000000"}
    result = CardMatcher(make_payload([BANK_ROW], [mp_different])).run()
    assert len(result["matched"])   == 0
    assert len(result["unmatched"]) == 1
```

---

## 8. Telemetry Events

Add to `infra_telemetry`:

```elixir
# New MW-Recon telemetry events
:telemetry.execute([:mw_recon, :session, :started],   %{count: 1}, %{recon_type: ..., tenant_id: ...})
:telemetry.execute([:mw_recon, :session, :completed], %{duration_ms: ms}, %{matched: n, unmatched: m})
:telemetry.execute([:mw_recon, :session, :failed],    %{count: 1}, %{step: step, tenant_id: ...})
:telemetry.execute([:mw_recon, :job, :rejected],      %{count: 1}, %{tenant_id: ...})
```

These auto-appear in the existing Prometheus metrics and Monitoring dashboard.

---

## 9. Recon Dashboard Card on Main Dashboard

Add a summary card to `DashboardLive` showing today's recon runs:

```elixir
# In DashboardLive.load_db_stats/0
recon_today: InfraRepo.Repo.aggregate(
  from(s in ReconSession,
    where: fragment("DATE(?)", s.inserted_at) == ^Date.utc_today()
  ),
  :count
),
recon_completed_today: InfraRepo.Repo.aggregate(
  from(s in ReconSession,
    where: fragment("DATE(?)", s.inserted_at) == ^Date.utc_today()
      and s.status == "completed"
  ),
  :count
)
```

---

## Acceptance Criteria

- [ ] All ExUnit tests pass (`mix test`)
- [ ] Python unit tests pass (`pytest services/recon_engine_py/tests/`)
- [ ] Uploading a file with missing required columns shows a clear message naming the specific missing columns
- [ ] Uploading a file > 20MB shows "File exceeds 20MB limit" (not a 500 error)
- [ ] Uploading an empty XLSX (headers only) shows "No data rows found"
- [ ] Attempting a second run while one is active shows "already running" message
- [ ] All 3 audit events (`recon.session.started`, `.completed`, `.failed`) appear in `/admin/audit`
- [ ] `mw_recon_session_completed_duration_ms` Prometheus metric is published
- [ ] 5 concurrent recon runs from 5 different tenants complete correctly with no cross-contamination
- [ ] Recon card visible on the main dashboard showing today's run count