# ADR 0009: Data Partitioning and Retention Policy

- Status: Accepted
- Date: 2026-03-11
- Owners: Data Platform Team, Compliance Team, Architecture Group
- Related:
  - `docs/adr/0001-app-boundaries.md`
  - `docs/adr/0003-financial-ledger-invariants.md`
  - `docs/adr/0007-observability-and-audit-traceability-standard.md`
  - `docs/adr/0008-integration-adapter-contract-and-failure-policy.md`
  - `docs/non-functional-slo.md`
  - `docs/phase-tracker.md`

## Context
The wallet platform processes financial records, identity-related data, audit evidence, and operational telemetry. Without explicit partitioning and retention rules, storage costs, performance degradation, and compliance risk increase over time.

This ADR defines data classification, partitioning strategy, and retention/deletion controls across core domains.

## Decision
Adopt domain-owned data partitioning with policy-driven retention tiers.

Core decisions:
1. Each domain app owns retention policy for its data classes, within compliance guardrails.
2. Financial ledger and audit data are append-only and retained per regulatory minimums.
3. Operational/diagnostic data has shorter retention and aggregation roll-up.
4. Partitioning strategy must support query performance and lifecycle operations (archive/purge).
5. Data deletion/anonymization for privacy requests must not violate financial record obligations.

## Data Classification Tiers
Tier A: Financial SoR data
- Journal entries
- Posting references
- Account balance snapshots and derived financial statements

Tier B: Compliance and audit evidence
- KYC/KYB case records
- AML/SAR records
- Immutable audit event stream

Tier C: Transactional operational data
- Transfer workflow states
- Idempotency records
- Adapter request/response metadata (sanitized)

Tier D: Observability data
- Metrics time series
- Structured logs
- Trace spans

Tier E: Cache and ephemeral state
- Locks
- Session caches
- Temporary reconciliation working sets

## Partitioning Strategy
General rules:
- Partition large, append-heavy tables by time (monthly recommended baseline).
- Add secondary partition keys where tenancy or provider segmentation is required.
- Prefer predictable partition naming conventions for automation.

Recommended partition examples:
1. Ledger entries:
- Partition by `posted_at` month.
- Optional sub-partition by `currency` or tenant where scale demands.

2. Audit events:
- Partition by `occurred_at` month.

3. Reconciliation records:
- Partition by `reconciliation_date` month.

4. Logs/traces:
- Index lifecycle management with hot/warm/cold tiers.

## Retention Policy Baseline
Note: exact durations must be aligned with jurisdiction/regulatory counsel.

1. Financial ledger (Tier A):
- Minimum retention: 7 years (or stricter jurisdictional requirement).
- Deletion: not permitted within legal retention window.
- Post-retention: archive strategy and controlled destruction process.

2. Compliance/audit (Tier B):
- Minimum retention: 7 years baseline, policy-adjustable.
- Audit stream remains tamper-evident during retention lifecycle.

3. Operational transactional state (Tier C):
- Active detailed retention: 12-24 months baseline.
- Older records: archive or aggregate where allowed.

4. Observability data (Tier D):
- Raw logs/traces: 30-180 days based on security and operations policy.
- Aggregated metrics: 13-24 months baseline for trend analysis.

5. Ephemeral/cache (Tier E):
- TTL-driven expiry in minutes/hours/days per use case.
- No long-term retention.

## Privacy and Deletion Rules
- Privacy deletion requests (where legally applicable) handled by:
  - direct deletion for non-regulated personal data
  - irreversible anonymization/pseudonymization where legal retention blocks deletion
- Financial/legal records remain retained per law, with restricted access controls.
- Deletion and anonymization actions must be audited.

## Archival and Restore Policy
- Archive format must preserve integrity and replay/audit utility.
- Archived datasets require checksum/hash validation.
- Restore drills required for selected datasets at regular intervals.
- Archive access must be least privilege and auditable.

## Performance and Cost Controls
- Use partition pruning in query plans for large historical datasets.
- Enforce query guardrails on unbounded historical scans.
- Implement scheduled compaction/maintenance tasks.
- Track storage growth and retention-policy effectiveness monthly.

## Security and Access Controls
- Tier-based access controls:
  - Tier A/B strictest controls, break-glass procedures for exceptional access.
- Encryption at rest and in transit for all retained data classes.
- Key rotation compatibility for encrypted historical data.

## Operational Requirements
- Automated jobs for partition creation, archival, and purge.
- Retention enforcement reports produced monthly.
- Alert on failed retention, purge, or archive tasks.

## Test and Verification Requirements
1. Partition creation/rollover integration tests.
2. Retention purge/anonymization workflow tests.
3. Archive checksum and restore validation tests.
4. Access control tests by data tier.
5. Query performance tests on recent and historical partitions.

## Consequences
Positive:
- Better long-term performance and predictable storage operations.
- Improved compliance posture for retention and privacy obligations.
- Clear ownership by domain teams.

Trade-offs:
- More lifecycle automation and operational overhead.
- Requires careful legal-policy alignment across jurisdictions.

## Acceptance Criteria
1. Tier classification map approved by architecture, compliance, and security.
2. Partitioning scheme implemented for large append-heavy datasets.
3. Retention and archival jobs are automated and monitored.
4. Privacy deletion/anonymization flow documented and auditable.
5. Phase 7 and Phase 8 governance gates include retention evidence.