# Cache Schema: Two-Tier ETS + Redis Feature Store

Version: 1.0 | Date: 2026-05-21

---

## 1. Architecture Overview (Jube Local LRU Pattern)

Tier 1 — ETS (in-process, sub-millisecond):
  GenServer owns :risk_feature_cache ETS table.
  Keyed by {tenant_id, entity_type, entity_value, horizon, metric}.
  LRU eviction when table exceeds configured byte limit.

Tier 2 — Redis (cluster-shared, 1–3ms):
  Three Redis data structures (matching Jube's schema):
    Hash Tables   → TtlCounter (velocity counts), PayloadLatest
    Sorted Sets   → Journal (payload refs ordered by timestamp)
    HyperLogLog   → Distinct value counts (unique cards, emails)

Read path:
  1. Check ETS → hit: return immediately
  2. Miss: query Redis → populate ETS → broadcast invalidation to peers
  3. Miss in Redis: entity has no history → return zero values

Write path (velocity update):
  1. Redis atomic write (INCRBYFLOAT / ZADD / PFADD)
  2. PubSub broadcast "feature_cache:invalidated:<key>"
  3. All nodes delete affected ETS entry (next read re-hydrates from Redis)

---

## 2. Redis Data Structures

### 2a. TtlCounter (Hash Table)

Key:    risk:<tid>:counter:<entity_seg>:<horizon>
Fields: count, count_success, count_failed, count_chargeback,
        sum_eur, sum_eur_success, sum_eur_failed,
        count_same_amount_<amount>, var_sum, var_count
TTL:    horizon_seconds * 2  (auto-expire at 2× window)

Example:
  Key:  risk:1:counter:card:abc123token:1h
  HGET  count_failed  → "7"
  HGET  sum_eur       → "1250.50"

Elixir: Redix.command!(:redis, ["HINCRBYFLOAT", key, "sum_eur", amount_str])

---

### 2b. Journal (Sorted Set)

Key:    risk:<tid>:journal:<entity_seg>
Score:  Unix timestamp (float)
Member: <payload_hash>:<horizon_flags>

Enables ZRANGEBYSCORE window queries:
  ZRANGEBYSCORE key (now - window_seconds) +inf

Members are payload hashes pointing to PayloadLatest or archive.
Journal is pruned by PruningWorker: ZREMRANGEBYSCORE key -inf (now - max_horizon).

Example:
  Key:   risk:1:journal:card:abc123token
  ZADD   1748700000.123  "sha256abc:1h,1d,2w"

---

### 2c. PayloadLatest (Hash Table)

Key:    risk:<tid>:latest:<entity_seg>
Fields: ts, amount_eur, result, three_ds, cvv_used, avs_used,
        merchant_id, tx_country, currency, card_access_method,
        failure_reason
TTL:    30 days (longest window we query "previous transaction" features)

Used for "time since last transaction", "prev tx was 3DS", "prev amount" features.

Example:
  Key:  risk:1:latest:card:abc123token
  HGETALL → {ts: "1748700000", amount_eur: "150.00", result: "success",
              three_ds: "true", merchant_id: "M9876"}

---

### 2d. HyperLogLog (Distinct Counts)

Key:    risk:<tid>:hll:<entity_seg>:<horizon>:<dimension>
TTL:    horizon_seconds * 2

Used for: unique cards, unique cardholders, distinct countries, distinct amounts.

Example:
  Key:  risk:1:hll:merchant:M9876:1d:cards
  PFADD card_token_value
  PFCOUNT → ~distinct_card_count

---

### 2e. LruJournal (Sorted Set, cache warming)

Key:    risk:<tid>:lru_journal:<entity_seg>
Score:  last_accessed_unix_ts
Member: entity_key

Used by FeaturePrecalcWorker to identify hot entities and pre-warm ETS.
Matches Jube's LruJournal concept exactly.

---

## 3. ETS Table Schema

Table name: :risk_feature_cache
Type:       :set (hash)
Access:     :public (read from any process)
Owner:      InfraFeatureStore.EtsTierCache (GenServer)

Key:   {tenant_id, entity_type, entity_value, horizon, metric}
Value: {value, fetched_at_unix_ms}

Eviction: LRU — when byte_size(ets) > @max_bytes, delete least-recently-used entries.
Invalidation: PubSub message "feature_cache:invalidated" → :ets.delete(table, key)

---

## 4. Key Naming Reference Table

| Purpose           | Redis Pattern                                      |
|-------------------|----------------------------------------------------|
| Velocity counter  | risk:<tid>:counter:<entity>:<val>:<horizon>        |
| Journal           | risk:<tid>:journal:<entity>:<val>                  |
| Latest payload    | risk:<tid>:latest:<entity>:<val>                   |
| Distinct count    | risk:<tid>:hll:<entity>:<val>:<horizon>:<dim>      |
| LRU journal       | risk:<tid>:lru_journal:<entity>:<val>              |
| Sanction cache    | risk:<tid>:sanction:<list_type>:<name_hash>        |

Note: GUIDs and hashes strip hyphens (matching Jube convention).
Tenant ID is always the first segment after "risk:".

---

## 5. Capacity Planning

Assumptions: 1M transactions/day, 10 tenants, 15 entity dimensions, 15 horizons.

Estimated Redis memory:
  TtlCounter entries:  10 * 15 * 1M * ~200 bytes = ~30 GB (with TTL expiry reducing active set)
  Journal entries:     1M * ~100 bytes = ~100 MB/day (pruned at 90 days)
  PayloadLatest:       10 * (unique_entities) * ~500 bytes ≈ depends on cardinality
  HyperLogLog:         12 bytes per HLL structure (very memory efficient)

Recommendation: Start with Redis 16GB, monitor with redis-cli INFO memory.
Use KeyDB or DragonflyDB (Jube recommendation) for multi-threaded performance
and flash fallback if needed.