# Rule Expression Engine

This is the most consequential architectural gap in the whole comparison: **Jube's rule
"expression" is real, compiled code; mw-core's is a small hand-rolled interpreter.** Almost every
other gap in this document set (AND/OR parsing, arithmetic, field-to-field comparison,
sanction/tag/dict wiring) traces back to this one difference.

## Jube

**Rules are VB.NET source code, compiled at sync/load time via Roslyn into real .NET delegates.**

- Source: `Jube.Parser/Parser.cs`, `Jube.Parser/Compiler/Compile.cs`,
  `Jube.Engine/.../SyncEntityAnalysisModelGatewayRulesExtensions.cs:220-240`.
- A rule's `BuilderRuleScript`/`CoderRuleScript` field holds VB.NET-flavored source, e.g.:
  ```vb
  If (Payload.CurrencyAmount > 0) Then Return True End If
  ```
- `Parser.TranslateFromDotNotation/0` (`Parser.cs:641-1062`) rewrites the dotted namespace
  prefixes into calls against the function's injected parameters, then the result is wrapped in a
  class definition and **compiled to a real assembly** with Roslyn, producing a `Match` delegate
  whose actual signature is:
  ```csharp
  Function Match(Data As DictionaryNoBoxing(Of String),
                  List As Dictionary(Of String, List(Of String)),
                  KVP As PooledDictionary(Of String, Double),
                  Log As ILog) As Boolean
  ```
- **Because it's real compiled code**, it gets — for free, with no special-casing anywhere in the
  evaluator — full arithmetic (`+ - * /`), nested boolean logic (`If/Then/AND/OR/NOT`, arbitrary
  nesting), string methods (`.Contains()`, `.StartsWith()`, etc.), type casts (`.AsDouble()`,
  `.AsBool()`, `.AsDateTime()`), and **native field-to-field comparison** (both sides of any
  comparison are just VB.NET expressions evaluated against the same `Data` dictionary — comparing
  `Payload.MerchantCountry <> Payload.IssuerCountry` requires zero special code).
- **Dotted namespace prefixes** recognized by the translator: `Payload.*` → `Data("field").As[T]()`,
  `Abstraction.*` → `Abstraction("name")`, `TTLCounter.*` → `TTLCounter("name")`, `Sanction.*` →
  `Sanctions("name")`, `Dictionary.*`/`KVP.*` → `KVP("key")`, `List.*` → `List("name")`.
- **Sanction/list/dictionary checks are NOT part of the expression language as special operators**
  — they are ordinary **injected parameters** (`List`, `KVP`, and an implicit sanctions context)
  that the compiled code calls like any other dictionary/collection (e.g.
  `List("HighRiskBins").Contains(Payload.Bin)`). The expression language has no dedicated
  "sanction op" — it's just a method call against data the surrounding system populated before
  invoking the delegate.
- The `Parser` class does pre-validate tokens against an allow-list before compilation (so
  arbitrary code execution is bounded), but within that allow-list, the full power of a real
  language is available.

## mw-core

**`MwRisk.RuleExpression` is a hand-rolled interpreter** over a small JSON tree format (with a
legacy single-line text fallback), not a compiler. Every capability Jube gets "for free" from
being real code had to be individually designed, implemented, and in several cases **was
silently missing or broken** until specifically found and fixed during this engagement.

| Capability | Jube | mw-core, as originally found | mw-core, after this engagement's fixes |
|---|---|---|---|
| Logical AND/OR/NOT, arbitrary nesting | Native (`If/Then`, full boolean expressions) | Tree form: full support. **Legacy single-line text form: none at all** — `parse_legacy_tokens/1` only recognized a single comparator, with no concept of `AND`/`OR`. A string like `"X > 5000 AND Y != \"USA\""` parsed into one bogus condition by matching whichever comparator happened to appear in a fixed priority list, regardless of where it actually occurred in the string. | Added `AND`/`OR` splitting to the legacy-text parser (`rule_expression.ex`, `from_legacy/1`/`legacy_group/2`), with whitespace-delimited splitting to avoid misreading literals like dates. |
| Arithmetic on a comparison value | Native | **None.** `resolve_operand/2`'s prefix-detection treated `"Abstraction.X * 3"` as one (nonexistent) literal field path. | Added arithmetic (`+ - * /`, whitespace-required around each operator) to `resolve_operand/2` — initially single-operator only, then extended for O8 to full chains with standard precedence (`*`/`/` before `+`/`-`, left-to-right within a level). Still value-side only — `field` is never run through arithmetic, matching `StdDevAnomalyReview`'s shape (plain field compared against a formula). |
| Field-to-field comparison (`Payload.X != Payload.Y`) | Native (just two variable reads) | **None.** The right-hand side of any `cmp` was always treated as a literal value. `CrossBorderActivity`'s `Payload.MerchantCountry != Payload.IssuerCountry` was comparing a country code against the **literal string `"Payload.IssuerCountry"`** — always true. | Added a namespace-prefix heuristic (`@field_ref_prefixes`) in `resolve_operand/2`: if a value string starts with `Payload.`/`Abstraction.`/etc., resolve it as a field path instead of a literal. Heuristic-based, not structurally guaranteed like Jube's. |
| Sanction / tag / dictionary lookups | Injected parameters, called as ordinary methods — no expression-language-level concept | Embedded as JSON tree node types: `{"op":"sanction", ...}`, `{"op":"tag", ...}`, `{"op":"dict", ...}`. **These were parsed (the UI could create them, `normalise/1` preserved them) but `evaluate/2`'s catch-all clause (`evaluate(_, _payload), do: false`) meant they could never actually fire** — silently, with no error. | Implemented `eval_sanction/3`, `eval_tag/3`, `eval_dict/3`, wired into `evaluate/3` (which also now threads `tenant_id`, required by the underlying `ReferenceDataCache`/`SanctionsCache` lookups). |
| String functions (`contains`/`starts_with`/`ends_with`) | Native | Implemented as explicit comparators (`@comparators`) | 🟢 Parity |
| Comparators | Full native operator set | `== != > < >= <= in contains starts_with ends_with` | 🟢 Parity for the supported set |
| `in` (list membership) in legacy text form | N/A (native arrays) | **Missing** — `" in "` was not in `@legacy_ops`, so e.g. `"Payload.MCC in [7995,5816,6051]"` never tokenized; bracket-list literals (`[a,b,c]`) also weren't stripped of their brackets. | Added `" in "` to `@legacy_ops` and a `strip_quotes("[" <> rest)` clause to trim the trailing `]`. |
| Code execution model | Compiled assembly, cached per-rule-version, native CPU execution | Tree walked/interpreted on every evaluation, no compilation/caching of the parsed form itself (though the *rules* are cached — see [07](07-MULTI-MODEL-SCOPING.md)) | 🟡 Intentional divergence — performance profile differs, not evaluated as a current bottleneck |

## Why this matters going forward

Because mw-core's evaluator is hand-rolled rather than a real language, **every new expression
capability someone wants to add to a rule has to be deliberately built into the interpreter** —
unlike Jube, where "can my rule expression do X" is almost always "yes, because it's just VB.NET
code." This is the single largest source of "silent gap" risk in the system: a rule can be
authored and saved through the UI, look syntactically valid, and simply never fire (or always
fire) because the interpreter doesn't support what was written — with no error surfaced anywhere.
Every fix listed above was found this way: a rule that looked correctly configured but produced
the wrong decision when actually exercised, not from a static read of the code.

## Open items

- **O8 closed.** Chained arithmetic (`A * 2 + B`) is now supported — tokenizes on whitespace,
  resolves each operand (numeric literal or field reference), then reduces in two passes
  (`*`/`/` first, then `+`/`-`) for standard left-to-right precedence within each level. Still
  value-side only (see the gap table above).
- The field-to-field and arithmetic detection are both **string-shape heuristics** (does the value
  start with a known prefix / contain a whitespace-delimited operator), not a structural parse —
  a literal string that happens to match one of these shapes would be misinterpreted. Low
  practical risk given the current rule set, but worth knowing.
- No equivalent of Jube's pre-compilation allow-list validation step — mw-core's interpreter is
  inherently bounded to what it implements, so this isn't a security gap, just a noted asymmetry.