Sanctions/PEP Screening — Fuzzy Name Matching & Alert Triage

A sanctions/PEP watchlist screening pipeline: five fuzzy name-matching algorithms compared honestly against each other and against two ensemble strategies, a recall-anchored threshold (the only defensible choice when missing a true match is a regulatory failure, not just a missed opportunity), and an alert-triage layer that ranks the resulting low- precision queue for analyst review.

Dataset: synthetic but structurally realistic — a 300-entry sanctions/ PEP watchlist (OFAC SDN/UN Consolidated/UK HMT/EU Consolidated/Domestic PEP style) and an 8,000-name customer/counterparty book to screen against it. True matches are generated by applying one real-world name corruption (transliteration, typo, word-order swap, nickname, initial, dropped middle name) to a genuine watchlist entry — never a verbatim copy. A deliberate slice of legitimate customers also collide with watchlist names purely by coincidence, which is what keeps precision realistically low no matter how good the matcher is (see Results, section 4).

Stack: Python 3.11 · rapidfuzz · jellyfish (Metaphone) · pandas · scikit-learn-adjacent evaluation code (no ML model — this is a name- matching and decision-threshold problem, not a classification one)

---

Why this project

1. Watchlist screening is a fuzzy-matching problem, not a classification one — there's no labelled training set to fit a model on, only a name and a list to check it against, so the interesting work is in the matching algorithm and threshold choice, not in model selection.
2. No single algorithm dominates. Levenshtein, Jaro-Winkler, token- based, and phonetic matching each catch a different real-world name corruption and miss others — the actual finding this project exists to demonstrate (Results, section 1), not an assumption baked in.
3. Ensembling isn't free. The obvious move — combine all five algorithms and alert if any one flags a match — turns out to perform worse than simply using the single best algorithm alone, because the naive maximum inherits the noisiest algorithm's false positives (Results, section 2). Reported honestly rather than assumed away.
4. Precision is capped by a real phenomenon, not a modelling failure. Coincidental name collisions between legitimate customers and watchlist entries are irreducible — reviewed in Results, section 4, including cases that fool all five algorithms simultaneously.

---

Project Structure

sanctions-pep-screening/
├── src/
│   ├── generator.py    # synthetic watchlist + customer book, 7 name-corruption types
│   ├── matching.py     # 5 fuzzy algorithms + composite (max) + agreement (2nd-highest) scores
│   ├── evaluation.py   # threshold sweeps, recall-anchored precision, per-transform diagnostics
│   └── triage.py       # alert ranking (agreement + list-source weight), precision/recall@k
├── notebooks/
│   ├── 01_eda.ipynb                          # watchlist/customer book composition, corruption examples
│   ├── 02_matching_algorithm_comparison.ipynb # per-algorithm precision at 95% recall + per-transform heatmap
│   ├── 03_threshold_optimization.ipynb       # ensemble vs single-algorithm, operating threshold
│   └── 04_alert_triage.ipynb                 # ranked alert queue, precision/recall@k, limitations
├── tests/  # 27 tests: generator, matching, evaluation, triage
└── pyproject.toml

---

Key Visualisations

Watchlist Composition	Customer Book by Corruption Type

Precision at 95% Recall, by Algorithm	Recall by Transform Type

Precision-Recall Trade-off by Strategy	Alert Triage: Precision/Recall by Review Depth

---

Results

1. No single algorithm wins on every corruption type

Each algorithm evaluated at its own threshold needed to reach ≥95% recall independently:

Algorithm	Threshold	Precision	Recall	Alerts
Phonetic (Metaphone)	60	3.4%	95.0%	4,454
Jaro-Winkler	68	6.4%	95.6%	2,407
Levenshtein ratio	56	14.2%	95.6%	1,080
Token-sort ratio	72	42.9%	95.6%	357
Token-set ratio	80	49.8%	96.3%	309

Token-set ratio is the strongest single algorithm by a wide margin. Phonetic matching alone is the weakest: Metaphone codes collapse enough unrelated names to the same phonetic key that reaching 95% recall requires accepting 4,454 alerts — 14× more than token-set ratio needs for the same recall.

Per-transform recall (at each algorithm's own threshold above) shows why:

Transform	Levenshtein	Jaro-Winkler	Token-sort	Token-set	Phonetic
token_reorder	0.61	0.61	1.00	1.00	0.56
initial	1.00	1.00	0.50	0.83	1.00
nickname	1.00	1.00	0.92	0.83	1.00
transliteration	1.00	1.00	1.00	0.95	1.00
exact / typo / drop_middle	1.00	1.00	1.00	1.00	1.00

Token-based algorithms hold recall on word-order swaps exactly because they tokenise before comparing — the entire reason they exist — while character- based algorithms lose almost 40% of those cases.

2. Ensembling underperforms the single best algorithm

Strategy	Threshold	Precision	Recall	Alerts
Ensemble: max across 5 algorithms	85	39.2%	95.6%	390
Ensemble: 2nd-highest (agreement)	80	42.4%	97.5%	368
Single algorithm: token-set ratio	80	49.8%	96.3%	309

The naive "alert if any algorithm flags it" ensemble is the worst of the three at matched recall — taking the max means every customer inherits phonetic and Jaro-Winkler's false-positive tendencies too. Requiring two algorithms to agree is a real, if modest, improvement over naive max, but neither ensemble beats simply using token-set ratio alone on this corruption mix. Token-set ratio is adopted as the operating algorithm for the rest of this project on that basis, not by default assumption.

3. The initial-only blind spot

Even under the adopted token-set operating threshold, initial (e.g. "J. Smith" for "John Smith") and nickname corruptions are caught at only 83.3% recall — clearly worse than every other corruption type (95–100%). There simply isn't enough surviving character information for any text- similarity metric to close this gap alone; a real deployment would need a structural rule (single-letter first token) or additional identifiers (DOB, nationality) — out of scope for a name-only screening exercise and stated as a limitation rather than solved here.

4. Precision is capped by genuine coincidental collisions

At the operating threshold, the alert queue (309 alerts, 154 true matches, 49.8% baseline precision) contains legitimate customers who share an exact or near-exact name with a real watchlist entry by pure chance — and some of these fool all five algorithms simultaneously (5/5 agreement, top priority_score tier), indistinguishable from a genuine hit on name alone. This is not a matching-algorithm failure: it is the actual, unavoidable reason sanctions screening runs at low precision in practice, and it is why a real system needs a secondary identifier (DOB, nationality) to resolve what name matching alone cannot.

5. Ranking the queue recovers most of the value under a review budget

k (alerts reviewed)	Precision@k	Recall@k
10	60.0%	3.9%
25	72.0%	11.7%
50	64.0%	20.8%
100	61.0%	39.6%
200	58.5%	76.0%
309 (full queue)	49.8%	100%

priority_score combines the composite match score, how many of the five algorithms independently agree (weighted heavily), and the matched watchlist entry's list source (OFAC/UN weighted above domestic PEP at an equal match score). Reviewing the top 100 ranked alerts — under a third of the full queue — already recovers close to 40% of all true matches at higher precision than the queue's baseline rate. This changes investigation order, not total workload: the 95%-recall guarantee from section 2 still requires eventually clearing the entire queue, not just the top-k.

---

Quickstart

# 1. install
pip install -e ".[dev]"

# 2. run the tests
pytest tests/ -v

# 3. open the notebooks for the full analysis, in order
jupyter notebook notebooks/01_eda.ipynb

---

Technical Notes

• Synthetic but honest: every true match traces back to one of seven documented, testable name-corruption functions in src/generator.py; every reported precision/recall number is computed the same way any hidden test data would be, never tuned to look better after the fact.
• Ground truth (is_true_match) is never fed to the matching algorithms — only src/evaluation.py and src/triage.py reference it, the same discipline the sibling sql-banking-fraud-monitoring repo applies to its is_fraud column.
• The general-population name pool is deliberately disjoint from the watchlist's name-group pool (GENERAL_FIRST_NAMES/GENERAL_SURNAMES vs. FIRST_NAME_GROUPS/SURNAME_GROUPS in src/generator.py). An earlier version of this generator sampled ordinary customers from the same restricted pool the watchlist itself is built from, which manufactured near-100% coincidental collision rates and drove precision down to ~2% regardless of matching quality — a data-generation artefact, not a genuine finding. Fixed before any result in this README was produced.
• list_source severity weighting in triage is illustrative, not calibrated against real regulatory guidance — stated as a limitation in notebooks/04_alert_triage.ipynb rather than presented as authoritative.
• No fairness/bias audit was carried out on which name backgrounds generate disproportionate false positives. The sibling transaction-anomaly-detection project's addr1_freq proxy-feature flag is the closest precedent in this portfolio for why that would matter here too — scoped out explicitly, not silently ignored.