Model Risk Management and Validation (SR 11-7)

Here's a reframing the exam expects: your transaction-monitoring system and your sanctions-screening engine are models. SR eleven dash seven, the Federal Reserve and OCC's supervisory guidance on model risk management, defines a model broadly as a quantitative method that turns inputs into estimates used for decisions, and AML systems fit squarely inside that definition. Model risk is the risk of adverse outcomes from a model that's either fundamentally wrong, producing bad results, or used wrongly, applied outside its valid range.

Because regulators treat AML detection systems as models, audit must apply model-risk discipline to them, and that's a distinct, technical skill set the certification tests.

SR eleven dash seven rests on three pillars, and they're worth memorizing. First, sound development, implementation, and use: the model should be built on solid theory, properly implemented, and used only for its intended purpose. Second, effective validation: an independent, robust challenge to the model's conceptual soundness, its ongoing monitoring, and its outcomes.

Third, governance: policies, roles, controls, and crucially a complete inventory of every model in use, because you can't govern a model you've forgotten you have. For an AML auditor, these three pillars become your testing program: did development follow standards, was validation truly independent, and is governance real rather than nominal?

Validation is the pillar audit scrutinizes most, and its defining feature is independence: the validator must be independent of the people who built the model, or it's just self-review again. Effective validation covers three things. Conceptual soundness asks whether the model's design and assumptions are reasonable for its purpose, are the monitoring scenarios theoretically capable of catching the targeted behavior?

Ongoing monitoring asks whether the model still performs as the business and data evolve, since a model tuned three years ago may have quietly degraded. And outcomes analysis tests results against reality through back-testing and benchmarking, comparing the model's output to known answers or to alternative approaches. Audit confirms all three actually happened and were performed independently.

Let's name the findings that recur, because the exam draws scenarios from them. First, no independent validation, or validation performed by the very team that built the model, which fails the independence requirement. Second, an incomplete model inventory, where end-user spreadsheets and side tools quietly make decisions outside any governance.

Third, stale tuning, thresholds set years ago and never re-validated against current data, the link back to our below-the-line testing. And fourth, the vendor black box: a purchased TM or screening engine accepted on faith, with no one inside the institution able to explain or challenge how it works. SR eleven dash seven doesn't let you outsource understanding; the institution still owns the model risk, exactly as it owns the program.

A fast-growing area the exam is starting to touch is artificial intelligence and machine-learning models in AML, increasingly used to score risk, prioritize alerts, and detect novel patterns. The same SR eleven dash seven discipline applies, but these models raise extra demands. Explainability is the first: can the institution actually explain why the model flagged or cleared a given case, or is it an opaque box that even its owners can't interpret?

Regulators expect decisions affecting customers and filings to be understandable, not magic. Then there's model drift, the tendency of a machine-learning model to degrade as the world changes away from its training data, which makes ongoing monitoring even more important. And there's bias and training-data quality: a model trained on skewed or incomplete data can systematically miss or mistreat certain customers.

So when you audit an AI-driven AML control, you apply model risk management plus a heightened focus on explainability, drift, bias, and the governance to manage all three. The exam rewards recognizing that newer technology doesn't escape model-risk discipline; it needs more of it.

Recapping: transaction-monitoring and sanctions-screening systems are models, and SR eleven dash seven governs them through three pillars, sound development and use, effective and independent validation, and real governance with a complete model inventory. Audit focuses hardest on validation independence and on outcomes analysis like back-testing and benchmarking, and watches for stale tuning and unchallenged vendor black boxes. Next, we stay technical and audit the sanctions and OFAC screening function, where the matching logic itself becomes the thing under test.

Take the model-risk practice questions first.