Design Effectiveness vs. Operating Effectiveness

The heart of fieldwork is a pair of questions you must never collapse into one. First, design effectiveness: is the control built right? If it operated exactly as designed, would it actually prevent or detect the risk it targets?

Second, operating effectiveness: does the control actually run the way it was designed, consistently, over the period? These are genuinely different. A control can be beautifully designed and never executed.

It can be faithfully executed and yet poorly designed, so it catches the wrong things. The exam constantly tests whether you can tell these apart, because the remediation is different for each, and naming the wrong one sends management to fix the wrong thing.

To test design, you walk the control through end to end, often a single walkthrough, and ask the key question: assuming this control ran perfectly every time, would it actually catch the risk? You look for design gaps, a monitoring scenario that targets the wrong behavior, a CDD form that never asks for beneficial ownership, an approval step that any junior staffer can bypass. The defining feature of a design flaw is that running the control more diligently won't fix it.

If the rule is built to detect the wrong pattern, executing it flawlessly still misses the real activity. Design failures are often the more serious, because they mean the risk was never truly covered.

To test operating effectiveness, one walkthrough isn't enough; you sample items across the whole period, because a control might work in January and quietly break by June. You re-perform where you can: did the alert actually generate, did the review actually happen, did the approver actually approve before the account opened? You check consistency, completeness, and timeliness, and you hunt for overrides and skipped steps, the manager who waved through approvals, the queue that was cleared without review to hit a deadline.

An operating failure means the control was well-designed but didn't run as intended, and the fix is usually about execution, training, capacity, or discipline, not redesign.

Why does the distinction matter so much in practice? Because it drives the fix. A design flaw means you must change the control itself, rebuild the scenario, add the missing field, redesign the approval.

Throwing more staff at a badly designed control just produces more diligent failure. An operating flaw means the design is sound but execution slipped, so you fix capacity, training, monitoring, or accountability. Mislabel the problem and management spends money on the wrong remedy.

Occasionally both fail at once, a control that's poorly designed and also inconsistently run, and a good auditor names both rather than forcing a single label. The exam rewards the precise diagnosis.

Sharpen your evaluation by classifying controls. Preventive controls stop a risk before it occurs, blocking a payment to a sanctioned party at the point of processing. Detective controls catch a problem after the fact, transaction monitoring flagging suspicious activity that already happened.

Compensating controls cover a gap left by a weaker primary control, extra manual review where an automated check is known to be limited. This classification matters because you evaluate the whole control environment, not a single control in isolation. A weakness in one control may be acceptable if a strong compensating control covers it, or it may be fatal if nothing backs it up.

So when you find a deficiency, ask what else stands between the risk and a loss; the residual exposure depends on the layers together. The exam may give you a weak control and a strong compensating one and ask for the net risk, the answer turns on whether the layers genuinely cover the gap.

Recapping: fieldwork answers two separate questions. Design effectiveness asks whether the control is built right, tested with a walkthrough; operating effectiveness asks whether it runs as built, tested by sampling across the period and re-performing. A design flaw needs a redesign; an operating flaw needs better execution; and getting the diagnosis right is what sends management to the correct fix.

And evaluate controls as layers, preventive, detective, and compensating, so you judge the net residual risk rather than any single control in isolation. Next, we apply this lens to the first big fieldwork target: auditing the enterprise-wide AML risk assessment itself. Test yourself on design versus operating effectiveness first, because it underpins everything that follows.