← All Terms

Minimum Detectable Effect

MDE

The smallest change in an outcome that a given measurement design can reliably distinguish from normal variation, at a chosen level of confidence.

AI Strategy

The minimum detectable effect is set by three things: how much the outcome moves on its own, how many observations the comparison has, and the confidence required. For a simple comparison of weekly averages before and during a pilot, it is roughly 2.8 times the weekly spread multiplied by the square root of one over the baseline weeks plus one over the pilot weeks, at the usual 80 per cent power and 5 per cent significance.

It belongs in the business case because it shows, before any money is spent, whether the pilot can prove the gain it promises. A measure whose weekly spread is 6 per cent, compared over eight weeks either side, cannot reliably show anything smaller than about 8 per cent.

When the target sits below the minimum detectable effect there are only a few honest options: run longer, add a control group, use pre-pilot data to reduce the noise, or accept that the benefit cannot be measured and decide on other grounds.