petralLEARNDesk ↗
CHAPTER 10LESSON 013 MIN READ

Z-scores done honestly

Standardisation, and the guard that stops it inventing a signal.

WHY IT MATTERS

A z-score says how far a value sits from a chosen baseline. If the baseline barely varies, a tiny move can look falsely extraordinary.

A z-score expresses a value as the number of standard deviations it sits from its own mean. It is the standard way to make quantities of different scale comparable: a spread in dollars, an inventory level in millions of barrels, and a freight rate in Worldscale points cannot be ranked against each other until they have been standardised.

z = (x - mean) / sd
x
the current observation
mean
sample mean over the lookback window
sd
sample standard deviation over the same window
Two choices are hidden here: the lookback window, and what to do when sd is small.

The second choice is where the damage happens. The obvious guard is to require a non-zero standard deviation before dividing, and it is wrong. Consider a spread that has been almost perfectly flat for the lookback window, moving by fractions of a cent from measurement noise. Its standard deviation is tiny but strictly positive, so the guard passes. A move of a few cents then divides by that tiny number and produces a z-score of eight, rendered on screen as an extreme, confident dislocation. Nothing happened.

The lookback window is the other choice and it is less dangerous but more consequential. A short window makes a z-score responsive and unstable, and it also mechanically caps the achievable score: with twenty observations you cannot be four standard deviations from the mean, because the sample standard deviation is computed from the same twenty points. Short-window z-scores are therefore not comparable to long-window ones, and a screen that mixes them is ranking on window length as much as on dislocation.

There is also a distributional caveat that applies to all of this. A z-score is interpretable as a probability only under an assumption about the distribution, and oil returns have fat tails. A three-sigma event in a fat-tailed series is far more common than the normal-distribution intuition suggests, so a z-score is best read as a rank within its own history rather than as a probability statement.

ZSCZ-ScoresLIVE FROM THE DESKOpen on the desk
Ranked dislocations with the dispersion guard applied, so a flat series cannot manufacture a sigma.