petralLEARNDesk ↗
CHAPTER 10LESSON 023 MIN READ

Correlation and spurious regression

Why regressing two oil series in levels produces relationships that do not exist.

WHY IT MATTERS

Two trending price series can appear related even when their changes are independent. Test the relationship before trading the fitted line.

The most common quantitative error in commodities is regressing two price series on each other in levels and reading the r-squared as evidence of a relationship. It is worth understanding exactly why this fails, because the failure is severe, systematic, and produces very convincing numbers.

Prices, inventory levels and the dollar index are all close to integrated of order one: their changes are roughly stationary but their levels wander without reverting. Two independent series with that property, regressed on each other, will produce a high r-squared and a significant t-statistic a large fraction of the time. This is the Granger-Newbold spurious regression result, and it is not a small-sample curiosity.

WORKED EXAMPLEMeasured on independent random walks
Test300 independent random-walk pairs, no relationship by construction
Regressed in levels, r-squared floor of 0.25113 pairs admitted as related
Regressed in changes, same floor0 pairs admitted
A levels fit admitted 38 percent of pure noise; a changes fit admitted none
These are petral's own measurements, recorded in `analytics/relations.py`. The fix is to difference both legs.

So the first rule is to difference both legs. Regress changes on changes, not levels on levels. A relationship that survives differencing is a relationship between the movements, which is what you almost always wanted anyway. A Durbin-Watson statistic on the residuals is a useful additional gate: strong positive serial correlation in the residuals is the signature of a levels regression pretending to be a relationship.

The second rule is subtler and cost petral a working feature. A fit-quality threshold answers the question 'is this real', not 'is this strong', and the answer depends on the sample size. A fixed r-squared floor conflates the two. A floor of 0.25, calibrated for levels and never re-derived after the fit moved to changes, rejected four of five genuinely live relationships, including crude against the dollar at an r-squared of 0.074 on 299 observations, which is roughly twenty times what unrelated series produce at that length.

r2 floor = t^2 / (t^2 + n - 2)
t
critical t-value at the chosen significance, e.g. the 99.9th percentile
n
number of observations
The floor falls as n rises, because more data makes a smaller effect distinguishable from noise. Strength is then carried separately, so a weak but real relation is discounted rather than hidden.
RVRelative ValueLIVE FROM THE DESKOpen on the desk
Brent-WTI against the level its drivers imply, with the fit quality stated rather than assumed.