petralLEARNDesk ↗
CHAPTER 11LESSON 072 MIN READ

How a survey is built

Sampling, response rates and the balancing item, underneath every official number.

WHY IT MATTERS

A survey reports what a sample and a method estimate. Response rates, late submissions and balancing items explain why the first print can change.

Behind most official energy statistics is a survey: a defined population of respondents, a questionnaire, a submission deadline, and a procedure for what to do about the ones who did not answer in time. Understanding that machinery explains most of the odd behaviour in the resulting series.

Coverage is the first design choice. Some surveys attempt a census of a defined population, such as all refineries above a size threshold, and some sample. A census of a threshold-defined population still misses everything below the threshold, and that missing tail is estimated rather than counted, which is why a series can be described as complete and still contain an estimate.

Non-response is the second, and it is handled by imputation: filling the gap with a modelled value based on the respondent's own history or on similar respondents. Imputation is a reasonable procedure and it means a published figure is partly a measurement and partly a model, in a proportion that varies week to week and is not usually published alongside the number.

The balancing item is the third and most revealing feature. When a survey collects supply, demand and stocks independently, the three will not be consistent, because each is measured with its own error. The publisher then either reports the inconsistency as an explicit adjustment or distributes it across the components. Reporting it is the more honest choice and is what makes an adjustment term a feature rather than an embarrassment. See the balance identity.

Seasonal adjustment deserves a specific warning. An adjusted series has had a fitted seasonal pattern removed, and that pattern is re-estimated as new data arrives, which means historical values of an adjusted series change even when no underlying data was revised. A chart of a seasonally adjusted series is not a chart of what was published at the time.

BALWeekly BalanceLIVE FROM THE DESKOpen on the desk
The identity behind the weekly print, with its adjustment term shown as a term rather than distributed away.