When the Distribution Lies

The previous lesson ended on a dependency worth restating, because everything here follows from it. A risk number is a functional applied to a distribution. The functional is a choice between quantile and tail average, and it is the easy part. The distribution is where the content is, and it is never observed, only estimated.

So the interesting question is not which measure to use but where the estimated distribution differs from the real one, and in which direction.

Three differences dominate, and they are not independent. Returns have fatter tails than a normal distribution. Volatility is not constant but clusters in time. And correlations between assets are not stable, rising sharply in exactly the conditions that make them matter.

Each one alone would cause a risk model to understate danger. Arriving together, as they do, they compound.

The normal distribution is not slightly wrong about extreme returns. It is wrong by amounts that make the events it describes effectively impossible.

Run the arithmetic. Under a normal distribution, a move of five standard deviations in one direction has a probability of roughly one in three and a half million. Counting both directions and 252 trading days per year, that is a wait of several thousand years for a single occurrence.

Moves of that size and larger appear repeatedly in the recorded history of most liquid markets. Whatever produced them, it was not a process the normal distribution describes.

from statistics import NormalDist

nd = NormalDist()
for k in (3, 4, 5, 6):
    p = 2 * nd.cdf(-k)                 # two-tailed
    print(k, f"p={p:.3e}", f"once per {1/p/252:,.0f} years")
# 3 p=2.700e-03 once per 1 years
# 4 p=6.334e-05 once per 63 years
# 5 p=5.733e-07 once per 6,922 years
# 6 p=1.973e-09 once per 2,011,101 years

The practical reading is that under a normal assumption, the losses capable of ending a firm are assigned a probability of zero. Not small. Zero, for planning purposes.

The second failure concerns timing rather than magnitude. Returns are not independent draws from a fixed distribution: large moves are followed by large moves, of either sign, and quiet periods are followed by quiet periods.

This is volatility clustering, and it has two consequences that pull against each other.

The helpful one is that volatility is forecastable in the short run. Models that let today's variance depend on recent squared returns produce genuinely better estimates than a long-run average, and they are standard in risk systems for that reason.

The harmful one concerns what a calm period licenses. If volatility is estimated from a recent quiet window, the estimate is low, the risk number is small, and the position permitted by a fixed limit is large. Then the regime changes, and the enlarged position meets the higher volatility.

So a risk model that adapts quickly to calm and slowly to turbulence maximises exposure at the worst moment, which is the procyclicality of the previous lesson in mechanical form.

The third failure is the most consequential for a diversified book, because diversification is the one defence that a portfolio is built around.

A correlation matrix estimated over normal conditions describes normal conditions. Under stress, correlations across risky assets tend to rise together, and positions that were offsetting become positions that lose simultaneously.

The mechanism is not mysterious. In calm markets, prices are driven by asset-specific news, which is idiosyncratic and diversifies away. Under stress, a common factor dominates: participants reduce risk across everything at once, and forced selling is indifferent to what an instrument is.

The consequence is precise and unpleasant. Measured diversification benefit is largest when it is least needed and smallest when it is most needed. A portfolio constructed to be robust using an estimated correlation matrix is robust to the conditions in the sample, and the sample is mostly calm because calm is mostly what happens.

This is why a hedge failing in a crisis is a design property rather than bad luck.

The three failures are not a list of independent modelling defects. They are symptoms of one thing: the distribution is not fixed, because it depends on what participants are doing, and what they are doing depends on the distribution.

The loop is the point. A shock raises volatility and correlations, which raises measured risk, which forces position reduction, which is itself selling pressure, which is another shock.

That feedback is why extreme moves are not the tail of a stable distribution but the output of a different regime. It also explains why the three failures coincide: they are the same event observed through three different statistics.

A risk model estimating a fixed distribution from history cannot represent this, because it treats the market as something that generates returns rather than something that responds to positions. The next lesson follows the loop through the funding channel, where it becomes concrete.

Given all that, the standard complement to a risk number is a stress test, and its value comes from what it does not attempt.

A VaR or expected shortfall figure asks: given a distribution, what loss corresponds to this probability? A stress test asks: given this scenario, what is the loss? No probability is assigned, and that omission is the feature.

That sidesteps the entire problem above. You are no longer required to know how likely a 20% equity fall with credit spreads doubling and correlations at one might be. You only need to compute what it would cost you, which is a valuation exercise rather than a statistical one.

Scenarios come in two flavours, and a serious programme uses both. Historical scenarios replay specific past episodes through the current book. Hypothetical scenarios are constructed, and they matter because the next crisis will not be a repeat of the last one.

The discipline in hypothetical design is to move the variables that are usually assumed stable: set correlations to one, widen bid-ask spreads, and assume the position cannot be exited on schedule.

The most useful variant inverts the question, and it is the one most often skipped because the answer is uncomfortable.

Ordinary stress testing picks a scenario and computes the loss. Reverse stress testing fixes the loss at the level that would end the business, then asks what would have to happen to produce it.

The reframing matters because it removes the step where scenarios are chosen. A scenario library is assembled by people who work at the firm, share its assumptions, and are unlikely to include the event nobody there considers plausible. Starting from the fatal loss and working backwards does not depend on having imagined the cause.

The output is a description of the firm's actual vulnerability, stated concretely: this set of moves, held for this long, with this position, is what would do it.

That is directly actionable in a way a percentile is not. It gives a specific thing to hedge, a specific concentration to reduce, or an explicit decision to accept a known failure mode, which is a legitimate choice provided it is made deliberately.

The pattern in the third column is that every tool except the last depends on someone having anticipated the danger, either in the sample, in the distributional form, or in the scenario list.

That is the argument for running the whole set rather than choosing among them. They fail in different places, and the combination is informative precisely because the failures do not overlap.

What none of them does is stop a loss. Measuring risk accurately and surviving it are different problems, and the remaining two lessons are about the second.

Pulling this together gives a defensible stance on risk models, avoiding both of the common overreactions.

The first overreaction is to trust the number. A risk figure is a summary of an estimated distribution, and the estimate is wrong in known directions: too thin in the tails, too stable in its volatility, too optimistic about diversification.

The second is to dismiss the number. The failures are systematic, which means they are known and can be compensated for. A model that is understood to understate tails is more useful than no model, because the direction of the error is information.

The workable position is to treat risk measures as instruments with a known bias. Read them for change rather than level: a doubling of expected shortfall is meaningful even if its absolute value is not. Pair them with stress tests that do not depend on the distribution. And size positions so that being wrong about the tail is survivable.

That last clause is doing most of the work, and it belongs to position sizing rather than to measurement. It is where this path ends.