Features: Stationarity Without Erasing the Memory

Statistical learning wants stationary inputs: a feature whose distribution is stable, so that a relationship fitted on old data still applies to new. A price series is not that. It wanders, has no fixed mean, and a model trained when a stock was at 40 has never seen 400.

The universal fix is to difference: model returns instead of prices. That works, and it costs something rarely acknowledged.

Key idea: Differencing removes the non-stationarity by removing the memory. A return series has almost no information about where the price actually is, and level matters: distance from a 52-week high, position within a range, whether a level has been tested before. The standard fix deletes exactly the features many strategies are built on.

The question this lesson answers is whether the trade has to be all or nothing. It does not.

Write the backshift operator $B$, where $B x_t = x_{t-1}$. First differencing is $(1 - B)$, and differencing twice is $(1 - B)^2$. Nothing requires the exponent to be a whole number.

Expand $(1 - B)^d$ as a binomial series for real $d$:

$(1 - B)^d = \sum_{k=0}^{\infty} \binom{d}{k}(-B)^k = 1 - dB + \frac{d(d-1)}{2!}B^2 - \frac{d(d-1)(d-2)}{3!}B^3 + \cdots$

At $d = 1$ the series terminates after two terms and you get the ordinary difference $x_t - x_{t-1}$. At $d = 0$ every term after the first vanishes and you get the series back unchanged. Between them the expansion does not terminate: the result is a weighted sum over the entire history, with weights that decay.

Definition: Fractional differentiation applies $(1-B)^d$ for non-integer $d$. It is a continuous dial between the original series and its first difference, and it is what makes stationarity a quantity you buy exactly as much of as you need.

The coefficients follow a simple recursion, which is all you need to implement it:

$w_0 = 1, \qquad w_k = -w_{k-1}\,\frac{d - k + 1}{k}$

import numpy as np

def weights(d, size):
    w = [1.0]
    for k in range(1, size):
        w.append(-w[-1] * (d - k + 1) / k)
    return np.array(w[::-1])

def frac_diff(series, d, thresh=1e-4):
    w = weights(d, len(series))
    w = w[np.abs(w) > thresh]          # fixed-width window
    width = len(w)
    out = np.full(len(series), np.nan)
    for i in range(width, len(series) + 1):
        out[i-1] = np.dot(w, series[i-width:i])
    return out

For $d$ between 0 and 1 the weights alternate in sign and shrink toward zero, so distant history contributes less and less. That decay is what makes the method usable: past some depth the weights are negligible and can be dropped, giving a fixed-width window rather than an expanding one.

Gotcha: Use a fixed-width window, not an expanding one. An expanding window applies a different number of terms at every point, so the transformed series has a drifting variance and is not comparable across time, which reintroduces a non-stationarity through the back door.

Take a simulated random-walk log price of 5,000 points, apply $(1-B)^d$ across a range of $d$, and measure the correlation between the transformed series and the original level.

At $d = 1$, ordinary differencing, the correlation with the original level is 0.02. Essentially all of the level information is gone, which is the cost nobody quotes when they take returns.

The curve is not linear. Most of the memory survives the first part of the range and collapses in the second half, which is precisely what makes a partial dose worthwhile.

In practice: The procedure is mechanical. Sweep $d$ upward in small steps, run an ADF test at each, and stop at the smallest $d$ whose statistic clears the critical value. That value is the minimum you must pay, and paying more is a pure loss.

One honest caveat about this number. The example is a clean random walk, where a very small $d$ suffices. Real price series carry additional structure, so the minimum $d$ is usually higher, often somewhere in the range of a few tenths. The procedure is unchanged; only the answer moves.

It is worth being concrete about what is lost, because if none of it matters to your strategy then returns are fine and this whole apparatus is unnecessary.

The last row is the sharpest case. A pairs or basket strategy trades the deviation of a spread from its equilibrium, so the level of the spread is the entire signal. Differencing it to stationarity in the ordinary way removes the strategy.

In practice: If your features are all short-horizon and momentum-flavoured, returns lose you little. If anything in your thesis is about where the price is rather than how it has been moving, ordinary differencing is deleting your signal before the model ever sees it.

Feature construction is where the future gets inserted into the past, usually through a library convenience rather than a conceptual error.

• Centred windows. A rolling mean with center=True uses future observations. It is the default in some plotting workflows and is fatal in a feature.
• Fitting the scaler on everything. Calling fit_transform on the full series before splitting leaks the future distribution's mean and variance into every training row.
• Full-sample statistics. Any normalisation by a global mean, a global standard deviation, or a full-history quantile uses data that did not exist at the time.
• Backfilled gaps. fillna(method='bfill') copies future values backwards, which is exactly what it says and rarely what was intended.
• Resampling and alignment. Aggregating to a coarser frequency can stamp a bar with a timestamp at the start of a period whose data runs to the end of it.

Gotcha: Every one of these produces a stronger model with a better validation score and no error message. The only reliable defence is a rule rather than an audit: every feature at time $t$ must be computable from a stream that has only seen data up to $t$, and the cheapest way to enforce it is to build features causally and test them against a replayed stream.

Having built features, the natural next step is to ask which ones matter. In this setting the standard tools mislead, and the reasons are worth knowing.

The substitution problem is the one that produces the worst decisions. Two highly correlated features split the importance between them, so each looks weak and both get dropped, removing a signal that either alone would have carried.

In practice: Cluster correlated features and measure importance at the cluster level rather than per feature. And whatever the method, compute it under the validation scheme from the next lesson: an importance ranking measured with a leaky split is a ranking of which features best exploit the leak.

The previous lesson treated non-stationarity as something to be transformed away. There is a second use for it: the moments when a series changes behaviour are themselves informative.

Statistical tests exist to detect this, including CUSUM tests on residuals and the supremum-augmented Dickey-Fuller family used to identify explosive or bubble-like behaviour. Two applications follow.

• As a sampling trigger. Label bars where a break is detected, rather than on a clock. This connects to the event-based sampling from the labelling lesson and concentrates the dataset on moments where something changed.
• As a feature. Time since the last detected break, or the current test statistic, tells a model which regime it is standing in.

Key idea: Fractional differentiation and break detection are two responses to the same fact and they are not alternatives. One makes a series usable by a model that assumes stability; the other tells the model when that assumption has just stopped holding. Using both is normal, and the regime question they open is the subject of a separate course in this catalogue.

Putting it together, the order of operations is itself a design decision.

1. Sample. Choose bars, on activity or events rather than on the clock by default.
2. Transform for stationarity. Sweep $d$, take the smallest value that passes an ADF test, and use a fixed-width window. Record the chosen $d$: it is a parameter of your dataset and should be reported alongside results.
3. Build features causally. Every value at time $t$ from data at or before $t$. No centred windows, no full-sample statistics, no backfill.
4. Fit any transform inside the fold. Scalers, PCA, quantile bins and imputers are all model components and must be fitted on training data only.
5. Assess importance under purged validation, at the cluster level for correlated features.

Gotcha: Step 4 is the one that is skipped most often, because scaling feels like preprocessing rather than modelling. It is not. A scaler carries the mean and variance of whatever it was fitted on, and fitting it before the split hands the test set's distribution to the training set. The next lesson makes the splitting itself the subject.