Convolution and Filters: Shaping a Signal

A system is linear if scaling the input scales the output and inputs add independently. It is time-invariant if delaying the input just delays the output. Amplifiers, cables, rooms, lenses and every filter in this lesson are close enough to both.

That pair of properties has a consequence out of proportion to how modest it sounds. Poke the system once with a single unit impulse, record what comes back, and you know everything about it. Any other input is a sum of scaled, shifted impulses, so its output is the same sum of scaled, shifted copies of that one recording.

Definition: The impulse response $h[n]$ is the output of an LTI system given a single unit impulse. It is a complete description. Clap once in a cathedral and the recording of the echo is that cathedral's impulse response, which is exactly how convolution reverb plugins work.

Applying the system is the operation of laying a flipped copy of the impulse response over the signal at every position and summing the overlap:

$y[n] = (x * h)[n] = \sum_{k} x[k] \, h[n - k]$

The flip is not decoration. The sample that arrived first has had the longest to be processed, so it is furthest along the impulse response by the time you read the output.

The cost is visible in the formula: every output sample touches every tap of $h$. For a signal of length $N$ and a kernel of length $K$, that is $N \times K$ multiply-accumulates. A 10,000-tap filter on a minute of 48 kHz audio is 29 billion operations, which is where the next idea earns its place.

Convolution in one domain is multiplication in the other:

$x * h \quad \Longleftrightarrow \quad X \cdot H$

This is the payoff from the frequency-domain lesson. Sinusoids pass through an LTI system unchanged in frequency, so in the frequency domain the system does nothing but scale each component by a complex number. That number, as a function of frequency, is $H$: the frequency response, and it is just the Fourier transform of the impulse response.

Key idea: "Filter" and "impulse response" and "frequency response" are three views of one object. Designing a filter means choosing a shape for $H$; running it means either convolving with $h$ or multiplying by $H$. Which route is cheaper is an arithmetic question with a definite answer.

Direct convolution costs $N \times K$. The frequency-domain route costs three transforms, forward on the signal, forward on the kernel, inverse on the product, and one cheap multiply, so its cost barely depends on $K$ at all.

Under this model the crossover is a kernel of about 126 taps. Below it, the loop wins and the transform overhead is wasted; above it, the FFT route pulls away without limit. That is why short kernels are convolved directly and long reverb impulse responses, which run to hundreds of thousands of taps, are never applied any other way.

IIR filters are dramatically cheaper: an 8th-order Butterworth does work that would take an FIR hundreds of taps. You pay for it with feedback, which means poles that can sit outside the unit circle and make the filter explode, and phase that varies with frequency.

FIR is the default when correctness matters more than cycles, which in software is most of the time.

A filter's effect on phase decides whether it delays every frequency by the same amount. If it does not, components arrive misaligned and a sharp edge turns into a lopsided wobble even though the magnitude response is exactly what you asked for.

Group delay, $-d\phi/d\omega$, is the quantity to watch. Constant group delay means the whole signal is delayed and otherwise intact. A symmetric FIR kernel achieves this exactly, with a delay of $(K-1)/2$ samples.

In practice: For measurement, ECG traces, or anything where waveform shape is the data, use a linear-phase FIR, or run an IIR forwards then backwards (scipy.signal.filtfilt) so the phase distortion cancels. The second trick doubles the effective filter order and is non-causal, so it works offline and never in real time.

The obvious design is a brick wall: pass everything below the cutoff untouched, block everything above it completely.

The ripple is not a small numerical blemish. Truncating the series overshoots the edge of a step by about 9 percent of the jump, and that overshoot does not shrink as you add taps, it only gets narrower. This is the Gibbs phenomenon, and it is the same effect that made Fourier's contemporaries suspicious of his square wave.

Averaging the last $M$ samples is the filter everybody reaches for first. Its impulse response is a rectangle of height $1/M$, so its frequency response is the transform of a rectangle: a main lobe with slowly decaying ripples either side.

Those ripples are the problem. The first one sits only 13.3 dB below the passband, so a moving average lets through roughly a fifth of the amplitude of frequencies it is supposed to be rejecting, and it does so at specific frequencies rather than uniformly. Worse, the response has exact nulls at multiples of $f_s/M$ and peaks between them, so which noise it removes depends on arithmetic nobody checked.

Gotcha: A moving average is a rectangular window wearing a different hat. Every criticism of rectangular windows in the next lesson applies to it directly. If you need smoothing, a short designed low-pass filter costs about the same and behaves predictably.

You do not derive taps by hand. You state a specification and let a design routine solve for it.

import numpy as np
from scipy import signal

fs = 1000
# 101-tap linear-phase low-pass, cutoff 100 Hz, Hamming-windowed
taps = signal.firwin(101, cutoff=100, fs=fs, window="hamming")
y = signal.lfilter(taps, 1.0, x)          # group delay = 50 samples

# IIR equivalent: 4th-order Butterworth, zero net phase distortion
sos = signal.butter(4, 100, btype="low", fs=fs, output="sos")
y2 = signal.sosfiltfilt(sos, x)

The specification is four numbers, and they trade against each other: passband ripple, stopband attenuation, transition width, and filter length. Tighten any one and at least one other must give. Asking for 80 dB of rejection across a 1 Hz transition at a 1 kHz sample rate will produce either a very long filter or a design failure, and both are the specification telling you something.

The same operation runs far outside audio, because the definition never mentioned time.

• Image processing. A Gaussian blur is a 2D convolution; sharpening subtracts a blurred copy; edge detection convolves with a kernel that responds to change. Separable kernels split one 2D convolution into two 1D ones, cutting cost from $K^2$ to $2K$ per pixel.
• Convolutional networks. The same operation with learned rather than designed kernels. Most libraries implement cross-correlation, the unflipped version, which is harmless because the weights are learned either way.
• Physics and probability. The distribution of a sum of two independent random variables is the convolution of their distributions, which is why characteristic functions, Fourier transforms of distributions, multiply.

Key idea: Convolution is what "applying a local, uniform rule everywhere" looks like written as arithmetic. Whenever a system treats every position alike and responds to a neighbourhood, you are convolving.