From Scene to Pixel: The Transform Chain

A scene is a set of surfaces in three dimensions. An image is a grid of pixels. Rendering is the map between them, and it splits cleanly into two problems.

The first is visibility: for this pixel, which surface is seen? The second is shading: given that surface, what colour is it? Shading is where the physics lives, and the lesson Rendering as an Integral treats it as the integral equation it really is.

This cursus is about the first problem, because visibility is what forces the architecture. The two dominant renderer designs are two different answers to it, and almost every performance characteristic of a graphics system follows from which answer it picked.

Before either answer can be given, the geometry has to be expressed in a space where the question is easy to ask. That is what the transform chain is for.

A vertex is authored once and then re-expressed several times on its way to the screen. Each space exists because some operation is trivial in it and awkward everywhere else.

Object space is where the model was built, with its own origin. World space places every object into one shared scene. View space re-anchors that scene on the camera, putting the eye at the origin looking down one axis, which is what makes the next step expressible at all. Clip space applies the projection. After dividing through, normalised device coordinates put everything visible inside a fixed cube. Screen space scales that to pixels.

The payoff of this staging is that each step is a matrix multiply, so the whole chain composes into a single matrix per object. A vertex is transformed once, not five times.

Everything up to clip space is a linear map, so those stages collapse into one matrix. The step that cannot collapse is the divide.

That asymmetry is the reason clip space exists as a named stage at all. Projection loads the fourth coordinate with the view-space depth, and the perspective effect is produced by dividing the other three by it. Division is not a linear operation, so it cannot be folded into the matrix product. It has to happen after.

Clipping is done before the divide rather than after, which is where the name comes from. Geometry behind the eye has a negative fourth coordinate, and dividing by it would fold those points back into view as though they were in front.

Points are carried as four numbers, (x, y, z, w), and the reason is structural rather than cosmetic.

Translation is not a linear map on three-dimensional vectors: no 3x3 matrix moves the origin, because a linear map always fixes it. Rotation and scale are linear, translation is not, so a chain mixing them cannot be one 3x3 product. Adding a fourth component with w = 1 embeds the points in a space where translation is linear, and a 4x4 matrix expresses all three uniformly.

The same extra component then does a second job. A perspective projection writes view-space depth into w, so the divide that follows scales x and y down in proportion to distance. That is the entire mechanism of perspective foreshortening: distant things are divided by a larger number.

The matrix looks arbitrary until you watch what each row does.

import math

def perspective(fov_y_deg, aspect, near, far):
    f = 1.0 / math.tan(math.radians(fov_y_deg) / 2)
    return [
        [f / aspect, 0,  0,                              0],
        [0,          f,  0,                              0],
        [0,          0,  (far + near) / (near - far),   (2 * far * near) / (near - far)],
        [0,          0, -1,                              0],   # w = -z_view
    ]

def project(m, p):
    x, y, z, w = (sum(m[r][c] * p[c] for c in range(4)) for r in range(4))
    return (x / w, y / w, z / w)          # the perspective divide

M = perspective(60, 16 / 9, 0.1, 100.0)
project(M, (1, 1, -2, 1))     # near   -> roughly (0.49, 0.87, -0.90)
project(M, (1, 1, -20, 1))    # far    -> roughly (0.05, 0.09,  0.99)

The same object at ten times the distance lands ten times closer to the centre. The last row is the whole trick: it copies negated view depth into w.

The x and y results are intuitive. The depth value is not, and this is where a real bug lives.

Because z is divided by w, and w is view depth, the stored depth is proportional to 1/distance rather than to distance. Run the numbers from the example above: an object at 2 units maps to about -0.90, and at 20 units to about 0.99. Ninety percent of the available depth range was consumed by the first tenth of the scene.

The consequence is uneven precision. Near the camera, depth values are finely separated. Far away they crowd together, so two surfaces a metre apart can round to the same stored value and flicker against each other as the camera moves. That is z-fighting.

The standard mitigations are to push the near plane out as far as the scene allows, and to use a floating-point depth buffer with the range reversed, which places the dense floating-point values where the crowding happens.

After this chain, every visible surface is described in a common cube, and each point carries a depth. The projection has been solved.

Visibility has not. Two surfaces can still land on the same pixel, and nothing so far decides which one wins. The next two lessons are the two ways to decide, and they differ in a single architectural choice: whether to walk the geometry and see which pixels it covers, or walk the pixels and see which geometry they hit.

It is worth noticing how much has been made cheap before any renderer-specific work begins.

The transform chain is per vertex, and a vertex is transformed by one matrix multiply regardless of how many pixels the triangle it belongs to eventually covers. A model with 50,000 vertices costs 50,000 small matrix products, whether it fills the screen or occupies four pixels.

That independence is what allows the same geometry front end to serve both architectures. Rasterisers and ray tracers disagree about visibility, not about projection, and hybrid systems exploit exactly that: the geometry is prepared once and then queried two different ways.

It also explains a common performance surprise. Adding vertices costs vertex work, adding screen coverage costs pixel work, and the two budgets are separate. A scene can be bound by either, and the fix for one does nothing for the other.

Four things in this chain break silently rather than loudly.

• Order of multiplication. The chain composes in a fixed order, and swapping two matrices produces a plausible-looking scene that is subtly wrong. Rotation then translation is not translation then rotation.
• Non-uniform scale and normals. Scaling an object unevenly and transforming its normals with the same matrix tilts them off the surface. Normals need the inverse transpose.
• The near plane is not free. Setting it very small to avoid clipping nearby geometry is exactly what destroys depth precision everywhere else.
• Clipping happens before the divide for a reason. Geometry behind the eye has negative w, and dividing by it silently maps those points into the visible range.

Each of these produces an image rather than an error, which is why they cost time.