The 3D Representation Zoo: Meshes, NeRFs, Gaussians, SDFs

Every modern 3D AI system — Meshy, Tripo, Rodin, Hunyuan3D, Trellis — has to answer one question before any clever modelling: what is the output, mathematically? Pick the wrong representation and the model is either impossible to train, ugly to render, or useless to the downstream pipeline.

The choices are not interchangeable. A NeRF can produce a stunning photoreal flythrough but ships zero polygons into Unreal Engine. A mesh is what a game, a 3D printer, or an animator can actually consume. A 3D Gaussian Splat is the fastest way to capture a real-world scene today, but you still need to convert it to a mesh to do anything that isn't view-only rendering. The rest of this lesson is the map of those tradeoffs.

A polygon mesh is the production format for 3D. It's three lists:

• Vertices — points in 3D space (x, y, z).
• Edges — connections between two vertices.
• Faces — usually triangles, sometimes quads, defined by 3-4 vertex indices.

Plus auxiliary buffers: per-vertex normals, per-vertex UV coordinates (the 2D map for textures), and material assignments.

# A unit triangle, the smallest valid mesh.
vertices = [[0,0,0], [1,0,0], [0,1,0]]
faces    = [[0, 1, 2]]

A modern character is ~30k-100k triangles. A hero film asset can be in the millions. The thing that matters for AI generation: meshes are discrete and unstructured — variable face counts, variable topology — which makes them hostile to a vanilla feed-forward neural network. That single difficulty drives most of the architectural exotica we'll meet later.

Two representations that dominated 3D deep learning before 2022 and have largely been displaced for generation, but still matter as intermediates.

• Point clouds. Just a set of 3D points, optionally with color or normal. Easy to learn (PointNet, PointNet++), trivial to sample. The catch: no surface, no topology. You can render dots, but you can't ray-trace, simulate, or print until you reconstruct a surface from the cloud (Poisson reconstruction, ball-pivoting, etc.).
• Voxels. A 3D grid where each cell is occupied / empty / has a signed distance. Conceptually clean — it's just a 3D image, so you can throw 3D convolutions at it. The fatal cost: memory is $O(n^3)$. A 512³ voxel grid is 128M cells. That ceiling killed voxels as a final output representation, though they're still used inside generators as a coarse latent.

Both are useful intermediates: many native-3D generators internally produce a point cloud or voxel volume and then extract a mesh from it.

An SDF represents a shape implicitly: a function $f(x, y, z) \to \mathbb{R}$ that returns the signed distance from a point to the nearest surface (negative inside, positive outside, zero on the surface). The surface is the zero level set of $f$.

Why this is useful for AI:

• Continuous. No discretization artifacts; you can sample at any resolution.
• Differentiable. The whole training pipeline is gradient-friendly.
• Compact. A small neural network (DeepSDF, occupancy networks) can encode an entire shape.

The related occupancy field replaces signed distance with a binary inside/outside probability — same idea, simpler target.

The price you pay: to render or use an SDF, you have to either ray-march it (slow) or extract a mesh from it (with marching cubes, a 1987 algorithm still doing heavy lifting in 2026). Many native-3D generators output an SDF and run marching cubes as the final step.

A Neural Radiance Field (NeRF, 2020) represents a scene as a function $F_\Theta(x, y, z, \theta, \phi) \to (\text{RGB}, \sigma)$ — at each 3D point and viewing direction, return a color and a density. Render by integrating along camera rays. The breakthrough wasn't 3D generation; it was novel view synthesis — give me 30 photos of a statue and I'll show you a smooth flythrough, with view-dependent reflections, in photoreal quality.

What NeRF actually changed:

• It legitimized implicit, neural scene representations as a production option.
• It seeded the explosion of "radiance field" methods that dominate scene capture today.
• It exposed the painful gap: stunning to render, expensive to train (hours), and not a mesh — useless if you need physics or editing.

Most commercial "NeRF" tools by 2026 are actually using its faster successor, 3D Gaussian Splatting, but the conceptual debt is to NeRF.

3D Gaussian Splatting (3DGS, SIGGRAPH 2023) represents a scene as millions of small 3D Gaussians — each one a point with a position, color, opacity, and a 3x3 covariance that gives it size and orientation. Rendering is rasterizing these splats, fast and differentiable, producing 100+ FPS where NeRF managed seconds per frame.

Why 3DGS won the capture battle in 2024-2025:

• Real-time rendering, even on mobile.
• Fast training — minutes to a high-quality scene, vs. NeRF's hours.
• Plug into existing engines as a custom rasterizer.

What it still isn't, as of 2026:

• Not a mesh. Same downstream problem as NeRF.
• Not great for sharp surfaces. Gaussians are fuzzy by nature — edges, thin features, and small text are hard.
• Hard to edit. "Move that chair" is much harder than it would be on a mesh.

A growing class of methods (SuGaR, 2DGS, GOF) constrains the Gaussians to align with surfaces and extracts a mesh from the result. That's how 3DGS capture flows into a mesh-output pipeline.

Decision flow for picking a representation in 2026.

Almost every modern 3D generator works internally with a non-mesh representation (SDF, voxel grid, latent triplane, Gaussian cloud) and faces the same final step: turn it into a mesh.

The workhorses:

• Marching cubes (1987). Given a scalar field on a 3D grid, walk every cube and stitch in triangles wherever the field crosses zero. The default for SDFs. Output topology is determined by the grid resolution; finer grids → more triangles.
• Dual Contouring. Like marching cubes, but uses Hermite data to preserve sharp features. Better edges, more complex.
• Shape As Points / SAP. Learn a differentiable Poisson reconstruction from points to surface. Modern, but heavier.
• Surface-aware 3DGS (SuGaR, 2DGS). Constrain Gaussians during training so a clean mesh falls out at the end.

Mesh extraction is rarely the bottleneck on quality — it's the representation upstream that determines whether the result is clean or noisy. But it's where every pipeline finally meets the production format.

Despite the radiance-field revolution, the production downstream of every 3D AI tool that sells today (Meshy, Tripo, Rodin, CSM, Luma Genie) ends in a mesh. The reasons are stubborn and structural.

• Game engines. Unreal and Unity import meshes. Native 3DGS support is improving but not the default authoring format.
• Animation. Rigging, skinning, and blend shapes are all defined on a mesh's vertices. Animating a NeRF or a Gaussian cloud is an active research problem, not a production one.
• 3D printing and CAD. Slicers want a watertight mesh (STL/OBJ/PLY). No alternative.
• AR / mobile. GLB/glTF — meshes plus PBR textures — is the de facto interchange format and is what every AR framework consumes.
• Simulation. Physics engines compute collisions, fluids, and softbody on meshes.

The radiance-field methods are world-class for capture and rendering. Meshes are still world-class for everything else. A modern AI 3D platform has to live in both worlds and bridge them well.

The 2026 line of sight, in three trends.

• Hybrid representations. Models that output both a mesh and a Gaussian cloud (or an SDF and a Gaussian cloud), so the same asset can be used for high-quality rendering and for downstream production. SuGaR and 2DGS were the early proofs; current pipelines (Trellis, Hunyuan3D-2) treat dual output as standard.
• Native mesh generation closing the gap. Models like MeshGPT, MeshXL, and MeshAnything that emit triangles directly are starting to compete with the multi-view-and-reconstruct school on quality — and they produce much cleaner topology, which matters downstream. We'll look at them in lesson 3.
• Mesh + texture + rig in one pipeline. The product trend is bundling: Meshy added rigging in 2025, Rodin shipped fast PBR texturing, Tripo integrated retopology. The asset that exits a 2026 platform is more game-ready than the one that exited a 2023 platform — but "production-ready" still requires the cleanup steps we'll cover in lesson 5.