AI Texturing and PBR for Generated Meshes

A perfectly accurate mesh of a knight is still gray without textures. The geometry tells you what shape something is; the textures tell you what it's made of, how the light hits it, and whether it looks real or stylized. In production, the gap between 'AI-generated' and 'looks real' is mostly a texture gap.

This is why every modern 3D platform — Meshy, Tripo, Rodin, Hunyuan3D-2 — treats texturing as a first-class second stage, often with its own dedicated model, separate UX, and re-roll workflow. A user can change the texture without regenerating the shape, and most users will. The texture model is not an afterthought.

Modern real-time renderers (Unreal, Unity, Three.js, every AR engine) use Physically Based Rendering (PBR). A PBR material is a small bundle of 2D images, each describing one physical property:

• Albedo (Base Color). The 'true color' of the surface, with no lighting baked in.
• Normal map. Per-texel surface direction; gives the impression of bumps and detail without adding geometry.
• Roughness. 0 = mirror, 1 = chalk. Drives how blurry reflections are.
• Metallic. 0 = dielectric (plastic, wood, skin), 1 = metal. A binary in practice, gradients are stylization.
• Ambient Occlusion (AO). Darkening in crevices, computed once per asset.

Optional: emissive, height, opacity. Together these are usually called the PBR stack. The job of an AI texture pipeline is to produce all of them, aligned to the mesh's UV layout. Albedo only is the starter pack and what early AI texture tools shipped; full PBR is table stakes in 2026.

Before you can paint a mesh, you need a 2D map of its surface — a UV layout. Imagine cutting the mesh along its seams, peeling it flat, and packing the pieces into a square. Each vertex gets a (u, v) coordinate that maps to a pixel on the texture.

The constraints that make this hard:

• Minimize stretching. A face that gets squashed in UV space will show texture stretching when rendered.
• Minimize seams. Every cut is a visible discontinuity unless padded carefully.
• Pack efficiently. Wasted UV space is wasted texture resolution.

The automated options:

• XAtlas / UVAtlas — open-source automatic unwrappers, the default in most pipelines.
• Blender's Smart UV / Unwrap. Standard, decent quality, slow on dense meshes.
• Neural UV (recent). Models that learn to predict good cuts; promising on character meshes, mature for production by 2026 on hard-surface objects.

For AI-generated meshes the assumption is automatic unwrap, always. Asking the user to UV-unwrap manually defeats the point of the tool. The cost is occasional stretching artifacts visible in the final texture.

The dominant recipe for AI texturing in 2026 reuses the same idea that powered shape generation: multi-view diffusion.

1. Render the untextured mesh from N fixed viewpoints (4-8 typically).
2. For each view, generate a textured rendering with a depth-conditioned diffusion model (e.g. ControlNet on depth, or a custom multi-view model).
3. Project each view's pixels back onto the mesh — every visible texel gets a color from the view that sees it most directly.
4. Bake the projected colors into the UV map.
5. Repeat with the mesh now partially textured, in-painting the unseen regions.

Landmark papers in this line: TEXTure (2023), Text2Tex (2023), Paint3D (2024), MVDiffusion, TexFusion. Production texture pipelines (Meshy's, Tripo's) are evolved versions of this recipe with proprietary multi-view models and better consistency tricks.

The single hardest thing about multi-view texturing: adjacent views must agree. If the front view paints a creature's chest red and the side view paints it slightly orange, the seam between them will be visible forever.

The mitigations, in roughly increasing sophistication:

• Sequential generation with conditioning. Generate front first, then condition each subsequent view on the already-painted texture. Reduces but doesn't eliminate drift.
• Joint multi-view diffusion. A single diffusion process operating over all views simultaneously with cross-view attention. The Wonder3D / SyncDreamer trick, applied to textures.
• Bidirectional consistency loss. During training, penalize disagreement between view pairs after projection.
• Texture-space blending. After projection, blend overlapping regions using weighted averaging in UV space, with the weight proportional to the projection angle.
• Diffusion-time UV decoding. Newer work (e.g. UV3D-D, late-2025) runs diffusion directly in UV space after a few multi-view warm-up steps, avoiding seams entirely.

A well-tuned pipeline produces seams you have to look for. A poorly-tuned one produces visible bands every time the camera orbits.

From untextured mesh to PBR-ready GLB.

Diffusion-painted textures come out lit. The model was trained on rendered images that include shadows, highlights, and ambient occlusion baked into the pixels. If you ship that as the albedo map, every rendered view will look wrong — the new scene's lighting will fight with the lighting already painted into the texture.

The job of delighting is to recover the underlying base color from a lit photograph or render. Approaches in production:

• Delighting networks — supervised CNN/transformer models trained on (lit, unlit) pairs. The standard production trick.
• Diffusion-based delighting. A diffusion model conditioned on the lit image, generating the delit albedo. Higher quality, slower.
• Photometric stereo when multiple views with different lighting are available — classical, accurate, hard to obtain in practice.
• Skip delighting and embrace stylization. Many platforms ship with the lit look on purpose because it's cheaper and a chunk of users (concept art, indie games) actually prefer it.

A telltale sign of weak delighting: the asset looks 'painterly' under one lighting condition and obviously wrong under another. A good test is rotating an HDRI in the viewer and watching how the highlights move.

Albedo is half the work. The remaining PBR maps are produced by map-conditioned predictors rather than by separate generative runs.

• Normal map. Either baked from a higher-res version of the geometry (classic) or predicted by a network from the albedo + shading cues. Modern pipelines: a U-Net or diffusion model fed (albedo, geometry) and trained to output a tangent-space normal map.
• Roughness + metallic. Usually predicted jointly from the albedo, sometimes with auxiliary inputs like the prompt ('a rusty iron sword' implies roughness pattern + metallic = 1). Models like MaterialPaletteNet and PBR-GPT specialize in this.
• Ambient occlusion. Computed analytically from the mesh geometry — no learning needed. Standard libraries (xatlas-bake, Blender baking).

The quality bar in 2026: the platform-emitted PBR stack is good enough for prototyping, indie games, AR, and most product visualization. For hero AAA assets it still gets passed to a human texture artist for cleanup.

Once shape and texture are decoupled (as they are in Hunyuan3D-2 and every production tool), re-texturing without re-shaping is just another diffusion run.

Product features built on this:

• 'Re-roll texture' button. Same geometry, new texture seed. Lets a user try variations without losing the shape they liked.
• Style prompts. 'Make it look like jade', 'low-poly Wind Waker style', 'matte clay sculpt'. The texture diffusion conditioning supports this directly.
• Reference-image texturing. Drop in an image of a material — leather, hammered brass, knit fabric — and the texture model conditions on it.
• Bring-your-own-mesh re-texturing. Upload an existing OBJ, get an AI-generated texture. Both Meshy and Tripo expose this; it's a useful retrofit for unwrapped legacy assets.

The one feature that's harder than it looks: partial re-texturing — 'change just the helmet, keep the body'. Requires region-aware masking in UV space, which the more polished platforms have started shipping. Less polished tools require you to re-roll the whole texture and hope it preserves the parts you liked.

Honest gaps in 2026 between AI textures and human-painted ones.

• High-frequency detail in semantically right places. A painter knows the worn spots on a sword are on the edges and pommel, not the flat. AI texturing scatters wear globally.
• Stylized consistency. Matching a specific game's art style precisely — the AI averages to 'generic stylized'. Studios can fine-tune on their assets, but most platforms don't expose this to users yet.
• Material handoffs. Where leather meets metal, hand-authored textures have crisp boundaries with subtle bleed. Diffusion textures often produce soft mush at those boundaries.
• Tileable / seamless materials. Generated textures are per-asset, not tileable. Producing a tileable wood/brick/fabric texture is a different problem and a different family of models.
• Resolution at scale. Most pipelines top out at 2048² or 4096² per asset. Hero-grade textures are commonly 8192² with udim tiles. Closing this is bandwidth more than algorithm.

The practical reality: AI textures are excellent for prototyping, indie production, and storytelling; they're a starting point for AAA and product visualization where a texture artist still polishes the result.