A Deep Dive into Nanite Virtualized Geometry

For real. The talent in the game industry is insane. Recently, I got my mind blown by the Unreal Nanite team: [VIDEO] The core of it is a respectably sophisticated LOD building algorithm, as you might expect, but the sheer amount of engineering horsepower they put into driving it into production just boggles the mind.

If this UE5 demo is using Nanite, then it really may well just be modeled geometry. Nanite really blurs the line between geometry and texture - in a sense it’s a shader that uses triangle mesh data as if it were a texture source. This siggraph session will expand your mind: [VIDEO]

It attracts truly extraordinary talent. One of many examples: [VIDEO] . There is more technical ambition in a single slide of that project than an entire department at places I've worked. However, it also tends to demand more and pay less, so there is churn and burnout and when it comes time to triage, the triage can cut very deep. If I were to dive into the JIRA for any one of these "missing compression" incidents, I wouldn't expect to find an easy win, I would expect to find "missing compression" beneath a pile of much worse bugs like "level 2 crashes on AMD cards." So it goes.

Nanite has a large pre-render step, but it also changes the rendering quite significantly. (Which in some cases includes a software rasterizer running on the GPU) This is a really good deep dive if you’re interested: [VIDEO]

Their YouTube channel has the excellent deep dive on the unreal engine's nanite virtual geometry system: [VIDEO] The one on idTech8 global illumination from last year is pretty interesting too.

If you want to see this sort of brilliant hacking on a modern system, I recommend taking a closer look at Unreal Nanite. It isn't just auto-LOD. Oh no. That's the core, but they had to do this same sort of "work around the tools they are given" to make it actually happen. Tiny triangles chug in modern GPU hardware raster pipelines -- so they wrote a GPGPU software rasterizer. They needed more bandwidth, so they wrote a compression engine. Their shaders needed better scheduling, so they abused the Z-test to do it. It's nuts! [VIDEO]

No, but it is a full time job. It's getting much more complicated. Here's an hour long talk on how Unreal Engine 5's Nanite works.[1] This is a huge breakthrough in level of detail theory. Things that used to be O(N) are now O(1). With enough pre-computation, rendering cost does not go up with scene size. See this demo.[2] You can download the demo for Xbox X/S and PlayStation 5, and soon, in source form for PCs. Explore 16 square kilometers of photorealistic city in which you can see a close-up view of anything. The format of assets in memory for Nanite becomes far more complicated. GPUs need a redesign again; UE5 has to do more of the rendering on the CPUs than they'd like. What they need to do is simple but not well suited to current GPU designs. It's such a huge win this approach will take over, despite that. New game engines will need to have something comparable. Maybe using the same format. Epic doesn't seem to be making patent claims, and the code for all this is in the Unreal Engine 5 download. (Basic concept of how this works: A mesh becomes a hierarchy of meshes, allowing zoom in and out of on level of detail. The approach to subdivision is seamless. You divide the mesh into cells of about 20 triangles. Then, each cell is cut by one or two new long edges across the cell. These become the boundaries of a simpler cell with about half as many triangles, but no new vertices. The lower-resolution cell has no more edges than the higher-resolution cell. There's a neat gra

Yeah, but they had to go through hell to make it happen. It took a lot more effort than you might think. [VIDEO] Some highlights: Seam staggering between levels for smooth swaps Current hardware rasterizers chug on small polys so they made a GPU software rasterizer (!) Actually it's a hybrid rasterizer, and gnarly features like derivatives are supported on both sides The GPU scheduler chugged so they abused Z-test wave repacking Streaming compression