Jul 29, 2013

SIGGRAPH Follow-Up

So I’m hoping that if you’re reading this, you’ve already attended or read the slides from my presentation about The Order: 1886 that was part of the Physically Based Shading Course at SIGGRAPH last week. If not, go grab them and get started! If you haven’t read through the course notes already there’s a lot of good info there, in fact there’s almost 30 pages worth! The highlights include:

Full description of our Cook-Torrance and Cloth BRDF’s, including a handy optimization for the GGX Smith geometry term (for which credit belongs to Steve McAuley)
Analysis of our specular antialiasing solution
Plenty of details regarding the material scanning process
HLSL sample code for the Cook-Torrance BRDF’s as well as the specular AA roughness modification
Lots of beautiful LaTeX equations

If you did attend, I really appreciate you coming and I hope that you found it interesting. It was my first time presenting in front of such a large audience, so please forgive me if I seemed nervous. Also I’d like to give another thank you to anyone that came out for drinks later that night, I had a really great time talking shop with some of the industry’s best graphics programmers. And of course the biggest thanks goes out to Stephen Hill and Stephen McAuley for giving us the opportunity to speak at the best course at SIGGRAPH.

Anyhow…now that SIGGRAPH is finished and I have some time to sit down and take a breath, I wanted to follow up with some additional remarks about the topics that we presented. I also thought that if I post blogs more frequently, it might inspire a few other people to do the same.

Physically Based BRDF’s

I didn’t even mention anything about the benefits of physically based BRDF’s in our presentation because I feel like it’s no longer an issue that’s worth debating. We’ve been using some form of Cook-Torrance specular for about 2 years now, and it’s made everything in our game look better. Everything works more like it should work, and requires less fiddling and fudging on the part of the artists. We definitely had some issues to work through when we first switched (I can’t tell you how many times they asked me for direct control over the Fresnel curve), but there’s no question as to whether it was worth it in the long run. Next-gen consoles and modern PC GPU’s have lots of ALU to throw around, and sophisticated BRDF’s are a great way to utilize it.

Compositing

First of all, I wanted to reiterate that our compositing system (or “The Smasher” as it’s referred to in-house) is a completely offline process that happens in our content build system. When a level is a built we request a build of all materials referenced in that level, which then triggers a build of all materials used in the compositing stack of that material. Once all of the source materials are built, the compositing system kicks in and generates the blended parameter maps. The compositing process itself is very quick since we do it in a simple pixel shader run on the GPU, but it can take some time to build all of the source materials since doing that requires processing the textures referenced by that material. We also support using composited materials as a source material in a composite stack, which means that a single runtime material can potentially depend on a arbitrarily complex tree of source materials. To alleviate this we aggressively cache intermediate and output assets on local servers in our office, which makes build times pretty quick once the cache has been primed. We also used to compile shaders for every material, which caused long build times when changing code used in material shaders. This forced us to change things so that we only compile shaders directly required by a mesh inside of a level.

We also support runtime parameter blending, which we refer to as layer blending. Since it’s at runtime we limit it to 4 layers to prevent the shader from becoming too expensive, unlike the compositing system where artists are free to composite in as many materials as they’d like. It’s mostly used for environment geo as a way to add in some break-up and variety to tiling materials via vertex colors, as opposed to blending in small bits of a layer using blend maps. One notable exception is that we use the layer blending to add a “detail layer” to our cloth materials. This lets us keep the overall low-frequency material properties in lower-resolution texture maps, and then blend in tiling high-frequency data (such as the textile patterns acquired from scanning).

One more thing that I really wanted to bring up is that the artists absolutely love it. The way it interacts with our template library allows us to keep low-level material parameter authoring in the hands of a few technical people, and enables everyone else to just think in terms of combining high-level “building blocks” that form an object’s final appearance. I have no idea how we would make a game without it.

Maya Viewer

A few people noticed that we have our engine running in a Maya viewport, and came up to ask me about it. I had thought that this was fairly common and thus uninteresting, but I guess I was wrong! We do this by creating a custom Viewport 2.0 render override (via MRenderOverride) that uses our engine to render the scene using Maya’s DX11 device (Maya 2013.5 and up support a DX11 mode for Viewport 2.0, for 2013 you gave to use OpenGL). With their their render pass API you can basically specify what things you want Maya to draw, so we render first (so that we can fill the depth buffer) and then have Maya draw things like HUD text and wireframe overlays for UI. The VP 2.0 API actually supports letting you hook into their renderer and data structure, which basically lets you specify things like vertex data and shader code while allowing Maya to handle the actual rendering. We don’t do that…we basically just give Maya’s device to our renderer and let our engine draw things the same way that it does in-game. To do this, we have a bunch of Maya plugin code that tracks objects in the scene (meshes, lights, plugin nodes, etc.) and handles exporting data off of them and converting them to data that can be consumed by our renderer. Maintaining this code has been a lot of work, since it basically amounts to an alternate path for scene data that’s in some ways very different from what we normally do in the game. However it’s huge workflow enhancement for all of our artists, so we put up with it. I definitely never thought I would know so much about Maya and its API!

Some other cool things that we support inside of Maya:

We can embed our asset editors (for editing materials, particle systems, lens flares, etc.) inside of Maya, which allows for real-time editing of those assets while they’re being viewed
GI bakes are initiated from Maya and then run on the GPU, which allows the lighting artists to keep iterating inside of a single tool and get quick feedback
Our pre-computed visibility can also be baked inside of Maya, which allows artists to check for vis dropouts and other bugs without running the game

One issue that I had to work around in our Maya viewer was using the debug DX11 device. Since Maya creates the device we can’t control the creation flags, which means no helpful DEBUG mode to tell us when we mess something up. To work around this, I had to make our renderer create its own device and use that for rendering. Then when presenting the back buffer and depth buffer to Maya, we have to use DXGI synchronization to copy texture data from our device’s render targets to Maya’s render targets. It’s not terribly hard, but it requires reading through a lot of obscure DXGI documentation.

Sample Code

You may not have noticed, but there’s a sample app to go with the presentation! Dave and I always say that it’s a little lame to give a talk about something without providing sample code, so we had to put our money where our mouth is. It’s essentially a working implementation of our specular AA solution as well as our Cook-Torrance BRDF’s that uses my DX11 sample framework. Seeing for yourself with the sample app is a whole lot better than looking at pictures, since the primary concern is avoiding temporal aliasing as the camera or object changes position. These are all of the specular AA techniques available in the sample for comparison:

LEAN
CLEAN
Toksvig
Pre-computed Toksvig
In-shader vMF evaluation of Frequency Domain Normal Map Filtering
Pre-computed vMF evaluation (what we use at RAD)

For a ground-truth reference, I also implemented in-shader supersampling and texture-space lighting. The shader supersampling works by interpolating vertex attributes to random sample points surrounding the pixel center, computing the lighting, and then applying a bicubic filter. Texture-space lighting works exactly like you think it would: the lighting result is rendered to a FP16 texture that’s 2x the width and height of the normal map, mipmaps are generated, and the geometry is rendered by sampling from the lighting texture with anisotropic filtering. Since linear filtering is used both for sampling the lighting map and generating the mipmaps, the overall results don’t always look very good (especially under magnification). However the results are completely stable, since the sample positions never move relative to the surface being shaded. The pixel shader supersampling technique still suffers from some temporal flickering because of this, although it’s obviously significantly reduced with higher sample counts.

Dave and I had also intended to implement solving for multiple vMF lobes, but unfortunately we ran out of time and we weren’t able to include it. I’d like to revisit it at some point and release an updated sample that has it implemented. I don’t think it would actually be worth it for a game to store so much additional texture data, however I think it would be useful as a reference. It might also be interesting to see if the data could be used to drive a more sophisticated pre-computation step that bakes the anisotropy into the lower-resolution mipmaps.

Like I mentioned before, the sample also has implementations of the GGX and Beckmann-based specular BRDF’s described in our course notes. We also implemented our GGX and Cloth BRDF’s as .brdf files for Disney’s BRDF Explorer, which you can download here.

Comments:

Sam Martin - Jul 1, 2013

This really is an exceptionally useful set of resources you’ve put together here. I saw your talk at siggraph and was very impressed. It’s clearly had a lot of love and effort put into it!

#### [Simon Brown]( "") - May 4, 2014

Gah, cannot type, I mean 1/(2*kappa) in course notes and 1/kappa in example project…

#### [Simon Brown]( "") - May 4, 2014

Great resource! I’m a bit confused by a difference between the course notes and sample code when computing effective roughness using existing roughness and kappa. The Han et el paper and course notes lists sigmap=sqrt(sigma*sigma + 2/kappa), but the sample code consistently uses sigmap=sqrt(sigma*sigma + 1/kappa) in GenerateMaps.hlsl and Mesh.hlsl. Is this a typo or the application of some errata? Thanks!

#### [MJP](http://mynameismjp.wordpress.com/ "mpettineo@gmail.com") - Apr 0, 2014

Hi Kostas, I’m glad you’ve found it useful! I prefer to keep the division by Pi in the diffuse term, as opposed to baking that into the light intensity and then multiplying the specular term by Pi. For me that just fits with my mental model of how I think about the diffuse BRDF. In practice I don’t think it should matter as long as you’re careful with multiplying your Pi factor through, so that you have the right relative intensity between your diffuse and specular. -Matt

#### [Readings on Physically Based Rendering | Interplay of Light](http://interplayoflight.wordpress.com/2013/12/30/readings-on-physically-based-rendering/ "") - Dec 1, 2013

[…] Pettineo’s follow up to the very interesting The Order: 1886 Siggraph 2013 presentation expands on some PBR topics and […]

#### [Kostas Anagnostou](http://interplayoflight.wordpress.com "kostas.anagnostou@gmail.com") - Apr 0, 2014

Hi Matt, thanks for the great presentation and following write-up as well as the sample application which is extremely useful! A quick question, in the sample’s implementation of the BRDFs, the division by Pi is still applied even for punctual lights. Shouldn’t it be removed by the Pi in the lighting equation for punctual lights? (or I am reading something wrong?).

#### [MJP](http://mynameismjp.wordpress.com/ "mpettineo@gmail.com") - May 5, 2014

Hi Simon, I think that’s just a typo that we never noticed. Thank you for pointing it out! -Matt

#### [Notes on Anti-Aliasing : Texture filtering (part 3) « Mehdi NS](https://mehdins.wordpress.com/2017/08/29/notes-on-anti-aliasing-texture-filtering-part-3/ "") - Aug 1, 2017

[…] implementation details, the reference is MJP specularAA project, in which the main specular anti aliasing techniques used in games are […]