Lumen Overview

Two angles for Lumen

To better understand the design philosophy of Lumen, I recommend imagining its working principles from two perspectives:

  • Trace: Lumen employs ray tracing technology for global illumination calculations.
  • Fall-back: In order to make the most of existing information and save computational resources, Lumen is designed with a series of fallback processes from screen-space tracing to global distance field tracing. Instead of performing tracing entirely, this approach helps to save resources.

Among them, Trace is the core method used by Lumen. However, ray tracing inevitably needs to solve the problem of excessive consumption. That is:

  • If ray tracing is performed on the entire scene for every frame and every pixel, Lumen will not be able to achieve 30 frames per second performance on mainstream GPUs.

In order to meet the performance requirements, Lumen's choice is simplification. That is:

  • Sacrificing a certain degree of accuracy → reducing computational complexity → improving performance

In terms of reducing computational complexity, Lumen adopts two approaches:

  1. It utilizes the common sampling technique in ray tracing algorithms to replace the calculation of multiple pixels with the calculation of key sample points.
  2. It employs the fall-back concept, which we will discuss in the following process.

Trace

image

When we discuss Ray Tracing, we should actually discuss two main topics:

  1. How to determine if a ray hits an object and determine the point of intersection.
  2. How to calculate the lighting information at the intersection point.

The first topic is about Hit checking, while the second topic is about lighting calculations.

ℹ️
It should be noted that the answer to these two questions is different for each level of Fall-back, and I will explain in detail in the following content.

Cost of the light calculation

It should be noted that the calculation of the lighting for the hit points is a very resource-intensive process.

This consumption mainly comes from the following points:

  1. The materials of the hit points may be completely different, which means that we need to load corresponding material shaders, textures, and other resources in order to calculate the lighting result for a point.
    2. If we want to evaluate all three points, we need to load all three shaders and all referenced textures
    If we want to evaluate all three points, we need to load all three shaders and all referenced textures
  2. Light rays will bounce repeatedly in the environment, so the hit points may be distributed in various areas of the scene, and the number of hit points may even exceed the number of pixels on the screen.
    3. Although we cannot see the yellow area, but in order to calculate the lighting bouncing, we still need to calculate the rays hit this area
    Although we cannot see the yellow area, but in order to calculate the lighting bouncing, we still need to calculate the rays hit this area

Therefore, it is necessary to introduce a Cache system to cache the results of this calculation. Different Fall-back mechanisms use different Cache solutions. This will be further discussed in the Fall-back section.

Fall-back

image

I have divided Lumen's Fall-back into 4 stages:

  • Screen Space: The current Ray hits a pixel in screen space that has already been rasterized.
  • Short Range: The current Ray does not hit a pixel in screen space, but instead hits some nearby cache information for short-range distances. In the case of SWRT, it is Mesh SDF, and in the case of HWRT, it is TLAS/BLAS.
    • Short Range Mesh SDF example, each mesh has its own SDF and not mixed
    Short Range Mesh SDF example, each mesh has its own SDF and not mixed
  • Mid Range: The cache for Short Range is not hit, but instead hits a cache for a slightly farther region. This distance range is larger than Short Range, but we can afford more precision loss due to the greater distance.
    • Mid-range Global SDF example. The nearby mesh’s SDF is mixed
    Mid-range Global SDF example. The nearby mesh’s SDF is mixed
  • Not Hit: Farther than Mid Range, at this point, it can be determined that there is no hit at all, and Skylight is directly sampled as the lighting result.

Caches

As mentioned earlier, in order to reduce the cost of calculating illumination for hit points, Lumen has multiple levels of Cache.

Screen space

For screen space, we already have the GBuffer information for the current position, so this information is the best Cache.

ℹ️
Due to the limited information available on the screen space, this cache is not always reliable. This has raised some issues with Lumen lighting. In certain situations, disabling the screen space solution may actually be a better choice.

Short-range Surface Cache

For slightly further distances, Lumen precomputes the Mesh and caches the illumination information in the form of Surface Cache.

image

World Space Radiance Cache

For even further distances, even the Surface Cache becomes costly to compute and store. Therefore, Lumen uses a sparser cache.

image