three-gpu-pathtracer
Path tracing project using three-mesh-bvh and WebGL 2 to accelerate high quality, physically based rendering on the GPU. Features include support for GGX surface model, material information, textures, normal maps, emission, environment maps, tiled rendering, and more!
More features and capabilities in progress!
Examples
Setup
Beauty Demos
Features
Test Scenes
Model Viewer Fidelity Scene Comparisons
Tools
Looking Glass Portrait Quilt Render
Running examples locally
To run and modify the examples locally, make sure you have Node and NPM installed. Check the supported versions in the test configuration.
In order to install dependencies, you will need make
and a C++ compiler available.
On Debian or Ubuntu, run sudo apt install build-essential
. It should just work on MacOS.
- To install dependencies, run
npm install
- To start the demos run
npm start
- Visit
http://localhost:1234/<demo-name.html>
Use
Basic Renderer
import * as THREE from 'three';
import { FullScreenQuad } from 'three/examples/jsm/postprocessing/Pass.js';
import {
PathTracingSceneGenerator,
PathTracingRenderer,
PhysicalPathTracingMaterial,
} from 'three-gpu-pathtracer';
// init scene, renderer, camera, controls, etc
renderer.outputEncoding = THREE.sRGBEncoding;
renderer.toneMapping = THREE.ACESFilmicToneMapping;
// initialize the path tracing material and renderer
const ptMaterial = new PhysicalPathTracingMaterial();
const ptRenderer = new PathTracingRenderer( renderer );
ptRenderer.setSize( window.innerWidth, window.innerHeight );
ptRenderer.camera = camera;
ptRenderer.material = ptMaterial;
// if rendering transparent background
ptRenderer.alpha = true;
// init quad for rendering to the canvas
const fsQuad = new FullScreenQuad( new THREE.MeshBasicMaterial( {
map: ptRenderer.target.texture,
// if rendering transparent background
blending: THREE.CustomBlending,
} ) );
// ensure scene matrices are up to date
scene.updateMatrixWorld();
// initialize the scene and update the material properties with the bvh, materials, etc
const generator = new PathTracingSceneGenerator();
const { bvh, textures, materials, lights } = generator.generate( scene );
// update bvh and geometry attribute textures
ptMaterial.bvh.updateFrom( bvh );
ptMaterial.attributesArray.updateFrom(
geometry.attributes.normal,
geometry.attributes.tangent,
geometry.attributes.uv,
geometry.attributes.color,
);
// update materials and texture arrays
ptMaterial.materialIndexAttribute.updateFrom( geometry.attributes.materialIndex );
ptMaterial.textures.setTextures( renderer, 2048, 2048, textures );
ptMaterial.materials.updateFrom( materials, textures );
// update the lights
ptMaterial.lights.updateFrom( lights );
// set the environment map
const texture = await new RGBELoader().setDataType( THREE.FloatType ).loadAsync( envMapUrl );
ptRenderer.material.envMapInfo.updateFrom( texture );
animate();
// ...
function animate() {
// if the camera position changes call "ptRenderer.reset()"
// update the camera and render one sample
camera.updateMatrixWorld();
ptRenderer.update();
// if using alpha = true then the target texture will change every frame
// so we must retrieve it before render.
fsQuad.material.map = ptRenderer.target.texture;
// copy the current state of the path tracer to canvas to display
fsQuad.render( renderer );
}
Blurred Environment Map
Using a pre blurred envioronment map can help improve frame convergence time at the cost of sharp environment reflections. If performance is concern then multiple importance sampling can be disabled and blurred environment map used.
import { BlurredEnvMapGenerator } from 'three-gpu-pathtracer';
// ...
const envMap = await new RGBELoader().setDataType( THREE.FloatType ).loadAsync( envMapUrl );
const generator = new BlurredEnvMapGenerator( renderer );
const blurredEnvMap = generator.generate( envMap, 0.35 );
// render!
Dynamic Scenes
Using the dynamic scene generator the same, frequently updated scene can be converted into a single reusable geometry multiple times and BVH refit which greatly improves subsequent scene updates. See DynamicPathTracingSceneGenerator
docs for more info.
import { DynamicPathTracingSceneGenerator } from 'three-gpu-pathtracer';
// ... initialize scene etc
const generator = new DynamicPathTracingSceneGenerator( scene );
const { bvh, textures, materials } = generator.generate( scene );
// ... update path tracer and render
Asynchronous Scene Generation
NOTE WebWorker syntax is inconsistently supported across bundlers and sometimes not supported at all so the PathTracingSceneWorker class is not exported from the package root. If needed the code from src/worker can be copied and modified to accomodate a particular build process.
import { PathTracingSceneWorker } from 'three-gpu-pathtracer/src/workers/PathTracingSceneWorker.js';
// ...
// initialize the scene and update the material properties with the bvh, materials, etc
const generator = new PathTracingSceneWorker();
const { bvh, textures, materials, lights } = await generator.generate( scene );
// ...
Exports
PathTracingRenderer
Utility class for tracking and rendering a path traced scene to a render target.
.samples
readonly samples : Number
Number of samples per pixel that have been rendered to the target.
.target
readonly target : WebGLRenderTarget
The target being rendered to. The size of the target is updated with setSize
and is initialized to a FloatType texture.
Note Target will swap render targets after every full sample when alpha is enabled.
.camera
camera = null : Camera
The camera to render with. The view offset of the camera will be updated every sample to enable anti aliasing.
.material
material = null : ShaderMaterial
The Path Tracing material to render. This is expected to be a full screen quad material that respects the "opacity" field for every pixel so samples can be accumulated over time. The material is also expected to have cameraWorldMatrix
and invProjectionMatrix
fields of type Matrix4
.
.tiles
tiles = ( 1, 1 ) : Vector2
Number of tiles on x and y to render to. Can be used to improve the responsiveness of a page while still rendering a high resolution target.
.stableNoise
stableNoise = false : Boolean
Whether to reset the random seed to 0
when restarting the render. If true then a consistent random sample pattern will appear when moving the camera, for example.
.stableTiles
stableTiles = true : Boolean
Whether the initial tile is reset to the top left tile when moving the camera or if it should continue to shift every frame.
.alpha
alpha = false : Boolean
Whether to support rendering scenes with transparent backgrounds. When transparent backgrounds are used two extra render targets are used, custom blending is performed, and PathTracingRenderer.target will change on every completed sample.
Note When a transparent background is used the material used for the final render to the canvas must use the same "premultipliedAlpha" setting as the canvas otherwise the final image may look incorrect.
constructor
constructor( renderer : WebGLRenderer )
.setSize
setSize( size : Vector2 ) : void
Sets the size of the target to render to.
.update
update()
Renders a single sample to the target.
.reset
reset() : void
Resets and restarts the render from scratch.
QuiltPathTracingRenderer
Renderer that supports rendering to a quilt renderer to rendering on displays such as the Looking Glass display.
.viewCount
viewCount = 48 : Number
The number of views to be rendered. If this is less than the product of the quiltDimensions then there will be gaps at the end of the quilt.
.quiltDimensions
quiltDimensions = Vector2( 8, 6 ) : Vector2
The number of quilt patches in each dimension.
.viewCone
viewCone = 35 * DEG2RAD : Number
The total angle sweep for the camera views rendered across the quilt.
.viewFoV
viewFoV = 14 * DEG2RAD : Number
The camera field of view to render.
.displayDistance
displayDistance = 1 : Number
The distance of the viewer to the display.
.displayAspect
displayAspect = 0.75 : Number
The aspect ratio of the display.
.setFromDisplayView
setFromDisplayView(
displayDistance : Number,
displayWidth : Number,
displayHeight : Number,
) : void
Updates the displayDistance
, displayAspect
, and the viewFoV
from viewer and display information.
PathTracingSceneGenerator
Utility class for generating the set of data required for initializing the path tracing material with a bvh, geometry, materials, and textures.
.generate
generate( scene : Object3D | Array<Object3D>, options = {} : Object ) : {
bvh : MeshBVH,
materials : Array<Material>,
textures : Array<Texture>,
lights : Array<Light>
}
Merges the geometry in the given scene with an additional "materialIndex" attribute that references the associated material array. Also produces a set of textures referenced by the scene materials.
PathTracingSceneWorker
extends PathTracingSceneGenerator
See note in Asyncronous Generation use snippet.
.generate
async generate( scene : Object3D | Array<Object3D>, options = {} : Object ) : {
bvh : MeshBVH,
materials : Array<Material>,
textures : Array<Texture>,
lights : Array<Light>
}
.dispose
dispose() : void
PhysicalCamera
extends THREE.PerspectiveCamera
An extension of the three.js PerspectiveCamera with some other parameters associated with depth of field. These parameters otherwise do not affect the camera behavior are are for convenience of use with the PhysicalCameraUniform and pathtracer.
.focusDistance
focusDistance = 25 : Number
The distance from the camera in meters that everything is is perfect focus.
.fStop
fStop = 1.4 : Number
The fstop value of the camera. If this is changed then the bokehSize
field is implicitly updated.
.bokehSize
bokehSize : Number
The bokeh size as derived from the fStop and focal length in millimeters. If this is set then the fStop is implicitly updated.
.apertureBlades
apertureBlades = 0 : Number
The number of sides / blades on the aperture.
.apertureRotation
apertureRotation = 0 : Number
The rotation of the aperture shape in radians.
.anamorphicRatio
anamorphicRatio = 1 : Number
The anamorphic ratio of the lens. A higher value will stretch the bokeh effect horizontally.
EquirectCamera
extends THREE.Camera
A class indicating that the path tracer should render an equirectangular view. Does not work with three.js raster rendering.
PhysicalSpotLight
extends THREE.SpotLight
.radius
radius = 0 : Number
The radius of the spotlight surface. Increase this value to add softness to shadows.
.iesTexture
iesTexture = null : Texture
The loaded IES texture describing directional light intensity. These can be loaded with the IESLoader
.
Premade IES profiles can be downloaded from [ieslibrary.com]. And custom profiles can be generated using CNDL.
ShapedAreaLight
extends THREE.RectAreaLight
.isCircular
isCircular = false : Boolean
Whether the area light should be rendered as a circle or a rectangle.
DynamicPathTracingSceneGenerator
A variation of the path tracing scene generator intended for quickly regenerating a scene BVH representation that updates frequently. Ie those with animated objects or animated skinned geometry.
In order to quickly update a dynamic scene the same BVH is reused across updates by refitting rather than regenerating. This is significantly faster but also results in a less optimal BVH after significant changes.
If geometry or materials are added or removed from the scene then reset
must be called.
constructor
constructor( scene : Object3D | Array<Object3D> )
Takes the scene to convert.
.generate
generate() : {
bvh : MeshBVH,
materials : Array<Material>,
textures : Array<Texture>
}
Generates and refits the bvh to the current scene state. The same bvh, materials, and textures objects are returns after the initial call until reset
is called.
.reset
reset() : void
Resets the generator so a new BVH is generated. This must be called when geometry, objects, or materials are added or removed from the scene.
IESLoader
extends Loader
Loader for loading and parsing IES profile data. Load and parse functions return a DataTexture
with the profile contents.
BlurredEnvMapGenerator
Utility for generating a PMREM blurred environment map that can be used with the path tracer.
constructor
constructor( renderer : WebGLRenderer )
.generate
generate( texture : Texture, blur : Number ) : DataTexture
Takes a texture to blur and the amount to blur it. Returns a new DataTexture
that has been PMREM blurred environment map that can have distribution data generated for importance sampling.
.dispose
dispose() : void
Disposes of the temporary files and textures for generation.
GradientEquirectTexture
.exponent
exponent = 2 : Number
.topColor
topColor = 0xffffff : Color
.bottomColor
bottomColor = 0x000000 : Color
constructor
constructor( resolution = 512 : Number )
.update
update() : void
MaterialBase
extends THREE.ShaderMaterial
Convenience base class that adds additional functions and implicitly adds object definitions for all uniforms of the shader to the object.
.setDefine
setDefine( name : string, value = undefined : any ) : void
Sets the define of the given name to the provided value. If the value is set to null or undefined then it is deleted from the defines of the material. If the define changed from the previous value then Material.needsUpdate
is set to true
.
PhysicalPathTracingMaterial
extends MaterialBase
Uniforms
{
// The number of ray bounces to test. Higher is better quality but slower performance.
// TransmissiveBounces indicates the number of additional transparent or translucent surfaces
// the ray can pass through.
bounces = 3 : Number,
transmissiveBounces = 10 : Number,
// The number of additional transmissive ray bounces to allow on top of existing bounces for object opacity / transmission.
transmissiveBounces = 5 : Number,
// The physical camera parameters to use
physicalCamera : PhysicalCameraUniform,
// Geometry and BVH information
bvh: MeshBVHUniformStruct,
attributesArray: AttributesTextureArray,
materialIndexAttribute: UIntVertexAttributeTexture,
materials: MaterialsTexture,
textures: RenderTarget2DArray,
// Light information
lights: LightsInfoUniformStruct,
iesProfiles: IESProfilesTexture,
// Environment Map information
envMapInfo: EquirectHdrInfoUniform,
environmentRotation: Matrix4,
environmentIntensity = 1: Number,
// background blur
backgroundBlur = 0: Number,
// Factor for alleviating bright pixels from rays that hit diffuse surfaces then
// specular surfaces. Setting this higher alleviates fireflies but will remove some
// specular caustics.
filterGlossyFactor = 0: Number,
// The equirectangular map to render as the background.
backgroundMap = null: Texture,
// The transparency to render the background with. Note that the "alpha" option
// must be set to true on PathTracingRenderer for this field to work properly.
backgroundAlpha: 1.0,
}
Defines
{
// Whether to use multiple importance sampling to help the image converge more quickly.
FEATURE_MIS = 1 : Number,
// Whether to use russian roulette path termination. Path termination will kick in after
// a minimum three bounces have been performed.
FEATURE_RUSSIAN_ROULETTE = 1 : Number,
}
FogVolumeMaterial
extends MeshStandardMaterial
A material used for rendering fog-like volumes within the scene. The color
, emissive
, and emissiveIntensity
fields are all used in the render.
NOTE Since fog models many particles throughout the scene and cause many extra bounces fog materials can dramatically impact render time.
.density
The particulate density of the volume.
DenoiseMaterial
extends MaterialBase
Denoise material based on BrutPitt/glslSmartDeNoise intended to be the final pass to the screen. Includes tonemapping and color space conversions.
Uniforms
{
// sigma - sigma Standard Deviation
// kSigma - sigma coefficient
// kSigma * sigma = radius of the circular kernel
sigma = 5.0 : Number,
kSigma = 1.0 : Number,
// edge sharpening threshold
threshold = 0.03 : Number,
}
RenderTarget2DArray
extends WebGLArrayRenderTarget
A convenience extension from WebGLArrayRenderTarget
that affords easily creating a uniform texture array from an array of textures.
.setTextures
setTextures(
renderer : WebGLRenderer,
width : Number,
height : Number,
textures : Array<Texture>
) : void
Takes the rendering context to update the target for, the target dimensions of the texture array, and the array of textures to render into the 2D texture array. Every texture is stretched to the dimensions of the texture array at the same index they are provided in.
PhysicalCameraUniform
Uniform for storing the camera parameters for use with the shader.
.updateFrom
updateFrom( camera : PerspectiveCamera | PhysicalCamera ) : void
Copies all fields from the passed PhysicalCamera if available otherwise the defaults are used.
AttributesTextureArray
A combined texture array used to store normal, tangent, uv, and color attributes in the same texture sampler array rather than separate samplers. Necessary to save texture slots.
Normals, tangents, uvs, and color attribute data are stored in the 1st, 2nd, 3rd, and 4th layers of the array respectively.
.updateNormalAttribute
updateNormalAttribute( attr : BufferAttribute ) : void
.updateTangentAttribute
updateTangentAttribute( attr : BufferAttribute ) : void
.updateUvAttribute
updateUvAttribute( attr : BufferAttribute ) : void
.updateColorAttribute
updateColorAttribute( attr : BufferAttribute ) : void
.updateFrom
updateFrom(
normal : BufferAttribute,
tangent : BufferAttribute,
uv : BufferAttribute,
color : BufferAttribute
) : void
MaterialsTexture
extends DataTexture
Helper texture uniform for encoding materials as texture data.
.threeCompatibilityTransforms
threeCompatibilityTransforms = false : Boolean
Three.js materials support only a single set of UV transforms in a certain fallback priority while the pathtracer supports a unique set of transforms per texture. Set this field to true in order to use the same uv transform handling as three.js materials.
See fallback order documentation here.
.setMatte
setMatte( index : Number, matte : Boolean ) : void
Sets whether or not the material of the given index is matte or not. When "true" the background is rendered in place of the material.
.setCastShadow
setCastShadow( index : Number, enabled : Boolean ) : void
Sets whether or not the material of the given index will cast shadows. When "false" materials will not cast shadows on diffuse surfaces but will still be reflected. This is a good setting for lighting enclosed interiors with environment lighting.
.updateFrom
updateFrom( materials : Array<Material>, textures : Array<Texture> ) : void
Updates the size and values of the texture to align with the provided set of materials and textures.
The "matte" and "side" values must be updated explicitly.
Note In order for volume transmission to work the "attenuationDistance" must be set to a value less than Infinity or "thickness" must be set to a value greater than 0.
LightsInfoUniformStruct
Helper uniform for encoding lights as texture data with count.
.updateFrom
updateFrom( lights : Array<Light>, iesTextures = [] : Array<Texture> ) : void
Updates the size and values of the texture to align with the provided set of lights and IES textures.
EquirectHdrInfoUniform
Stores the environment map contents along with the intensity distribution information to support multiple importance sampling.
.updateFrom
updateFrom( environmentMap : Texture ) : void
Takes an environment map to process into something usable by the path tracer. Is expected to be a DataTexture so the data can be read.
Functions
mergeMeshes
mergeMeshes( meshes : Array<Mesh> ) : {
materials : Array<Material>,
textures : Array<Textures>,
geometry : BufferGeometry
}
Merges the set of meshes into a single geometry with a materialIndex
vertex attribute included on the geometry identifying the associated material in the returned materials
array.
CompatibilityDetector
Detects whether the path tracer can run on the current device by checking whether struct precision is reliable and the material shader will compile.
constructor
constructor( renderer : WebGLRenderer, material : Material )
Takes a WebGLRenderer to use and material to test again.
.detect
detect() : {
pass: Boolean,
message: String
}
Returns pass === true
if the path tracer can run. If it cannot run then a message is returned indicating why.
Shader Chunks
shaderMaterialSampling
Set of functions for performing material scatter and PDF sampling. See the implementation for full list of functions.
shaderLightSampling
Set of functions for performing light sampling. See the implementation for full list of functions.
shaderStructs
Material and light struct definition for use with uniforms. See the implementation for full list of functions.
shaderUtils
Set of randomness and other light transport utilities for use in a shader. See the implementation for full list of functions.
Gotchas
- The project requires use of WebGL2.
- All textures must use the same wrap and interpolation flags.
- SpotLights, DirectionalLights, and PointLights are only supported with MIS.
- Only MeshStandardMaterial and MeshPhysicalMaterial are supported.
- Instanced geometry and interleaved buffers are not supported.
- Emissive materials are supported but do not take advantage of MIS.
Screenshots
Sample materials
"SD Macross City Standoff Diorama" scene by tipatat
"Interior Scene" model by Allay Design
Perseverance Rover, Ingenuity Helicopter models by NASA / JPL-Caltech
Gelatinous Cube model by glenatron
Lego models courtesy of the LDraw Official Model Repository
Octopus Tea model by AzTiZ
Botanists Study model by riikkakilpelainen
Japanese Bridge Garden model by kristenlee
Resources
Raytracing in One Weekend Book