Matt Pharr

I am a distinguished research scientist at NVIDIA, where I am working on a variety of projects related to path tracing on GPUs.

My book on rendering, Physically Based Rendering: From Theory to Implementation, is widely used in university courses and by graphics researchers and developers. Greg Humphreys, Pat Hanrahan, and I were awarded a Scientific and Technical Academy Award in recognition of the book's impact on CGI in movies—never before has a book received an Academy Award. The third edition of the book was released in the Fall of 2016 and work on the fourth edition of the book was released on March 28, 2023.

I have previously worked at Google, co-founded Neoptica (which was acquired by Intel), and co-founded Exluna (which was acquired by NVIDIA). I have a Ph.D. from the Stanford Graphics Lab under the supervision of Pat Hanrahan and a B.S. in Computer Science from Yale.


PBR Book

Greg Humphreys, Wenzel Jakob, and I are the authors of the book Physically Based Rendering: From Theory to Implementation. The book dates to the late 1990s, when Greg and I were graduate students and Pat Hanrahan suggested that we write a book on how to write a ray tracer. He offered to use it in the rendering class he was teaching at Stanford and Greg and I agreed, having no idea what we were getting ourselves into. The manuscript greatly benefitted from both Pat's approach to rendering as well as being battle tested in the course. The first edition was published in 2004 and the second edition came out in 2010. Wenzel joined for the third edition in 2016, and the fourth edition was recently released.

The book is distinguished by being a literate program. Literate programming is an approach invented by Donald Knuth where programs are broken down into short sections of code that are intermixed with text describing them. Writing the book in this way has allowed us to start from the theory and mathematics of rendering and go all the way to its implementation. Doing so makes it possible to discuss subtle implementation issues that aren't often mentioned in papers, for example, and also lets us say something about the design and structure of a moderately complex software system. pbrt, the rendering system described in the book, is capable of rendering highly realistic imagery, such as the image shown here.

As its title suggests, the book's focus is on physically-based approaches to rendering. It implements a ray-optics based simulation of light transport and scattering; all of the light transport algorithms it describes are based on Monte Carlo path tracing. For more information, see both the book Website, as well as the online verision of the book.


Papers

Decorrelating ReSTIR Samplers via MCMC Mutations

Rohan Sawhney, Daqi Lin, Markus Kettunen, Benedikt Bitterli, Ravi Ramamoorthi, Chris Wyman, Matt Pharr
ACM Transactions on Graphics
Abstract: Monte Carlo rendering algorithms often utilize correlations between pixels to improve efficiency and enhance image quality. For real-time applications in particular, repeated reservoir resampling offers a powerful framework to reuse samples both spatially in an image and temporally across multiple frames. While such techniques achieve equal-error up to 100× faster for real-time direct lighting [Bitterli et al. 2020] and global illumination [Ouyang et al. 2021; Lin et al. 2021], they are still far from optimal. For instance, unchecked spatiotemporal resampling often introduces noticeable correlation artifacts, while reservoirs holding more than one sample suffer from impoverishment in the form of duplicate samples. We demonstrate how interleaving Markov Chain Monte Carlo (MCMC) mutations with reservoir resampling helps alleviate these issues, especially in scenes with glossy materials and difficult-to-sample lighting. Moreover, our approach does not introduce any bias, and in practice we find considerable improvement in image quality with just a single mutation per reservoir sample in each frame.
Comments: None so far.
Citation: Rohan Sawhney, Daqi Lin, Markus Kettunen, Benedikt Bitterli, Ravi Ramamoorthi, Chris Wyman, Matt Pharr. 2024. Decorrelating ReSTIR Samplers via MCMC Mutations, ACM Transactions on Graphics, January 2024.

ART-Owen Scrambling

Abdalla G. M. Ahmed, Matt Pharr, Peter Wonka
ACM Transactions on Graphics (SIGGRAPH Asia)
Abstract: We present a novel algorithm for implementing Owen-scrambling, combining the generation and distribution of the scrambling bits in a single self-contained compact process. We employ a context-free grammar to build a binary tree of symbols, and equip each symbol with a scrambling code that affects all descendant nodes. We nominate the grammar of adaptive regular tiles (ART) derived from the repetition-avoiding Thue-Morse word, and we discuss its potential advantages and shortcomings. Our algorithm has many advantages, including random access to samples, fixed time complexity, GPU friendliness, and scalability to any memory budget. Further, it provides two unique features over known methods: it admits optimization, and it is invertable, enabling screen-space scrambling of the high-dimensional Sobol sampler.
Comments: None other than that I love the naming of this paper (which is not due to me) and it's incredible that the two Arts lined up.
Citation: Abdalla G. M. Ahmed, Matt Pharr, Peter Wonka. 2023. ART-Owen Scrambling. ACM Transactions on Graphics (SIGGRAPH Asia), 42(6) 258:1-11.

Stochastic Texture Filtering

Marcos Fajardo, Bartlomiej Wronski, Marco Salvi, and Matt Pharr
Abstract: 2D texture maps and 3D voxel arrays are widely used to add rich detail to the surfaces and volumes of rendered scenes, and filtered texture lookups are integral to producing high-quality imagery. We show that filtering textures after evaluating lighting, rather than before BSDF evaluation as is current practice, gives a more accurate solution to the rendering equation. These benefits are not merely theoretical, but are apparent in common cases. We further show that stochastically sampling texture filters is crucial for enabling this approach, which has not been possible previously except in limited cases. Stochastic texture filtering offers additional benefits, including efficient implementation of high-quality texture filters and efficient filtering of textures stored in compressed and sparse data structures, including neural representations. We demonstrate applications in both real-time and offline rendering and show that the additional stochastic error is minimal. Furthermore, this error is handled well by either spatiotemporal denoising or moderate pixel sampling rates.
Comments: Rejected by HPG 2023.
Citation: Marcos Fajardo, Bartlomiej Wronski, Marco Salvi, and Matt Pharr. 2023. Stochastic Texture Filtering. arXiv: 2305.05810 [cs.GR]

A Generalized Ray Formulation For Wave-Optics Rendering

Shlomi Steinberg, Ravi Ramamoorthi, Benedikt Bitterli, Eugene d'Eon, Ling-Qi Yan, and Matt Pharr
Abstract: Under ray-optical light transport, the classical ray serves as a local and linear “point query” of light’s behaviour. Such point queries are useful, and sophisticated path tracing and sampling techniques enable efficiently computing solutions to light transport problems in complex, real-world settings and environments. However, such formulations are firmly confined to the realm of ray optics, while many applications of interest, in computer graphics and computational optics, demand a more precise understanding of light. We rigorously formulate the generalized ray, which enables local and linear point queries of the wave-optical phase space. Furthermore, we present sample-solve: a simple method that serves as a novel link between path tracing and computational optics. We will show that this link enables the application of modern path tracing techniques for wave-optical rendering, improving upon the state-of-the-art in terms of the generality and accuracy of the formalism, ease of application, as well as performance. Sampling using generalized rays enables interactive rendering under rigorous wave optics, with orders-of-magnitude faster performance compared to existing techniques.
Comments: ...
Citation: Shlomi Steinberg, Ravi Ramamoorthi, Benedikt Bitterli, Eugene d'Eon, Ling-Qi Yan, and Matt Pharr. 2023. A Generalized Ray Formulation For Wave-Optics Rendering. arXiv: 2303.15762 [cs.GR]

ReSTIR GI: Path Resampling for Real-Time Path Tracing

Yaobin Ouyang, Shiqiu Liu, Markus Kettunen, Matt Pharr, and Jacopo Pantaleoni
High Performance Graphics 2021 (Computer Graphics Forum)
Abstract: Even with the advent of hardware-accelerated ray tracing in modern GPUs, only a small number of rays can be traced at each pixel in real-time applications. This presents a significant challenge for path tracing, even when augmented with state-of-the art denoising algorithms. While the recently-developed ReSTIR algorithm [Bitterli et al. 2020] enables high-quality renderings of scenes with millions of light sources using just a few shadow rays at each pixel, there remains a need for effective algorithms to sample indirect illumination. We introduce an effective path sampling algorithm for indirect lighting that is suitable to highly parallel GPU architectures. Building on the screen-space spatio-temporal resampling principles of ReSTIR, our approach resamples multi-bounce indirect lighting paths obtained by path tracing. Doing so allows sharing information about important paths that contribute to lighting both across time and pixels in the image. The resulting algorithm achieves a substantial error reduction compared to path tracing: at a single sample per pixel every frame, our algorithm achieves MSE improvements ranging from 9.3x to 166x in our test scenes. In conjunction with a denoiser, it leads to high-quality path traced global illumination at real-time frame rates on modern GPUs
Comments: ReSTIR: the gift that keeps on giving.
Citation: Yaobin Ouyang, Shiqiu Liu, Markus Kettunen, Matt Pharr, and Jacopo Pantaleoni. 2021. ReSTIR GI: Path Resampling for Real-Time Path Tracing. Computer Graphics Forum (High Performance Graphics 2021).

Spatiotemporal Reservoir Resampling for Real-Time Ray Tracing with Dynamic Direct Lighting

Benedikt Bitterli, Chris Wyman, Matt Pharr, Peter Shirley, Aaron Lefohn, and Wojciech Jarosz
SIGGRAPH 2020
Abstract: Efficiently rendering direct lighting from millions of dynamic light sources using Monte Carlo integration remains a challenging problem, even for off-line rendering systems. We introduce a new algorithm—ReSTIR—that renders such lighting interactively, at high quality, and without needing to maintain complex data structures. We repeatedly resample a set of candidate light samples and apply further spatial and temporal resampling to leverage information from relevant nearby samples. We derive an unbiased Monte Carlo estimator for this approach, and show that it achieves equal-error 6-60× faster than state-of-the-art methods. A biased estimator reduces noise further and is 35-65× faster, at the cost of some energy loss. We implemented our approach on the GPU, rendering complex scenes containing up to 3.4 million dynamic, emissive triangles in under 50ms per frame while tracing at most 8 rays per pixel.
Comments: Temporal reuse has been an important part of the real-time rendering toolbox in recent years, notably with techniques like TAA. What I like most about this paper is that it applies temporal reuse to samples rather than final shaded pixel colors. In doing so, it can deliver an unbiased Monte Carlo estimate of the scene's lighting and in turn, avoids some of the issues that come with temporal reuse of final shaded pixel values. The algorithm is spooky effective at sampling direct lighting from up to millions of light sources thanks to massive sharing of light samples across both space and time.
Citation: Benedikt Bitterli, Chris Wyman, Matt Pharr, Peter Shirley, Aaron Lefohn, and Wojciech Jarosz. 2020. Spatiotemporal reservoir resampling for real-time ray tracing with dynamic direct lighting. ACM Transactions on Graphics (Proceedings of SIGGRAPH), 39 (4).

Practical Product Sampling by Fitting and Composing Warps

David Hart, Matt Pharr, Thomas Müller, Ward Lopes, Morgan McGuire, and Peter Shirley.
Eurographics Symposium on Rendering 2020
Abstract: We introduce a Monte Carlo importance sampling method for integrands composed of products and show its application to rendering where direct sampling of the product is often difficult. Our method is based on warp functions that operate on the primary samples in [0,1)^n, where each warp approximates sampling a single factor of the product distribution. Our key insight is that individual factors are often well‐behaved and inexpensive to fit and sample in primary sample space, which leads to a practical, efficient sampling algorithm. Our sampling approach is unbiased, easy to implement, and compatible with multiple importance sampling. We show the results of applying our warps to projected solid angle sampling of spherical triangles, to sampling bilinear patch light sources, and to sampling glossy BSDFs and area light sources, with efficiency improvements of over 1.6× on real‐world scenes.
Comments: This paper grew out of a clever idea from Pete Shirley about warping uniform samples before using them with traditional inversion-method based approaches for importance sampling. It turns out that you can do a surprisingly decent approximation to product sampling by doing so; we show results for incorporating the cosine factor in sampling techniques for light sources as well as for sampling the product of BSDF and illumination. A nice thing about this technique is that it's roughly 50 lines of code and can easily be added to incorporated into Monte Carlo rendering systems; pbrt-v4 has an implementation of it.
Subsequent to publication, I realized that both Lafortune And Willems (1995) and Jensen (1995) used a related technique for product sampling of BRDFs with indirect lighting: they both took indirect lighting samples, applied the inverse of the importance sampling mapping, and then tabularized sample counts in primary sample space. Samples in [0,1]^2 were then generated based on that distribution before being used for BRDF importance sampling. This work should have been included in our previous work discussion. (It does, however, share some similarities with the earlier work of Booth (1986), which is cited.)
Citation: David Hart, Matt Pharr, Thomas Müller, Ward Lopes, Morgan McGuire, and Peter Shirley. 2020. Practical product sampling by fitting and composing warps. Computer Graphics Forum (Proceedings of EGSR 2020), 39 (4), 149—158.

Dynamic Many-Light Sampling for Real-Time Ray Tracing

Pierre Moreau, Matt Pharr, and Petrik Clarberg
High Performance Graphics 2019
Abstract: Monte Carlo ray tracing offers the capability of rendering scenes with large numbers of area light sources—lights can be sampled stochastically and shadowing can be accounted for by tracing rays, rather than using shadow maps or other rasterization-based techniques that do not scale to many lights or work well with area lights. Current GPUs only afford the capability of tracing a few rays per pixel at real-time frame rates, making it necessary to focus sampling on important light sources. While state-of-the-art algorithms for offline rendering build hierarchical data structures over the light sources that enable sampling them according to their importance, they lack efficient support for dynamic scenes. We present a new algorithm for maintaining hierarchical light sampling data structures targeting real-time rendering. Our approach is based on a two-level BVH hierarchy that reduces the cost of partial hierarchy updates. Performance is further improved by updating lower-level BVHs via refitting, maintaining their original topology. We show that this approach can give error within 6% of recreating the entire hierarchy from scratch at each frame, while being two orders of magnitude faster, requiring less than 1 ms per frame for hierarchy updates for a scene with thousands of moving light sources on a modern GPU. Further, we show that with spatiotemporal filtering, our approach allows complex scenes with thousands of lights to be rendered with ray-traced shadows in 16.1 ms per frame.
Comments: This paper brought Alex Conty Estevez and Chris Kulla's BVH-based many light sampling technique to the GPU for real-time rendering. The main contribution was showing that the light hierarchy could be efficiently updated for dynamic scenes by maintaining a two-level BVH and applying refitting. Not too long afterward, ReSTIR showed much better results for this problem, though only for first-hit rays from the camera.
Citation: Pierre Moreau, Matt Pharr, and Petrik Clarberg. 2019. Dynamic many-light sampling for real-time ray tracing. High Performance Graphics (Short Papers), 21—26.

Efficient Generation of Points that Satisfy Two-Dimensional Elementary Intervals

Matt Pharr
Journal of Computer Graphics Techniques
Abstract: Precomputing high-quality sample points has been shown to be a useful technique for Monte Carlo integration in rendering; doing so allows optimizing properties of the points without the performance constraints of generating samples during rendering. A particularly useful property to incorporate is stratification across elementary intervals, which has been shown to reduce error in Monte Carlo integration. This is a key property of the recently-introduced progressive multi-jittered, pmj02 and pmj02bn points [Christensen et al. 2018]. For generating such sets of sample points, it is important to be able to efficiently choose new samples that are not in elementary intervals occupied by existing samples. Random search, while easy to implement, quickly becomes infeasible after a few thousand points. We describe an algorithm that efficiently generates 2D sample points that are stratified with respect to sets of elementary intervals. If a total of n sample points are being generated, then for each sample, our algorithm uses O(n^1/2) time to build a data structure that represents the regions where a next sample may be placed. Given this data structure, valid samples can be generated in O(1) time. We demonstrate the utility of our method by generating much larger sets of pmj02bn points than were feasible previously
Comments: I really liked the 2018 paper by Per Christensen, Andrew Kensler, and Charlie Kilpatrick on progressive multi-jittered sample sets that also satisfied the 2D elementary intervals and had blue noise characteristics. Their approach used a relatively slow search for valid points and so I spent some time thinking about whether that part of their algorithm could be done more efficiently. After this paper was published, I realized that the paper (t,m,s)-nets and maximized minimum distance by Leo Grüschloß et al. introduces a very similar trick to the one this paper described.
Citation: Matt Pharr. 2019. Efficient generation of points that satisfy two-dimensional elementary intervals. Journal of Computer Graphics Techniques (JCGT), 8 (1) 56—68.

A System for Acquiring, Processing, and Rendering Panoramic Light Field Stills for Virtual Reality

Ryan Overbeck, Daniel Erickson, Daniel Evangelakos, Matt Pharr, and Paul Debevec
SIGGRAPH Asia 2018
Abstract: We present a system for acquiring, processing, and rendering panoramic light field still photography for display in Virtual Reality (VR). We acquire spherical light field datasets with two novel light field camera rigs designed for portable and efficient light field acquisition. We introduce a novel real-time light field reconstruction algorithm that uses a per-view geometry and a disk-based blending field. We also demonstrate how to use a light field prefiltering operation to project from a high-quality offline reconstruction model into our real-time model while suppressing artifacts. We introduce a practical approach for compressing light fields by modifying the VP9 video codec to provide high quality compression with real-time, random access decompression. We combine these components into a complete light field system offering convenient acquisition, compact file size, and high-quality rendering while generating stereo views at 90Hz on commodity VR hardware. Using our system, we built a freely available light field experience application called Welcome to Light Fields featuring a library of panoramic light field stills for consumer VR which has been downloaded over 15,000 times.
Comments: There is an entertaining retrospective to write about Google's VR efforts and the light field project specifically, but this space is too narrow to contain it.
Citation: Ryan Overbeck, Daniel Erickson, Daniel Evangelakos, Matt Pharr, and Paul Debevec. 2018. A system for acquiring, processing, and rendering panoramic light field stills for virtual reality. ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2018) 37 (6).

View-Region Optimized Image-Based Scene Simplification

Puneet Lall, Silviu Borac, Dave Richardson, Matt Pharr, and Manfred Ernst
High Performance Graphics 2018
Third Place, Best Paper Award
Abstract: We present a new algorithm for image-based simplification of complex scenes for virtual reality (VR). The algorithm transforms geometrically-detailed environments into a layered quad tile representation that is optimized for a specified viewing region and is renderable on low-power mobile-class VR devices. A novel constrained optimization formulation ensures that the scene can be rendered within a predetermined compute budget, with limits on both primitive count and fill rate. Furthermore, we introduce a new method for texturing from point samples of the original scene geometry that generates high-quality silhouettes without the drawbacks of traditional point splatting.
The resulting representation achieves a visual fidelity that was previously impossible on mobile graphics hardware; our algorithm can typically generate a high-quality representation of visually-rich scenes with billions of triangles using just 72k triangles and a single high-resolution texture map (with generally only about 50% more texels than a stereo panorama). The effectiveness of the approach is demonstrated with a set of challenging test cases.
Comments: Google's VR effort was focused on battery-powered HMDs, which in turn implied limited power and thermal budgets and mobile-class GPUs. Under these constraints, rendering high-quality content was especially challenging. This paper came out of the Seurat project, which was working on finding ways to efficiently render complex static models while still maintaining correct parallax when viewed from some specified head box. We were lucky to be able to work with ILM and show off Star Wars assets rendered in VR on a battery-powered headset.
Citation: Puneet Lall, Silviu Borac, Dave Richardson, Matt Pharr, and Manfred Ernst. 2018. View-region optimized image-based scene simplification. Proceedings of the ACM on Computer Graphics and Interactive Techniques.

Sequences With Low‐Discrepancy Blue‐Noise 2‐D Projections

Hélène Perrier, David Coeurjolly, Feng Xie, Matt Pharr, Pat Hanrahan, and Victor Ostromoukhov
Eurographics 2018
Abstract: Distributions of samples play a very important role in rendering, affecting variance, bias and aliasing in Monte-Carlo and Quasi-Monte Carlo evaluation of the rendering equation. In this paper, we propose an original sampler which inherits many important features of classical low-discrepancy sequences (LDS): a high degree of uniformity of the achieved distribution of samples, computational efficiency and progressive sampling capability. At the same time, we purposely tailor our sampler in order to improve its spectral characteristics, which in turn play a crucial role in variance reduction, anti-aliasing and improving visual appearance of rendering. Our sampler can efficiently generate sequences of multidimensional points, whose power spectra approach so-called Blue-Noise (BN) spectral property while preserving low discrepancy (LD) in certain 2-D projections. In our tile-based approach, we perform permutations on subsets of the original Sobol LDS. In a large space of all possible permutations, we select those which better approach the target BN property, using pair-correlation statistics. We pre-calculate such “good” permutations for each possible Sobol pattern, and store them in a lookup table efficiently accessible in runtime. We provide a complete and rigorous proof that such permutations preserve dyadic partitioning and thus the LDS properties of the point set in 2-D projections. Our construction is computationally efficient, has a relatively low memory footprint and supports adaptive sampling. We validate our method by performing spectral/discrepancy/aliasing analysis of the achieved distributions, and provide variance analysis for several target integrands of theoretical and practical interest.
Comments: (none)
Citation: Hélène Perrier, David Coeurjolly, Feng Xie, Matt Pharr, Pat Hanrahan, and Victor Ostromoukhov. 2018. Sequences with Low-Discrepancy Blue-Noise 2-D Projections Computer Graphics Forum (Proceedings of Eurographics) 37 (2), 339—353.

ispc: A SPMD compiler for high-performance CPU programming

Matt Pharr and William R. Mark
Innovative Parallel Computing (InPar) 2012
Best Paper Award
Abstract: SIMD parallelism has become an increasingly important mechanism for delivering performance in modern CPUs, due its power efficiency and relatively low cost in die area compared to other forms of parallelism. Unfortunately, languages and compilers for CPUs have not kept up with the hardware's capabilities. Existing CPU parallel programming models focus primarily on multi-core parallelism, neglecting the substantial computational capabilities that are available in CPU SIMD vector units. GPU-oriented languages like OpenCL support SIMD but lack capabilities needed to achieve maximum efficiency on CPUs and suffer from GPU-driven constraints that impair ease of use on CPUs.
We have developed a compiler, the Intel SPMD Program Compiler (ispc), that delivers very high performance on CPUs thanks to effective use of both multiple processor cores and SIMD vector units. ispc draws from GPU programming languages, which have shown that for many applications the easiest way to program SIMD units is to use a single-program, multiple-data (SPMD) model, with each instance of the program mapped to one SIMD lane. We discuss language features that make ispc easy to adopt and use productively with existing software systems and show that ispc delivers up to 35x speedups on a 4-core system and up to 240x speedups on a 40-core system for complex workloads (compared to serial C++ code).
Comments: None beyond 12 blog posts about the history of ispc.
Citation: Matt Pharr and William R. Mark. 2012. ispc: A SPMD compiler for high-performance CPU programming. Proceedings of Innovative Parallel Computing (InPar) 2012.

Monte Carlo evaluation of non-linear scattering equations for subsurface reflection

Matt Pharr and Pat Hanrahan
SIGGRAPH 2000
Abstract: We describe a new mathematical framework for solving a wide variety of rendering problems based on a non-linear integral scattering equation. This framework treats the scattering functions of complex aggregate objects as first-class rendering primitives; these scattering functions accurately account for all scattering events inside them. We also describe new techniques for computing scattering functions from the composition of scattering objects. We demonstrate that solution techniques based on this new approach can be more efficient than previous techniques based on radiance transport and the equation of transfer and we apply these techniques to a number of problems in rendering scattering from complex surfaces.
Comments: (None).
Citation: Matt Pharr and Pat Hanrahan. 2000. Monte Carlo evaluation of non-linear scattering equations for subsurface reflection. Proceedings of the 27th annual conference on Computer graphics and interactive techniques (SIGGRAPH '00), 75–84.

Realistic modeling and rendering of plant ecosystems

Oliver Deussen, Pat Hanrahan, Bernd Lintermann, Radomir Mech, Matt Pharr, and Przemyslaw Prusinkiewicz.
SIGGRAPH 1998
Abstract: Modeling and rendering of natural scenes with thousands of plants poses a number of problems. The terrain must be modeled and plants must be distributed throughout it in a realistic manner, reflecting the interactions of plants with each other and with their environment. Geometric models of individual plants, consistent with their positions within the ecosystem, must be synthesized to populate the scene. The scene, which may consist of billions of primitives, must be rendered efficiently while incorporating the subtleties of lighting in a natural environment. We have developed a system built around a pipeline of tools that address these tasks. The terrain is designed using an interactive graphical editor. Plant distribution is determined by hand (as one would do when designing a garden), by ecosystem simulation, or by a combination of both techniques. Given parameterized procedural models of individual plants, the geometric complexity of the scene is reduced by approximate instancing, in which similar plants, groups of plants, or plant organs are replaced by instances of representative objects before the scene is rendered. The paper includes examples of visually rich scenes synthesized using the system.
Comments: As I remember it, after the 1997 paper on memory-coherent ray tracing, Pat and Przemek got to talking. The combination of Przemek's ability to generate complex scenes full of plants and our work on rendering complex scenes led to this collaboration and soon enough I was off to Calgary for a few weeks in November 1997. I remember that in Calgary it got dark awfully early and that it was mighty chilly on walks back to my hotel in the evenings.
Citation: Oliver Deussen, Pat Hanrahan, Bernd Lintermann, Radomir Mech, Matt Pharr, and Przemyslaw Prusinkiewicz. 1998. Realistic modeling and rendering of plant ecosystems. Proceedings of the 25th annual conference on Computer graphics and interactive techniques (SIGGRAPH '98), 275–286.

Rendering complex scenes with memory-coherent ray tracing

Matt Pharr, Craig Kolb, Reid Gershbein, and Pat Hanrahan
SIGGRAPH 1997
Abstract: Simulating realistic lighting and rendering complex scenes are usually considered separate problems with incompatible solutions. Accurate lighting calculations are typically performed using ray tracing algorithms, which require that the entire scene database reside in memory to perform well. Conversely, most systems capable of rendering complex scenes use scan-conversion algorithms that access memory coherently, but are unable to incorporate sophisticated illumination. We have developed algorithms that use caching and lazy creation of texture and geometry to manage scene complexity. To improve cache performance, we increase locality of reference by dynamically reordering the rendering computation based on the contents of the cache. We have used these algorithms to compute images of scenes containing millions of primitives, while storing ten percent of the scene description in memory. Thus, a machine of a given memory capacity can render realistic scenes that are an order of magnitude more complex than was previously possible.
Comments: The follow-on to the EGSR geometry caching paper, where we showed that caching worked really well if you also reorder the ray tracing work to access the cache coherently. It is a fairly disruptive change to a renderer to reorder the traced rays and as it has turned out, available RAM has grown enough that it is generally possible to store whatever scene you want to render in memory, perhaps with a bit of work to ensure it fits and perhaps with some caching but no reordering. Nevertheless, this is probably my favorite paper, both for being my first SIGGRAPH paper and for the complexity of the scenes we were path tracing, given available memory at the time.
Citation: Matt Pharr, Craig Kolb, Reid Gershbein, and Pat Hanrahan. 1997. Rendering complex scenes with memory-coherent ray tracing. Proceedings of the 24th annual conference on Computer graphics and interactive techniques (SIGGRAPH '97), 101–108.

Geometry caching for ray-tracing displacement maps

Matt Pharr and Pat Hanrahan
Eurographics Workshop on Rendering 1996
Abstract: We present a technique for rendering displacement mapped geometry in a ray-tracing renderer. Displacement mapping is an important technique for adding detail to surface geometry in rendering systems. It allows complex geometric variation to be added to simpler geometry, without the cost in geometric complexity of completely describing the nuances of the geometry at modeling time and with the advantage that the detail can be added adaptively at rendering time. The cost of displacement mapping is geometric complexity. Renderers that provide it must be able to efficiently render scenes that have effectively millions of geometric primitives. Scan-line renderers process primitives one at a time, so this complexity doesn't tax them, but traditional ray-tracing algorithms require random access to the entire scene database, so any part of the scene geometry may need to be available at any point during rendering. If the displaced geometry is fully instantiated in memory, it is straightforward to intersect rays with it, but displacement mapping has not yet been practical in ray-tracers due to the memory cost of holding this much geometry. We introduce the use of a geometry cache in order to handle the large amounts of geometry created by displacement mapping. By caching a subset of the geometry created and rendering the image in a coherent manner, we are able to take advantage of the fact that the rays spawned by traditional ray-tracing algorithms are spatially coherent. Using our algorithm, we have efficiently rendered highly complex scenes while using a limited amount of memory.
Comments: Caching a subset of the scene geometry in memory works out well when you're doing Whitted ray tracing and trying to render complex scenes with a few tens of megabytes of RAM. (Note: less RAM than a current CPU has in cache memory.) This paper was published in the olden days when you couldn't have a color image here there and everywhere in your paper. Therefore, the paper itself includes no images, but there are separate “color plates” for the reader's elucidation. At the conference, which was held in Porto, we found a restaurant named “Monte Carlo” that naturally had to be tried; Phil Dutré took a wonderful picture of assembled rendering nerds there.
Citation: Matt Pharr and Pat Hanrahan. 1996. Geometry caching for ray-tracing displacement maps. Proceedings of the Eurographics Workshop on Rendering.

Projects

Committees

  • Papers Chair, Eurographics Symposium on Rendering 2020 (with Carsten Dachsbacher)
  • Academy of Motion Picture Arts and Sciences Digital Imaging Technology Surrogate Committee, 2015, 2016, 2017, 2018, 2022, 2023
  • Program Chair, High Performance Graphics 2014
  • Conference Chair, High Performance Graphics 2011 (with John Owens)
  • Papers Chair, High Performance Graphics 2009 (with David McAllister and Ingo Wald)
  • Editorial board, Journal of Computer Graphics Techniques

TOG Production Renderers

In 2018, I guest-edited a special issue of ACM Transactions on Graphics on production rendering. The developers of five widely-used renderers wrote comprehensive systems papers, describing the challenges they face, the constraints they work under, and the solutions they have developed:

I wrote a short introduction that discusses the industry's transition from Reyes to path tracing.

Reports / Drafts

Here are a handful of additional documents that are not formal publications as such.

  • Physically Based Image Synthesis: Design and Implementation of a Rendering System (2003) (with Greg Humphreys) [PDF]
    For posterity, a 2003 snapshot of the manuscript that became the Physically Based Rendering book.
  • Infinite Area Light Source With Importance Sampling (2004) (with Greg Humphreys) [PDF]
    A short literate program implementation of an importance sampling algorithm for infinite light sources, too late for the first edition of Physically Based Rendering, but included in subsequent editions.
  • The Implementation of a Hair Scattering Model (2016) [PDF]
    This is an implementation of Chiang et al's A Practical and Controllable Hair and Fur Model for Production Path Tracing, implemented as a literate program. The fourth edition of Physically Based Rendering includes an updated version of this.
  • The Implementation of a Scalable Texture Cache (2017) [PDF]
    I spent some time trying to figure out how to implement a texture cache for CPU rendering that scaled well with many threads; this is a problem that has clearly been solved in production but one where no one has published about their approaches. This write-up this is the result of my efforts—an effective approach that is based on the "read-copy update" technique, implemented as a literate program.

Code

  • skicka: Partially as an exercise to learn go, I wrote a command-line tool that makes it easy to work with files and directories on Google Drive (including uploading/downloading, listing files in folders, etc). Google was happy to let me open source it. (“skicka” is Swedish for “to send”, which vaguely alludes to what the tool does.) [source]
  • bk: After near disaster from an Apple Time Capsule that silently failed, I wrote my own backup system featuring just enough functionality for my needs while not being too complicated: it features deduplication, encryption, direct cloud storage, and Reed-Solomon encoding for local backup integrity. [source]
  • vice: In the "software with a very small audience" department, I've lately been hacking on an air traffic control simulator, patterened on the STARS radar scope used in the US. The intent is for it to be useful for practicing for TRACON controlling on VATSIM. [documentation] [source]
  • Apple ][ works: Not to brag, but in middle school I wrote a pretty awesome D&D character generator in Applesoft Basic as well as a disk track/sector editor in 6502 assembly language.

Selected Talks

  • Porting pbrt to the GPU While Preserving its Soul [slides]
    Invited talk, High Performance Graphics 2020.
  • Path Tracing for Future Games (??!!?) [slides]
    SIGGRAPH 2019 “Open Problems in Real-Time Rendering” course.
    Comments: A year after the launch of RTX GPUs, it was easier to talk about ray tracing. The “open problems” course was a great opportunity to talk about some of the early successes with real-time ray tracing and to sketch out some of the biggest unsolved challenges.
  • Adopting Lessons from Offline Ray-Tracing to Real-Time Ray Tracing for Practical Pipelines [slides]
    SIGGRAPH 2018 “Advances in Real-Time Rendering in Games” course.
    Comments: The goal of this talk was to advance the idea of the importance of sampling well when doing Monte Carlo ray tracing in the real-time rendering community. It was a slightly tricky talk in that there was a risk of the topic seeming irrelevant to the realities of real-time rendering at that time. The public announcement of RTX GPUs half an hour after the end of the talk was helpful in that regard, at least after the fact.
  • Warp and Effect [slides]
    SIGGRAPH 2018 “My Favorite Samples” course.
  • Mobile VR: Challenges and Opportunities [slides]
    Keynote, Symposium on Interactive 3D Graphics and Games (i3d) 2017.
    Comments: “Challenges” seems to have won.
  • ispc: A SPMD Compiler for High-Performance CPU Programming [slides]
    University of Illinois Urbana-Champaign I2PC Distinguished Speaker Series, 2012.
    Comments: This is representative of a number of talks about ispc that I gave around this time.
  • Interactive Rendering in the Post-GPU Era [slides]
    Keynote, Graphics Hardware 2006.
    Comments: Based on the observation that some of the most effective real-time graphics algorithms were based on using sophisticated data structures in shaders, this talk argued for the importance of closely coupled CPU+GPU architectures under the premise that it was too difficult to build such data structures on the GPU. As it has turned out, enough clever people have subsequently figured out how to do that on GPUs that keeping it all on the GPU seems just fine. I believe that at least this talk coined the term “programmable graphics” and made the distinction with “programmable shading”.
  • The Quiet Revolution in Interactive Rendering [slides]
    Invited talk, Stanford University ee380 Computer Systems Colloquium, 2005.
    Comments: Graphics got a lot better even just a few years after the arrival of programmable GPUs. This talk was an effort to break down what had happened in how people were using programmability and to argue that they would be able to reach film-quality imagery well before a naive comparison based on FLOPS required to render a image film suggested.

Miscellanea

  • Film credits: For work I did in Pixar's Rendering R&D group as an intern in graduate school, I have movie credits for A Bug's Life and Toy Story 2. Using a very loose definition of the Bacon number, this means that I have an Erdős–Bacon number of 6.
  • Stanford cs348b: I've taught Image Synthesis Techniques, the graduate-level rendering class at Stanford seven times, co-teaching it with Pat Hanrahan for the past four years and doing it a few times solo in years previous.
  • GPU Gems 2: While at NVIDIA the first time, I edited the book GPU Gems 2: Programming Techniques for High-Performance Graphics and General Purpose Computation. The first half of the book is comprised of twenty-four chapters about the state-of-the-art in interactive rendering, and the second half is devoted to general purpose computation on graphics processors (GPGPU)—the first book covering this topic.
  • CACM Technical Perspective: The May 2013 issue of Communications of the ACM had a paper on OptiX by Steve Parker et al. for which I wrote an introduction and technical perspective, framing the work for a wider audience.

Social