CS184 Final Project Proposal

Realistic Real-time Rendering of Real Scenes using Ray-Tracing

By Adam Chang, Leo Adberg, and Mitchell Chase

Goal: Perform 3D reconstructions of real-scenes, process them, and ray-trace them in real-time. Use texture baking and hardware acceleration to allow real-time virtual scene traversal with reflective surfaces.

Using standard 3D reconstruction techniques, we have the capability to transform real scenes into a geometric representation similar to the unordered triangle list introduced in class. We wish to allow users to traverse this virtual scene, rendering it realistically and with reflective surfaces in real-time. This requires detection of reflective surfaces (either manual or autonomous), and other techniques to speed up the ray-tracing process. One possible technique is texture baking, where we take advantage of the properties of diffuse surfaces, allowing us to rasterize instead of ray trace.

Realistic Goals:

• Convert 3D real scene reconstructions into ray-traceable scene
  • Build a tool to automatically do the conversion, or modify the pathtracer to load that type of file
• Add textured surfaces to the pathtracer
  • Maybe add normal maps
• Mesh simplification
• Perform hardware acceleration + texture baking to allow faster rendering
  • With texture baking, some of the scene could be rasterized instead of raytraced without a decrease in quality (and a huge increase in speed).

Stretch Goals:

• Automatically detect (or hand paint) non-diffuse surfaces to be ray-traced (windows, mirrors, glass)
• Fast enough that user can walk through scene in real-time (like a videogame)

Quality Measurement:

• Quality: realistic, real-time
• How many reflections can we have?

Analysis:

• Tradeoffs between speed and quality
• e.g. What fraction of the scene needs to be diffuse?
• Real-time ray tracing vs. tricks
• How does it compare to non-raytracing methods? (https://youtu.be/FtLkF8HVEco?t=170)

Schedule:

• Most work can be done in parallel because the new features are relatively orthogonal
• Weeks 1 & 2:
  • Conversion from unordered list triangle meshes into readable format for pathtracer
  • Texture baking and materials with texture & normal maps (diffuse objects)
• Weeks 3 & 4:
  • Hardware acceleration in CUDA or OpenCL (if other methods don’t speed it up enough)
  • Reflective surface ray-tracing using reconstructed scenes
  • Relighting a reconstructed real scene
  • Environment mapping windows?

Resources:

• Pathtracer (Project 3) will be our core system, and we will build off of it
• 3D Reconstruction from OpenARK, https://github.com/augcog/openark
• Languages: C++, CUDA
• Technologies: Intel Realsense, OpenGL, …