Projects

Importance Sampling

Importance Sampling for Many-Light Rendering   CG AI
Importance sampling is a popular variance reduction technique for Monte Carlo ray tracing. However, rendering with a large number of light sources brings up challenges for importance sampling, since the sampling needs to take the geometry, visibility, light intensity, and material properties into consideration. We have proposed three importance sampling algorithms to improve the efficiency of rendering, especially for scenes with many light sources.


Adaptive Sampling and Filtering

Adaptive Rendering for Monte Carlo Ray Tracing   CG
Monte Carlo (MC) integration is a mainstream technique for rendering images with distributed effects such as antialiasing, depth of field, motion blur, and global illumination. It simulates a variety of sophisticated light transport paths in a unified fashion: using stochastic point samples in the integral domain. Despite its simplicity and generality; however, MC integration converges slowly and suffers from noises due to sampling variance. We have proposed a method that can determine the sampling distribution or exploit spatial coherence for reducing image noise.


VR Games and MR Tools

VR Games and AR/MR Tools   XR CG CV
Extended reality (XR) including VR, AR, and MR have attracted lots of attention in the past decade. Recently the idea of Metaverse proposed by Meta have aroused another lively discussion. There is no doubt that XR techniques will become one of the most important computer technology in the near future. I have participated in production of two VR games and several MR techniques when I worked in HTC.

  • TrueColor, © 2011-2021 HTC Corporation
  • Arcade Saga, © 2011-2021 HTC Corporation
  • Electronic device, method for displaying an augmented reality scene and non-transitory computer-readable medium, ROC Patent No. I711966
  • Virtual reality device, image processing method, and non-transitory computer-readable medium, ROC Patent No. I684163


Virtual Studio

Virtual Studio System for Movie Production   XR CV IM CG
We built a virtual studio system for movie production at Toppano. With our system, movie directors can preview the real-time compositing of real performers, virtual environment, and digital visual effects in the progress of filming. Such that, they can minimize the gaps between the final production and their imagination. To build this system, we have developed several techniques including RGB-D video enhancement, virtual lighting augmentation, real-time matting, and multiple object tracking.


Image/Video Enhancement

Image and Video Enhancement   CV IM CG
Sometimes the images or videos captured by acquisition devices contain defects due to imperfect hardware or an uncontrolled environment. Software algorithms are employed to enhance the captured images or videos. We have proposed a novel real-time depth completion algorithm for filling the holes of depth maps captured by consumer-level RGB-D cameras. Our method can better preserve depth boundaries than previous methods; while maintaining real-time frame rate.


Intuitive Image Editing

Intuitive Image Editing   CV AI IM CG
Creating or modifying digital content usually requires professional skills in editing tools. To allow more intuitive editing for general users, we leverage AI techniques to design efficient image editing systems. For general users, our systems can assist them in creating their desired content, which is very difficult to do on their own. For professional artists, our systems can greatly reduce the time and effort to finish the jobs.


Grants