Fusion4D: real-time performance capture of challenging scenes

Mingsong Dou, Sameh Khamis, Yury Degtyarev, Philip L. Davidson, Sean Ryan Fanello, Adarsh Kowdle, Sergio Orts-Escolano, Christoph Rhemann, David Kim, Jonathan Taylor, Pushmeet Kohli, Vladimir Tankovich, Shahram Izadi

Abstract

We contribute a new pipeline for live multi-view performance capture, generating temporally coherent high-quality reconstructions in real-time. Our algorithm supports both incremental reconstruction, improving the surface estimation over time, as well as parameterizing the nonrigid scene motion. Our approach is highly robust to both large frame-to-frame motion and topology changes, allowing us to reconstruct extremely challenging scenes. We demonstrate advantages over related real-time techniques that either deform an online generated template or continually fuse depth data nonrigidly into a single reference model. Finally, we show geometric reconstruction results on par with offline methods which require orders of magnitude more processing time and many more RGBD cameras.
BibTeX
@article{dou2016fusion,
  author = {Mingsong Dou and Sameh Khamis and Yury Degtyarev and Philip L. Davidson and Sean Ryan Fanello and Adarsh Kowdle and Sergio Orts-Escolano and Christoph Rhemann and David Kim and Jonathan Taylor and Pushmeet Kohli and Vladimir Tankovich and Shahram Izadi},
  title = {{Fusion4D: real-time performance capture of challenging scenes}},
  journal = {ACM Trans. Graphics},
  year = {2016},
  url = {https://openreview.net/forum?id=dAv4waq3KzN},
}

Loading related content...

Figure