Abstract

Reconstructing humans in the 3D space has gained significant attention from the computer vision and computer graphics communities as it is a core component for human-interacting systems, such as fitness assistant, motion capture, virtual avatar, and AR/VR. In particular, recovering the human body, hands, and face together can make the 3D human highly expressive; therefore, it can represent delicate motions, intention, and feelings in addition to body-driven large motions. Taking a single image as an input makes the system applicable to in-the-wild environments without any additional sensors; however, it is greatly challenging due to the 2D-to-3D depth ambiguity, complicated articulations of body and hands, and small sizes of hands. In this talk, various learning-based 3D human reconstruction methods will be presented. Starting with 3D human body-only and hand-only reconstruction methods, expressive whole-body 3D human reconstruction methods will be presented, which aim to reconstruct the 3D body, hands, and face together. The last part of the talk includes remaining research problems for seamless human-computer interaction and virtual 3D space, recently called metaverse.