Gengshan Yang | 杨庚山

I'm a PhD (2019-) student at CMU Robotics, advised by Prof. Deva Ramanan. My research is focused on computer vision and machine learning. I’m particularly interested in learning structures (e.g., 3D, motion, physics) from partial observations (e.g., monocular videos).

I obtained a MS degree (2017-2019) at CMU Robotics and a BEng degree (2013-2017) at Xi'an Jiaotong University. I was also fortunate to intern at FAIR, Google Research, Argo AI, TuSimple, and SVCL at UC San Diego.

Github /  Google Scholar  /  Email

Research
PPR: Physically Plausible Reconstruction from Monocular Videos
Gengshan Yang, Shuo Yang, Ziyang Zhang, Zachary Manchester, Deva Ramanan
Preprint

Given monocular videos, PPR builds 3D models of the object and the environment whose physical configurations satisfy dynamics and contact constraints.

RAC: Reconstructing Animatable Categories from Videos
Gengshan Yang, Chaoyang Wang, N Dinesh Reddy, Deva Ramanan
CVPR, 2023

RAC learns category-level deformable 3D models from monocular videos. It disentangles morphology and motion and allows for motion retargeting.

Distilling Neural Fields for Real-Time Articulated Shape Reconstruction
Jeff Tan, Gengshan Yang, Deva Ramanan
CVPR, 2023

We distill offline-optimized dynamic NeRFs into efficient video shape, pose, and appearance predictors.

3D-aware Conditional Image Synthesis
Kangle Deng, Gengshan Yang, Deva Ramanan, Jun-Yan Zhu
CVPR, 2023

A 3D-aware conditional generative model for controllable image synthesis. Given a 2D label map, such as a segmentation or edge map, our model learns to synthesize images consistent from different viewpoints.

BANMo: Building Animatable 3D Neural Models from Many Casual Videos
Gengshan Yang, Minh Vo, Natalia Neverova, Deva Ramanan, Andrea Vedaldi, Hanbyul Joo
CVPR, 2022 (Oral)

Given casual videos capturing a deformable object, BANMo reconstructs an animatable 3D model in a differentiable volume rendering framework.

ViSER: Video-Specific Surface Embeddings for Articulated 3D Shape Reconstruction
Gengshan Yang, Deqing Sun, Varun Jampani, Daniel Vlasic, Forrester Cole, Ce Liu Deva Ramanan
NeurIPS, 2021 (Spotlight)

Given a long video or multiple short videos, ViSER jointly optimizes articulated 3D shapes and a pixel-surface embedding to establish dense correspondences over video frames.

NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild
Jason Y. Zhang, Gengshan Yang, Shubham Tulsiani*, Deva Ramanan*
NeurIPS, 2021

Given several (8-16) unposed images of the same instance, NeRS optimizes for a textured 3D reconstruction along with the illumination parameters at test-time.

LASR: Learning Articulated Shape Reconstruction from a Monocular Video
Gengshan Yang, Deqing Sun, Varun Jampani, Daniel Vlasic, Forrester Cole, Huiwen Chang, Deva Ramanan, William T. Freeman, Ce Liu
CVPR, 2021

A template-free approach for articulated shape reconstruction from a single video by combining differentiable rendering and data-driven correspondence and segmentation priors.

Learning to Segment Rigid Motions from Two Frames
Gengshan Yang, Deva Ramanan
CVPR, 2021

We propose a neural architecture powered by geometric reasoning that decomposes two frames into a rigid background and multiple moving rigid bodies, parameterized by 3D rigid transformations and depth.

Upgrading Optical Flow to 3D Scene Flow through Optical Expansion
Gengshan Yang, Deva Ramanan
CVPR, 2020 (Oral)

We describe a neural architecture to upgrade 2D optical flow to 3D scene flow using optical expansion, which reveals changes in depth of scene elements over frames, e.g., things moving closer will get bigger.

Volumetric Correspondence Networks for Optical Flow
Gengshan Yang, Deva Ramanan
NeurIPS, 2019

We introduce several simple modifications to the optical flow volumetric layers that: 1) significantly reduces computation and parameters, 2) enables test-time adaptation of cost volume size, and 3) converges much faster.

Hierarchical Deep Stereo Matching on High-resolution Images
Gengshan Yang, Joshua Manela, Michael Happold, Deva Ramanan
CVPR, 2019

To adress the problem of real-time stereo matching on high-res imagery, an end-to-end framework that searches for correspondences incrementally over a coarse-to-fine hierarchy is proposed.

Inferring Distributions Over Depth from a Single Image
Gengshan Yang, Peiyun Hu, Deva Ramanan
IROS, 2019

We recast the continuous problem of depth regression as discrete binary classification, whose output is the occupancy probabilities on a 3D voxel grid. Such output reliably and efficiently captures multi-modal depth distributions in ambiguous cases.