Realistic Full-Body Motion Generation from Sparse Tracking with State Space Model

Oct 28, 2024·
Kun Dong
,
Jian Xue
Zehai Niu
Zehai Niu
,
Xing Lan
,
Ke Lu
,
Qingyuan Liu
,
Xiaoyu Qin
· 0 min read
Abstract
A state-space-model-based framework that generates realistic full-body motion from sparse VR/AR tracker signals (head and hands), achieving state-of-the-art quality with significantly lower computational cost.
Type
Publication
Proceedings of the 32nd ACM International Conference on Multimedia (ACM MM)
Status
Peer-reviewed
publications
Motion Generation VR/AR State Space Models
Zehai Niu
Authors
Zehai Niu (he/him)
Assistant Professor · Shenzhen University

Zehai Niu is an Assistant Professor at Shenzhen University. His research focuses on edge-device inference acceleration for Vision-Language Models (VLM) and VLM interpretability — building VLMs that run fast on consumer devices, and that we actually understand.

He received his PhD from the University of Chinese Academy of Sciences (UCAS) in 2024. From July 2024 to May 2026, he was an AI Researcher at HONOR Device, where he led innovation for the gaming business — including M-DLSS 3.0, the industry’s first NPU-GPU heterogeneous frame interpolation system shipped to consumer phones — and the embodied intelligence stack of RobotPhone. During this period, he worked closely with Google’s Gemma team, Qualcomm, and Alibaba’s Qwen (Tongyi Qianwen) Lab.

He has published 20+ papers at ICCV, ECCV, ACM MM, IEEE TCSVT, IEEE TMM and other top venues, holds 10+ patents, and serves as a reviewer for ICLR, ICCV, ECCV, and ACM MM.