Hezhen Hu

I am a postdoctoral fellow in VITA group in University of Texas at Austin (UT Austin), working with Prof. Zhangyang Wang and Prof. Georgios Pavlakos. My research aims to build human-centered AI systems that treat human communication and interaction as first-class capabilities, grounded in embodiment and real-world context, enabling physically and socially meaningful interactions between people and AI. This vision has translated into sustained real-world deployments for Deaf communities. In parallel, I have deeply engaged in education and outreach.

I got my PhD degree from University of Science and Technology of China (USTC), supervised by Prof. Wengang Zhou and Prof. Houqiang Li. I served as a leader in the VSLRG research team of sign language understanding. From May 2022 to Feburary 2023, I am fortune to work as a research intern at Microsfot Research Asia, supervised by Jianmin Bao, Dongdong Chen, Lu Yuan, Dong Chen, and Fang Wen.

Please feel free to contact me if you are interested in the above topics and want to collaborate with me.

Email  /  Google Scholar

• 2026.01: We are organizing the 1st GenSign workshop at CVPR 2026.
• 2025.10: We are organizing the 3rd AI3DCC workshop at ICCV 2025.

Representative papers are highlighted. For the full list, please refer to Google Scholar.

Expressive animatable avatar (hand & face), learn from in-the-wild monocular RGB video (no SMPL-X annotation required).

[Extension of SignBERT] The first self-supervised pre-training framework in sign language understanding.

We present a new task, i.e., hand-object interaction image generation. This task is challenging and research-worthy in many potential application scenarios, such as online shopping.

We propose a large-scale generative pre-training strategy, that eliminates the gap between pre-training and downstream SLU tasks.

We propose a novel cross-modality augmentation learning paradigm with prior incorporated for continuous SLR.

We are the first to explore the multilingual topic in continuous SLR and propose a unified framework targeting this problem.

The first self-supervised pre-training framework in isolated SLR. It conducts masked modeling with hand-prior incorporated for better capturing context in the sign language domain.

The first model-aware framework in gesture-to-gesture translation.

The first model-aware framework in isolated SLR.

The first dataset explicitly exploring the importance of non-manual features in sign language.

Deaf-facing deployment: Our sign-language foundation models support real-world interactions in public services and robotics.

K–12 education: Co-authored an AI textbook series adopted nationwide, distributed across 500+ schools and reaching 300,000+ students.

I serve as Area Chair (AC) for International Conference on Acoustics, Speech, and Signal Processing (ICASSP) and have reviewed the following Journals and Conferences:

• IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
• International Conference on Computer Vision (ICCV)
• European Conference on Computer Vision (ECCV)
• Conference on Neural Information Processing Systems (NeurIPS)
• International Conference on Learning Representations (ICLR)
• International Conference on Machine Learning (ICML)
• AAAI Conference on Artificial Intelligence (AAAI)
• International Joint Conference on Artificial Intelligence (IJCAI)
• IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
• International Journal of Computer Vision (IJCV)
• IEEE Transactions on Multimedia (TMM)
• IEEE Transactions on Circuits and Systems for Video Technology (TCSVT)

I am happy to serve as a reviewer. Please feel free to contact me.

• MSRA "Stars of Tomorrow" Award, 2023
• University Excellent Doctoral Dissertation, 2023
• Outstanding Graduate Student, 2023
• ECCV Looking at People CSLR Challenge, Ranked 1st, 2022
• National Scholarship, 2021, 2017, 2015

Last Update: January 21th, 2025
Template borrowed from Jon Barron. Thanks!