Hezhen Hu

I am a postdoctoral fellow in VITA group in University of Texas at Austin (UT Austin), supervised by Prof. Zhangyang Wang.

I got my PhD degree from University of Science and Technology of China (USTC), supervised by Prof. Wengang Zhou and Prof. Houqiang Li. I served as a leader in the VSLRG research team of sign language understanding. From May 2022 to Feburary 2023, I am fortune to work as a research intern at Microsfot Research Asia, supervised by Jianmin Bao, Dongdong Chen, Lu Yuan, Dong Chen, and Fang Wen.

I'm interested in sign language understanding, large-scale pre-training, human-centric visual understanding, and etc. Please feel free to contact me if you are interested in the above topics and want to collaborate with me.

Email  /  Google Scholar

profile photo
Selected Publications

Representative papers are highlighted. For the full list, please refer to Google Scholar.

b3do SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understanding
Hezhen Hu, Weichao Zhao, Wengang Zhou, Houqiang Li,
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023
Project

[Extension of SignBERT] The first self-supervised pre-training framework in sign language understanding.

b3do Hand-Object Interaction Image Generation
Hezhen Hu, Weilun Wang, Wengang Zhou, Houqiang Li,
Conference on Neural Information Processing Systems (NeurIPS), 2022
Project

We present a new task, i.e., hand-object interaction image generation. This task is challenging and research-worthy in many potential application scenarios, such as online shopping.

b3do Prior-aware Cross Modality Augmentation Learning for Continuous Sign Language Recognition
Hezhen Hu, Junfu Pu, Wengang Zhou, Hang Fang, Houqiang Li,
IEEE Transactions on Multimedia (TMM), 2023

We propose a novel cross-modality augmentation learning paradigm with prior incorporated for continuous SLR.
b3do Collaborative Multilingual Sign Language Recognition: A Unified Framework
Hezhen Hu, Junfu Pu, Wengang Zhou, Houqiang Li,
IEEE Transactions on Multimedia (TMM), 2022

We are the first to explore the multilingual topic in continuous SLR and propose a unified framework targeting this problem.

b3do SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition
Hezhen Hu, Weichao Zhao, Wengang Zhou, Houqiang Li,
IEEE International Conference on Computer Vision (ICCV), 2021

The first self-supervised pre-training framework in isolated SLR. It conducts masked modeling with hand-prior incorporated for better capturing context in the sign language domain.
b3do Model-Aware Gesture-to-Gesture Translation
Hezhen Hu, Weilun Wang, Wengang Zhou, Weichao Zhao, Houqiang Li,
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021

The first model-aware framework in gesture-to-gesture translation.
b3do Hand-Model-Aware Sign Language Recognition
Hezhen Hu, Wengang Zhou, Houqiang Li,
AAAI Conference on Artificial Intelligence (AAAI), 2021

The first model-aware framework in isolated SLR.
b3do Global-local Enhancement Network for NMFs-aware Sign Language Recognition
Hezhen Hu, Wengang Zhou, Junfu Pu, Houqiang Li,
ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 2021

The first dataset explicitly exploring the importance of non-manual features in sign language.
Academic Services

I am happy to serve as a reviewer. Please feel free to contact me. Till now, I have been invited to review the following Journals and Conferences:

  • IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • European Conference on Computer Vision (ECCV)
  • AAAI Conference on Artificial Intelligence (AAAI)
  • International Joint Conference on Artificial Intelligence (IJCAI)
  • IEEE Transactions on Multimedia (TMM)
  • IEEE Transactions on Circuits and Systems for Video Technology (TCSVT)
Awards
  • National Scholarship, Education Ministry of China, 2021, 2017, 2015
  • Presidential Scholarship, Chinese Academy of Sciences (CAS), 2023
  • "Stars of Tomorrow" Excellent Internship Award, MSRA, 2023
  • Outstanding Graduate Student of USTC, 2023

Last Update: December 22th, 2023
Template borrowed from Jon Barron. Thanks!