About Me
I am currently a second-year Ph.D. student at Show Lab, National University of Singapore (NUS), supervised by Prof. Mike Shou. Previously, I was an AI researcher at Netease Fuxi AI Lab, working with Dr. Lincheng Li. I received my bachelor’s degree and master’s degree from the Image Processing Center of Beihang University, supervised by Prof. Zhenwei Shi, also working closely with Prof. Zhengxia Zou.
My current research interests lie in multimodal and AIGC, including text-driven video generation and avatar generation.
News
- 2023.10: Two co-authored papers, Mix-of-Show and DatasetDM, accepted by NeurIPS 2023.
- 2023.07: Invited talk at OPPO Research Institute, “Text-Driven Avatar Auto-Creation”.
- 2023.03: One first-authored paper accepted by CVPR 2023.
- 2022.08: One first-authored paper selected as ESI Highly Cited Paper.
- 2022.07: One paper accepted by IEEE Transactions on Image Processing (TIP).
- 2022.01: Awarded the Outstanding Graduates of Beijing (Top 1%).
- 2021.09: Awarded the National Scholarship.
Selected Publications
Video Generation & Editing:
MotionDirector: Motion Customization of Text-to-Video Diffusion Models
arXiv, 2023
Rui Zhao, Yuchao Gu, Jay Zhangjie Wu, David Junhao Zhang, Jiawei Liu, Weijia Wu, Jussi Keppo, Mike Zheng Shou
[Project Page][arXiv]
[Github 
Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation
arXiv, 2023
David Junhao Zhang†, Jay Zhangjie Wu†, Jia-Wei Liu†, Rui Zhao, Lingmin Ran, Yuchao Gu, Difei Gao, Mike Zheng Shou (†equal contribution)
[Project Page][arXiv]
[Github 
Castle in the Sky: Dynamic Sky Replacement and Harmonization in Videos
IEEE Transactions on Image Processing, 2022
Zhengxia Zou, Rui Zhao, Tianyang Shi, Shuang Qiu, and Zhenwei Shi
[PDF][Project Page]
[Github 
Featured apps: Weights & Biases, a ML developer tool with 100,000+ practitioners.
Media coverage:
[TNW] This open-source AI tool can make your video spectacular with sky replacement effects ||
[Better Programming] The Top 10 Trending ML Projects of 2020.
Avatar Generation:
Zero-Shot Text-to-Parameter Translation for Game Character Auto-Creation
The IEEE Conference on Computer Vision and Pattern Recognition, 2023
Rui Zhao, Wei Li, Zhipeng Hu, Lincheng Li, Zhengxia Zou, Zhenwei Shi, and Changjie Fan
[PDF]
Media coverage: [NewScientist] Character creator AI puts Barack Obama – or anyone – in a video game || [游戏日报] 网易伏羲“文字捏脸”方向论文入选CVPR会议，称已应用于《逆水寒手游》 || CVPR 2023 | 网易伏羲5篇论文入选 包含文字捏脸等业内首创工作 || [AUTOMATON] Researchers report technological breakthroughs in using AI to “make characters by asking with words”. (in Japanese, for English version, please refer to reprint 1 and reprint 2) || [Nazology] "Character creation support AI" that creates the ideal face specified in the sentence is now available! .
Image Captioning & Generation:
High-resolution remote sensing image captioning based on structured attention
IEEE Transactions on Geoscience and Remote Sensing, 2021
Rui Zhao, Zhenwei Shi, and Zhengxia Zou
[PDF]
🏆️ ESI Highly Cited Paper*
* received enough citations to place in the top 1% of the academic field of Geosciences based on publication year. [Clarivate]
Text to Remote Sensing Image Generation With Structured Generative Adversarial Networks
IEEE Geoscience and Remote Sensing Letters, 2022
Rui Zhao, Zhenwei Shi
[PDF]
Selected Honors
- 2022 The Outstanding Graduates of Beijing (Top 1%)
- 2021 National Scholarship, Ministry of Education of China (Top 1%)
- 2019 The Outstanding Graduates of Beijing (Top 1%)
- …
Academic Service
-
Conference Reviewer: ICCV 2023, CVPR 2024, ICML 2024, ECCV 2024.
-
Journal Reviewer: IEEE Transactions on Geoscience and Remote Sensing (Q1); ISPRS Journal of Photogrammetry and Remote Sensing (Q1); IEEE Geoscience and Remote Sensing Letters (Q1); IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing (Q1); Remote Sensing (Q1); International Journal of Digital Earth (Q1).