About Me

I am currently a second-year Ph.D. student at Show Lab, National University of Singapore (NUS), supervised by Prof. Mike Shou. Previously, I was an AI researcher at Netease Fuxi AI Lab, working with Dr. Lincheng Li. I received my bachelor’s degree and master’s degree from the Image Processing Center of Beihang University, supervised by Prof. Zhenwei Shi, also working closely with Prof. Zhengxia Zou.

My current research interests lie in multimodal and GenAI, including customizing and improving foundational generation models.

News

  • 2024.07: MotionDirector got accepted for oral presentation at ECCV 2024.
  • 2023.10: Foundational T2V generation model Show-1 released!
  • 2023.07: Invited talk at OPPO Research Institute, “Text-Driven Avatar Auto-Creation”.
  • 2023.04: A collection of papers on video diffusion models: Awesome-Video-Diffusion .
  • 2023.03: Our paper for avatar auto-creation was accepted by CVPR 2023.
  • 2022.08: One first-authored paper selected as ESI Highly Cited Paper.
  • 2022.07: Castle in the Sky accepted by IEEE Transactions on Image Processing.
  • 2022.01: Awarded the Outstanding Graduates of Beijing.
  • 2021.09: Awarded the National Scholarship.

Selected Publications

NeurIPS 2024
sym

EvolveDirector: Approaching Advanced Text-to-Image Generation with Large Vision-Language Models
The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024
Rui Zhao, Hangjie Yuan, Yujie Wei, Shiwei Zhang, Yuchao Gu, Lingmin Ran, Xiang Wang, Jay Wu, David Zhang, Yingya Zhang, Mike Zheng Shou.
[arXiv][Hugging Face] [Github ]

ECCV 2024
sym

MotionDirector: Motion Customization of Text-to-Video Diffusion Models
Proceedings of the European Conference on Computer Vision, 2024
Rui Zhao, Yuchao Gu, Jay Zhangjie Wu, David Junhao Zhang, Jiawei Liu, Weijia Wu, Jussi Keppo, Mike Zheng Shou
[Project Page][arXiv] [Github ]
🎙️ Oral Presentation, Acceptance Rate: 2.3%.
🤗 Featured in Hugging Face “Spaces of the Week 🔥” trending list.
📃 Featured in “Top 40 most cited papers of ECCV 2024” list by AIR-SUN.

IJCV 2024
sym

Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation
International Journal of Computer Vision, 2024
David Junhao Zhang, Jay Zhangjie Wu, Jia-Wei Liu, Rui Zhao, Lingmin Ran, Yuchao Gu, Difei Gao, Mike Zheng Shou (equal contribution)
[Project Page][arXiv] [Github ]

CVPR 2023
sym

Zero-Shot Text-to-Parameter Translation for Game Character Auto-Creation
The IEEE Conference on Computer Vision and Pattern Recognition, 2023
Rui Zhao, Wei Li, Zhipeng Hu, Lincheng Li, Zhengxia Zou, Zhenwei Shi, and Changjie Fan
[PDF]

TIP 2022
sym

Castle in the Sky: Dynamic Sky Replacement and Harmonization in Videos
IEEE Transactions on Image Processing, 2022
Zhengxia Zou, Rui Zhao, Tianyang Shi, Shuang Qiu, and Zhenwei Shi
[PDF][Project Page] [Github ]

Featured apps: Weights & Biases, a ML developer tool with 100,000+ practitioners.

Media coverage: [TNW] This open-source AI tool can make your video spectacular with sky replacement effects || [Better Programming] The Top 10 Trending ML Projects of 2020.

TGRS 2021
sym

High-resolution remote sensing image captioning based on structured attention
IEEE Transactions on Geoscience and Remote Sensing, 2021
Rui Zhao, Zhenwei Shi, and Zhengxia Zou
[PDF]

🏆️ ESI Highly Cited Paper*
* received enough citations to place in the top 1% of the academic field of Geosciences based on publication year. [Clarivate]

Selected Honors

  • 2022 The Outstanding Graduates of Beijing (Top 1%)
  • 2021 National Scholarship, Ministry of Education of China (Top 1%)
  • 2019 The Outstanding Graduates of Beijing (Top 1%)

Academic Service

  • Conference Reviewer: ICCV 2023, CVPR 2024-2025, ECCV 2024, NeurIPS 2024, ICLR 2025.

  • Journal Reviewer: IEEE Transactions on Geoscience and Remote Sensing (Q1); ISPRS Journal of Photogrammetry and Remote Sensing (Q1); IEEE Geoscience and Remote Sensing Letters (Q1); IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing (Q1); Remote Sensing (Q1); International Journal of Digital Earth (Q1); IEEE Internet of Things Journal (Q1).