About Me

I am currently a second-year Ph.D. student at Show Lab, National University of Singapore (NUS), supervised by Prof. Mike Shou. Previously, I was an AI researcher at Netease Fuxi AI Lab, working with Dr. Lincheng Li. I received my bachelor’s degree and master’s degree from the Image Processing Center of Beihang University, supervised by Prof. Zhenwei Shi, also working closely with Prof. Zhengxia Zou.

My current research interests lie in multimodal and AIGC, including text-driven video generation and avatar generation.

News

  • 2023.10: Two co-authored papers, Mix-of-Show and DatasetDM, accepted by NeurIPS 2023.
  • 2023.07: Invited talk at OPPO Research Institute, “Text-Driven Avatar Auto-Creation”.
  • 2023.03: One first-authored paper accepted by CVPR 2023.
  • 2022.08: One first-authored paper selected as ESI Highly Cited Paper.
  • 2022.07: One paper accepted by IEEE Transactions on Image Processing (TIP).
  • 2022.01: Awarded the Outstanding Graduates of Beijing (Top 1%).
  • 2021.09: Awarded the National Scholarship.

Selected Publications

Video Generation & Editing:

arXiv
sym

MotionDirector: Motion Customization of Text-to-Video Diffusion Models
arXiv, 2023
Rui Zhao, Yuchao Gu, Jay Zhangjie Wu, David Junhao Zhang, Jiawei Liu, Weijia Wu, Jussi Keppo, Mike Zheng Shou
[Project Page][arXiv] [Github ]

arXiv
sym

Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation
arXiv, 2023
David Junhao Zhang, Jay Zhangjie Wu, Jia-Wei Liu, Rui Zhao, Lingmin Ran, Yuchao Gu, Difei Gao, Mike Zheng Shou (equal contribution)
[Project Page][arXiv] [Github ]

TIP 2022
sym

Castle in the Sky: Dynamic Sky Replacement and Harmonization in Videos
IEEE Transactions on Image Processing, 2022
Zhengxia Zou, Rui Zhao, Tianyang Shi, Shuang Qiu, and Zhenwei Shi
[PDF][Project Page] [Github ]

Featured apps: Weights & Biases, a ML developer tool with 100,000+ practitioners.

Media coverage: [TNW] This open-source AI tool can make your video spectacular with sky replacement effects || [Better Programming] The Top 10 Trending ML Projects of 2020.

Avatar Generation:

CVPR 2023
sym

Zero-Shot Text-to-Parameter Translation for Game Character Auto-Creation
The IEEE Conference on Computer Vision and Pattern Recognition, 2023
Rui Zhao, Wei Li, Zhipeng Hu, Lincheng Li, Zhengxia Zou, Zhenwei Shi, and Changjie Fan
[PDF]

Image Captioning & Generation:

TGRS 2021
sym

High-resolution remote sensing image captioning based on structured attention
IEEE Transactions on Geoscience and Remote Sensing, 2021
Rui Zhao, Zhenwei Shi, and Zhengxia Zou
[PDF]

🏆️ ESI Highly Cited Paper*
* received enough citations to place in the top 1% of the academic field of Geosciences based on publication year. [Clarivate]

TGRS 2022
sym

Remote Sensing Image Change Captioning With Dual-Branch Transformers: A New Method and a Large Scale Dataset
IEEE Transactions on Geoscience and Remote Sensing, 2022
Chenyang Liu, Rui Zhao, Hao Chen, Zhengxia Zou, and Zhenwei Shi
[PDF] [Dataset] [Code]

GRSL 2022
sym

Text to Remote Sensing Image Generation With Structured Generative Adversarial Networks
IEEE Geoscience and Remote Sensing Letters, 2022
Rui Zhao, Zhenwei Shi
[PDF]

RS 2019
sym

Ensemble-based cascaded constrained energy minimization for hyperspectral target detection
Remote Sensing, 2019
Rui Zhao, Zhenwei Shi, Zhengxia Zou, and Zhou Zhang
[PDF][Code]

Selected Honors

  • 2022 The Outstanding Graduates of Beijing (Top 1%)
  • 2021 National Scholarship, Ministry of Education of China (Top 1%)
  • 2019 The Outstanding Graduates of Beijing (Top 1%)

Academic Service

  • Conference Reviewer: ICCV 2023, CVPR 2024, ECCV 2024, NeurIPS 2024.

  • Journal Reviewer: IEEE Transactions on Geoscience and Remote Sensing (Q1); ISPRS Journal of Photogrammetry and Remote Sensing (Q1); IEEE Geoscience and Remote Sensing Letters (Q1); IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing (Q1); Remote Sensing (Q1); International Journal of Digital Earth (Q1).