Official Pytorch implementation for the “Event-Enhanced Blurry Video Super-Resolution” paper (AAAI 2025).
🌐 Project | 📃 Paper
Authors: Dachun Kai:email:️, Yueyi Zhang, Jin Wang, Zeyu Xiao, Zhiwei Xiong, Xiaoyan Sun, University of Science and Technology of China
Feel free to ask questions. If our work helps, please don’t hesitate to give us a :star:!
:rocket: News
:bookmark: Table of Contents
- Video Demos
- Code
- Citation
- Contact
- License and Acknowledgement
:fire: Video Demos
A $4\times$ blurry video upsampling results on the synthetic dataset GoPro and real-world dataset NCER test sets.
https://github.com/user-attachments/assets/df54a750-25fd-4ac1-9980-20ef7f73c738
https://github.com/user-attachments/assets/4d58c85f-1a47-4292-8e4a-4ea0ccfe1b0d
https://github.com/user-attachments/assets/cb7c3a62-5927-4f5a-8aec-258d7e1d513e
https://github.com/user-attachments/assets/0c030756-f2a0-4a9d-81a2-99943a0f881f
Code
Installation
-
Dependencies: Miniconda, CUDA Toolkit 11.1.1, torch 1.10.2+cu111, and torchvision 0.11.3+cu111.
-
Run in Conda (Recommended)
conda create -y -n ev-deblurvsr python=3.7
conda activate ev-deblurvsr
pip install torch-1.10.2+cu111-cp37-cp37m-linux_x86_64.whl
pip install torchvision-0.11.3+cu111-cp37-cp37m-linux_x86_64.whl
git clone https://github.com/DachunKai/Ev-DeblurVSR
cd Ev-DeblurVSR && pip install -r requirements.txt && python setup.py develop
-
Run in Docker :clap:
Note: We use the same docker image as our previous work EvTexture.
[Option 1] Directly pull the published Docker image we have provided from Alibaba Cloud.
docker pull registry.cn-hangzhou.aliyuncs.com/dachunkai/evtexture:latest
[Option 2] We also provide a Dockerfile that you can use to build the image yourself.
cd EvTexture && docker build -t evtexture ./docker
The pulled or self-built Docker image contains a complete conda environment named evtexture. After running the image, you can mount your data and operate within this environment.
source activate evtexture && cd EvTexture && python setup.py develop
Test
-
Download the pretrained models from (Releases / Baidu Cloud (n8hg)) and place them to experiments/pretrained_models/EvDeblurVSR/. The network architecture code is in evdeblurvsr_arch.py.
- Synthetic dataset model:
- EvDeblurVSR_GOPRO_BIx4.pth: trained on GoPro dataset with Blur-Sharp pairs and BI degradation for $4\times$ SR scale.
- EvDeblurVSR_BSD_BIx4.pth: trained on BSD dataset with Blur-Sharp pairs and BI degradation for $4\times$ SR scale.
- Real-world dataset model:
- EvDeblurVSR_NCER_BIx4.pth: trained on NCER dataset with Blur-Sharp pairs and BI degradation for $4\times$ SR scale.
-
Download the preprocessed test sets (including events) for GoPro, BSD, and NCER from (Baidu Cloud (n8hg) / Google Drive), and place them to datasets/.
-
GoPro_h5: HDF5 files containing preprocessed test datasets for the GoPro test set.
-
BSD_h5: HDF5 files containing preprocessed test datasets for the BSD dataset.
-
NCER_h5: HDF5 files containing preprocessed test datasets for the NCER dataset.
-
Run the following command:
- Test on GoPro for 4x Blurry VSR:
./scripts/dist_test.sh [num_gpus] options/test/EvDeblurVSR/test_EvDeblurVSR_GoPro_x4.yml
- Test on BSD for 4x Blurry VSR:
./scripts/dist_test.sh [num_gpus] options/test/EvDeblurVSR/test_EvDeblurVSR_BSD_x4.yml
- Test on NCER for 4x Blurry VSR:
./scripts/dist_test.sh [num_gpus] options/test/EvDeblurVSR/test_EvDeblurVSR_NCER_x4.yml
This will generate the inference results in results/. The output results on GoPro, BSD and NCER datasets can be downloaded from (Releases / Baidu Cloud (n8hg)).
-
Test the number of parameters, runtime, and FLOPs:
python test_scripts/test_params_runtime.py
-
Both video and event data are required as input, as shown in the snippet. We package each video and its event data into an HDF5 file.
-
Example: The structure of GOPR0384_11_00.h5 file from the GoPro dataset is shown below.
GOPR0384_11_00.h5
├── images
│ ├── 000000 # frame, ndarray, [H, W, C]
│ ├── ...
├── vFwd
│ ├── 000000 # inter-frame forward event voxel, ndarray, [Bins, H, W]
│ ├── ...
├── vBwd
│ ├── 000000 # inter-frame backward event voxel, ndarray, [Bins, H, W]
│ ├── ...
├── vExpo
│ ├── 000000 # intra-frame exposure event voxel, ndarray, [Bins, H, W]
│ ├── ...
:blush: Citation
If you find the code and pre-trained models useful for your research, please consider citing our paper. :smiley:
@inproceedings{kai2025event,
title={Event-{E}nhanced {B}lurry {V}ideo {S}uper-{R}esolution},
author={Kai, Dachun and Zhang, Yueyi and Wang, Jin and Xiao, Zeyu and Xiong, Zhiwei and Sun, Xiaoyan},
booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
volume={39},
number={4},
pages={4175--4183},
year={2025}
}
If you meet any problems, please describe them in issues or contact:
License and Acknowledgement
This project is released under the Apache 2.0 License. Our work builds significantly upon our previous project EvTexture. We would also like to sincerely thank the developers of BasicSR, an open-source toolbox for image and video restoration tasks. Additionally, we appreciate the inspiration and code provided by BasicVSR++, RAFT and event_utils.
评论