ShadowDiffusion: When Degradation Prior Meets Diffusion Model for Shadow Removal
Lanqing Guo, Chong Wang, Wenhan Yang, Siyu Huang, Yufei Wang, Hanspeter Pfister, Bihan Wen
In CVPR'2023
- Aug 03, 2024: Release the complete code, pretrained model for ISTD, ISTD+, and evaluation results. Sorry for updating late 🙏.
- Aug 08, 2023: Release the pretrained model for SRD and evaluation results.
- Aug 05, 2023: Release the training and testing codes.
Our work addresses shadow removal by proposing a unified diffusion framework, dubbed ShadowDiffusion, that integrates both the image and degradation priors for highly effective shadow removal. ShadowDiffusion progressively refines the estimated shadow mask as an auxiliary task of the diffusion generator, which leads to more accurate and robust shadow-free image generation. For more details, please refer to our original paper.
- Python 3.7
- Pytorch 1.7
- CUDA 11.1
pip install -r requirements.txt
Please download the corresponding pretrained model and modify the resume_state
and degradation_model_path
(optional) in shadow.json
.
You can directly test the performance of the pre-trained model as follows
- Modify the paths to dataset and pre-trained model. You need to modify the following path in the
shadow.json
resume_state # pretrain model or training state -- Line 12
dataroot # validation dataset path -- Line 30
- Test the model
python sr.py -p val -c config/shadow_SRD.json
We use the DDIM sampling to speed up the inference stage. The number of steps can be set by T_sampling
as 5 or 25.
- Download datasets and set the following structure
|-- SRD_Dataset
|-- train
|-- train_A # shadow image
|-- train_B # shadow mask
|-- train_C # shadow-free GT
|-- test
|-- test_A # shadow image
|-- test_B # shadow mask
|-- test_C # shadow-free GT
- You need to modify the following terms in
option.py
"resume_state": null # if train from scratch
"dataroot" # training and testing set path
"gpu_ids": [0] # Our model can be trained using a single RTX A5000 GPU. You can also train the model using multiple GPUs by adding more GPU ids in it.
- Train the network
python sr.py -p train -c config/shadow.json
The results reported in the paper are calculated by the matlab
script used in previous method. Details refer to evaluation/measure_shadow.m
.
The testing results on dataset ISTD, ISTD+, SRD are: results
Our implementation is based on SR3 and WeatherDiffusion. We would like to thank them.
Preprint available here.
In case of use, please cite our publication:
L. Guo, C. Wang, W. Yang, S. Huang, Y. Wang, P. Hanspeter, and B. Wen, "Shadowdiffusion: When degradation prior meets diffusion model for shadow removal," CVPR 2023.
Bibtex:
@inproceedings{guo2023shadowdiffusion,
title={Shadowdiffusion: When degradation prior meets diffusion model for shadow removal},
author={Guo, Lanqing and Wang, Chong and Yang, Wenhan and Huang, Siyu and Wang, Yufei and Pfister, Hanspeter and Wen, Bihan},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={14049--14058},
year={2023}
}
If you have any questions, please contact [email protected]