Restormer: Efficient Transformer for High-Resolution Image Restoration
Task: Denoising, Deblurring, Deraining
Since convolutional neural networks (CNNs) perform well at learning generalizable image priors from large-scale data, these models have been extensively applied to image restoration and related tasks. Recently, another class of neural architectures, Transformers, have shown significant performance gains on natural language and high-level vision tasks. While the Transformer model mitigates the shortcomings of CNNs (i.e., limited receptive field and inadaptability to input content), its computational complexity grows quadratically with the spatial resolution, therefore making it infeasible to apply to most image restoration tasks involving high-resolution images. In this work, we propose an efficient Transformer model by making several key designs in the building blocks (multi-head attention and feed-forward network) such that it can capture long-range pixel interactions, while still remaining applicable to large images. Our model, named Restoration Transformer (Restormer), achieves state-of-the-art results on several image restoration tasks, including image deraining, single-image motion deblurring, defocus deblurring (single-image and dual-pixel data), and image denoising (Gaussian grayscale/color denoising, and real image denoising).
Evaluated on Y channels. The metrics are PSNR / SSIM .
| Model | Dataset | PSNR (Y) | SSIM (Y) | Training Resources | Download |
|---|---|---|---|---|---|
| restormer_official_rain13k | Rain100H | 31.4804 | 0.9056 | 1 | model | log |
| restormer_official_rain13k | Rain100L | 39.1023 | 0.9787 | 1 | model | log |
| restormer_official_rain13k | Test100 | 32.0287 | 0.9239 | 1 | model | log |
| restormer_official_rain13k | Test1200 | 33.2251 | /0.9272 | 1 | model | log |
| restormer_official_rain13k | Test2800 | 34.2170 | 0.9451 | 1 | model | log |
Evaluated on RGB channels for GoPro and HIDE, and Y channel for ReakBlur-J and ReakBlur-R. The metrics are PSNR / SSIM .
| Model | Dataset | PSNR/SSIM (RGB) | PSNR/SSIM (Y) |
Training Resources | Download |
|---|---|---|---|---|---|
| restormer_official_gopro | DatasetGoPro | 32.9295/0.9496 | - | 1 | model | log |
| restormer_official_gopro | HIDE | 31.2289/0.9345 | - | 1 | model | log |
| restormer_official_gopro | RealBlur-J | - | 28.4356/0.8681 | 1 | model | log |
| restormer_official_gopro | RealBlur-R | - | 35.9141/0.9707 | 1 | model | log |
Evaluated on RGB channels. The metrics are PSNR / SSIM / MAE / LPIPS.
| Model | Dataset | PSNR | SSIM | MAE | Training Resources | Download |
|---|---|---|---|---|---|---|
| restormer_official_dpdd-single | Indoor Scenes | 28.8681 | 0.8859 | 0.0251 | 1 | model | log |
| restormer_official_dpdd-single | Outdoor Scenes | 23.2410 | 0.7509 | 0.0499 | 1 | model | log |
| restormer_official_dpdd-single | Combined | 25.9805 | 0.8166 | 0.0378 | 1 | model | log |
| restormer_official_dpdd-dual | Indoor Scenes | 26.6160 | 0.8346 | 0.0354 | 1 | model | log |
| restormer_official_dpdd-dual | Outdoor Scenes | 26.6160 | 0.8346 | 0.0354 | 1 | model | log |
| restormer_official_dpdd-dual | Combined | 26.6160 | 0.8346 | 0.0354 | 1 | model | log |
Test Grayscale Gaussian Noise
Evaluated on grayscale images. The metrics are PSNR / SSIM .
training a separate model for each noise level
| Model | Dataset | PSNR | SSIM | Training Resources | Download | |
|---|---|---|---|---|---|---|
| restormer_official_dfwb-gray-sigma15 | Set12 | 15 | 34.0182 | 0.9160 | 1 | model | log |
| restormer_official_dfwb-gray-sigma15 | BSD68 | 15 | 32.4987 | 0.8940 | 1 | model | log |
| restormer_official_dfwb-gray-sigma15 | Urban100 | 15 | 34.4336 | 0.9419 | 1 | model | log |
| restormer_official_dfwb-gray-sigma25 | Set12 | 25 | 31.7289 | 0.8811 | 1 | model | log |
| restormer_official_dfwb-gray-sigma25 | BSD68 | 25 | 30.1613 | 0.8370 | 1 | model | log |
| restormer_official_dfwb-gray-sigma25 | Urban100 | 25 | 32.1162 | 0.9140 | 1 | model | log |
| restormer_official_dfwb-gray-sigma50 | Set12 | 50 | 28.6269 | 0.8188 | 1 | model | log |
| restormer_official_dfwb-gray-sigma50 | BSD68 | 50 | 27.3266 | 0.7434 | 1 | model | log |
| restormer_official_dfwb-gray-sigma50 | Urban100 | 50 | 28.9636 | 0.8571 | 1 | model | log |
learning a single model to handle various noise levels
| Model | Dataset | PSNR | SSIM | Training Resources | Download | |
|---|---|---|---|---|---|---|
| restormer_official_dfwb-gray-sigma15 | Set12 | 15 | 33.9642 | 0.9153 | 1 | model | log |
| restormer_official_dfwb-gray-sigma15 | BSD68 | 15 | 32.4994 | 0.8928 | 1 | model | log |
| restormer_official_dfwb-gray-sigma15 | Urban100 | 15 | 34.3152 | 0.9409 | 1 | model | log |
| restormer_official_dfwb-gray-sigma25 | Set12 | 25 | 31.7106 | 0.8810 | 1 | model | log |
| restormer_official_dfwb-gray-sigma25 | BSD68 | 25 | 30.1486 | 0.8360 | 1 | model | log |
| restormer_official_dfwb-gray-sigma25 | Urban100 | 25 | 32.0457 | 0.9131 | 1 | model | log |
| restormer_official_dfwb-gray-sigma50 | Set12 | 50 | 28.6614 | 0.8197 | 1 | model | log |
| restormer_official_dfwb-gray-sigma50 | BSD68 | 50 | 27.3537 | 0.7422 | 1 | model | log |
| restormer_official_dfwb-gray-sigma50 | Urban100 | 50 | 28.9848 | 0.8571 | 1 | model | log |
Test Color Gaussian Noise
Evaluated on RGB channels. The metrics are PSNR / SSIM .
training a separate model for each noise level
| Model | Dataset | PSNR (RGB) | SSIM (RGB) | Training Resources | Download | |
|---|---|---|---|---|---|---|
| restormer_official_dfwb-color-sigma15 | CBSD68 | 15 | 34.3506 | 0.9352 | 1 | model | log |
| restormer_official_dfwb-color-sigma15 | Kodak24 | 15 | 35.4900 | 0.9312 | 1 | model | log |
| restormer_official_dfwb-color-sigma15 | McMaster | 15 | 35.6072 | 0.9352 | 1 | model | log |
| restormer_official_dfwb-color-sigma15 | Urban100 | 15 | 35.1522 | 0.9530 | 1 | model | log |
| restormer_official_dfwb-color-sigma25 | CBSD68 | 25 | 31.7457 | 0.8942 | 1 | model | log |
| restormer_official_dfwb-color-sigma25 | Kodak24 | 25 | 33.0489 | 0.8943 | 1 | model | log |
| restormer_official_dfwb-color-sigma25 | McMaster | 25 | 33.3260 | 0.9066 | 1 | model | log |
| restormer_official_dfwb-color-sigma25 | Urban100 | 25 | 32.9670 | 0.9317 | 1 | model | log |
| restormer_official_dfwb-color-sigma50 | CBSD68 | 50 | 28.5569 | 0.8127 | 1 | model | log |
| restormer_official_dfwb-color-sigma50 | Kodak24 | 50 | 30.0122 | 0.8238 | 1 | model | log |
| restormer_official_dfwb-color-sigma50 | McMaster | 50 | 30.2608 | 0.8515 | 1 | model | log |
| restormer_official_dfwb-color-sigma50 | Urban100 | 50 | 30.0230 | 0.8902 | 1 | model | log |
learning a single model to handle various noise levels
| Model | Dataset | PSNR (RGB) | SSIM (RGB) | Training Resources | Download | |
|---|---|---|---|---|---|---|
| restormer_official_dfwb-color-sigma15 | 15 | 34.3422 | 0.9356 | 1 | model | log | |
| restormer_official_dfwb-color-sigma15 | 15 | 35.4544 | 0.9308 | 1 | model | log | |
| restormer_official_dfwb-color-sigma15 | 15 | 35.5473 | 0.9344 | 1 | model | log | |
| restormer_official_dfwb-color-sigma15 | 15 | 35.0754 | 0.9524 | 1 | model | log | 1 |
| restormer_official_dfwb-color-sigma25 | 25 | 31.7391 | 0.8945 | 1 | model | log | |
| restormer_official_dfwb-color-sigma25 | 25 | 33.0380 | 0.8941 | 1 | model | log | |
| restormer_official_dfwb-color-sigma25 | 25 | 33.3040 | 0.9063 | 1 | model | log | |
| restormer_official_dfwb-color-sigma25 | 25 | 32.9165 | 0.9312 | 1 | model | log | |
| restormer_official_dfwb-color-sigma50 | 50 | 28.5582 | 0.8126 | 1 | model | log | |
| restormer_official_dfwb-color-sigma50 | 50 | 30.0074 | 0.8233 | 1 | model | log | |
| restormer_official_dfwb-color-sigma50 | 50 | 30.2671 | 0.8520 | 1 | model | log | |
| restormer_official_dfwb-color-sigma50 | 50 | 30.0172 | 0.8898 | 1 | model | log | 1 |
Evaluated on RGB channels. The metrics are PSNR / SSIM .
| Model | Dataset | PSNR | SSIM | Training Resources | Download |
|---|---|---|---|---|---|
| restormer_official_sidd | SIDD | 40.0156 | 0.9225 | 1 | model | log |
Train
You can refer to Train a model part in train_test.md.
Test
Test Instructions
You can use the following commands to test a model with cpu or single/multiple GPUs.
# cpu test
# Deraining
CUDA_VISIBLE_DEVICES=-1 python tools/test.py configs/restormer/restormer_official_rain13k.py https://download.openmmlab.com/mmediting/restormer/restormer_official_rain13k-2be7b550.pth
# Motion Deblurring
CUDA_VISIBLE_DEVICES=-1 python tools/test.py configs/restormer/restormer_official_gopro.py https://download.openmmlab.com/mmediting/restormer/restormer_official_gopro-db7363a0.pth
# Defocus Deblurring
# Single
CUDA_VISIBLE_DEVICES=-1 python tools/test.py configs/restormer/restormer_official_dpdd-dual.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dpdd-single-6bc31582.pth
# Dual
CUDA_VISIBLE_DEVICES=-1 python tools/test.py configs/restormer/restormer_official_dpdd-single.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dpdd-dual-52c94c00.pth
# Gaussian Denoising
# Test Grayscale Gaussian Noise
# sigma15
CUDA_VISIBLE_DEVICES=-1 python tools/test.py configs/restormer/restormer_official_dfwb-gray-sigma15.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-gray-sigma15-da74417f.pth
CUDA_VISIBLE_DEVICES=-1 python tools/test.py configs/restormer/restormer_official_dfwb-gray-sigma15.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-gray-blind-5f094bcc.pth
# sigma25
CUDA_VISIBLE_DEVICES=-1 python tools/test.py configs/restormer/restormer_official_dfwb-gray-sigma25.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-gray-sigma25-08010841.pth
CUDA_VISIBLE_DEVICES=-1 python tools/test.py configs/restormer/restormer_official_dfwb-gray-sigma25.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-gray-blind-5f094bcc.pth
# sigma50
CUDA_VISIBLE_DEVICES=-1 python tools/test.py configs/restormer/restormer_official_dfwb-gray-sigma50.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-gray-sigma50-ee852dfe.pth
CUDA_VISIBLE_DEVICES=-1 python tools/test.py configs/restormer/restormer_official_dfwb-gray-sigma50.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-gray-blind-5f094bcc.pth
# Test Color Gaussian Noise
# sigma15
CUDA_VISIBLE_DEVICES=-1 python tools/test.py configs/restormer/restormer_official_dfwb-color-sigma15.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-color-sigma15-012ceb71.pth
CUDA_VISIBLE_DEVICES=-1 python tools/test.py configs/restormer/restormer_official_dfwb-color-sigma15.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-color-blind-dfd03c9f.pth
# sigma25
CUDA_VISIBLE_DEVICES=-1 python tools/test.py configs/restormer/restormer_official_dfwb-color-sigma25.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-color-sigma25-e307f222.pth
CUDA_VISIBLE_DEVICES=-1 python tools/test.py configs/restormer/restormer_official_dfwb-color-sigma25.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-color-blind-dfd03c9f.pth
# sigma50
CUDA_VISIBLE_DEVICES=-1 python tools/test.py configs/restormer/restormer_official_dfwb-color-sigma50.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-color-sigma50-a991983d.pth
CUDA_VISIBLE_DEVICES=-1 python tools/test.py configs/restormer/restormer_official_dfwb-color-sigma50.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-color-blind-dfd03c9f.pth
# single-gpu test
# Deraining
python tools/test.py configs/restormer/restormer_official_rain13k.py https://download.openmmlab.com/mmediting/restormer/restormer_official_rain13k-2be7b550.pth
# Motion Deblurring
python tools/test.py configs/restormer/restormer_official_gopro.py https://download.openmmlab.com/mmediting/restormer/restormer_official_gopro-db7363a0.pth
# Defocus Deblurring
# Single
python tools/test.py configs/restormer/restormer_official_dpdd-dual.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dpdd-single-6bc31582.pth
# Dual
python tools/test.py configs/restormer/restormer_official_dpdd-single.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dpdd-dual-52c94c00.pth
# Gaussian Denoising
# Test Grayscale Gaussian Noise
# sigma15
python tools/test.py configs/restormer/restormer_official_dfwb-gray-sigma15.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-gray-sigma15-da74417f.pth
python tools/test.py configs/restormer/restormer_official_dfwb-gray-sigma15.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-gray-blind-5f094bcc.pth
# sigma25
python tools/test.py configs/restormer/restormer_official_dfwb-gray-sigma25.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-gray-sigma25-08010841.pth
python tools/test.py configs/restormer/restormer_official_dfwb-gray-sigma25.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-gray-blind-5f094bcc.pth
# sigma50
python tools/test.py configs/restormer/restormer_official_dfwb-gray-sigma50.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-gray-sigma50-ee852dfe.pth
python tools/test.py configs/restormer/restormer_official_dfwb-gray-sigma50.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-gray-blind-5f094bcc.pth
# Test Color Gaussian Noise
# sigma15
python tools/test.py configs/restormer/restormer_official_dfwb-color-sigma15.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-color-sigma15-012ceb71.pth
python tools/test.py configs/restormer/restormer_official_dfwb-color-sigma15.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-color-blind-dfd03c9f.pth
# sigma25
python tools/test.py configs/restormer/restormer_official_dfwb-color-sigma25.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-color-sigma25-e307f222.pth
python tools/test.py configs/restormer/restormer_official_dfwb-color-sigma25.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-color-blind-dfd03c9f.pth
# sigma50
python tools/test.py configs/restormer/restormer_official_dfwb-color-sigma50.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-color-sigma50-a991983d.pth
python tools/test.py configs/restormer/restormer_official_dfwb-color-sigma50.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-color-blind-dfd03c9f.pth
# multi-gpu test
# Deraining
./tools/dist_test.sh configs/restormer/restormer_official_rain13k.py https://download.openmmlab.com/mmediting/restormer/restormer_official_rain13k-2be7b550.pth
# Motion Deblurring
./tools/dist_test.sh configs/restormer/restormer_official_gopro.py https://download.openmmlab.com/mmediting/restormer/restormer_official_gopro-db7363a0.pth
# Defocus Deblurring
# Single
./tools/dist_test.sh configs/restormer/restormer_official_dpdd-dual.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dpdd-single-6bc31582.pth
# Dual
./tools/dist_test.sh configs/restormer/restormer_official_dpdd-single.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dpdd-dual-52c94c00.pth
# Gaussian Denoising
# Test Grayscale Gaussian Noise
# sigma15
./tools/dist_test.sh configs/restormer/restormer_official_dfwb-gray-sigma15.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-gray-sigma15-da74417f.pth
./tools/dist_test.sh configs/restormer/restormer_official_dfwb-gray-sigma15.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-gray-blind-5f094bcc.pth
# sigma25
./tools/dist_test.sh configs/restormer/restormer_official_dfwb-gray-sigma25.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-gray-sigma25-08010841.pth
./tools/dist_test.sh configs/restormer/restormer_official_dfwb-gray-sigma25.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-gray-blind-5f094bcc.pth
# sigma50
./tools/dist_test.sh configs/restormer/restormer_official_dfwb-gray-sigma50.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-gray-sigma50-ee852dfe.pth
./tools/dist_test.sh configs/restormer/restormer_official_dfwb-gray-sigma50.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-gray-blind-5f094bcc.pth
# Test Color Gaussian Noise
# sigma15
./tools/dist_test.sh configs/restormer/restormer_official_dfwb-color-sigma15.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-color-sigma15-012ceb71.pth
./tools/dist_test.sh configs/restormer/restormer_official_dfwb-color-sigma15.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-color-blind-dfd03c9f.pth
# sigma25
./tools/dist_test.sh configs/restormer/restormer_official_dfwb-color-sigma25.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-color-sigma25-e307f222.pth
./tools/dist_test.sh configs/restormer/restormer_official_dfwb-color-sigma25.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-color-blind-dfd03c9f.pth
# sigma50
./tools/dist_test.sh configs/restormer/restormer_official_dfwb-color-sigma50.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-color-sigma50-a991983d.pth
./tools/dist_test.sh configs/restormer/restormer_official_dfwb-color-sigma50.py https://download.openmmlab.com/mmediting/restormer/restormer_official_dfwb-color-blind-dfd03c9f.pth
For more details, you can refer to Test a pre-trained model part in train_test.md.
@inproceedings{Zamir2021Restormer,
title={Restormer: Efficient Transformer for High-Resolution Image Restoration},
author={Syed Waqas Zamir and Aditya Arora and Salman Khan and Munawar Hayat
and Fahad Shahbaz Khan and Ming-Hsuan Yang},
booktitle={CVPR},
year={2022}
}