Reconstruction of super-resolution image using wavelet residual convolutional neural networks

Abstract

Image super-resolution is a classic low-level vision and image processing task that aims to generate a high-resolution image from its low-resolution counterpart. As of now, most of the existing methods in image super-resolution use different neural network-based learning algorithms that are usually optimized in the spatial domain. While working in the spatial domain is a perfectly sound approach and produces images with a high peak signal-to-noise (PSNR) ratio, it often leads to producing images with poor frequency domain characteristics, i.e., they lack the natural high- frequency details, resulting in super-resolved images with blurry regions. In this work, we propose a novel wavelet-based residual convolutional neural network ar- chitecture, referred to as WaveSRResNet, that learns the image features both in the spatial and wavelet domains. The WaveSRResNet is also optimized by minimizing the loss function in the said both domains in the MAE sense to further harness the power of the discrete wavelet transform. Specifically, we design the WaveSRResNet with the development of a novel wavelet residual block (WaveRB), which is capable of performing the forward and the inverse transform inside the CNN based network. This helps the model to be regularized in the wavelet domain as well, and produce super-resolution images with better high-frequency details. Extensive experiments are carried out in order to show the effectiveness of the proposed wavelet residual block in the CNN network for its performance in the image super-resolution algorithm. The results demonstrate that the proposed WaveSRRes- Net outperforms recent image super-resolution methods in terms of both quantita- tive and perceptual metrics. On average, the WaveSRResNet achieved 4.8% higher PSNR (↑) and 8% higher SSIM (↑) objective scores than the most successful meth- ods for the 8× image super-resolution, which is the most challenging scenario. It also attained 9% lower perceptual score, LPIPS (↓) on average for the 4× up-scaling factor. Visual outputs also indicate that the proposed WaveSRResNet is able to re- construct high-quality, edge-preserving images at high-resolution. The outcomes of this research clearly show the effectiveness of the proposed method for the problem of image super-resolution.