Project Proposal
- Mahalakshmi Sundaresan
- Feb 14, 2020
- 3 min read
Analysis of Single Image Super-Resolution Using Deep Learning
Mahalakshmi Sundaresan ( msundaresan2@wisc.edu)
Pratiksha Pai( ppai3@wisc.edu )
14 February 2020
1 Overview and Problem Statement
The problem we are attempting to analyze is super-resolution (SR) from a single image using deep learning. Super Resolution is the process of enhancing an image to generate a higher resolution image, preserving the original image details while introducing more detail. SR may be classified into a single image and multiple images SR. SISR is more efficient, and the evolution of neural network architectures has led to deep learning models such as the Super-Resolution Convolutional Neural Network (SRCNN), Very Deep Super Resolution (VDSR), and Enhanced Deep Super-Resolution (EDSR), among others.
1.1 Why SISR?
Multiple scientific studies, research, and general image dependent applications are constrained by an optical resolution limit established by cameras/optical instruments. Examples of such applications include microscopy or biological imaging systems, satellite imagery and surveillance applications, to name a few. We may resolve structures that are beyond the diffraction limit of optical microscopy using SR.
Two methods for realizing super-resolution are Classical SR and Example-Based SR. Classical SR relies on detecting similar patches in multiple low-resolution images of a scene and using that to generate a high-resolution image, while Example-Based SR relies on detecting similar patches in multiple image scales.
We focus on SISR, which uses a combination of existing methods. A well-known contribution to single image super-resolution was given by Glasner et al.[1], in which both Classical and Example-Based SR were combined to obtain Super-Resolution from a single image. This paper has gone on to be widely cited with respect to work on SISR. One of the downfalls of this kind of SISR is induced hallucinations from high-frequency estimation processes.
Fig 1: Cross-Patch Redundancy when combining Classical and Example-Based SR
1.2 Current State-of-the-art:
The current state of the art aims to reduce the downfalls of this approach using deep learning.
Deep learning aims to use neural networks with several layers to understand the hierarchical representation of data and approximate desired results. In recent years especially, deep learning has managed to overcome quite a few shortcomings of earlier approaches to SISR.
2 Project Outline
We are planning to re-implement the existing solution to single image super-resolution using Very Deep Super Resolution (VDSR) [3] with 20 layers [4] and also Enhanced Deep Super-Resolution [5] to perform an analysis of the advantages and shortfalls of using these two approaches. We shall study and implement feasible changes to improve the performance by increasing the PSNR /SSIM value or detect cases where these methods could fail.
2.1 Need for a New Approach
The current Peak Signal to Noise Ratio (PSNR) / Structural Similarity Index (SSIM) for VDSR [3] is 31.35/0.8838 and EDSR [5] is 32.62/0.8984 [6]. We can try to enhance the PSNR and SSIM [7] values by adopting a new approach. It could be implemented by varying the number of layers, or by using a combination of existing approaches.
2.2 Evaluation
We are planning to analyze the results quantitatively by calculating the PSNR, SSIM [7] and computing speed.
PSNR: By minimizing the mean squared error (MSE) between the original image and the low-resolution image, we seek to improve the PSNR.
SSIM: It is a perceptual metric to compare the original and processed image containing a similar scene content.[8]
We would analyze the drawbacks existing in the chosen two models and explore cases where they might fail.
2.3 Timeline
References:
[1] D. Glasner, S. Bagon and M. Irani, "Super-resolution from a single image," in 2009 IEEE 12th International Conference on Computer Vision (ICCV), Kyoto, 2009 pp. 349-356.
doi: 10.1109/ICCV.2009.5459271
[2] Yang, Wenming et al. “Deep Learning for Single Image Super-Resolution: A Brief Review.” IEEE Transactions on Multimedia 21.12 (2019): 3106–3121. Crossref. Web.
[3] J. Kim, J. Kwon Lee, and K. Mu Lee. Accurate image super-resolution using very deep convolutional networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1646–1654, 2016.
[5] B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee. Enhanced deep residual networks for single image super-resolution. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 1132–1140, 2017
[6] Yang, Wenming, Xuechen Zhang, Yapeng Tian, Wei Wang, Jing-Hao Xue, and Qingmin Liao. “Deep Learning for Single Image Super-Resolution: A Brief Review.” IEEE Transactions on Multimedia 21 (2018): 3106-3121.
[7] Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, Eero P. Simoncelli.”Image Quality Assessment: From Error Visibility to Structural Similarity”.IEEE TRANSACTIONS ON IMAGE PROCESSING(2004).
Comments