International Journal of Research in Signal Processing, Computing & Communication System Design

Open Access Open Access  Restricted Access Subscription Access
Open Access Open Access Open Access  Restricted Access Restricted Access Subscription Access

A Survey on Blind Super Resolution of Real-Life Video Sequences


Affiliations
1 Malla Reddy College of Engineering & Technology, Secunderabad, Telangana, India
2 ECE Department, Malla Reddy College of Engineering & Technology, Hyderabad, Telangana, India
     

   Subscribe/Renew Journal


Super Resolution (SR) for real-life video sequences is a challenging problem due to complex nature of the motion fields. In this paper, a novel blind SR method is proposed to improve the spatial resolution of video sequences, while the overall pointspread function of the imaging system, motion fields, and noise statistics are unknown. To estimate the blur(s), first, a nonuniform interpolation SR method is utilized to upsample the frames, and then, the blur(s) is(are) estimated through a multiscale process. The blur estimation process is initially performed on a few emphasized edges and gradually on more edges as the iterations continue. Also for faster convergence, the blur is estimated in the filter domain rather than the pixel domain. The high-resolution frames are estimated using a cost function that has the fidelity and regularization terms of type Huber-Markov random field to preserve edges and fine details. The fidelity term is adaptively weighted at each iteration using a masking operation to suppress artifacts due to inaccurate motions. Very promising results are obtained for real-life videos containing detailed structures, complex motions, fast-moving objects, deformable regions, or severe brightness changes. The proposed method outperforms the state of the art in all performed experiments through both subjective and objective evaluations.

Keywords

Blind Estimation, Blur Deconvolution, Huber Markov Random Field (HMRF), Video Super Resolution.
User
Subscription Login to verify subscription
Notifications
Font Size

  • E. Faramarzi, V. R. Bhakta, D. Rajan, and M. P. Christensen, “Super resolution results in PANOPTES, an adaptive multi-aperture folded architecture,” In Proc. 17th IEEE Int. Conf. Image Process. (ICIP), pp. 2833-2836, Sep. 2010.

  • E. Faramarzi, D. Rajan, and M. P. Christensen, “Unified blind method for multi-image super-resolution and single/multi-image blur deconvolution,” IEEE Trans. Image Process., vol. 22, no. 6, pp. 2101-2114, Jun. 2013.

  • S. Borman, and R. L. Stevenson, “Spatial resolution enhancement of lowresolution image sequences: A comprehensive review with directions for future research,” Dept. Elect. Eng., Univ. Notre Dame, Notre Dame, IN, USA, Tech. Rep., Jul. 1998.

  • S. Borman, and R. L. Stevenson, “Super-resolution from image sequences: A review,” In Proc. Midwest Symp. Circuits Syst., Notre Dame, IN, USA, pp. 374-378, Aug. 1998.

  • S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: A technical overview,” IEEE Signal Processing Magazine, vol. 20, no. 3, pp. 21-36, May 2003.

  • R. R. Schultz, L. Meng, and R. L. Stevenson, “Subpixel motion estimation for super-resolution image sequence enhancement,” Journal of Visual Communication and Image Representation, vol. 9, no. 1, pp. 38-50, Mar. 1998.

  • A. M. Tekalp, “Digital video processing,” Englewood Cliffs, NJ, USA: Prentice-Hall: Signal Processing Series 1995.

  • Y. Caspi, and M. Irani, “Spatio-temporal alignment of sequences,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 11, pp. 1409-1424, Nov. 2002.

  • O. Shahar, A. Faktor, and M. Irani, “Space-time super-resolution from a single video,” In Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 3353-3360, Jun. 2011.

  • V. Cheung, B. J. Frey, and N. Jojic, “Video epitomes,” International Journal of Computer Vision, vol. 76, no. 2, pp. 141-152, 2008.

  • R. Fergus, B. Singh, A. Hertzmann, S. T. Roweis, and W. T. Freeman, “Removing camera shake from a single photograph,” ACM Trans. Graph., vol. 25, no. 3, pp. 787-794, 2006.

  • Q. Shan, J. Jia, and A. Agarwala, “High-quality motion deblurring from a single image,” ACM Trans. Graph., vol. 27, no. 3, p. 73, 2008.

  • S. Cho, and S. Lee, “Fast motion deblurring,” ACM Transactions on Graphics, Art. ID 145, vol. 28, no. 5, 2009.

  • L. Xu, and J. Jia, “Two-phase kernel estimation for robust motion deblurring,” In Proc. 11th Eur. Conf. Comput. Vis., pp. 157-170, 2010.

  • T. F. Chan, and C.-K. Wong, “Total variation blind deconvolution,” IEEE Transactions on Image Processing, vol. 7, no. 3, pp. 370-375, Mar. 1998.


Abstract Views: 337

PDF Views: 7




  • A Survey on Blind Super Resolution of Real-Life Video Sequences

Abstract Views: 337  |  PDF Views: 7

Authors

A. Susmitha
Malla Reddy College of Engineering & Technology, Secunderabad, Telangana, India
V. Kiran Kumar
ECE Department, Malla Reddy College of Engineering & Technology, Hyderabad, Telangana, India
P. Anitha
ECE Department, Malla Reddy College of Engineering & Technology, Hyderabad, Telangana, India

Abstract


Super Resolution (SR) for real-life video sequences is a challenging problem due to complex nature of the motion fields. In this paper, a novel blind SR method is proposed to improve the spatial resolution of video sequences, while the overall pointspread function of the imaging system, motion fields, and noise statistics are unknown. To estimate the blur(s), first, a nonuniform interpolation SR method is utilized to upsample the frames, and then, the blur(s) is(are) estimated through a multiscale process. The blur estimation process is initially performed on a few emphasized edges and gradually on more edges as the iterations continue. Also for faster convergence, the blur is estimated in the filter domain rather than the pixel domain. The high-resolution frames are estimated using a cost function that has the fidelity and regularization terms of type Huber-Markov random field to preserve edges and fine details. The fidelity term is adaptively weighted at each iteration using a masking operation to suppress artifacts due to inaccurate motions. Very promising results are obtained for real-life videos containing detailed structures, complex motions, fast-moving objects, deformable regions, or severe brightness changes. The proposed method outperforms the state of the art in all performed experiments through both subjective and objective evaluations.

Keywords


Blind Estimation, Blur Deconvolution, Huber Markov Random Field (HMRF), Video Super Resolution.

References