ICTACT Journal on Image and Video Processing

Open Access Open Access  Restricted Access Subscription Access
Open Access Open Access Open Access  Restricted Access Restricted Access Subscription Access

Patch Based Stereo Matching Using Convolutional Neural Network


Affiliations
1 Department of Computer Science and Engineering, Jai Narain Vyas University, India
2 Department of Production and Industrial Engineering, Jai Narain Vyas University, India
     

   Subscribe/Renew Journal


The paper presents a new Convolutional Neural Network (CNN) architecture, called stacked stereo CNN, for computing disparity map from stereo images. In stacked stereo CNN, left and right image patches are stacked back-to-back and fed to a single tower CNN. This is in contrast to Siamese network where two towers are used, one for the left patch and other for the right patch. The proposed network is trained on a large set of similar and dissimilar image patches, which are generated from stereo images and their ground truth images from Middlebury stereo datasets. The network returns a dissimilarity score for a pair of image patch which is used to compute the cost volume. The cost volume is further refined using post processing steps before generating the final disparity map. The proposed network is evaluated on Middlebury datasets and achieves comparable results to the state-of-art algorithms.

Keywords

Stereo Vision, Patch Matching, Disparity Map, CNN.
Subscription Login to verify subscription
User
Notifications
Font Size

  • K.Y. Kok and P. Rajendran, “A Review on Stereo Vision Algorithms: Challenges and Solutions”, ECTI Transactions on Computer and Information Technology, Vol. 13, No. 2, pp. 134-150, 2019.

  • J. Zbontar and Y. Le Cuny, “Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches”, Journal of Machine Learning Research, Vol. 17, No. 2, pp. 1-32, 2016.

  • S. Zagoruyko and N. Komodakis, “Learning to Compare Image Patches via Convolutional Neural Networks”, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 4353-4361, 2015.

  • A. Tonioni, F. Tosi, M. Poggi, S. Mattoccia and L. di Stefano, “Real-Time Self-Adaptive Deep Stereo”, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-13, 2019.

  • X. Song, X. Zhao, L. Fang and H. Hu, “EdgeStereo: An Effective Multi-Task Learning Network for Stereo Matching and Edge Detection”, International Journal of Computer Vision, Vol. 128, pp. 910-930, 2020.

  • Middlebury Stereo Datasets, Available at https://vision.middlebury.edu/stereo/data/ last, Accessed at 2020.

  • R.A. Hamzah and H. Ibrahim, “Literature Survey on Stereo Vision Disparity Map Algorithms”, Journal of Sensors, Vol. 2016, pp. 1-23, 2016.

  • K. Zhou, X. Meng and Bo Cheng, “Review of Stereo Matching Algorithms Based on Deep Learning”, Computational Intelligence and Neuroscience, Vol. 2020, pp. 1-18, 2020.

  • W. Luo, A.G. Schwing and Raquel Urtasun, “Efficient Deep Learning for Stereo Matching”, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 5695-5703, 2016.

  • H. Park and K.M. Lee, “Look Wider to Match Image Patches with Convolutional Neural Networks”, IEEE Signal Processing Letters, Vol. 24, pp. 1788-1792, 2017.

  • X. Ye, J. Lib, H. Wang and X. Zhang, “Feature Ensemble Network with Occlusion Disambiguation for Accurate Patch-Based Stereo Matching”, IEICE Transactions on Information and Systems, Vol. 100, No. 12, pp. 3077-3080, 2017.

  • P. Brandao, E. Mazomenos and D. Stoyanov, “Widening Siamese Architectures for Stereo Matching”, Pattern Recognition Letters, Vol. 120, pp. 75-81, 2019.

  • B. Chen and C. Jung, “Patch-Based Stereo Matching using 3D Convolutional Neural Networks”, Proceedings of IEEE International Conference on Image Processing, pp. 3633-3637, 2018.

  • H. Hirschmuller, “Stereo Processing by Semiglobal Matching and Mutual Information”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 30, No. 2, pp. 328-341, 2008.

  • R. Verma, H.S. Singh and A.K. Verma, “Depth Estimation from Stereo Images Based on Adaptive Weight and Segmentation”, Journal of the Institution of Engineers (India): Series B - Electrical, Electronics and Telecommunication and Computer Engineering, Vol. 93, No. 4, pp. 223-229, 2013.

  • R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua and S. Süsstrunk, “SLIC Superpixels Compared to State-of-the-Art Superpixel Methods”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 34, No. 11, pp. 2274-2282, 2012.


Abstract Views: 299

PDF Views: 0




  • Patch Based Stereo Matching Using Convolutional Neural Network

Abstract Views: 299  |  PDF Views: 0

Authors

Rachna Verma
Department of Computer Science and Engineering, Jai Narain Vyas University, India
Arvind Kumar Verma
Department of Production and Industrial Engineering, Jai Narain Vyas University, India

Abstract


The paper presents a new Convolutional Neural Network (CNN) architecture, called stacked stereo CNN, for computing disparity map from stereo images. In stacked stereo CNN, left and right image patches are stacked back-to-back and fed to a single tower CNN. This is in contrast to Siamese network where two towers are used, one for the left patch and other for the right patch. The proposed network is trained on a large set of similar and dissimilar image patches, which are generated from stereo images and their ground truth images from Middlebury stereo datasets. The network returns a dissimilarity score for a pair of image patch which is used to compute the cost volume. The cost volume is further refined using post processing steps before generating the final disparity map. The proposed network is evaluated on Middlebury datasets and achieves comparable results to the state-of-art algorithms.

Keywords


Stereo Vision, Patch Matching, Disparity Map, CNN.

References