ICTACT Journal on Soft Computing

Open Access Open Access  Restricted Access Subscription Access
Open Access Open Access Open Access  Restricted Access Restricted Access Subscription Access

Detection of Accurate Facial Detection using Hybrid Deep Convolutional Recurrent Neural Network


Affiliations
1 Department of Information Technology, Lebanese French University, Iraq
     

   Subscribe/Renew Journal


Facial Landmark discovery is an imperative issue in numerous PC vision applications about appearances. It is extremely testing as human faces in wild conditions regularly present expansive varieties fit as a fiddle because of various stances, impediments or demeanors. Profound neural systems have been connected to take in the guide from face pictures to confront shapes. To the best of our insight, Recurrent Neural Network (RNN) has not been utilized in this issue yet. In this paper, we propose a technique which uses RNN and Deep Neural Network (DNN) to take in the face shape. To start with, we design a system utilizing Convolutional Neural Network (CNN) to get the underlying Landmark estimation of appearances. At that point, we utilize feed-forward neural systems for neighborhood look where a segment based seeking technique is investigated. By utilizing LSTM- CNN-RNN, the underlying estimation is more dependable which makes the accompanying segment based pursuit doable and exact. Tests demonstrate that the worldwide system utilizing CNN-LSTM-RNN shows signs of improvement results than past systems in the two recordings and single picture. Our technique beats the cutting edge calculations particularly regarding fine estimation of Landmark spots.

Keywords

Facial landmark, Deep Neural Network, Recurrent Neural Network, Convolutional Neural Network.
Subscription Login to verify subscription
User
Notifications
Font Size

  • T. Weise, S. Bouaziz, H. Li and M. Pauly, “Realtime Performance-based Facial Animation”, ACM Transactions on Graphics, Vol.30, No. 4, pp. 71-77, 2011.

  • Q. Cai, D. Gallup, C. Zhang and Z. Zhang, “3D Deformable Face Tracking with a Commodity Depth Camera”, Proceedings of 11th International Conference on European Conference on Computer Vision, pp. 229-242, 2010.

  • G. Fanelli, M. Dantone, and L.V. Gool, “Real Time 3D Face Alignment with Random Forests-based Active Appearance Models”, Proceedings of 10th International Conference and Workshops on Automatic Face and Gesture Recognition, pp. 1-8, 2013.

  • Z.Z. Zhang, W. Zhang, J.Z. Liu and X.O. Tang, “Multiview Facial Landmark Localization in RGB-D Images via Hierarchical Regression With Binary Patterns”, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 24, No. 9, pp. 1475-1485, 2014.

  • T. Cootes, C.J. Taylor, D.H. Cooper and J. Graham, “Active Shape Models-Their Training and Application”, Computer Vision and Image Understanding, Vol. 61, No. 1, pp. 38-59, 1995.

  • T. Cootes, G.J. Edwards and C.J. Taylor, “Active Appearance Models”, Available at: https://www.cs.cmu.edu/~efros/courses/AP06/Papers/cootes-eccv-98.pdf.

  • I. Matthews and S. Baker, “Active Appearance Models Revisited”, International Journal of Computer Vision, Vol. 6, No. 2, pp. 135-164, 2004.

  • T. Cootes, G.J. Edwards and C.J. Taylor. “Active Appearance Models”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No. 6, pp. 681-685, 2001.

  • X. Liu, “Generic Face Alignment using Boosted Appearance Model”, Proceedings of International Conference on Computer Vision and Pattern Recognition, pp. 1-8, 2007.

  • D. Cristinacce and T. Cootes, “Automatic Feature Localization with Constrained Local Models”, Pattern Recognition, Vol. 41, No. 10, pp. 3054-3067, 2008.

  • D. Cristinacce and T. Cootes, “Feature Detection and Tracking with Constrained Local Models”, Proceedings of International Conference on British Machine Vision Conference, pp. 929-938, 2006.

  • J. Saragih, S. Lucey and J. Cohn, “Deformable Model Fitting by Regularized Landmark Mean-Shift”, International Journal of Computer Vision, Vol. 91, No. 2, pp. 200-215, 2011.

  • Y. Wang, S. Lucey and J. Cohn, “Enforcing Convexity for Improved Alignment with Constrained Local Model”, Proceedings of International Conference on Computer Vision and Pattern Recognition, pp. 1-8, 2008.

  • L. Gu, and T. Kanade, “A Generative Shape Regularization Model for Robust Face Alignment”, Proceedings of International Conference on Computer Vision and Pattern Recognition, pp. 413-426, 2008.

  • P. Dollar, P. Welinder and P. Perona, “Cascaded Pose Regression”, Proceedings of International Conference on Computer Vision and Pattern Recognition, pp. 1078-1085, 2010.

  • T.C. Patrick Sauer and C. Taylor, “Accurate Regression Procedures for Active Appearance Models”, Proceedings of International Conference on European Conference on Computer Vision, 2011, pp. 1-30, 2011.

  • J. Saragih and R. Goecke, “A Nonlinear Discriminative Approach to AAM Fitting”, Proceedings of 7th IEEE International Conference on Computer Vision, pp. 1-8, 2007.

  • X.D. Cao, Y.C. Wei, F. Wen and J. Sun, “Face Alignment by Explicit Shape Regression”, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2887-2894, 2012.

  • F. Visin et al., “ReNet: A Recurrent Neural Network based Alternative to Convolutional Networks”, Available at: https://arxiv.org/pdf/1505.00393.pdf.

  • Y. Sun, Q. Liu, H. Lu, “Low Rank Driven Robust Facial Landmark Regression”, Neurocomputing, Vol. 151, pp. 196-206, 2015.

  • M. Ozuysal, M. Calonder, V. Lepetit and P. Fua, “Fast Key-Point Recognition using Random Ferns”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 32, No. 3, pp. 448-461, 2009.

  • N. Duffy and D.P. Helmbold, “Boosting methods for Regression”, Machine Learning, Vol. 47, No. 2, pp. 153-200, 2002.

  • J.H. Friedman, “Greedy Function Approximation: A Gradient Boosting Machine”, Available at: https://statweb.stanford.edu/~jhf/ftp/trebst.pdf.

  • D.G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints”, International Journal of Computer Vision, Vol. 60, No. 2, pp. 91-110, 2004.

  • M. Calonder, V. Lepetit, C. Strecha and P. Fua, “Brief: Binary Robust Independent Elementary Features”, Proceedings of 10th IEEE International Conference on Computer Vision, pp. 778-792, 2010.

  • V. Lepetit, P. Lagger and P. Fua, “Randomized Trees for Real-Time Keypoint Recognition”, Proceedings of International Conference on Computer Vision and Pattern Recognition, pp. 775-781, 2005.

  • E. Rublee, V. Rabaud, K. Konolige and G. Bradski, “ORB: An Efficient Alternative to SIFT or SURF”, Proceedings of International Conference on Computer Vision, pp. 2564-2571, 2011.

  • S. Leutenegger, M. Chli and R. Siegwart, “Brisk: Binary Robust Invariant Scalable Keypoints”, Proceedings of International Conference on Computer Vision, pp. 2548-2555, 2011.

  • A. Alahi, R. Ortiz and P. Vandergheynst, “FREAK: Fast Retina Keypoint”, Proceedings of International Conference on Computer Vision and Pattern Recognition, pp. 510-517, 2012.

  • X.P. Burgos-Artizzu, P. Perona and P. Dollar. “Robust Face Landmark Estimation Under Occlusion”, Proceedings of International Conference on Computer Vision, pp. 1513-1520, 2013.

  • Y. Sun, X.G. Wang and X.O. Tang, “Deep Convolutional Network Cascade for Facial Point Detection”, Proceedings of International Conference on Computer Vision, pp. 3476-3483, 2013.

  • J. Zhang, S.G. Shan, M.N. Kan and X.L. Chen, “Coarse-to-Fine Auto-Encoder Networks (CFAN) for Real-Time Face Alignment, Proceedings of International Conference on European Conference on Computer Vision, pp. 1-16, 2014.

  • Y. Chen, W. Luo and J. Yang. “Facial Landmark Detection via Pose-Induced Auto-Encoder Networks”, Proceedings of International Conference on Image Processing, pp. 27-30, 2015.

  • Z.P. Zhang, P. Luo, C.C Loy and X.O. Tang, “Facial Landmark Detection by Deep Multi-Task Learning”, Proceedings of International Conference on European Conference on Computer Vision, pp. 94-108, 2014.

  • S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory”, Neural Computing, Vol. 9, No. 8, pp. 1735-1780, 1997.

  • A. Graves et al., “A Novel Connectionist System for Unconstrained Handwriting Recognition”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 31, No. 5, pp. 855-868, 2009.

  • C. Plahl, M. Kozielski, R. Schluter and H. Ney, “Feature Combination and Stacking of Recurrent and Non-Recurrent Neural Networks for LVCSR”, Proceedings of International Conference on Acoustics, Speech and Signal Processing, pp. 6714-6718, 2013.

  • M. Wollmer et al., “Online Driver Distraction Detection using Long Short-Term Memory”, IEEE Transactions on Intelligent Transportation Systems, Vol. 12, No. 2, pp. 574-582, 2011.

  • S. Hochreiter, Y. Bengio, P. Frasconi and J. Schmidhuber, “Gradient Flow in Recurrent Nets: The Difficulty of Learning Long-Term Dependencies”, IEEE Press, 2001.

  • M. Schuster and K.K. Paliwal, “Bidirectional Recurrent Neural Networks”, IEEE Transactions on Signal Processing, Vol. 45, No. 11, pp. 2673-2681, 1997.

  • A. Graves and J. Schmidhuber, “Framewise Phoneme Classification with Bidirectional LSTM and other Neural Network Architectures”, Neural Networks, Vol. 18, No. 5-6, pp. 602-610, 2005.

  • A. Maas et al., “Recurrent Neural Networks for Noise Reduction in Robust ASR”, Available at: http://www1.icsi.berkeley.edu/~vinyals/Files/rnn_denoise_2012.pdf.

  • C. Cao, Q. Hou and K. Zhou, “Displaced Dynamic Expression regression for Real-Time Facial Tracking and Animation”, Proceedings of ACM Conference on Special Interest Group on Computer Graphics, Vol. 33, No. 4, pp. 142-147, 2014.

  • C. Sagonas, G. Tzimiropoulos, S. Zafeiriou and M. Pantic, “The First Facial Landmark Localization Challenge”, Proceedings of IEEE International Conference on Computer Vision, pp. 41-52, 2013.

  • A. Asthana, S. Zafeiriou, S. Cheng and M. Pantic, “Robust Discriminative Response Map Fitting with Constrained Local Models”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3444-3451, 2013.

  • X. Yu, J. Huang, S. Zhang, W. Yan, D. N. Metaxas, “Pose-Free Facial Landmark Fitting via Optimized Part Mixtures and Cascaded Deformable Shape Model”, Proceedings of International Conference on Computer Vision and Pattern Recognition, pp. 1944-1951, 2013.

  • X. Zhu and D. Ramanan, “Face Detection, Pose Estimation, and Landmark Localization in the Wild”, Proceedings of International Conference on Computer Vision and Pattern Recognition, pp. 2879-2886, 2012.

  • F. Weninger, J. Bergmann and B. Schuller, “Introducing Currennt-The Munich Open-Source CUDA Recurrent Neural Network Toolkit”, Journal of Machine Learning Research, Vol. 16, pp. 547-551, 2015.

  • P.N. Belhumeur, D.W. Jacobs, D.J. Kriegman and N. Kumar, “Localizing Parts of Faces using a Consensus of Exemplars”, Proceedings of International Conference on Computer Vision and Pattern Recognition, pp. 545-552, 2013.

  • V. Le, J. Brandt, Z. Lin, L. Bourdev and T.S. Huang, “Interactive Facial Feature Localization”, Proceedings of International Conference on European Conference on Computer Vision, pp. 679-692, 2012.

  • J. Yang, J.K. Deng, K.H. Zhang and Q.S. Liu, “Facial Shape Tracking Via Spatio-Temporal Cascade Shape Regression”, Proceedings of the IEEE International Conference on Computer Vision Workshop, pp. 41-49, 2015.

  • G.S. Chrysos, E. Antonakos, S. Zafeiriou and P. Snape. “Offline Deformable Face Tracking in Arbitrary Videos, Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 32-35, 2015.

  • J. Shen, S. Zafeiriou, G.S. Chrysos, J. Kossaifi, G. Tzimiropoulos and M. Pantic, “The First Facial Landmark Tracking in-the-Wild Challenge: Benchmark and Results”, Proceedings of IEEE International Conference and Workshop on Computer Vision, pp. 11-17, 2015.

  • G. Tzimiropoulos, “Project-Out Cascaded Regression with an Application to Face Alignment”, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3659-3667, 2015.

  • C. Sagonas, E. Antonakos, G, Tzimiropoulos, S. Zafeiriou and M. Pantic, “300 faces In-the-wild challenge: Database and Results”, Image and Vision Computing, Vol. 47, pp. 3-18, 2016.

  • C. Sagonas, G. Tzimiropoulos, S. Zafeiriou and M. Pantic, “A Semi-Automatic Methodology for Facial Landmark Annotation”, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2111-2117, 2013.

  • S. Bell, K. Bala, L. Zitnick and R. Girshick, “Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2874-2883, 2016.

  • V. Kazemi and J. Sullivan, “One Millisecond Face Alignment with an Ensemble of Regression Trees”, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1867-1874, 2014.

  • A. Graves and J. Schmidhuber, “Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks”, Proceedings of the 21st International Conference on Neutral Information Processing Systems, pp. 545-552, 2008.

  • W. Byeon, T.M. Breuel, F. Raue and M. Liwicki, “Scene Labeling with LSTM Recurrent Neural Networks”, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 111-117, 2015.


Abstract Views: 280

PDF Views: 0




  • Detection of Accurate Facial Detection using Hybrid Deep Convolutional Recurrent Neural Network

Abstract Views: 280  |  PDF Views: 0

Authors

M. Sivaram
Department of Information Technology, Lebanese French University, Iraq
V. Porkodi
Department of Information Technology, Lebanese French University, Iraq
Amin Salih Mohammed
Department of Information Technology, Lebanese French University, Iraq
V. Manikandan
Department of Information Technology, Lebanese French University, Iraq

Abstract


Facial Landmark discovery is an imperative issue in numerous PC vision applications about appearances. It is extremely testing as human faces in wild conditions regularly present expansive varieties fit as a fiddle because of various stances, impediments or demeanors. Profound neural systems have been connected to take in the guide from face pictures to confront shapes. To the best of our insight, Recurrent Neural Network (RNN) has not been utilized in this issue yet. In this paper, we propose a technique which uses RNN and Deep Neural Network (DNN) to take in the face shape. To start with, we design a system utilizing Convolutional Neural Network (CNN) to get the underlying Landmark estimation of appearances. At that point, we utilize feed-forward neural systems for neighborhood look where a segment based seeking technique is investigated. By utilizing LSTM- CNN-RNN, the underlying estimation is more dependable which makes the accompanying segment based pursuit doable and exact. Tests demonstrate that the worldwide system utilizing CNN-LSTM-RNN shows signs of improvement results than past systems in the two recordings and single picture. Our technique beats the cutting edge calculations particularly regarding fine estimation of Landmark spots.

Keywords


Facial landmark, Deep Neural Network, Recurrent Neural Network, Convolutional Neural Network.

References