ICTACT Journal on Soft Computing

Open Access Open Access  Restricted Access Subscription Access
Open Access Open Access Open Access  Restricted Access Restricted Access Subscription Access

Efficientnet for Human Fer Using Transfer Learning


Affiliations
1 Department of Electronic Science, Kurukshetra University, India
2 CSIR-Central Electronics Engineering Research Institute, Pilani, India
     

   Subscribe/Renew Journal


Automatic facial expression recognition (FER) remained a challenging problem in computer vision. Recognition of human facial expression is difficult for machine learning techniques since there is a variation in emotional expression from person to person. With the advancement in deep learning and the easy availability of digital data, this process has become more accessible. We proposed an efficient facial expression recognition model based EfficientNet as backbone architecture and trained the proposed model using the transfer learning technique. In this work, we have trained the network on publicly available emotion datasets (RAF-DB, FER-2013, CK+). We also used two ways to compare our trained model: inner and cross-data comparisons. In an internal comparison, the model achieved an accuracy of 81.68 % on DFEW and 71.02 % on FER-2013. In a cross-data comparison, the model trained on RAF-DB and tested on CK+ achieved 78.59%, while the model trained on RAF-DB and tested on FER-2013 achieved 56.10% accuracy. Finally, we generated an t-SEN distribution of our model on both datasets to demonstrate the model's inter-class discriminatory power.

Keywords

FER, Deep Convolution Neural Network, EfficientNet, Transfer Learning
Subscription Login to verify subscription
User
Notifications
Font Size

  • P. Ekman, “Universal Facial Expressions of Emotion”, Mental Illness Journal, Vol. 8, pp. 151-158, 1970.

  • P.S. Suchitra and S. Tripathi, “Real-Time Emotion Recognition from Facial Images using Raspberry Pi II”, Proceedings of International Conference on Signal Processing and Integrated Networks, pp. 666-670, 2016.

  • A. Fernandez-Caballero, R. Zangroniz and J.M. Latorre, “A Smart Environment Architecture for Emotion Detection and Regulation”, Biomed Research International, Vol. 2016, pp. 55-73, 2016.

  • U. Thonse and S.K. Sharma, “PSVN Facial Emotion Recognition, Socio-Occupational Functioning and Expressed Emotions in Schizophrenia versus Bipolar Disorder”, Psychiatry Research, Vol. 264, pp. 354-360, 2018.

  • N.K. Mehta and S. Singh, “Three-Dimensional DenseNet Self-Attention Neural Network for Automatic Detection of Student's Engagement”, Applied Intelligence, Vol. 18, pp.1-21, 2022.

  • M.Z. Alom and V.K. Asari, “A State-of-the-Art Survey on Deep Learning Theory and Architectures”, Electronics, Vol. 8, pp. 292-298, 2019.

  • M. Sahu and R. Dash, “A Survey on Deep Learning: Convolution Neural Network (CNN)”, Proceedings of International Conference on Smart Innovation, Systems and Technologies, pp. 317-325, 2021.

  • X. Zhao and S. Zhang, “Facial Expression Recognition via Deep Learning”, IETE Technical Review, Vol. 32, pp. 347-355, 2015.

  • M.A. Akhand and T. Shimamura, “Facial Emotion Recognition using Transfer Learning in the Deep CNN”, Electronics, Vol. 10, No. 9, pp. 1036-1045, 2021

  • C.F. Liew and T. Yairi, “Facial Expression Recognition and Analysis: A Comparison Study of Feature Descriptors”, IPSJ Transactions on Computer Vision and Applications, Vol. 7, pp. 104-120, 2015.

  • H. Alshamsi and H. Meng, “Stacked Deep Convolutional Auto-Encoders for Emotion Recognition from Facial Expressions”, Proceedings of International Conference on Neural Networks, pp. 1586-1593, 2017.

  • M. Oquab and J. Sivic, “Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks”, Proceedings of International Conference on Computer Vision and Pattern Recognition, pp. 1717-1724, 2014.

  • I.J. Goodfellow, “Challenges in Representation Learning: A Report on Three Machine Learning Contests”, Proceedings of International Conference on Neural Networks, pp. 117-124, 2013.

  • S. Li and W. Deng, “Reliable Crowdsourcing and Deep Locality-Preserving Learning for Unconstrained Facial Expression Recognition”, IEEE Transactions on Image Processing, Vol. 28, No. 1, pp. 356-370, 2018.

  • P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar and I. Matthews, “The Extended Cohn-Kanade Dataset (CK+): A Complete Dataset for Action Unit and Emotion-Specified Expression”, Proceedings of International Conference on Computer Vision and Pattern Recognition, pp. 94-101, 2010.

  • X. Zhao and S. Zhang, “Facial Expression Recognition via Deep Learning”, IETE Technical Review, Vol. 32, pp. 347-355, 2015.

  • H. Ding, S.K. Zhou and R. Chellappa, “FaceNet2ExpNet: Regularizing a Deep Face Recognition Net for Expression Recognition”, Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition, pp. 118-126, 2017.

  • S. Shaees and H. Aldabbas, “Facial Emotion Recognition using Transfer Learning”, Proceedings of International Conference on Computing and Information Technology, pp. 1-5, 2020.

  • Y. Fan, J.C. Lam and V.O. Li, “Multi-Region Ensemble Convolutional Neural Network for Facial Expression Recognition”, Proceedings of International Conference on Artificial Neural Networks, pp. 84-94, 2018.

  • S. Zhao, H. Cai, H. Liu, J. Zhang and S. Chen, “Feature Selection Mechanism in CNNs for Facial Expression Recognition”, Proceedings of International Conference on Computer Vision, pp. 317-328, 2018.

  • K. Wang and Y. Qiao, “Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition”, IEEE Transactions on Image Processing, Vol. 29, pp. 4057-4069, 2020.

  • K. Wang, “Suppressing Uncertainties for Large-Scale Facial Expression Recognition”, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6897-6906, 2020.

  • G. Wen, Z. Hou, H. Li, D. Li, L. Jiang and E. Xun, “Ensemble of Deep Neural Networks with Probability-Based Fusion for Facial Expression Recognition”, Cognitive Computing, Vol. 9, pp. 597-610, 2017.

  • R. Breuer and R. Kimmel, “A Deep Learning Perspective on the Origin of Facial Expressions”, Proceedings of International Conference on Computer Vision, pp. 1-4, 2017.

  • J. Cai and Z. Li, “Probabilistic Attribute Tree in Convolutional Neural Networks for Facial Expression Recognition”, Proceedings of International Conference on Computer Vision, pp. 1-9, 2017.

  • C.J.L. Flores, A.E.G. Cutipa and R.L. Enciso, “Application of Convolutional Neural Networks for Static Hand Gestures Recognition under Different Invariant Features”, Proceedings of International Conference on Electronics, Electrical Engineering and Computing, pp. 1-4, 2017.

  • I. Rocco, R. Arandjelovic and J. Sivic, “Convolutional Neural Network Architecture for Geometric Matching”, Proceedings of International Conference on Computer Vision and Pattern Recognition, pp. 39-48, 2017.

  • L.C. Yan and B. Yoshua, “Deep Learning”, Nature, Vol. 521, pp. 436-444, 2015.

  • M. Tan and Quoc V. Le, “Efficientnet: Improving Accuracy and Efficiency through AutoML and Model Scaling”,

  • Proceedings of International Conference on Computer Vision, pp. 1-9, 2019.

  • A.P. Fard and M.H. Mahoor, “Ad-Corre: Adaptive Correlation-Based Loss for Facial Expression Recognition in the Wild”, IEEE Access, Vol. 10, pp. 26756-26768, 2022.

  • R. Pecoraro and S. Gallo, “Local Multi-Head Channel Self-Attention for Facial Expression Recognition”, Proceedings of International Conference on Computer Vision, pp. 1-14, 2021.

  • S. Li and W. Deng, “A Deeper Look at Facial Expression Dataset Bias”, IEEE Transactions on Affective Computing, Vol. 13, No. 2, pp. 881-893, 2020.


Abstract Views: 78

PDF Views: 2




  • Efficientnet for Human Fer Using Transfer Learning

Abstract Views: 78  |  PDF Views: 2

Authors

Rajesh Singh
Department of Electronic Science, Kurukshetra University, India
Himanshu Sharma
CSIR-Central Electronics Engineering Research Institute, Pilani, India
Naval Kishore Mehta
Department of Electronic Science, Kurukshetra University, India
Anil Vohra
Department of Electronic Science, Kurukshetra University, India
Sanjay Singh
Department of Electronic Science, Kurukshetra University, India

Abstract


Automatic facial expression recognition (FER) remained a challenging problem in computer vision. Recognition of human facial expression is difficult for machine learning techniques since there is a variation in emotional expression from person to person. With the advancement in deep learning and the easy availability of digital data, this process has become more accessible. We proposed an efficient facial expression recognition model based EfficientNet as backbone architecture and trained the proposed model using the transfer learning technique. In this work, we have trained the network on publicly available emotion datasets (RAF-DB, FER-2013, CK+). We also used two ways to compare our trained model: inner and cross-data comparisons. In an internal comparison, the model achieved an accuracy of 81.68 % on DFEW and 71.02 % on FER-2013. In a cross-data comparison, the model trained on RAF-DB and tested on CK+ achieved 78.59%, while the model trained on RAF-DB and tested on FER-2013 achieved 56.10% accuracy. Finally, we generated an t-SEN distribution of our model on both datasets to demonstrate the model's inter-class discriminatory power.

Keywords


FER, Deep Convolution Neural Network, EfficientNet, Transfer Learning

References