International Journal of Advanced Networking and Applications

Open Access Open Access  Restricted Access Subscription Access

Fingerspelling Identification for American Sign Language Based on Resnet-18


Affiliations
1 College of Computer and Communication, Hunan Institute of Engineering, Xiangtan 411104, China
 

Sign language as the main communication channel for deaf and hearing people, plays a very important role in daily life. With the rapid development of the field of deep learning, the field of sign language recognition has ushered in new opportunities. Aiming at the small number of sign language samples and low detection accuracy, an American sign language detection method based on Resnet-18 and data augmentation is proposed. First, the sign language picture is adjusted to 64 × 64 size using the filling method, and then the data is augmented by methods such as chromaticity change, rotation, and noise addition to increase the diversity of data samples and improve the robustness of the detection method. The experimental results show that the accuracy of the sign language recognition method based on Resnet-18 can reach 99%, which provides a new method for sign language recognition

Keywords

Convolutional Neural Network, Data augmentation, Resnet-18, Sign Language Recognition.
User
Notifications
Font Size

  • Chen H X, Huang Y Y, Liu Y, Research and implementation of air gesture tracking and recognition based on Kinect, Video Engineering,39(21), 2015, 91-94.

  • Kong Y, Satarboroujeni B, Fu Y, Learning hierarchical 3D kernel descriptors for RGB-D action recognition, Computer Vision & Image Understanding, 144(5), 2016, 14-23.

  • Wang W, Zhang H J, Ren X Z, Single finger language recognition method based on SVM, Computer Engineering and Design, 39(10),2018,3234-3239.

  • MU Y, Research on finger language recognition method based on sparse coding, master diss., Shenyang University of Technology, Shen Yang, 2020.

  • Amaya C, Murray V, Real-time sign language recognition, 2020 IEEE XXVII International Conference on Electronics, Electrical Engineering and Computing, Lima, Peru, 2020, 1-4.

  • Pias Paul,Moh.Anwar-Ul-Azim Bhuiya,et al, A modern approach for sign language interpretation using convolutional neural network, PRICAI 2019: Trends in Artificial Intelligence 16th Pacific Rim International Conference on Artificial Intelligence, LNAI, 2019, 431-444.

  • D. AICH, A. AL ZUBAIR,et al, A deep learning approach for recognizing Bengali character sign language, 2020 11th International Conference on Computing, Communication and Networking Technologies, Kharagpur, India, 2020, 1-5.

  • JIANG X W, HU B,et al, Fingerspelling identification for chinese sign language via AlexNet-based transfer learning and adam optimizer, Scientific Programming , 2020, 1-12.

  • HASAN M M, SRIZON A Y, SAYEED A, et al, Classification of sign language characters by applying a deep convolutional neural network, 2020 2nd International Conference on Advanced Information and Communication Technology, Dhaka, Bangladesh, 2020,434-438.

  • Pugeault N, Bowden R, Spelling it out: Real-time ASL fingerspelling recognition, 2011 IEEE International conference on computer vision workshops, Barcelona, Spain, 2011, 1114-1119.

  • CUBUK E D,ZOPH B, et al, Autoaugment: Learning augmentation strategies from data,Proceedings of the IEEE conference on computer vision and pattern recognition, Long Beach, 2019, 113-123.

  • HE K, ZHANG X, et al, Deep residual learning for image recognition, Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, 2016, 770-778.

  • Dewinta Aryanie, Yaya Heryadi, American sign language-based finger-spelling recognition using k-Nearest Neighbors classifier, 2015 3rd International Conference on Information and Communication Technology, Nusa Dua, Bali, Indonesia, 2015, 533-536.


Abstract Views: 132

PDF Views: 1




  • Fingerspelling Identification for American Sign Language Based on Resnet-18

Abstract Views: 132  |  PDF Views: 1

Authors

Han-Wen Zhang
College of Computer and Communication, Hunan Institute of Engineering, Xiangtan 411104, China
Ying Hu
College of Computer and Communication, Hunan Institute of Engineering, Xiangtan 411104, China
Yong-Jia Zou
College of Computer and Communication, Hunan Institute of Engineering, Xiangtan 411104, China
Cheng-Yu Wu
College of Computer and Communication, Hunan Institute of Engineering, Xiangtan 411104, China

Abstract


Sign language as the main communication channel for deaf and hearing people, plays a very important role in daily life. With the rapid development of the field of deep learning, the field of sign language recognition has ushered in new opportunities. Aiming at the small number of sign language samples and low detection accuracy, an American sign language detection method based on Resnet-18 and data augmentation is proposed. First, the sign language picture is adjusted to 64 × 64 size using the filling method, and then the data is augmented by methods such as chromaticity change, rotation, and noise addition to increase the diversity of data samples and improve the robustness of the detection method. The experimental results show that the accuracy of the sign language recognition method based on Resnet-18 can reach 99%, which provides a new method for sign language recognition

Keywords


Convolutional Neural Network, Data augmentation, Resnet-18, Sign Language Recognition.

References