Journal of Scientific and Industrial Research

Open Access Open Access  Restricted Access Subscription Access

Using BiLSTM Structure with Cascaded Attention Fusion Model for Sentiment Analysis


Affiliations
1 Department of Computer Applications, Noorul Islam Centre for Higher Education, Kumaracoil, Thuckalay 629 180, Tamil Nadu, India
2 Department of Computer Science and Engineering, Noorul Islam Centre for Higher Education, Kumaracoil, Thuckalay 629 180, Tamil Nadu, India
 

In the last decade, sentiment analysis has been a popular research area in the domains of natural language processing and data mining. Sentiment analysis has several commercial and social applications. The technique is essential to analyse the customer experience to develop customer loyalty and maintenance through better assistance. Deep Neural Network (DNN) models have recently been used to do sentiment analysis tasks with promising results. The disadvantage of such models is that they value all characteristics equally. We propose a Cascaded Attention Fusion Model-based BiLSTM to address these issues (CAFM-BiLSTM). Multiple heads with embedding and BiLSTM layers are concatenated in the proposed CAFM-BiLSTM. The information from both deep multi-layers is merged and provided as input to the BiLSTM layer later in this paper. The results of our fusion model are superior to those of the existing models. Our model outperforms the competition for lengthier sentence sequences and pays special attention to referral words. The accuracy of the proposed CAFM-BiLSTM is 5.1%, 5.25%, 6.1%, 12.2%, and 13.7% better than RNN-LSTM, SVM, NB, RF and DT respectively.

Keywords

CAFM, Deep Learning, Deep Neural Network, Long Short-Term Memory, Natural Language Processing.
User
Notifications
Font Size

  • Mohbey K K, Sentiment analysis for product rating using a deep learning approach, Int Conf Artif Intell Smart Syst (IEEE) 2021, 121–126.

  • Balakrishnan V, Shi Z, Law C L, Lim R, Teh L L & Fan Y, A deep learning approach in predicting products’ sentiment ratings: a comparative analysis, J Supercomput, 78(5) (2022) 7206–7226.

  • Naresh A & Venkata Krishna P, An efficient approach for sentiment analysis using machine learning algorithm, Evol Intell, 14(2) (2021) 725–731.

  • Dubey T & Jain A, Sentiment analysis of keenly intellective smart phone product review utilizing SVM classification technique, In 2019 10th Int Conf Comput Commun Netw Technol (IEEE), 2019, 1–8.

  • Hasan MR, Maliha M & Arifuzzaman M, Sentiment analysis with NLP on Twitter Data, Int Conf Comput Commun Chem Mater Electron Eng (IEEE) 2019, 1–4.

  • Zhang Q, Lu R, Wang Q, Zhu Z & Liu P, Interactive multi-head attention networks for aspect-level sentiment classification, IEEE Access, 7 (2019) 160017–160028.

  • Li J, Wang X, Tu Z & Lyu M R, On the diversity of multi-head attention, Neurocomput, 454 (2021) 14–24.

  • Cheng L C, Chen Y L & Liao Y Y, Aspect-based sentiment analysis with component focusing multi-head co-attention networks, Neurocomput, 489 (2022) 9–17.

  • Wang X, Tang M, Yang T and Wang Z, A novel network with multiple attention mechanisms for aspect-level sentiment analysis, Knowl Based Syst, 227 (2021) 107196.

  • Lin Y, Fu Y, LiY, Cai G & Zhou A, Aspect-based sentiment analysis for online reviews with hybrid attention networks, World Wide Web, 24(4) (2021) 1215–1233.

  • Yang J, Zou X, Zhang W & Han H, Microblog sentiment analysis via embedding social contexts into an attentive LSTM, Eng Appli Artif Intell, 97 (2021) 104048.

  • Xu Q, Zhu L, Dai T & Yan C, Aspect-based sentiment classification with multi-attention network, Neurocomput, 388 (2020) 135–143.

  • Xu G, Meng Y, Qiu X, Yu Z & Wu X, Sentiment analysis of comment texts based on BiLSTM, IEEE Access, 7 (2019) 51522–51532.

  • Rehman A U, Malik A K, Raza B & Ali W, A hybrid CNN-LSTM model for improving accuracy of movie reviews sentiment analysis, Multimed Tools Appl, 78(18) (2019) 26597–26613.

  • Muhammad P F, Kusumaningrum R & Wibowo A, Sentiment analysis using Word2vec and long short-term memory (LSTM) for Indonesian hotel reviews, Proc Comput Sci, 179 (2021) 728–735.


Abstract Views: 106

PDF Views: 82




  • Using BiLSTM Structure with Cascaded Attention Fusion Model for Sentiment Analysis

Abstract Views: 106  |  PDF Views: 82

Authors

J Sangeetha
Department of Computer Applications, Noorul Islam Centre for Higher Education, Kumaracoil, Thuckalay 629 180, Tamil Nadu, India
U Kumaran
Department of Computer Science and Engineering, Noorul Islam Centre for Higher Education, Kumaracoil, Thuckalay 629 180, Tamil Nadu, India

Abstract


In the last decade, sentiment analysis has been a popular research area in the domains of natural language processing and data mining. Sentiment analysis has several commercial and social applications. The technique is essential to analyse the customer experience to develop customer loyalty and maintenance through better assistance. Deep Neural Network (DNN) models have recently been used to do sentiment analysis tasks with promising results. The disadvantage of such models is that they value all characteristics equally. We propose a Cascaded Attention Fusion Model-based BiLSTM to address these issues (CAFM-BiLSTM). Multiple heads with embedding and BiLSTM layers are concatenated in the proposed CAFM-BiLSTM. The information from both deep multi-layers is merged and provided as input to the BiLSTM layer later in this paper. The results of our fusion model are superior to those of the existing models. Our model outperforms the competition for lengthier sentence sequences and pays special attention to referral words. The accuracy of the proposed CAFM-BiLSTM is 5.1%, 5.25%, 6.1%, 12.2%, and 13.7% better than RNN-LSTM, SVM, NB, RF and DT respectively.

Keywords


CAFM, Deep Learning, Deep Neural Network, Long Short-Term Memory, Natural Language Processing.

References