Research Cell: An International Journal of Engineering Sciences

Open Access Open Access  Restricted Access Subscription Access

Comparative Analysis of Contextual Relation Extraction based on Deep Learning Models


Affiliations
1 Department of Computer Science, School of Engineering and Technology, Pondicherry University, Puducherry, India., India
 

Contextual Relation Extraction (CRE) is mainly used for constructing a knowledge graph with a help of ontology. It performs various tasks such as semantic search, query answering, and textual entailment. Relation extraction identifies the entities from raw texts and the relations among them. An efficient and accurate CRE system is essential for creating domain knowledge in the biomedical industry. Existing Machine Learning and Natural Language Processing (NLP) techniques are not suitable to predict complex relations from sentences that consist of more than two relations and unspecified entities efficiently. In this work, deep learning techniques have been used to identify the appropriate semantic relation based on the context from multiple sentences. Even though various machine learning models have been used for relation extraction, they provide better results only for binary relations, i.e., relations occurred exactly between the two entities in a sentence. Machine learning models are not suited for complex sentences that consist of the words that have various meanings. To address these issues, hybrid deep learning models have been used to extract the relations from complex sentence effectively. This paper explores the analysis of various deep learning models that are used for relation extraction.

Keywords

Contextual Relation Extraction, Word Embeddings, Bert, Deep Learning Model.
User
Notifications
Font Size

  • X. Chen and R. Badlani, “Proceedings of Deep Learning Inside Out (DeeLIO): The First Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pages Relation Extraction with Contextualized Relation Embedding (CRE).” [Online]. Available: https://developers.google.com/

  • P. Li, M. Wang, and J. Wang,“Named entity translation method based on machine-translation lexicon,” Neural Comput Appl, vol. 33, no. 9,pp. 3977–3985, May 2021, doi: 10.1007/s00521- 020-05509-y.

  • C. Gao, X. Zhang, M. Han, and H. Liu, “A review on cyber security named entity recognition,”Frontiers of Information Technology and Electronic Engineering, vol. 22, no. 9. Zhejiang University, pp. 1153–1168, Sep. 01, 2021. doi: 10.1631/FITEE.2000286.

  • R. Patra and S. K. Saha, “A hybrid approach for automatic generation of named entity distractors formultiple choice questions,” Educ Inf Technology (Dordr), vol. 24, no. 2, pp. 973–993, Mar. 2019, doi: 10.1007/s10639- 018-9814-3.

  • B. Qiao, Z. Zou, Y. Huang, K. Fang, X. Zhu, and Y. Chen, “A joint model for entity and relation extraction based on BERT,” Neural Comput Appl, vol. 34, no. 5, pp. 3471–3481, Mar. 2022, doi: 10.1007/s00521- 021-05815-z.

  • H. Zhu, I. Ch. Paschalidis, and A. Tahmasebi, “Clinical Concept Extraction with Contextual Word Embedding,” Oct. 2018, [Online]. Available: http://arxiv.org/abs/1810.10566

  • J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” Oct. 2018, [Online]. Available: http://arxiv.org/abs/1810.04805

  • Z. Zhong and D. Chen, “A Frustratingly Easy Approach for Entity and Relation Extraction,” Oct. 2020, [Online]. Available: http://arxiv.org/abs/2010.12812

  • S. Zheng, F. Wang, H. Bao, Y. Hao, P. Zhou, and B. Xu, “Joint extractionof entities and relations based on a novel tagging scheme,” in ACL2017 - 55th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers), 2017, vol.1, pp. 1227– 1236. doi: 10.18653/v1/P17-1113.

  • W. Xu, K. Chen, L. Mou, and T. Zhao, “Document-Level Relation Extraction with Sentences Importance Estimation and Focusing.” [Online]. Available: https://github.

  • S. Zeng, Y. Wu, and B. Chang, “SIRE: Separate Intra- and Inter- sentential Reasoning for Document- level Relation Extraction,” Jun. 2021, [Online]. Available: http://arxiv.org/abs/2106.01709

  • M. E. Peters et al., “Deep contextualized word representations,” Feb. 2018, [Online]. Available: http://arxiv.org/abs/1802.05365.

  • D. Christou and G. Tsoumakas, “Improving Distantly-Supervised Re- lation Extraction through BERT-Based Label and Instance Embed- dings,” IEEE Access, vol. 9, pp. 62574–62582, 2021, doi:10.1109/AC- CESS.2021.3073428.

  • I. Hendrickx et al., “SemEval-2010 Task 8: Multi-Way Classificationof Semantic Relations Between Pairs of Nominals,” Association for Computational Linguistics, 2010. [Online]. Available: http://docs.

  • Y. Yao et al., “DocRED: A Large-Scale Document-Level Relation Extraction Dataset.” [Online]. Available: https://spacy.io

  • X. Han et al., “FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation.” [Online]. Avail-able: http://zhuhao.me/fewrel

  • T. Gao et al,“FewRel 2.0: Towards More Challenging Few-Shot Relation Classification.” [Online]. Available: https://www.ncbi.nlm.nih.gov/pubmed/

  • Q. Cheng et al., “HacRED: A Large-Scale Relation Extraction Dataset Toward Hard Cases in Practical Applications.” [Online]. Available: http://lic2019.ccf.org.cn/kg

  • Haque, A. B., Islam, A. N., & Mikalef, P. (2023). Explainable Artificial Intelligence (XAI)from a user perspective: A synthesis of prior literature and problematizing avenues for future research. Technological Forecasting and Social Change, 186, 122120.

  • Rahman, S., Rahman, S., & Bahalul Haque, A. K. M. (2022). Automated detection of cardiac arrhythmia based on a hybrid CNN-LSTM network. In Emergent Converging Technologies and Biomedical Systems: Select Proceedings of ETBS 2021 (pp. 395-414). Singapore: Springer Singapore.

  • G. Kim, C. Lee, J. Jo, and H. Lim, “Automatic extraction of named entities of cyber threats using a deep Bi-LSTM-CRF network,” Inter- national Journal of Machine Learning and Cybernetics, vol. 11, no. 10, pp. 2341–2355, Oct. 2020, doi: 10.1007/s13042-020-01122-6.

  • Navid, S. M. A., Priya, S. H., Khandakar, N. H., Ferdous, Z., & Haque, A. B. (2019). Signature verification using convolutional neural network. In 2019 IEEE International Conference on Robotics, Automation, Artificial-intelligence and Internet-of-Things (RAAICON) (pp. 35-39). IEEE.

  • Siam, S. C., Faisal, A., Mahrab, N., Haque, A. B., & Suvon, M. N. I. (2021, February). Automated student review system with computer vision and convolutional neural network. In 2021 International Conference on Computing, Communication, and Intelligent Systems (ICCCIS) (pp. 493-497). IEEE.

  • P. Srivastava, S. Bej, K. Schultz, K. Yordanova, and O. Wolkenhauer, “Attention Retrieval Model for Entity Relation Extraction from Biolog- ical Literature,” IEEE Access, vol. 10, pp. 22429–22440,2022, doi: 10.1109/ACCESS.2022.3154820.

  • S. Banerjee and K. Tsioutsiouliklis, “Relation Extraction Using Multi- Encoder LSTM Network on a Distant Supervised Dataset,” in Proceedings - 12th IEEE International Conference on Semantic Computing, ICSC 2018, Apr. 2018, vol. 2018-January, pp. 235–238. doi: 10.1109/ICSC.2018.00040.

  • H. Zhu, I. Ch. Paschalidis, and A. Tahmasebi, “Clinical Concept Extraction with Contextual Word Embedding,” Oct. 2018, [Online]. Available: http://arxiv.org/abs/1810.10566

  • G. Wang, S. Liu, and F. Wei, “Weighted graph convolution over dependency trees for nontaxonomic relation extraction on public opinioninformation,” Applied Intelligence, vol. 52, no. 3, pp. 3403– 3417, Feb. 2022, doi: 10.1007/s10489-021- 02596-9.

  • Y. Yang, Z. Wu, Y. Yang, S. Lian, F. Guo, and Z. Wang, “A Survey of Information Extraction Based on Deep Learning,” Applied Sciences (Switzerland), vol. 12, no. 19. MDPI, Oct. 01, 2022. doi:10.3390/app12199691.

  • S. Mitra, S. Saha, and M. Hasanuzzaman, “A multi-view deep neural network model for chemical- disease relation extraction from imbalanceddatasets,” IEEE J Biomed Health Inform, vol. 24, no. 11, pp. 3315–3325,Nov. 2020, doi: 10.1109/JBHI.2020.2983365.

  • Y. Chen, W. Li, Y. Liu, D. Zheng, and T. Zhao, “Exploring DeepBelief Network for Chinese Relation Extraction.” [Online]. Available: http://www.nist.gov/speech/tests/ace/.

  • T. M. Alam and M. J. Awan, “Domain Analysis of Information Extraction Techniques,” INTERNATIONAL JOURNAL OF MULTIDISCI-PLINARY SCIENCES AND ENGINEERING, vol. 9, no. 6, 2018, [On-line]. Available: https://www.researchgate.net/publication/326463350.

  • C. Gao, X. Zhang, H. Liu, W. Yun, and J. Jiang, “A joint extraction model of entities and relations based on relation decomposition,” Inter- national Journal of Machine Learning and Cybernetics, vol. 13, no. 7,pp. 1833–1845, Jul. 2022, doi: 10.1007/s13042-021-01491-6.

  • O. A. Tarasova, A. v. Rudik, N. Y. Biziukova, D. A. Filimonov, and V. v.Poroikov, “Chemical named entity recognition in the texts of scientific publications using the na¨ıve Bayes classifier approach,” J Cheminform,vol. 14, no. 1, Dec. 2022, doi: 10.1186/s13321-022-00633-4.

  • T. Bai, H. Guan, S. Wang, Y. Wang, and L. Huang, “Traditional Chinese medicine entity relation extraction based on CNN with segmentattention,” Neural Comput Appl, vol. 34, no. 4, pp. 2739– 2748, Feb. 2022, doi: 10.1007/s00521- 021-05897-9.

  • Q. Wang, Q. Zhang, M. Zuo, S. He, and B. Zhang, “An Entity Relation Extraction Model with Enhanced Position Attention in Food Domain,” Neural Process Lett, vol. 54, no. 2, pp. 1449–1464, Apr. 2022, doi: 10.1007/s11063-021- 10690-9.

  • H. Wang, K. Qin, R. Y. Zakari, G. Lu, and J. Yin, “Deep neural network-based relation extraction: an overview,” Neural Comput Appl, vol. 34, no. 6, pp. 4781– 4801, Mar. 2022, doi: 10.1007/s00521-021-06667-3.

  • Y. Yang, Z. Wu, Y. Yang, S. Lian, F. Guo, and Z. Wang, “A Survey of Information Extraction Basedon Deep Learning,” Applied Sciences (Switzerland), vol. 12, no. 19. MDPI, Oct. 01, 2022. doi: 10.3390/app12199691.

  • Z. Zheng, Y. Liu, D. Li, and X. Zhang, “Distant supervised relation extraction based on residual attention,” Frontiers of Computer Science, vol. 16, no. 6. Higher Education Press Limited Company, Dec. 01, 2022. doi: 10.1007/s11704-021-0474-x

  • G. Wang, S. Liu, and F. Wei, “Weighted graph convolution over dependency trees for nontaxonomic relation extraction on public opinion information,” Applied Intelligence, vol. 52, no. 3, pp. 3403– 3417, Feb. 2022, doi: 10.1007/s10489-021- 02596-9.

  • C. Chantrapornchai and A. Tunsakul, “Information extraction on tourism domain using SpaCy and BERT,” ECTI Transactions on Computer and Information Technology, vol. 15, no. 1, pp. 108–122, Apr. 2021, doi: 10.37936/ecticit.2021151.228621.

  • W. Zhou, K. Huang, T. Ma, and J. Huang, “Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling,” 2021. [Online]. Available: www.aaai.org

  • P. Srivastava, S. Bej, K. Schultz, K. Yordanova, and O. Wolkenhauer, “Attention Retrieval Model for Entity Relation Extraction from Biolog- ical Literature,” IEEE Access, vol. 10, pp. 22429–22440,2021, doi: 10.1109/ACCESS.2022.3154820.

  • K. Liu, “A survey on neural relation extraction,” Science China Technological Sciences, vol. 63, no. 10. Springer Verlag, pp. 1971–1989, Oct. 01, 2020. doi: 10.1007/s11431-020-1673-6.

  • B. Hao, H. Zhu, and I. Ch Paschalidis, “Enhancing Clinical BERT Embedding using a Biomedical Knowledge Base,” Online, 2020. [Online]. Available: https://github.com/noc-lab/clinical-kb-bert

  • D.Sousa and F. M. Couto, “BiOnt: Deep learning using multiple biomed-ical ontologies for relation extraction,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2020, vol. 12036 LNCS, pp. 367–374. doi: 10.1007/978-3-030- 45442-546.

  • R. Patra and S. K. Saha, “A hybrid approach for automatic generation of named entity distractors formultiple choice questions,” Educ Inf Technol (Dordr), vol. 24, no. 2, pp. 973–993, Mar. 2019, doi: 10.1007/s10639-018-9814-3.

  • Chikka, Veera Raghavendra, and Kamalakar Karlapalem. ”A hybrid deep learning approach for medical relation extraction.” arXiv preprint arXiv:1806.11189 (2018).

  • S. Zeng, Y. Wu, and B. Chang, “SIRE: Separate Intra- and Inter- sentential Reasoning for Document- level Relation Extraction,” Jun. 2021, [Online]. Available: http://arxiv.org/abs/2106.01709

  • J. Qiu, Y. Chai, Y. Liu, Z. Gu, S. Li, and Z. Tian, “Automatic Non-Taxonomic Relation Extraction from Big Data in Smart City,” IEEE Access, vol. 6, pp. 74854–74864, 2018, doi: 10.1109/AC-CESS.2018.2881422.

  • H. Zhu, I. Ch. Paschalidis, and A. Tahmasebi, “Clinical Concept Extraction with Contextual Word Embedding,” Oct. 2018, [Online]. Available: http://arxiv.org/abs/1810.10566

  • L. Song, Y. Zhang, Z. Wang, and D. Gildea, “N-ary Relation Extraction using Graph State LSTM.”[Online]. Available: https://github.com/

  • Zhang X, Zhang Y, Zhang Q, Ren Y, Qiu T, Ma J, Sun Q, “ Extracting comprehensive clinical information for breast cancer using deep learning methods.” Int J Med Inform. 2019 Dec;132:103985. doi: 10.1016/j.ijmedinf.2019.103985. Epub 2019 Oct 2. PMID: 31627032.

  • L. Wang, Z. Cao, G. de Melo, and Z. Liu, “Relation Classification via Multi-Level Attention CNNs.”

  • S. Wu and Y. He, “Enriching Pre-trained Language Model with Entity Information for Relation Classification,” May 2019, [Online]. Available: http://arxiv.org/abs/1905.08284

  • D. Zeng, K. Liu, S. Lai, G. Zhou, and J. Zhao, “Relation Classification via Convolutional Deep Neural Network.” [Online]. Available: http://en.wikipedia.org/wiki/Bag-of-words

  • R. Cai, X. Zhang, and H. Wang, “Bidirectional Recurrent Convolutional Neural Network forRelation Classification.”

  • P. Zhou et al., “Attention-Based Bidirectional Long Short-Term Memory Networks for RelationClassification.”

  • S. Wu and Y. He, “Enriching Pre-trained Language Model with Entity Information for Relation Classification,” May 2019, [Online]. Available: http://arxiv.org/abs/1905.08284.

  • Y. Zhao, H. Wan, J. Gao, and Y. Lin, “Improving Relation Classification by Entity Pair Graph,”2019.


Abstract Views: 127

PDF Views: 0




  • Comparative Analysis of Contextual Relation Extraction based on Deep Learning Models

Abstract Views: 127  |  PDF Views: 0

Authors

R. Priyadharshini
Department of Computer Science, School of Engineering and Technology, Pondicherry University, Puducherry, India., India
G. Jeyakodi
Department of Computer Science, School of Engineering and Technology, Pondicherry University, Puducherry, India., India
P. Shanthi Bala
Department of Computer Science, School of Engineering and Technology, Pondicherry University, Puducherry, India., India

Abstract


Contextual Relation Extraction (CRE) is mainly used for constructing a knowledge graph with a help of ontology. It performs various tasks such as semantic search, query answering, and textual entailment. Relation extraction identifies the entities from raw texts and the relations among them. An efficient and accurate CRE system is essential for creating domain knowledge in the biomedical industry. Existing Machine Learning and Natural Language Processing (NLP) techniques are not suitable to predict complex relations from sentences that consist of more than two relations and unspecified entities efficiently. In this work, deep learning techniques have been used to identify the appropriate semantic relation based on the context from multiple sentences. Even though various machine learning models have been used for relation extraction, they provide better results only for binary relations, i.e., relations occurred exactly between the two entities in a sentence. Machine learning models are not suited for complex sentences that consist of the words that have various meanings. To address these issues, hybrid deep learning models have been used to extract the relations from complex sentence effectively. This paper explores the analysis of various deep learning models that are used for relation extraction.

Keywords


Contextual Relation Extraction, Word Embeddings, Bert, Deep Learning Model.

References