Journal of Information and Knowledge (Formerly SRELS Journal of Information Management)

Open Access Open Access  Restricted Access Subscription Access
Open Access Open Access Open Access  Restricted Access Restricted Access Subscription Access

Scholarly Communication and Machine-Generated Text: Is it Finally AI vs AI in Plagiarism Detection?


Affiliations
1 Assistant Librarian, DSMS College, Durgapur - 713206, Durgapur, West Bengal, India
2 Research Scholar, Banaras Hindu University, Varanasi – 221002, Uttar Pradesh, India
     

   Subscribe/Renew Journal


This study utilizes GPT (Generative Pre-Trained Transformer) language model-based AI writing tools to create a set of 80 academic writing samples based on the eight themes of the experiential sessions of the LTC 2023. These samples, each between 2000 and 2500 words long, are then analyzed using both conventional plagiarism detection tools and selected AI detection tools. The study finds that traditional syntactic similarity-based anti-plagiarism tools struggle to detect AI-generated text due to the differences in syntax and structure between machine-generated and human-written text. However, the researchers discovered that AI detector tools can be used to catch AI-generated content based on specific characteristics that are typical of machine-generated text. The paper concludes by posing the question of whether we are entering an era in which AI detectors will be used to prevent AI-generated content from entering the scholarly communication process. This research sheds light on the challenges associated with AI-generated content in the academic research literature and offers a potential solution for detecting and preventing plagiarism in this context.

Keywords

AI (Artificial Intelligence), GPT (Generative Pre-Training Transformer), Machine Learning, ChatGPT, Natural Language Processing (NLP), OpenAI, Plagiarism.
User
About The Authors

Patit Paban Santra
Assistant Librarian, DSMS College, Durgapur - 713206, Durgapur, West Bengal
India

Debasis Majhi
Research Scholar, Banaras Hindu University, Varanasi – 221002, Uttar Pradesh
India


Notifications

  • Birunda, S. S. and Devi, R. K. (2021). A review on word embedding techniques for text classification. In J. S. Raj, A. M. Iliyasu, R. Bestak, and Z. A. Baig (Eds.), Innovative Data Communication Technologies and Application, p. 267-281. https://doi.org/10.1007/978-981-15-9651-3_23

  • Boden, M. A. and Edmonds, E. A. (2009). What is generative art? Digital Creativity, 20(1-2), 21-46. https://doi.org/10.1080/14626260902867915

  • Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., … Amodei, D. (2020). Language models are few-shot learners, Available at: https://arxiv.org/abs/2005.14165.

  • Chan, A. (2023). GPT-3 and InstructGPT: Technological dystopianism, utopianism, and ‘Contextual’ perspectives in AI ethics and industry. AI and Ethics, 3(1), 53-64. https://doi.org/10.1007/s43681-022-00148-6

  • Chowdhury, H. A. and Bhattacharyya, D. K. (2018). Plagiarism: Taxonomy, tools and detection techniques.

  • Cortiz, D. (2022). Exploring transformers models for emotion recognition: A comparision of BERT, DistilBERT, RoBERTa, XLNET and ELECTRA. Proceedings of the 2022 3rd International Conference on Control, Robotics and Intelligent System, 230-234. https://doi.org/10.1145/3562007.3562051

  • Crothers, E., Japkowicz, N., and Viktor, H. (2023). Machine generated text: A comprehensive survey of threat models and detection methods. Available at: https://arxiv.org/abs/2210.07321

  • Das, A., Mandal, N., Rath, D. S. and Das, S. (2022). Trendline of open access publication by Indian Institute of Technology (IITs) researchers in India. SRELS Journal of Information Management, 399-409. https://doi.org/10.17821/srels/2022/v59i6/168621

  • King, M. R. and chatGPT. (2023). A conversation on artificial intelligence, chatbots, and plagiarism in higher education. Cellular and Molecular Bioengineering, 16(1), 1-2. https://doi.org/10.1007/s12195-022-00754-8

  • Labbé, C. and Labbé, D. (2013). Duplicate and fake publications in the scientific literature: How many SCIgen papers in computer science? Scientometrics, 94(1), 379-396. https://doi.org/10.1007/s11192-012-0781-y

  • Maity, D. and Dutta, B. (2022). Identifying the core and allied disciplines involved in the growth of virology: A linguistic analysis. SRELS Journal of Information Management, 363-371. https://doi.org/10.17821/srels/2022/v59i6/170750

  • Oberreuter, G. and Velásquez, J. D. (2013). Text mining applied to plagiarism detection: The use of words for detecting deviations in the writing style-ScienceDirect. Expert Systems with Applications, 40(9), 3756-3763. https://doi.org/10.1016/j.eswa.2012.12.082

  • O’Connor, S. and ChatGPT. (2023). Open artificial intelligence platforms in nursing education: Tools for academic progress or abuse? Nurse Education in Practice, 66. https://doi.org/10.1016/j.nepr.2022.103537

  • Oladokun, B. D., Seidu, A. E., Ogunbiyi, J. O., Aboyade, W. A., Yemi-Peters, O. E. and Elai, M. A. (2022). Utilization of Information and Communication Technologies (ICTs) for managing students’ academic records in Nigerian Schools. SRELS Journal of Information Management, 373-381. https://doi.org/10.17821/srels/2022/v59i6/168449

  • Oya, M. (2020). Syntactic similarity of the sentences in a multi-lingual parallel corpus based on the Euclidean distance of their dependency trees. Proceedings of the 34th Pacific Asia Conference on Language, Information, and Computation, 225-233.

  • Pal, A. and Mukhopadhyay, P. (2022). Fetching automatic authority data in ILS from Wikidata via OpenRefine. SRELS Journal of Information Management, 353-362. https://doi.org/10.17821/srels/2022/v59i6/170677

  • Parmar, R. D. and Nagi, P. K. (2022). Institutional knowledge repositories: Re-contextualization for accreditation and quality management. SRELS Journal of Information Management, 383-390. https://doi.org/10.17821/srels/2022/v59i6/170796

  • Pataranutaporn, P., Danry, V., Leong, J., Punpongsanon, P., Novy, D., Maes, P. and Sra, M. (2021). AI-generated characters for supporting personalized learning and well-being. Nature Machine Intelligence, 3(12). https://doi.org/10.1038/s42256-021-00417-9

  • Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y. and Miller, A. (2019). Language models as knowledge bases? Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2463-2473. https://doi.org/10.18653/v1/D19-1250

  • Roy, B. K., and Mukhopadhyay, P. (2022). Digital access brokers: Clustering and comparison (Part II - from Summarization to Citation Map). SRELS Journal of Information Management, 337-351. https://doi.org/10.17821/srels/2022/v59i6/170786

  • Topal, M. O., Bas, A. and van Heerden, I. (2021). Exploring transformers in natural language generation: GPT, BERT, and XLNet. Available at: https://arxiv.org/abs/2102.08036

  • Transformer, G. G. P., Thunström, A. O. and Steingrimsson, S. (2022). Can GPT-3 write an academic paper on itself, with minimal human input?

  • van Noorden, R. (2014). Publishers withdraw more than 120 gibberish papers. Nature. https://doi.org/10.1038/nature.2014.14763

  • Wani, Z. A. and Bhat, A. (2022). Figshare: A one-stop shop for research data management with diverse features and services. SRELS Journal of Information Management, 391-397. https://doi.org/10.17821/srels/2022/v59i6/170789

  • Weizenbaum, J. (1966). ELIZA-a computer program for the study of natural language communication between man and machine. Communications of the ACM, 36-45. https://doi.org/10.1145/365153.365168

  • Writer, B. (2019). Lithium-ion batteries: A machine-generated summary of current research. Springer International Publishing. https://doi.org/10.1007/978-3-030-16800-1


Abstract Views: 297

PDF Views: 3




  • Scholarly Communication and Machine-Generated Text: Is it Finally AI vs AI in Plagiarism Detection?

Abstract Views: 297  |  PDF Views: 3

Authors

Patit Paban Santra
Assistant Librarian, DSMS College, Durgapur - 713206, Durgapur, West Bengal, India
Debasis Majhi
Research Scholar, Banaras Hindu University, Varanasi – 221002, Uttar Pradesh, India

Abstract


This study utilizes GPT (Generative Pre-Trained Transformer) language model-based AI writing tools to create a set of 80 academic writing samples based on the eight themes of the experiential sessions of the LTC 2023. These samples, each between 2000 and 2500 words long, are then analyzed using both conventional plagiarism detection tools and selected AI detection tools. The study finds that traditional syntactic similarity-based anti-plagiarism tools struggle to detect AI-generated text due to the differences in syntax and structure between machine-generated and human-written text. However, the researchers discovered that AI detector tools can be used to catch AI-generated content based on specific characteristics that are typical of machine-generated text. The paper concludes by posing the question of whether we are entering an era in which AI detectors will be used to prevent AI-generated content from entering the scholarly communication process. This research sheds light on the challenges associated with AI-generated content in the academic research literature and offers a potential solution for detecting and preventing plagiarism in this context.

Keywords


AI (Artificial Intelligence), GPT (Generative Pre-Training Transformer), Machine Learning, ChatGPT, Natural Language Processing (NLP), OpenAI, Plagiarism.

References





DOI: https://doi.org/10.17821/srels%2F2023%2Fv60i3%2F171028