Indian Journal of Computer Science

Open Access Open Access  Restricted Access Subscription Access

Language Model Evolution : GPT and Impact Beyond


Affiliations
1 Assistant Professor, Amity School of Business, Amity University Kolkata, Major Arterial Road (South-East), AA II, Newtown, Kolkata, West Bengal - 700 135, India

   Subscribe/Renew Journal


Language models are considered a way of machine level of understanding and predicting human languages as a part of human communication relevant to the context. The present research paper tries to understand the growth of such language models popularly known as GPT or Generative Pre-trained Transformer. It tries to understand the meaning, growth, working of model, and some of the applications where GPT is being used. Several applications on business, website development, and conversational applications are now being powered by GPT with tremendous potential for future convergence. Artificial Intelligence and sub-domains. This research paper can be very useful for academicians, researchers, and professionals who handle business and information technology applications.

Keywords

Context, Predictive, Text, Trained, Transformer.

Manuscript Received : December 20, 2022; Revised : January 12, 2023 ; Accepted : January 16, 2023. Date of Publication : February 5, 2023.

User
Subscription Login to verify subscription
Notifications
Font Size

  • “OpenAI.” Wikipedia.org. https://en.wikipedia.org/wiki/OpenAI#GPT

  • J. Alammar, “The Illustrated GPT-2 (Visualizing Transformer Language Models).” Jalammar.github.io. https://jalammar.github.io/illustrated-gpt2/

  • N. Thompson and S. Spanuth, “The decline of computers as a general purpose technology: Why Deep Learning and the end of Moore’s Law are fragmenting computing,” 2018, doi: 10.2139/ssrn.3287769.

  • L. Floridi and M. Chiriatti, “GPT-3: Its nature, scope, limits, and consequences,” Minds and Machines, vol. 30, no. 4, pp. 681–694, 2020.

  • Z. Yang, Z. Gan, J. Wang, X. Hu, Y. Lu, Z. Liu, and L. Wang, “An empirical study of GPT-3 for few-shot knowledge-based VQA,” in Proc. AAAI Conf. Artif. Intell., 2021, doi: 10.48550/arXiv.2109.05014.

  • N. Dehouche, “Plagiarism in the age of massive Generative Pre-trained Transformers (GPT-3),” Ethics Sci. Environmental Politics, vol. 21, pp.17–23, 2021.

  • W. L. Benzon, “GPT-3: Waterloo or Rubicon? Here be Dragons, Version 4.1,” Working Paper, Version 2, 2020. https://ssrn.com/abstract=3667608

  • J. Liu, D. Shen, Y. Zhang, B. Dolan, L. Carin, and W. Chen, “What makes good in-context examples for GPT-3?,” 2022. http://arXiv:2101.06804

  • J. Klinger, J. C. Mateos-Garcia, and K. Stathoulopoulos, “Deep Learning, Deep Change? Mapping the development of the Artificial Intelligence general purpose technology,” 2018, doi: 10.2139/ssrn.3233463.

  • Y. Yang, Y. Li, and X. Quan, “UBAR: Towards fully end-to-end task-oriented dialog system with GPT-2,” in Proc. AAAI Conf. Artif. Intell., vol. 35, no.16, 2021, pp. 14230–14238.

  • J.-S. Lee and J. Hsiang, “Patent claim generation by fine-tuning OpenAI GPT-2,” World Patent Inform., vol. 62, 2020, 101983, doi: 10.1016/j.wpi.2020.101983.

  • D. Ham, J. G. Lee, Y. Jang, and K.-E. Kim, “End-to-end neural pipeline for goal-oriented dialogue systems using GPT-2,” in Proc. 58th Annu. Meeting Assoc. Computational Linguistics, 2020, pp. 583–592, doi: 10.18653/v1/2020.acl-main.54 .

  • K. Elkins and J. Chun, “Can GPT-3 pass a writer’s Turing test?,” J. Cultural Analytics, vol. 5, no. 2, pp. 1–16, 2020, doi: 10.22148/001c.17212.

  • T. Klein and M. Nabi, “Learning to answer by learning to ask: Getting the best of GPT-2 and BERT worlds,” 2019. [Online]. Available: http://arXiv:1911.02365

  • T. Goyal, J. J. Li, and G. Durrett, “News summarization and evaluation in the era of GPT-3,” 2022. [Online]. Available: http://arXiv:2209.12356

  • K. McGuffie and A. Newhouse, “The radicalization risks of GPT-3 and advanced neural language models,” 2020. [Online]. Available: http://arXiv:2009.06807

  • M. Mager, R. F. Astudillo, T. Naseem, M. A. Sultan, Y. S. Lee, R. Florian, and S. Roukos, “GPT-too: A language-model-first approach for AMR-to-text generation,” 2020. [Online]. Available: http://arXiv:2005.0912 3

  • W. de Vries and M. Nissim, “As good as new. How to successfully recycle English GPT-2 to make models for other languages,” 2020. [Online]. Available: http://arXiv: 2012.05628

  • Y. Liao, Y. Wang, Q. Liu, and X. Jiang, “GPT-based generation for classical Chinese poetry,” 2019. [Online]. Available: http://1907.00151

  • S. Black, S. Biderman, E. Hallahan, Q. Anthony, L. Gao, L. Golding, H. Horace, C. Leahy, C., K. McDonald, J. Phang, M. Pieter, U. S. Prashanth, S. Purohit, L. Reynolds, J. Tow, B. Wang, and S. Weinbach, “GPT-NeoX-20B: An open-source autoregressive language model,” 2022. [Online]. Available: http://arXiv:2204.06745

  • V. Kieuvongngam, B. Tan, and Y. Niu, "Automatic text summarization of COVID-19 medical research articles using BERT and GPT-2," 2020. [Online]. Available: http://arXiv:2006.01997

  • E. H. Mashalidis and S-Y. Lee, “Structures of bacterial MraY and human GPT provide insights into rational antibiotic design,” J. Mol. Biol., vol. 432, no. 18, pp. 4946–4963, 2020. [Online]. Available: https://www.sciencedirect.com/science/article/abs/pii/S0022283620302382

  • M. O. Topal, A. Bas, and I. van Heerdenx, “Exploring transformers in natural language generation: GPT, BERT, and XLNET,” 2021. [Online]. Available: http://arXiv:2102.08036

  • Y. Dou, M. Forbes, R. Koncel-Kedziorski, N. A. Smith, and Y. Choi, “Is GPT-3 text indistinguishable from human text? SCARECROW: A framework for scrutinizing machine text,” in Proc. 60th Annu. Meeting Assoc. Comput. Linguistics (Volume 1: Long Papers), pp. 7250–7274, Dublin, Ireland. Assoc. Comput. Linguistics, 2022, doi: 10.48550/arXiv.2107.01294.

  • K. Meng, D. Bau, A. J. Andonian, and Y. Belinkov, “Locating and editing factual associations in GPT,” in Advances Neural Inform. Proces. Syst., 2022.

  • D. Sinha, “Top 10 GPT-3 powered applications to know in 2022,” Analyticsinsight.net. https://www.analyticsinsight.net/top-10-gpt-3-powered-applications-to-know-in-2022/


Abstract Views: 143

PDF Views: 0




  • Language Model Evolution : GPT and Impact Beyond

Abstract Views: 143  |  PDF Views: 0

Authors

Sandeep Bhattacharjee
Assistant Professor, Amity School of Business, Amity University Kolkata, Major Arterial Road (South-East), AA II, Newtown, Kolkata, West Bengal - 700 135, India

Abstract


Language models are considered a way of machine level of understanding and predicting human languages as a part of human communication relevant to the context. The present research paper tries to understand the growth of such language models popularly known as GPT or Generative Pre-trained Transformer. It tries to understand the meaning, growth, working of model, and some of the applications where GPT is being used. Several applications on business, website development, and conversational applications are now being powered by GPT with tremendous potential for future convergence. Artificial Intelligence and sub-domains. This research paper can be very useful for academicians, researchers, and professionals who handle business and information technology applications.

Keywords


Context, Predictive, Text, Trained, Transformer.

Manuscript Received : December 20, 2022; Revised : January 12, 2023 ; Accepted : January 16, 2023. Date of Publication : February 5, 2023.


References





DOI: https://doi.org/10.17010/ijcs%2F2023%2Fv8%2Fi1%2F172680