Using GPT-2 Model and Hazm Library for Persian Text Generation

Soluki, M.; Askarinejadamiri, Z.; Zanjani, N.

doi:10.22061/jecei.2023.10101.690

Articles in Press

Document Type : Original Research Paper

Authors

Computer science of Refah University college, Tehran, Iran.

https://doi.org/10.22061/jecei.2023.10101.690

Abstract

Background and Objectives: This article explores a method for generating Persian texts using the GPT-2 language model and the Hazm library. Researchers and writers often require tools that can assist them in the writing process and even think on their behalf in various domains. By leveraging the GPT-2 model, it becomes possible to generate acceptable and creative texts, which increases writing speed and efficiency, thus mitigating the high costs associated with article writing.
Methods: In this research, the GPT-2 model is employed to generate and predict Persian texts. The Hazm library is utilized for natural language processing and automated text generation. The results of this study are evaluated using different datasets and output representations, demonstrating that employing the Hazm library with input data exceeding 1000 yields superior outcomes compared to other text generation methods
Results: Through extensive experimentation and analysis, the study demonstrates the effectiveness of this combination in generating coherent and contextually appropriate text in the Persian language. The results highlight the potential of leveraging advanced language models and linguistic processing tools for enhancing natural language generation tasks in Persian. The findings of this research contribute to the growing field of Persian language processing and provide valuable insights for researchers and practitioners working on text generation applications in similar languages.
Conclusion: Overall, this study showcases the promising capabilities of the GPT-2 model and Hazm library in Persian text generation, underscoring their potential for future advancements in the field This research serves as a valuable guide and tool for generating Persian texts in the field of research and scientific writing, contributing to cost and time reduction in article writing

Keywords

Main Subjects

Optimization

Open Access

This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit: http://creativecommons.org/licenses/by/4.0/

Publisher’s Note

JECEI Publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Publisher

Shahid Rajaee Teacher Training University

References

[1] D. McDonald, P. Proctor, W. Gill, S. Heaven, J. Marr, J. Young, "Increasing early childhood educators’ use of communication-facilitating and language-modelling strategies: Brief speech and language therapy training," Child Lang. Teach. Ther., 31(3): 305-322, 2015.

[2] E. S. Jo, T. Gebru, "Lessons from archives: Strategies for collecting sociocultural data in machine learning," in Proc. the 2020 Conference on Fairness, Accountability and Transparency: 306-316 2020.

[3] P. P. Ray, "ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope," Internet Things Cyber-Phys. Syst., 3: 121-154, 2023.

[4] A. Tamkin, M. Brundage, J. Clark, D. Ganguli, "Understanding the capabilities, limitations, and societal impact of large language models," arXiv preprint arXiv:2102.02503, 2021.

[5] Y. Yibin, C. Na, X. Chaoqian, Y. Junjian, "Accuracy assessment and analysis for GPT2," Acta Geod. Cartographica Sin., 44(7):726, 2015.

[6] A. S. George, A. H. George, "A review of ChatGPT AI's impact on several business sectors," Partners Univers. Int. Innovation J., 1(1): 9-23, 2023.

[7] X. Zheng, C. Zhang, P. C. Woodland, "Adapting GPT, GPT-2 and BERT language models for speech recognition," in Proc. 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU): 162-168, 2021.

[8] Z. Liu, R. A. Roberts, M. Lal-Nag, X. Chen, R. Huang, W. Tong, "AI-based language models powering drug discovery and development," Drug Discovery Today, 26(11): 2593-2607, 2021.

[9] N. De Cao, T. Kipf, "MolGAN: An implicit generative model for small molecular graphs," arXiv preprint arXiv:1805.11973, 2018.

[10] M. Mars, "From word embeddings to pre-trained language models: A state-of-the-art walkthrough," Appl. Sci., 12(17): 8805, 2022.

[11] Y. Qu, P. Liu, W. Song, L. Liu, M. Cheng, "A text generation and prediction system: pre-training on new corpora using BERT and GPT-2," in Proc. 2020 IEEE 10th International Conference on Electronics Information and Emergency Communication (ICEIEC): 323-326, 2020.

[12] O. Shatalov, N. Ryabova, "Towards Russian text generation problem using OpenAI’s GPT-2," in Proc. CEUR Workshop 2021.

[13] N. Alexandr, O. Irina, K. Tatyana, K. Inessa, P. Arina, "Fine-tuning gpt-3 for russian text summarization," in Data Science and Intelligent Systems: Proceedings of 5th Computational Methods in Systems and Software, 2: 748-757, 2021.

[14] T. Goyal, J. J. Li, G. Durrett, "News summarization and evaluation in the era of gpt-3," arXiv preprint arXiv:2209.12356, 2022.

[15] S. Kumari, T. Pushphavati, "Question answering and text generation using BERT and GPT-2 model," in Proc. ICCMDE 2021 Computational Methods and Data Engineering: 93-110, 2022.

[16] A. H. George, A. S. Hameed, A. S. George, T. Baskar, "Study on quantitative understanding and knowledge of farmers in trichy district," Partners Univer. Int. Res. J., 1(2): 5-8, 2022.

[17] J. Devlin, M. W. Chang, K. Lee, K. Toutanova, "Bert: Pre-training of deep bidirectional transformers for language understanding," arXiv preprint arXiv:1810.04805, 2018.

[18] A. Kraft, "Triggering models: Measuring and mitigating bias in german language generation," Universität Hamburg, 2021.

[19] M. Moila, T. Modipa, J. Manamela, "The analysis of a GPT-based Sepedi text generation model," in Proc. International Conference on Intelligent and Innovative Computing Applications: 144-152, 2022.

[20] S. P. Ramalepe, T. I. Modipa, M. H. Davel, “The development of a sepedi text generation model using transformers,” in Proc. Southern Africa Telecommunication Networks and Applications Conference (SATNAC)," 2022.

[21] L. Butgereit, A. van Staden, "Supporting home-language education in africa with multi-lingual mathematics tutoring using GPT-4," in Proc. International Conference on Artificial Intelligence and its Applications: 44-49, 2023.

[22] S. Yang, D. Feng, L. Qiao, Z. Kan, D. Li, "Exploring pre-trained language models for event extraction and generation," in Proc. the 57th Annual Meeting of the Association for Computational Linguistics: 5284-5294, 2019.

[23] J. Mellon, J. Bailey, R. Scott, J. Breckwoldt, M. Miori, " Do AIs know what the most important issue is? Using language models to code open-text social survey responses at scale," SSRN Electron. J., 2022.

[24] A. Radford et al., "Better language models and their implications. OpenAI," ed, 2019.

[25] R. Dale, "GPT-3: What’s it good for?," Nat. Lang. Eng., 27(1): 113-118, 2021.

[26] M. Zhang, J. Li, "A commentary of GPT-3 in MIT Technology Review 2021," Fundam. Res., 1(6): 831-833, 2021.

[27] B. Zhu, Z. Gu, Y. Qian, F. Lau, Z. Tian, "Leveraging transferability and improved beam search in textual adversarial attacks," Neurocomputing, 500: 135-142, 2022.

[28] D. Kim, J. Lee, "Designing an algorithm-driven text generation system for personalized and interactive news reading," Int. J. Human–Computer Interact., 35(2): 109-122, 2019.

[29] X. He et al., "Cater: Intellectual property protection on text generation apis via conditional watermarks," Adv. Neural Inf. Process. Syst., 35: 5431-5445, 2022.

[30] J. Li, Z. Li, L. Mou, X. Jiang, M. Lyu, I. King, "Unsupervised text generation by learning from search," Adv. Neural Inf. Process. Sys., 33: 10820-10831, 2020.

LETTERS TO EDITOR

Journal of Electrical and Computer Engineering Innovations (JECEI) welcomes letters to the editor for the post-publication discussions and corrections which allows debate post publication on its site, through the Letters to Editor. Letters pertaining to manuscript published in JECEI should be sent to the editorial office of JECEI within three months of either online publication or before printed publication, except for critiques of original research. Following points are to be considering before sending the letters (comments) to the editor.

[1] Letters that include statements of statistics, facts, research, or theories should include appropriate references, although more than three are discouraged.

[2] Letters that are personal attacks on an author rather than thoughtful criticism of the author’s ideas will not be considered for publication.

[3] Letters can be no more than 300 words in length.

[4] Letter writers should include a statement at the beginning of the letter stating that it is being submitted either for publication or not.

[5] Anonymous letters will not be considered.

[6] Letter writers must include their city and state of residence or work.

[7] Letters will be edited for clarity and length.

Name *

Email Address *

Affiliation *

Comments *

Security Code *

Journal of Electrical and Computer Engineering Innovations (JECEI)

Using GPT-2 Model and Hazm Library for Persian Text Generation

References

References

Send comment about this article

Articles in Press, Accepted Manuscript
Available Online from 25 December 2023

Using GPT-2 Model and Hazm Library for Persian Text Generation

References

References

Send comment about this article

Articles in Press, Accepted Manuscript Available Online from 25 December 2023

Articles in Press, Accepted Manuscript
Available Online from 25 December 2023