Proceedings of International Conference on Applied Innovation in IT
2024/03/07, Volume 12, Issue 1, pp.115-121

Bert Embedding and Scoring for Scientific Automatic Essay Grading


Abeer Abdulkarem and Anastasia Krivtsun


Abstract: The educational landscape is experiencing a surging demand for Automated Essay Grading (AEG), prompting the need for innovative solutions. This paper introduces a cutting-edge methodology that harnesses the power of Bidirectional Encoder Representation from Transformers (BERT) to embed and score essays in the scientific AEG domain. Tackling challenges such as Out-of-Vocabulary (OOV), BERT's contextual embedding proves instrumental. The study meticulously evaluates a hybrid architecture on a prototype incorporating non-English essay answers, establishing a benchmark against state-of-the-art studies. Beyond the expeditious grading of essays, particularly in scientific realms, this paper makes a substantial contribution to the ever-evolving field of educational technology. The AEG task revolves around the automation of essay response grading, where input data encompasses essay answers, and output data comprises assigned scores. The adopted mathematical model seamlessly integrates BERT for contextual embedding and subsequent scoring. The evaluation uncovers compelling results, underscoring the effectiveness of the proposed BERT-based model. The model's architecture, characterized by bidirectional layers and a dense output, encompasses a notable 2,243,401 parameters. Significantly, the Kappa Score achieved by the model impressively stands at 0.9725, highlighting its superiority over existing methodologies.

Keywords: Automatic Essay Grading, Word Embedding Techniques, BERT Techniques, Neural Network

DOI: 10.25673/115649; PPN 1884680852

Download: PDF

References:

  1. F. Li, X. Xi, Z. Cui, D. Li, and W. Zeng, “Automatic Essay Scoring Method Based on Multi-Scale Features,” Applied Sciences (Switzerland), vol. 13, no. 11, Jun. 2023, doi: 10.3390/app13116775.
  2. L. Blecher, G. Cucurull, T. Scialom, and R. Stojnic, “Nougat: Neural Optical Understanding for Academic Documents,” Aug. 2023, [Online]. Available: http://arxiv.org/abs/2308.13418.
  3. F. Nadeem, H. Nguyen, Y. Liu, and M. Ostendorf, “Automated Essay Scoring with Discourse-Aware Neural Models,” 2019, [Online]. Available: www.smashwords.com.
  4. M.V. Koroteev, “BERT: A Review of Applications in Natural Language Processing and Understanding.”
  5. Y. Farag, H. Yannakoudakis, and T. Briscoe, “Neural Automated Essay Scoring and Coherence Modeling for Adversarially Crafted Input,” Apr. 2018, [Online]. Available: http://arxiv.org/abs/1804.06898.
  6. Y.-J. Jong, Y.-J. Kim, and O.-C. Ri, “Improving Performance of Automated Essay Scoring by using back-translation essays and adjusted scores.” [Online]. Available: https://github.com/j-y-j-109/asap-back-translation.
  7. H. Zhang and D. Litman, “Co-Attention Based Neural Network for Source-Dependent Essay Scoring,” Aug. 2019, doi: 10.18653/v1/W18-0549.
  8. C. T. Lim, C. H. Bong, W. S. Wong, and N. K. Lee, “A comprehensive review of automated essay scoring (Aes) research and development,” Pertanika Journal of Science and Technology, vol. 29, no. 3. Universiti Putra Malaysia Press, pp. 1875-1899, 2021, doi: 10.47836/pjst.29.3.27.
  9. J. Liu, Y. Xu, and Y. Zhu, “Automated Essay Scoring based on Two-Stage Learning,” Jan. 2019, [Online]. Available: http://arxiv.org/abs/1901.07744.
  10. D. Ramesh and S. K. Sanampudi, “An automated essay scoring systems: a systematic literature review,” Artif Intell Rev, vol. 55, no. 3, pp. 2495–2527, Mar. 2022, doi: 10.1007/s10462-021-10068-2.
  11. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, 2018, arXiv e-prints, page arXiv:1810.04805.
  12. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, et. al., “Attention Is All You Need,” 2017, arXiv:1706.03762.
  13. K. Taghipour and H. T. Ng, “A Neural Approach to Automated Essay Scoring,” [Online]. Available: https://www.kaggle.com/c/asap-aes.
  14. B.F. Dhini, A.S. Girsang, U.U. Sufandi, and H. Kurniawati, “Automatic essay scoring for discussion forum in online learning based on semantic and keyword similarities,” Asian Association of Open Universities Journal, Dec. 2023, doi: 10.1108/AAOUJ-02-2023-0027.
  15. E. Mayfield and A.W. Black, “Should You Fine-Tune BERT for Automated Essay Scoring?,” 2020, [Online]. Available: https://course.fast.ai.

    HOME

       - Call for Papers
       - For authors
       - Important Dates
       - Conference Committee
       - Editorial Board
       - Reviewers
       - Last Proceedings


    PROCEEDINGS

       - Volume 12, Issue 1 (ICAIIT 2024)        - Volume 11, Issue 2 (ICAIIT 2023)
       - Volume 11, Issue 1 (ICAIIT 2023)
       - Volume 10, Issue 1 (ICAIIT 2022)
       - Volume 9, Issue 1 (ICAIIT 2021)
       - Volume 8, Issue 1 (ICAIIT 2020)
       - Volume 7, Issue 1 (ICAIIT 2019)
       - Volume 7, Issue 2 (ICAIIT 2019)
       - Volume 6, Issue 1 (ICAIIT 2018)
       - Volume 5, Issue 1 (ICAIIT 2017)
       - Volume 4, Issue 1 (ICAIIT 2016)
       - Volume 3, Issue 1 (ICAIIT 2015)
       - Volume 2, Issue 1 (ICAIIT 2014)
       - Volume 1, Issue 1 (ICAIIT 2013)


    PAST CONFERENCES

       ICAIIT 2024
         - Photos
         - Reports

       ICAIIT 2023
         - Photos
         - Reports

       ICAIIT 2021
         - Photos
         - Reports

       ICAIIT 2020
         - Photos
         - Reports

       ICAIIT 2019
         - Photos
         - Reports

       ICAIIT 2018
         - Photos
         - Reports

    ETHICS IN PUBLICATIONS

    ACCOMODATION

    CONTACT US

 

DOI: http://dx.doi.org/10.25673/115729


        

         Proceedings of the International Conference on Applied Innovations in IT by Anhalt University of Applied Sciences is licensed under CC BY-SA 4.0


                                                   This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License


           ISSN 2199-8876
           Publisher: Anhalt University of Applied Sciences

        site traffic counter

Creative Commons License
Except where otherwise noted, all works and proceedings on this site is licensed under Creative Commons Attribution-ShareAlike 4.0 International License.