EmoStudent: Developing a Dataset to Analyse Students' Emotional Well-Being

Breskina, Svitlana

doi:10.25673/112996

Proceedings of International Conference on Applied Innovation in IT · 2023/11/30 · Vol. 11 · Issue 2 · pp. 75–80

EmoStudent: Developing a Dataset to Analyse Students' Emotional Well-Being

Svitlana Antoshchuk and Anastasiia Breskina

📄 Download PDF DOI: 10.25673/112996

Abstract

This article introduces an initial version of a dataset designed to educate and assess models that concentrate on studying of the emotional condition of students throughout the remote learning process. This dataset comprises short video clips showing the faces of individuals from diverse ethnic backgrounds and age groups in front of the computer screen. No dataset specialising in the proctoring systems problem solving task was found (process of working in front of the computer, emotions during the educational process). As a result, existing datasets for solving problems in related fields were analysed: emotion classification, emotion recognition, and face recognition. Building on this analysis and the specifics of the chosen data source (YouTube videos with Creative Commons license), the previously established criteria for creating the dataset were modified and expanded. A more adaptable approach was introduced concerning the categorization based on age and ethnicity. A path for future endeavors was also delineated, proposing an enhancement of the current implementation to encompass a broader spectrum of emotions and individuals with various forms of disabilities in subsequent iterations.

Keywords

Dataset Computer Vision Emotion Understanding Artificial Intelligence-Based Proctoring Systems.

References

A. Nigam, R. Pasricha, T. Singh, and P. Churi, "A systematic review on AI-based proctoring systems: Past, present and future," Education and Information Technologies, vol. 26, pp. 6421-6445, Dec. 2021, doi: 10.1007/s10639-021-10597-x.
S. Motwani, C. Nagpal, M. Motwani, N. Nagdev, and A. Yeole, "AI-Based proctoring system for online tests," Proceedings of the 4th International Conference on Advances in Science & Technology (ICAST2021), Jun. 2021, doi: 10.2139/ssrn.3866446.
A. Breskina, "Development of an automated online proctoring system," Herald of Advanced Information Technology, vol. 6, no. 2, pp. 163-173, 2023, doi: 10.15276/hait.06.2023.11.
S. Jin et al., "Whole-Body human pose estimation in the wild," ECCV 2020: Computer Vision – ECCV, 2020, pp. 196-214, 2020, doi: 10.48550/arXiv.2007.11858.
S. Yang, P. Luo, C. C. Loy, and X. Tang, "WIDER FACE: A face detection benchmark," 2016 IEEE Conference on Computer Vision and Pattern Recognition, pp. 5525-5533, 2016, doi: 10.1109/CVPR.2016.596.
V. Jain and E. Learned-Miller, "FDDB: A benchmark for face detection in unconstrained settings.," Technical Report UM-CS-2010-009, Dept. Of Computer Science, University of Massachusetts, Amherst, 2010.
X. Zhu and D. Ramanan, "Face detection, pose estimation, and landmark localization in the wild," 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, pp. 2879- 2886, 2012, doi: 10.1109/CVPR.2012.6248014.
J. Yan, X. Zhang, Z. Lei, and S. Z. Li, "Face detection by structural models," Image and Vision Computing, vol. 32, no. 10, pp. 790-799, 2014, doi: 10.1016/j.imavis.2013.12.004.
S. R. Livingstone and F. A. Russo, "The ryerson audiovisual database of emotional speech and song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in north american english," PLoS ONE, vol. 13, no. 5, May 2018, doi: 10.1371/journal.pone.0196391.
D. McDuff, R. E. Kaliouby, T. Senechal, M. Amr, J. F. Cohn, and R. Picard, "Affectiva-MIT facial expression dataset (AM-FED): Naturalistic and spontaneous facial expressions collected “in-thewild”," 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops, OR, USA,, pp. 881-888, 2013, doi: 10.1109/CVPRW.2013.130.
P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews, "The Extended CohnKanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression," 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, San Francisco, CA, USA, pp. 94-101, 2010, doi: 10.1109/CVPRW.2010.5543262.
E. A. Clark et al., "The facial action coding system for characterization of human affective response to consumer product-based stimuli: A systematic review," Front Psychol, May 2020, doi: 10.3389/fpsyg.2020.00920.
M. Pantic, M. Valstar, R. Rademaker, and L. Maat, "Web-based database for facial expression analysis," 2005 IEEE International Conference on Multimedia and Expo, Amsterdam, Netherlands, pp. 5, 2005, doi: 10.1109/ICME.2005.1521424.
P. Lucey, J. F. Cohn, K. M. Prkachin, P. E. Solomon, and I. Matthews, "Painful data: The UNBC-McMaster shoulder pain expression archive database," 2011 IEEE International Conference on Automatic Face & Gesture Recognition (FG), Santa Barbara, CA, USA, pp. 57-64, 2011, doi: 10.1109/FG.2011.5771462.
M. Singh, X. Hoque, D. Zeng, Y. Wang, K. Ikeda, and A. Dhall, "Do I have your attention: A large scale engagement prediction dataset and baselines," 2023. doi: 10.48550/arXiv.2302.00431.
"Face dataset by generated photos (face dataset for academics by generated photos)," [Online]. Available: https://generated.photos/ datasets#research-dataset, [Accessed Jul. 3, 2023].
W. Zhang, X. Wang, and X. Tang, "Coupled information-theoretic encoding for face photo-sketch recognition," CVPR 2011, Colorado Springs, CO, USA, pp. 513-520, 2011, doi: 10.1109/CVPR.2011.5995324.
A. Kaur, A. Mustafa, L. Mehta, and A. Dhall, "Prediction and localization of student engagement in the wild," 2018 Digital Image Computing: Techniques and Applications (DICTA), Canberra, ACT, Australia, p. 1-8, 2018, doi: 10.1109/DICTA.2018.8615851.