Proceedings of International Conference on Applied Innovation in IT
2022/03/09, Volume 10, Issue 1, pp.43-49

Linguistic Difference of Human-Human and Human-Chatbot Dialogues about COVID-19 in the Russian Language

Aleksandr Perevalov, Aleksandr Vysokov, Andreas Both

Abstract: This work describes the quantitative analysis of the linguistic difference in human-human and human-chatbot dialogues. The research is based on conducting a set of experiments where respondents communicate with a human or a chatbot in the domain of COVID-19 questions. In the case of the human-human dialogues, the approach of the inverted "Wizard of Oz" experimental setting is used. During the experiments, 35 humanhuman and 68 human-chatbot dialogues in Russian language were performed. The dialogues were collected

Keywords: Quantitative Text Analysis, Human-Chatbot Interaction, Human-Computer Interaction, Chatbot Interaction Design, Language Features, Wizard of Oz, COVID-19

DOI: 10.25673/76931

Download: PDF


  1. U. Gnewuch, S. Morana, and A. Maedche, "Towards designing cooperative and social conversational agents for customer service." in Proceedings of the 38th International Conference on Information Systems (ICIS), 2017.
  2. K. K. Fitzpatrick, A. Darcy, and M. Vierhile, "Deliv-ering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully au-tomated conversational agent (woebot): a randomized controlled trial," JMIR mental health, vol. 4, no. 2, p. e7785, 2017.
  3. A. Androutsopoulou, N. Karacapilidis, E. Loukis, and Y. Charalabidis, "Transforming the communication between citizens and government through aiguided chatbots," Government Information Quarterly, vol. 36, no. 2, pp. 358-367, 2019.
  4. A. P. Chaves and M. A. Gerosa, "How should my chatbot interact? a survey on social characteristics in human–chatbot interaction design," International Journal of Human–Computer Interaction, vol. 37, no. 8, pp. 729-758, 2021.
  5. M. R. Mehl, "Quantitative text analysis." 2006.
  6. D. Duijst, "Can we improve the user experience of chatbots with personalisation," Master’s thesis. University of Amsterdam, 2017.
  7. M.-C. Jenkins, R. Churchill, S. Cox, and D. Smith, "Analysis of user interaction with service oriented
  8. chatbot systems," in International Conference on Human-Computer Interaction. Springer, 2007, pp. 76-83.
  9. J. Hill, W. R. Ford, and I. G. Farreras, "Real con-versations with artificial intelligence: A compari-son between human–human online conversations and human–chatbot conversations," Computers in human behavior, vol. 49, pp. 245-250, 2015.
  10. A. Schlesinger, K. P. O’Hara, and A. S. Taylor, "Let’s talk about race: Identity, chatbots, and ai," in Proceed-ings of the 2018 chi conference on human factors in computing systems, 2018, pp. 1-14.
  11. Y. Mou and K. Xu, "The media inequality: Comparing the initial human-human and human-ai social interactions,"
  12. Computers in Human Behavior, vol. 72, pp. 432-440, 2017.
  13. E. Silkej, "Linguistic differences in real conversations: Human to human vs human to chatbot," 2020.
  14. M. Oya, "Syntactic dependency distance as sentence complexity measure," Proceedings of the 16th International Conference of Pan-Pacific Association of Applied Linguistics, 01 2011.
  15. H. Liu, "Dependency distance as a metric of language comprehension difficulty," Journal of Cognitive Science, vol. 9, pp. 159-191, 09 2008.
  16. J. Ure, "Lexical density and register differentiation,"Applications of Linguistics, pp. 443–452, 1971.
  17. G. Herdan, Quantitative Linguistics. London: Butterworth, 1960.
  18. H. H. Sommers, "Statistical methods in literary analysis,"The computer and literary style, pp. 128-140, 1966.
  19. P. Guiraud, Probl`emes et M´ethodes de la Statistique Linguistique. Paris: Presses universitaires de France, 1960.
  20. R. Flesch, "A new readability yardstick," Journal of Applied Psychology, pp. 221–233, 1948.
  21. V. Solovyev, V. Ivanov, and M. Solnyshkina, "Assessment of reading difficulty levels in russian academic
  22. texts: Approaches and metrics," Journal of Intelligent and Fuzzy Systems, vol. 34, pp. 1-10, 04 2018.
  23. J. P. Kincaid, R. P. Fishburne Jr, R. L. Rogers, and B. S. Chissom, "Derivation of new readability formulas
  24. (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel," Naval Technical Training Command Millington TN Research Branch, Tech. Rep., 1975.
  25. R. Senter and E. A. Smith, "Automated readability index," CINCINNATI UNIV OH, Tech. Rep., 1967.
  26. E. B. Coleman, "Developing a technology of written instruction: Some determiners of the complexity of prose," Verbal learning research and the technology of written instruction, pp. 155-204, 1971.
  27. I. E. Fang, "The "easy listening formula"," Journal of Broadcasting & Electronic Media, vol. 11, no. 1,
  28. pp. 63-68, 1966.
  29. A. Both, A. Perevalov, J. R. Bartsch, P. Heinze, R. Iudin, J. R. Herkner, T. Schrader, J. Wunsch, A. K. Falkenhain, and R. Gurth, "A question answering system for retrieving german covid-19 data driven and quality-controlled by semantic technology," in Joint Proceedings of the Semantics colocated
  30. events: Poster&Demo track and Workshop on Ontology-Driven Conceptual Modelling of Digital
  31. Twins co-located with SEMANTiCS 2021, ser. CEUR Workshop Proc., vol. 2774,, 2021.
  32. L. D. Riek, "Wizard of oz studies in hri: a systematic review and new reporting guidelines," Journal of Human-Robot Interaction, vol. 1, no. 1, pp. 119-136, 2012.
  33. J. Newn, R. Singh, F. Allison, P. Madumal, E. Velloso, and F. Vetere, "Designing interactions with intentionaware gaze-enabled artificial agents," in IFIP Conference on Human-Computer Interaction. Springer, 2019, pp. 255-281



       - Committees
       - Proceedings


       - Volume 10, Issue 1 (ICAIIT 2022)
       - Volume 9, Issue 1 (ICAIIT 2021)
       - Volume 8, Issue 1 (ICAIIT 2020)
       - Volume 7, Issue 1 (ICAIIT 2019)
       - Volume 7, Issue 2 (ICAIIT 2019)
       - Volume 6, Issue 1 (ICAIIT 2018)
       - Volume 5, Issue 1 (ICAIIT 2017)
       - Volume 4, Issue 1 (ICAIIT 2016)
       - Volume 3, Issue 1 (ICAIIT 2015)
       - Volume 2, Issue 1 (ICAIIT 2014)
       - Volume 1, Issue 1 (ICAIIT 2013)


       ICAIIT 2022
         - Message

       ICAIIT 2021
         - Photos
         - Reports

       ICAIIT 2020
         - Photos
         - Reports

       ICAIIT 2019
         - Photos
         - Reports

       ICAIIT 2018
         - Photos
         - Reports





           ISSN 2199-8876
           Copyright © 2013-2021 Leonid Mylnikov, © 2022 at Anhalt University of Applied Sciences. All rights reserved.