The problem of processing large databases is important for solving many pressing problems of science and technology. In this paper, we have developed a computer program (Conan 3.0) for processing large text arrays. However, other applications are possible. We have applied the developed program for the analysis of large texts on the basis of Zipf's laws. The task, which was solved in this work, is connected with the laws of the evolution of languages; in particular, correlations in the development of different Slavic languages were traced. It was assumed that an important characteristic of the language is the Zipf's constant. As a result of calculating the changes in the short-circuit over the 18th, 19th and 20th centuries for the Ukrainian, Russian and Polish languages, no significant changes in the short-circuit were revealed. Small fluctuations in the short-circuit for these languages do not correlate.
Keywords
Program for Big Date TreatmentZipf’s LawsEvolution of Languages
References
A. Corral, G. Boleda, and R. Ferrer-i-Cancho, “Zipf’sLaw for Word Frequencies: Word Forms versus Lemmas in Long Texts”, PLoS ONE, vol. 10 (7), 2015.
M. Cristelli, M. Batty, and L. Pietronero, “There isMore than a Power Law in Zipf”, Scientific Report 2,no. 812, 2012.
A. Saichev, Y. Malevergne, and D. Sornette, “Theoryof Zipf ’s law and beyond”, Lecture Notes inEconomics and Mathematical Systems 632, Springer,Heidelberg, Germany, 2010.
V. Bochkarev and E. Lerner, “Calculation of PreciseConstants in a Probability Model of Zipf’s LawGeneration and Asymptotics of Sums of MultinomialCoefficients”, International Journal of Mathematicsand Mathematical Sciences, vol. 17, 2017.
M. Hauser, Ch. Yang, R. Berwick, I. Tattersall,M.Ryan, J. Watumull, N. Chomsky, andR.Lewontin, “The mystery of language evolution”,Frontiers in Psychology, vol. 5, 2014.
W. Fitch, “Empirical approaches to the study oflanguage evolution”, Psychonomic Bulletin &Review, vol. 24 (1), 2017, pp. 3-33.
Y. Zhao and J. Zobel, “Search with style: Authorshipattribution in classic literature”, In Proceedings of theThirtieth Australasian Computer Science Conference,Association for Computing Machinery, 2007.
K.Shilova, D.Goncharenko,L.Bodnar,O.Britavska, and A. Grechkosiy, “Zipf’s laws andtranslation approaches”, Proc. of the 7th Intern,Conference “Information Technologies andManagement”, Riga, 2009, pp. 61-62.
А. Kiv, D. Goncharenko, Ye. Sedov, L. Bodnar, andN.Yaremchuk, “Mathematical study of evolution ofRussian language”, Computer Modeling & NewTechnologies, vol. 12 (1), 2008, pp. 56-59.
A. Kiv, L. Bodnar, O. Britavska, E. Sedov,N.Yaremchuk, and M. Yakovleva, “Quantitativeanalysis of translation texts”, Computer Modeling &New Technologies, vol. 18 (12C), 2014, pp. 260-263.
Analyzing and Interpreting Large Datasets. Atlanta,GA: Centers for Disease Control and Prevention(CDC), 2013.
S. Bird, “Natural Language Processing with Python”,O'Reilly Media Inc, 2009, р. 504.
R. Dale, H. Moisl, and H. Somers, “Handbook ofNatural Language Processing”, Marcel Dekker, 2000.
D. Mertz, “Text Processing in Python”, Addison-Wesley, Boston, MA, 2003.
·
This work is licensed under a
Creative Commons Attribution-ShareAlike 4.0 International License
