COMPARATIVE ANALYSIS OF AUTOMATIC TEXT CLASSIFICATION METHODS IN BIBLIOGRAPHIC INFORMATION SYSTEMS - KRASNOSHLYK, Nataliya; BOHATYRENKO , Pavlo (Cherkasy University Bulletin: Applied Mathematics. Informatics, 2025)
Introduction. The rapid growth of digital technologies and the increasing intensity of scientific communication result in a constant expansion of textual data volumes accumulated in bibliographic information systems (BIS), electronic repositories, and scientometric databases. Traditional cataloguing approaches based on manual or semi-automatic indexing no longer meet the scalability and timeliness requirements of modern information environments. An experienced cataloguer can process only 20–30 documents per day, which is wholly insufficient given the continuous and rapid growth of scientific output. Research shows that accurate thematic classification can improve the relevance of search results by 30–40% compared to systems relying solely on full-text search. Despite a substantial body of work on text classification algorithms, systematic comparative studies specifically addressing the thematic categorisation of bibliographic records — with consideration of performance and resource requirements — remain relatively scarce in the literature. Purpose. The aim of this article is a comparative analysis and experimental evaluation of automatic text classification methods — from classical machine learning algorithms to neural network architectures based on transformers — with respect to their effectiveness, performance, and practical suitability for use in bibliographic information systems. Results. A modular classification system for bibliographic records was developed in Python using the scikit-learn, PyTorch, and FastAPI frameworks. Six models were evaluated using accuracy metrics (Accuracy, Precision, Recall, F1-Score), processing speed, and resource consumption. Among classical machine learning methods, Support Vector Machines (SVM) achieved the highest quality (F1-Score = 0.885) while Naive Bayes demonstrated the fastest processing speed (487 samples/sec). Among deep learning methods, BERT achieved the best classification quality (F1-Score = 0.912) but requires substantial computational resources (18.5 hours of training, GPU, 8 GB RAM). CNN provides a good balance between accuracy (F1-Score = 0.876) and training speed (3.2 hours). It was established that combining title, abstract, and keywords yields 8–12% better results than using any single field. For the morphologically rich Ukrainian language, lemmatisation using pymorphy2 provides a 2–5% accuracy improvement over stemming. Conclusion. BERT achieves the highest classification quality (F1-Score = 0.912) but demands significant resources. SVM provides the best balance for production real-time systems (F1-Score = 0.885, 199 samples/sec, 180 MB RAM). Naive Bayes is optimal for resource-constrained environments and rapid prototyping. Practical recommendations for choosing a classification method according to specific requirements for accuracy, speed, resource consumption, and interpretability are formulated. Multilingual transformer models (XLM-RoBERTa) are recommended for international systems supporting multiple languages due to their cross-lingual transfer capabilities.