COMPARATIVE ANALYSIS OF AUTOMATIC TEXT CLASSIFICATION METHODS IN BIBLIOGRAPHIC INFORMATION SYSTEMS

Abstract

Introduction. The rapid growth of digital technologies and the increasing intensity
of scientific communication result in a constant expansion of textual data volumes accumulated in
bibliographic information systems (BIS), electronic repositories, and scientometric databases.
Traditional cataloguing approaches based on manual or semi-automatic indexing no longer meet the
scalability and timeliness requirements of modern information environments. An experienced
cataloguer can process only 20–30 documents per day, which is wholly insufficient given the
continuous and rapid growth of scientific output. Research shows that accurate thematic classification
can improve the relevance of search results by 30–40% compared to systems relying solely on full-text
search. Despite a substantial body of work on text classification algorithms, systematic comparative
studies specifically addressing the thematic categorisation of bibliographic records — with
consideration of performance and resource requirements — remain relatively scarce in the literature.
Purpose. The aim of this article is a comparative analysis and experimental evaluation of
automatic text classification methods — from classical machine learning algorithms to neural network
architectures based on transformers — with respect to their effectiveness, performance, and practical
suitability for use in bibliographic information systems.
Results. A modular classification system for bibliographic records was developed in Python
using the scikit-learn, PyTorch, and FastAPI frameworks. Six models were evaluated using accuracy
metrics (Accuracy, Precision, Recall, F1-Score), processing speed, and resource consumption. Among
classical machine learning methods, Support Vector Machines (SVM) achieved the highest quality
(F1-Score = 0.885) while Naive Bayes demonstrated the fastest processing speed (487 samples/sec).
Among deep learning methods, BERT achieved the best classification quality (F1-Score = 0.912) but
requires substantial computational resources (18.5 hours of training, GPU, 8 GB RAM). CNN
provides a good balance between accuracy (F1-Score = 0.876) and training speed (3.2 hours). It was
established that combining title, abstract, and keywords yields 8–12% better results than using any
single field. For the morphologically rich Ukrainian language, lemmatisation using pymorphy2
provides a 2–5% accuracy improvement over stemming.
Conclusion. BERT achieves the highest classification quality (F1-Score = 0.912) but demands
significant resources. SVM provides the best balance for production real-time systems (F1-Score =
0.885, 199 samples/sec, 180 MB RAM). Naive Bayes is optimal for resource-constrained environments
and rapid prototyping. Practical recommendations for choosing a classification method according to
specific requirements for accuracy, speed, resource consumption, and interpretability are formulated.
Multilingual transformer models (XLM-RoBERTa) are recommended for international systems
supporting multiple languages due to their cross-lingual transfer capabilities.