APPLICATION OF NEURAL-ASSISTED DECISION TREES IN ARTIFICIAL INTELLIGENCE PROBLEMS

Abstract

Introduction.
The article investigates hybrid machine learning models that combine decision trees with neural
networks to simultaneously achieve high prediction accuracy and interpretability of decisions. The
modern architectures of the following models are analyzed: Neural Decision Trees (NDT),
Differentiable Decision Trees, Neural Oblivious Decision Trees (NODE), TabNet, and Neural-Backed
Decision Trees (NBDT). The practical implementation includes building a hybrid model on the Iris
and Nobel laureate datasets using SHAP analysis to interpret the results, which confirms the
effectiveness and practical applicability of the described approach.
Modern machine learning faces the fundamental challenge of balancing predictive accuracy
with model interpretability. Classical decision trees offer transparent, rule-based reasoning but are
limited in capturing complex nonlinear patterns. Neural networks achieve state-of-the-art accuracy but
function as opaque "black boxes." Hybrid models that combine the strengths of both approaches
represent a promising research direction, particularly for safety-critical applications where algorithmic
decisions must be explainable.
Purpose. The aim of this article is to investigate the theoretical foundations and modern
architectures of hybrid models combining decision trees with neural networks, to practically
implement such models on classification tasks, and to evaluate the interpretability of obtained
decisions using SHAP analysis.
Results. The paper systematizes and compares five classes of modern hybrid architectures:
Neural Decision Trees (NDT), Differentiable Decision Trees, Neural Oblivious Decision Trees
(NODE), TabNet, and Neural-Backed Decision Trees (NBDT). The components of hybrid decision
trees are examined in detail: soft differentiable branching, neural networks as split functions, attention
mechanisms, and ensemble approaches. Training algorithms – gradient descent, regularization types
(parametric, structural, stochastic, and entropy-based), pruning, and hyperparameter optimization – are
described. The NBDT library achieves 97.55% on CIFAR-10, 82.97% on CIFAR-100, and 76.60% on
ImageNet, surpassing previous hybrid methods by 3.23%, 6.73%, and 15.31% respectively. A twostage hybrid model is implemented, where the probabilistic outputs of an MLP are used as additional
features for a decision tree. The hybrid model achieves 100% accuracy on the Iris dataset,
outperforming both baseline models. SHAP analysis confirms that neural probability features play a
decisive role in uncertain nodes, while original features dominate in straightforward cases. Application
to the Nobel laureates dataset further validates the approach on real social data.
Conclusion. Hybrid models combining decision trees with neural network support effectively
resolve the accuracy–interpretability trade-off. The proposed neural support mechanism enriches the
feature space of a decision tree without sacrificing its structural interpretability, as confirmed through
experiments on two different datasets and quantitative SHAP-based explanations.