Multi-class probabilistic classification approach using Gaussian process and support vector machine

Abstract

In recent years, Support Vector Machines (SVMs) have demonstrated strong performance in efficiently and accurately handling large-scale classification tasks, often outperforming many existing classification techniques. Meanwhile, Gaussian Process Classification (GPC) offers the distinct advantage of generating probabilistic outputs, providing insights into prediction uncertainty—though this comes at a significant computational cost. This thesis introduces two novel algorithms tailored for multi-class probabilistic classification. These methods integrate the strengths of SVM and GPC into a unified framework, enabling the classification of large datasets in a relatively reduced timeframe while also delivering probabilistic predictions for each class. Such probabilistic outputs are valuable for deriving confidence measures and assessing uncertainty in classification decisions. The effectiveness of the proposed models is evaluated using six diverse classification datasets, ranging from 1,197 to 5,811 data points and 3 to 27 predictors. Each dataset features 3 to 7 categorical outcome classes across health, environmental, and industrial domains. The datasets are sourced from various platforms, such as UCI Machine Learning Repository and Kaggle. One dataset is synthetically generated using Python. Comparative analyses show that these new algorithms achieve substantial computational efficiency, particularly during the prediction phase on unseen data, and often outperform existing state-of-the-art methods. In terms of accuracy, the proposed approaches yield consistently competitive results, sometimes surpassing current leading algorithm’s performance.