Metaheuristic Hyperparameter Optimization for AdaBoost-Based Insider Threat Detection: A Comparative Study on CERT Logon Activity

Abstract

Insider threats are hard to detect because malicious actions are rare and closely resemble ordinary activity. This study asks how much of the achievable detection performance comes from tuning a classifier's hyperparameters, and how much depends on which optimizer performs the tuning. AdaBoost is applied to logon activity from the publicly available synthetic Computer Emergency Response Team (CERT) Insider Threat Test Dataset. The majority class was down-sampled to a 9:1 ratio, giving 1,980 observations split 70:30, and nine metaheuristics were compared under an identical budget of 1,020 objective evaluations across 30 independent runs, with hyperparameters selected by three-fold cross-validation on the training partition only and the held-out partition used once for final evaluation. Tuning is decisive: AdaBoost with default parameters reaches a Cohen's kappa of 0.281 and recovers 14 of 59 insider events, whereas tuned configurations reach approximately 0.59. The choice of optimizer matters far less. Only 0.0151 of Cohen's kappa separates the strongest from the weakest of the nine, and although the differences are statistically detectable, the ranking is largely stable across budgets. A hybrid Crayfish Optimization Algorithm variant combining quasi-reflective learning with a firefly-inspired step is also evaluated; it improves on the original algorithm but does not reach the leading methods. Analysis of false negative rates and precision-recall behaviour shows that a model of this kind is best deployed as one risk signal within a wider