ECSW: An Integrated Model for Expert-Based Criteria Evaluation, Selection, and Weighting in Multi-Criteria Decision-Making

Abstract

This paper proposes an integrated Expert Criteria Selection and Weighting (ECSW) model for expert-based criteria evaluation, selection, and weighting in multi-criteria decision-making (MCDM). The model is designed to preserve the expert-driven nature of the decision process while reducing the cognitive burden and potential inconsistency associated with pairwise comparisons and criteria ranking. It integrates four components: statistical criteria selection, assessment of the informativeness of expert ratings, determination of criteria weights based on their relative importance and rating stability, and sensitivity analysis. Candidate criteria with below-average ratings are evaluated by testing whether their expert-specific deviations from the experts’ average rating patterns are significantly below zero, using a one-sided one-sample t-test followed by the Holm–Bonferroni correction for multiple comparisons. The final weights are obtained by adjusting equal base weights according to the relative importance of each criterion and the dispersion of expert ratings. Model stability is examined using a leave-one-expert-out procedure and Pearson and Spearman correlation coefficients. The weighting procedure was preliminarily verified against results obtained using the Ordinal Priority Approach. A strong agreement was observed for both criteria weights (r = 0.956) and rankings (ρ = 0.929), with p < 0.001 in both cases. The two models also identified the same ten highest-ranked criteria, although some differences occurred in their ordering. The ECSW model is particularly suitable for decision problems involving a large number of criteria for which conventional pairwise-comparison or ranking-based procedures may be difficult to apply. A supplementary ECSW implementation tool is provided to facilitate practical application and reproducibility.