no. 16 · Tuesday 17 November 2026
Advancing Cognitive Pairwise Comparisons for Human-Centric Artificial Intelligence and Data Science: A Human-in-the-Loop Framework
Abstract
Human-centric artificial intelligence and data science (HCAD) systems increasingly involve human-in-the-loop (HITL) interventions to overcome the limitations of fully automated machine learning algorithms, such as automation bias and the inability to incorporate tacit domain knowledge, and continuously improve predictive performance. Data transformations, feature selection, feature weighting, and model selection usually require laborious human involvement across iterations of artificial intelligence and data science (AI&DS) pipelines, making development a challenging and time-consuming process. The Cognitive Pairwise Comparison (CPC) framework is proposed as a human-centric methodology to elicit preferences and priorities in a structured format. To establish essential facets of human-centricity, this research advances the CPC methodology so that user preference elicitation can be systematically quantified in these challenging AI&DS tasks. To demonstrate the usability and validity of these facets, this study presents several applications within movie recommender systems using the MovieLens dataset. It exemplifies how the extended CPC framework can facilitate interactions between human judgments and data across supervised learning, dimensionality reduction, and unsupervised learning algorithms.
- comments
- 33 pages, 19 tables, 8 figures; interactive web demos available at https://kkfyuen.github.io/cpcHCADDemos ; source code and datasets at https://github.com/kkfyuen/cpcHCADDemos ; archived at https://doi.org/10.5281/zenodo.23434993 ; earlier preprint version available at SSRN 4048692 (http://dx.doi.org/10.2139/ssrn.4048692 or https://ssrn.com/abstract=4048692)
- fields
- Optimization and Control