On the inconsistency of pairwise comparisons: an experimental study

Abstract

A problem of the inconsistency of pairwise comparisons is of focal interest in the analytic hierarchy process (AHP), but, up to date, we know only little about how much are real decision makers inconsistent, and whether the number of objects to be compared influences the inconsistency of their judgments. Therefore, the aim of this paper is to experimentally assess how the inconsistency of pairwise comparisons in the AHP framework changes when the number of objects to be compared (alternatives and/or criteria) increases. In our study, the method of a blind experiment was selected: subjects of the study, who were familiar with the AHP, were instructed to pairwise compare from 3 to 7 objects not knowing the true objective of the study. The main result obtained via ANOVA method is that the consistency ratio was not affected by the increasing number of compared objects, the result that might be likely attributed to the apparent redundancy of pairwise comparisons in the AHP which “corrects” inconsistent judgments. Also, it was found that only 3% of pairwise comparison matrices provided by decision makers were fully consistent, while for 36% of pairwise comparison matrices the consistency ratio CR exceeded the threshold of 0.10.