eComplex: validity and reliability of rubric for assessing reasoning for complexity competency

Abstract

In the absence of available instruments that measure reasoning-for-complexity competency in higher education, we designed a rubric to measure university students' levels of mastery of that competency. This article presents the process to estimate the validity and reliability that led to the final version of the "eComplex" rubric, which consists of 27 items. This validation process considered experts' consultation, the internal consistency estimation using Cronbach's alpha coefficient, and correlation matrix analysis, there was also qualitative analysis by reviewing the experts' comments in detail and making the appropriate adjustments. The results were: (a) a high concordance index of the judges for the evaluated criteria of the items, (b) an instrument with high internal consistency and reliability, and (c) improvement in the wording of the items thanks to the observations and contributions of the experts, this was reinforced by conducting a second phase of expert validation using the Delphi Method to provide further support for the Clarity criterion. The instrument is expected to validate measuring the development and appropriation of reasoning-for-complexity competency in university students.