Development and Simulation of a Computerized Adaptive Testing Algorithm Based on an Item Bank Measuring Verbal, Spatial, and Numerical Reasoning

Abstract

The main objective of this study is the development, specification, and subsequent simulation of the algorithm for a Computerized Adaptive Test (CAT) based on an Item Bank (IB) that measures numerical reasoning (NR), verbal reasoning (VR), and spatial reasoning (SR). The IB consists of 247 questions, of which 82 assess NR, 97 assess VR, and 68 assess SR. These items were calibrated in a previous study using the Rasch model. In this study, the difficulty parameters of the items were equated, and the final IB for each type of reasoning was established. Subsequently, specification rules were developed for the test algorithm, and finally, the CAT algorithm was simulated under different stopping rules based on the standard error of estimation. We present a preliminary analysis of the algorithm's performance by simulating 1,000 response patterns. Overall, this study allows us to conclude that this IB enables the precise estimation of an ability level of 0, using a stopping criterion of a standard error of estimation ≤ 0.5. Additionally, concerning this error and in comparison, to paper-and-pencil administration, the test length could be reduced by approximately 26 to 28%. While further studies in both real and simulated conditions are needed, the present study provides evidence of the benefits of incorporating an innovative technological system in Argentina for assessing different cognitive abilities in educational settings. Keywords-- Computerized Adaptive Testing, Cognitive Abilities, Simulation.