Validation of a matrix reasoning task for mobile devices

Abstract

Many cognitive tasks have been adapted for tablet-based testing, but tests to assess nonverbal reasoning ability, as measured by matrix-type problems that are suited to repeated testing, have yet to be adapted for and validated on mobile platforms. Drawing on previous research, we developed the University of California Matrix Reasoning Task (UCMRT)-a short, user-friendly measure of abstract problem solving with three alternate forms that works on tablets and other mobile devices and that is targeted at a high-ability population frequently used in the literature (i.e., college students). To test the psychometric properties of UCMRT, a large sample of healthy young adults completed parallel forms of the test, and a subsample also completed Raven's Advanced Progressive Matrices and a math test; furthermore, we collected college records of academic ability and achievement. These data show that UCMRT is reliable and has adequate convergent and external validity. UCMRT is self-administrable, freely available for researchers, facilitates repeated testing of fluid intelligence, and resolves numerous limitations of existing matrix tests.

Keywords: Fluid intelligence; Matrix problems; Mobile; Reasoning; UCMRT; Validity.

Figures

Figure 1:

Example of a logic problem generated by the Sandia Software. The 3×3 matrix on the left represents the problem with the lower right entry missing, and the 8 answer options are presented on the right.

Figure 2:

Examples of UCMRT problems using the same structure as Sandia matrices, apart from the answer options which are presented vertically. All types of problems are shown in the practice section: (1) 1-relation problem, (2) 2-relation problem, (3) logic, (4) 3-relation with one transformation, (5) 3-relation with two transformations, and (6) 3-relation with three transformations.

Figure 3:

Mean Accuracy based on problem type for A, B and C versions in participants who correctly solved 2-relation problems. 3-REL-1 = 3-relation with 1 transformation, 3-REL-2 = 3relation with 2 transformations, 3-REL-3 = 3-relation with 3 transformations.

Figure 4:

Average number of correctly solved problems on alternate versions of UCMRT at two time points. Each participant completed 2 out of 3 alternate versions. Error bars = SEM.

Figure 5:

Scatter plots illustrating the correlation between UCMRT and APM accuracy for the three groups of participants that solved A, B and C versions of UCMRT.

Figure 6:

Mean accuracy of UCMRT and APM in the three groups of participants that solved the alternate versions of UCMRT. Error bars = SEM.