©ISTOCK
Researchers from the Higher School of Economics and KU Leuven have developed a method of measuring growth in students’ proficiency in digital learning environments. It helps to see the progress of online course participants in dynamics, i.e., to understand how students study and how the course works. The results of the study have been published in the journal Behaviour Research Methods.
The expected and desired result of any course is growth in students’ proficiency, and an increase in their knowledge and skills. This growth is represented by a positive difference between knowledge and skills at the end and the beginning of the course. For many years, growth has been measured as the difference in the results of the final and entrance tests. However, this approach, which uses two control points, does not allow the dynamics of growth within the course to be traced and understood.
The new approach suggested by the HSE and KU Leuven researchers is based on students’ digital logged data. Logged data represent events recorded by the online learning platform, such as watching video lectures and attempting to solve tasks. By using these data, the researchers can see two types of proficiency growth: continuous (throughout the course) and local (in a certain area, associated with a particular task).
Continuous growth is estimated as a cumulative effect of watching video lectures at a certain point of the online course. Local growth is calculated as the effect of an attempt to solve a specific task. As the researchers note, both effects are specific for each student. In other words, the same number of material views results in different growth for two different students.
‘Thus, we see the results for each student at any time of the course, and we don’t waste their time on voluminous entrance and final tests. Our research is a conceptual transition from traditional analysis of test results to progressive analysis of digital traces in the educational environment,’ says Dmitry Abbakumov, author of the article and Head of the HSE Centre for Psychometrics in eLearning.
These models can be used in analytical panels on online learning platforms, while algorithms based on them are appropriate for browsing and recommendation solutions in digital education.
IQ