Data Analysis for Distance Learning Courses

David Halsted and Jackie Wickham

Courses: MSGH 405; MSGH 408; MSGH 417; MSGH 427; MSGH 452

Download the PDF – Halsted and Wickham: Data Analysis for Distance Learning Courses

Context

Traditional teaching puts the instructor in the same room as the students. This means that the instructor receives a great deal of real-time feedback from the students in the room and can adjust his or her instructional approach accordingly. Online instruction requires a different mechanism for the delivery of feedback. That feedback can be provided by the collection and analysis of the data generated when students interact with online resources.

The focus of this project is to use the data generated by student interactions with the Canvas LMS system to create meaningful feedback for instructors and course designers.

Project

This project involved collecting and analyzing student-access (clickstream) data from the Canvas LMS to provide meaningful information to instructors and course designers. Commercial websites have been working with data like this for years, but academic institutions are only beginning to understand its significance and how to work with it.

The project looked in detail at data from a group of online courses offered as part of the Master’s in Global Health program during Fall 2015.

Objectives & Outcomes

This project is aimed in the first instance at instructors and course designers. The first phase in understanding data of this type is to establish a context within which new data points can be interpreted. We are at the phase now of establishing that context. What data elements about student behavior are most helpful for faculty? How can design encourage students to engage in one way or another with online resources, whether in the context of a purely online course or a traditional course with online elements? A follow-on project might build on the information gathered here to create new types of resources for students; at that point we might be able to talk about course outcomes in a more meaningful way.

The results of this project can be viewed at http://dl.sps.northwestern.edu/data-analytics/.

Results

The results of this project met with an enthusiastic reception from Northwestern University faculty and staff. In fact, we have been approached to provide data to support faculty conference presentations and papers; that success went beyond our original goals, which were at the level of course design rather than research. Therefore, we were able to show that LMS data can be captured and visualized in a way that communicates usefully with instructors, designers, and other stakeholders.

Lessons Learned

The project must be considered a success insofar as we were able to generate meaningful and valid results for instructors. It would be very interesting to make the queries both broader and deeper. Broader queries—data analysis across more programs and courses—would provide a richer context for the data points and visualizations in individual courses. Deeper queries—drilling into the data and examining differences between courses in more detail—have the potential to provide more specific guidelines for course instructors and designers.