Kate LaBore, Center for Innovation in Teaching and Learning, Naj Shaik, Center for Innovation in Teaching and Learning
Fall 2014
Learning Analytics and Student Analytics
Deanna Raineri, Associate Dean for Instructional Technologies and Information Services at the College of Liberal Arts and Sciences, makes an important distinction regarding analytics in education today. Student analytics involves the analysis of data about students, which can help to identify trends and patterns for how various cohorts perform in certain classes of instruction – say, introductory STEM courses, where gender, incoming GPA, or other factors can impact student learning. Learning analytics is used to research learning experiences themselves, such as testing a particular pedagogical model, exploring the effectiveness of educational media, or examining different approaches toward teaching a particular concept.
Analytics Come to Academia
Interest in analytics has grown rapidly in the past few years, largely as a result of the growth of MOOCs (Massively Open Online Courses). The key players in MOOC platforms such as Coursera have incorporated analytics and made the data available to partners, and this has spurred interest in using such data beyond MOOCs themselves. Similar to how Google, Amazon, and other major online companies collect tremendous amounts of online behavior data, online systems such as MOOCs and LMS on campus are able to collect not only grade and performance data, but also data on every click, page view, time and area of interest on videos, and so forth. A particular student using an online learning environment for any length of time will, very soon, have generated 1000s if not tens of thousands of data points for potential analysis.
Expected Growth of Learning Analytics
We expect to see rapid growth in this area because of the availability of huge amounts of data. In the past, data was embedded in Learning Management Systems (LMS), but it was difficult to find and extract. MOOCs have changed that because they make large amounts of data available within a single MOOC, due, in part, to the sheer number of students as well as the broad demographic and geographic diversity of students within a single MOOC. Producers of MOOCs continue to make more and better data available to their educational partners. As a result, developers of LMS software have accelerated their efforts at making analytic data available from their more traditional online course environments. By capturing analytics across all courses within an LMS (not merely a single, smaller course and pool of students), these systems may one day offer equally robust and extensive information.
Recent Studies at Illinois
Research at Illinois has focused on the goal of being able to tweak course designs to optimize and improve learning outcomes and student performance. ATLAS is conducting several studies utilizing data obtained from Illinois MOOCs and LMS. All of these are ongoing, and in most cases, preliminary results are not significant. Analytics studies require time; study designs are tested and then modified to provide the most accurate interpretations of data. Among such studies are: An “instructor presence” study looking at the presence or absence of direct communication from instructors on student motivation (two sections, one with instructor presence in forums and emails to students, and the other without, were studied); an “aversion to loss” study, based on the theory that people will work harder to keep something than they will to earn it (two sections, one where students were given a certificate and had to work to keep it, and the other where they were told they had to earn the certificate); video studies testing optimal length of videos and the importance of student motivation of having the instructor’s face appear in the video. Upcoming will be studies on learning styles, and gamification (adding elements of competition and incentives to courses). In the area of student analytics, there is an ongoing study of underrepresented student populations in introductory STEM courses. The student advising office is interested in predictive analytics that could help advisors identify patterns that could enhance academic counseling to particular student cohorts.
Challenges
The biggest challenge to more and better use of analytics is the bottleneck created by a lack of skilled data analysts, survey design specialists and learning scientists. Currently, data comes to us in a very raw state. Data delivery continues to improve, but specialists will always be needed. Another challenge is the sensitivity of these studies: they can potentially show that certain practices are not very effective. We are careful to frame outcomes in such a way as to highlight the talents of our “teaching champions,” and put the blame for poor practices on those practices themselves rather than on people.
Future Prospects
We would like to see an infrastructure that would help instructors access all kinds of data on their students so they won’t have to rely so much on specialists to help them use these analytics to keep an eye on student progress and have intervention strategies. We are never going to have enough specialists, so we also need a simpler way of delivering this information directly. We are looking into different learning platforms that have better data and better tools for making sense of data.
Further Reading