Evidence of Impact

MOOC Research: What Can We Do with Big Data?

Gary Matkin

University of California, Irvine

We are now in the fourth year of the “Massive Open Online Course (MOOC) era” which began in July 2012 with Stanford’s two artificial intelligence courses. While some see MOOCs as a temporary phenomenon, doomed by higher education’s resistance to change and lack of financially sustainable models, it’s become clear that MOOCs are here to stay. As MOOCs evolve, sponsors and courses alike are growing at a rapid rate. Yet, the effect MOOCs are having on research in higher education is lost in the hype and the disillusionment. While in fact, MOOCs have stimulated higher education research. Harvard and MIT jointly invested $30 million in edX by using the MOOC platform to test new teaching and learning methods. Anything new will excite research questions. And those willing to pay for the answers to the MOOC phenomenon include some unusual patrons such as Google, Yahoo, the Gates Foundation, and UNESCO.

MOOCs offer research objects that have the potential to address many of the issues higher education researchers face. They present new and unique opportunities to understand how people learn across a broad spectrum of educational mediums. MOOCs cross the boundaries between formal and informal learning in an unprecedented way, with each MOOC course offering opportunities for researchers to study how people select and engage with learning resources. The very large numbers (or n’s) of students in most MOOCS, coming from different countries and educational backgrounds, provide statistical validity that most learning research lacks due to highly restricted populations and narrowly focused research objects.

The MOOC phenomenon itself has researchers generating interesting hypotheses of practical impact; questions that seek to create theories rather than prove or disprove them. In a course with thousands of enrollments and hundreds of completers, research questions can be answered and administered across thousands of students, particularly the typical “A/B” testing. Student diversity (age, nationality, educational background) allows for the creation of cells that contain many people. The relationship between formal and informal learning can be assessed. There are institutional dynamics pushing for replication. As an example, why is our completion rate so much lower than others? Much of the MOOC experimentation and data is itself co-developed (often between institutions and foundations or MOOC providers) which means that there is a greater impulse to share experimental results. MOOC research has attracted funding and, more importantly, has created large-scale efforts to organize and share data.

One of the most interesting research questions is to determine how learning pathways created by instructors match the pathways students naturally take in learning something. Because MOOCs are directed largely at informal learners and their platforms can trace learner behavior at very granular levels, researchers can begin to gage the gap in learning contexts between instructors, students, and diverse groups of learners. Formal and informal learning can be studied side by side. Cross national studies of learners from different countries can help us understand how MOOCs can be effective in delivering low cost education and how they must be localized. So far, no serious research has been done to understand what MOOCs might reveal in these areas. What has been done is a few studies on whether MOOCs can help students (at risk, low income, marginally qualified) do better academically in targeted academic courses.

For instance, the University of California, Irvine, (UCI) counseled at-risk, entering freshman to take a “pre-biology” MOOC that was offered through Coursera. These students were given an incentive to complete the Coursera course “with distinction” to be able to transfer into biology as a major more quickly than if they did not take the course. The incentive worked; about 450 UCI students joined the 37,000 who enrolled in the MOOC. UCI found that those who competed the MOOC did significantly better than the two control groups, one that engaged in, but did not complete the Coursera MOOC, and one group that did not enroll in the MOOC. A similar experiment using a pre-calculus course offered face-to-face, online, and as a MOOC found that those who fully engaged in the MOOC course were more successful that those who did not.

In addition, university studies have found that isolated underprivileged students are less likely to complete a MOOC, yet when the do they are more likely to do so with “distinction.” At risk students who had off line help with MOOCs were more successful.

Further, MOOCs also offer the opportunity for researchers to study how systems and institutions in higher education react to potentially disruptive change. Patterns of adoption and rejection reveal where innovation actually originates, how it spreads, where, and how it persists or dies. Another related avenue for MOOC research is towards identifying the optimal learning settings. Many institutions are experimenting with using MOOC course assets in parallel with on-ground, campus-based instruction. Research with careful controls for confounding factors will allow us to investigate learning environments at a scale not previously possible in our educational system.

While higher education research can be significantly advanced through analyses of MOOCs, there are some barriers that need to be addressed. These issues include personally identifiable information (PII), ownership of data, the “messy” nature of the data, and intellectual property constraints. As long as these issues are clearly understood, these barriers have a resolution. The questions, guidelines, and ethics that we have been using were designed for a different world of data production than we face now. MOOC research will push us into this new world. Data will come before theory and theory will be simultaneously developed and modified as results are tabulated. More importantly, the effects on individual learning will immediately feed back into research agendas as theories are rapidly tested in the field.

This presentation begins with a brief summary of the categories (illustrated by examples) of MOOC research. It then examines how MOOC research is being organized around the world. This presentation will identify important questions: how are these research efforts being focused? What are they trying to learn? What impact are they having? What are they revealing about higher education? It also will explore the current state of MOOC research, summarize the approaches being taken, highlight some of the results that are coming from the research, and make predictions about what we might expect in the future.

Download PDF of Paper


MOOC Research: What Can We Do with Big Data? from Open Education Consortium