One of the most vexing problems facing those offering MOOCs today is completion rates. Sure, the observation goes, lots of people sign up, but a very small percentage actually complete the courses. A small percent of a very large number still turns out to be a large number in and of itself, but what about the others, the non-completers? Are MOOCs discouraging them, turning them off to learning? One of our goals in offering this class is to understand exactly this issue.
So, a few early numbers: We have just over 5,500 learners who’ve signed up for the Mechanical MOOC mailing list. We’ve been sending e-mails out on the list for about three weeks and have delivered a total of just under 37,000 e-mails to participants. According to the mailing list analytics, just over 3,000 individuals have opened at least one of the e-mails sent to them (this does not count those with HTML disabled in their e-mail client), and about 2,000 have clicked on at least one link. 133 individuals have unsubscribed from the mailing list since we started e-mailing. And since we pointed to the 6.189 study group on OpenStudy, the participation has grown from 25 members to 455 members.
So in this, we have at least some clues as to the number of learners that have signed up and are at least somewhat committed to taking the course. If you look at each step as an increasing level of commitment–remaining on the list, opening an e-mail, clicking on a link, registering for the study group–you begin to get a sense of how many committed learners there are signed up for the course. Likely somewhere between 2,000 (those clicking links) and 450 (those signing up for the OpenStudy group), but probably closer to the smaller number.
One thing this might point to is that we’re using a flawed formula for “completion rate” in talking about MOOCs. When a student signs up for a traditional course, they have already made very significant commitments to the educational process–they’ve applied and been accepted, often moved to a new town or city, paid tuition, committed to a degree program–in other words, there are lots of background reasons why they are committed to completing the course they register for.
A concept we’d like to borrow from the college admissions world is “yield,”–that is the % of admitted students who actually end up enrolling at the college. About.com suggests for top colleges, this number can be as high as 80%, and for less competitive schools as low as 25%. The above statistics would indicate the “yield” for the Mechanical MOOC’s Gentle Introduction to Python (committed learners/subscribers) to be at about 9%.
This would indicate that a lot of the other subscribers are curiosity subscriptions (educators interested in how the course will work, people confused about what they are signing up for, etc.). We’ll continue monitoring these numbers as the course progresses, but right now, we’ll keep a dual definition of completion rate in mind, both completions/subscriptions and completions/yield. For many of the MOOCs for which stats have been released, completion rates have been in the single digits. If they have effective yields similar to the above, then those single digits start to look healthier calculated as completions/yield.
A more relevant question is do we have enough learners to offer an effective course? At the moment, if you visit the OpenStudy group for 6.189, there are usually about a dozen learners online at any one time, and questions are already being answered pretty quickly after being asked, so from that perspective, it appears to be a big enough group. We’re planning to cluster learners on e-mail lists in groups of 12, however, and if each group has a 10% yield, that will be a problem. Something for us to think about.
More data to come…