EDITOR’S NOTE (13 Nov 2012): We identified a flaw with the data below.
A while back, we noted that while just over 6,000 people had registered for the Mechanical MOOC’s Intro Python course, other data indicated the the number of engaged learners was actually probably significantly lower–just north of 450 we estimated. Having done this estimation (which is also consistent with the few published participation numbers we’ve seen from other MOOCs), we were thus prepared to not be alarmed by a significant drop-off in the number of registrants opening our second week’s e-mail as compared to the first week:
If you add the number of unique opens on Oct 15th and 16th, you get 2,367–a number likely slightly inflated because it would include at least a few unique opens of the “getting ready” e-mails sent out previously, but at least is a rough count of learners actually looking at the week one e-mail. If you add up unique opens for Oct 22nd and 23rd, you get 622, about a quarter of the unique opens that followed week one.
This is actually spot on to our estimate of the number of engaged learners we thought we had, so although you can always hope to do better than you think, at least the drop is not unexpected. And more evidence emerges that we are getting down to the truly engaged learners: If you take the total number of opens (as opposed to unique opens) for the week 1 e-mail and the week 2 e-mail (which counts each time any individual opens the mail, if a learner opens the e-mail four times to go back and look at assignments, this tallies four opens in the “total opens” calculation but only one in the “unique opens” calculation) you get some revealing ratios.
For the week one e-mail we have 6,326 opens divided by 2,367 unique opens, or 2.67 opens per unique open. For week 2, we have 2,815 opens by 622 unique opens, or 4.52 opens per unique open. This is a little apples to oranges, as unique opens may count unique opens of other e-mails we’ve sent and the total opens only counts the specific e-mail we’re addressing (unfortunately, mailgun doesn’t seem to provide unique opens on a per e-mail basis, only for date ranges). But this rough ratio indicates that the 622 users in week two are on average twice as engaged as the 2,367 in week one.
Another data point of triangulation on this issue is the size of the OpenStudy group for 6.189, which currently has 699 members. So we probably can take about 600 learners as our engaged set, a 10% yield from the 6,000 that signed up. Now the questions become:
- Is 600 big enough to create a robust community for the course? Happily, we are also backed up by the 17,000-strong 6.00 communiy on OpenStudy.
- Is the format engaging enough to carry a significant number of the 600 learners through the full eight weeks?