How do we count thee, oh students of Python? #mmooc

EDITOR’S NOTE (13 Nov 2012):  We identified a flaw with the data below.

A while back, we noted that while just over 6,000 people had registered for the Mechanical MOOC’s Intro Python course, other data indicated the the number of engaged learners was actually probably significantly lower–just north of 450 we estimated. Having done this estimation (which is also consistent with the few published participation numbers we’ve seen from other MOOCs), we were thus prepared to not be alarmed by a significant drop-off in the number of registrants opening our second week’s e-mail as compared to the first week:

Graph of unique opens and clicks

If you add the number of unique opens on Oct 15th and 16th, you get 2,367–a number likely slightly inflated because it would include at least a few unique opens of the “getting ready” e-mails sent out previously, but at least is a rough count of learners actually looking at the week one e-mail.  If you add up unique opens for Oct 22nd and 23rd, you get 622, about a quarter of the unique opens that followed week one.

This is actually spot on to our estimate of the number of engaged learners we thought we had, so although you can always hope to do better than you think, at least the drop is not unexpected.  And more evidence emerges that we are getting down to the truly engaged learners:  If you take the total number of opens (as opposed to unique opens) for the week 1 e-mail and the week 2 e-mail (which counts each time any individual opens the mail, if a learner opens the e-mail four times to go back and look at assignments, this tallies four opens in the “total opens” calculation but only one in the “unique opens” calculation) you get some revealing ratios.

For the week one e-mail we have 6,326 opens divided by 2,367 unique opens, or 2.67 opens per unique open.  For week 2, we have 2,815 opens by 622 unique opens, or 4.52 opens per unique open.  This is a little apples to oranges, as unique opens may count unique opens of other e-mails we’ve sent and the total opens only counts the specific e-mail we’re addressing (unfortunately, mailgun doesn’t seem to provide unique opens on a per e-mail basis, only for date ranges).  But this rough ratio indicates that the 622 users in week two are on average twice as engaged as the 2,367 in week one.

Another data point of triangulation on this issue is the size of the OpenStudy group for 6.189, which currently has 699 members.  So we probably can take about 600 learners as our engaged set, a 10% yield from the 6,000 that signed up.  Now the questions become:

  • Is 600 big enough to create a robust community for the course?  Happily, we are also backed up by the 17,000-strong 6.00 communiy on OpenStudy.
  • Is the format engaging enough to carry a significant number of the 600 learners through the full eight weeks?

Stay tuned….

Advertisements

About MOOC-E

The Mechanical MOOC, international man (or machine) of mystery. (Photo credit: Tinkerbots http://bit.ly/P029IR)
This entry was posted in Analysis and tagged . Bookmark the permalink.

6 Responses to How do we count thee, oh students of Python? #mmooc

  1. Pingback: MOOC mécanique, MOOC connectiviste « Tritypa

  2. Adam Stiles says:

    You forgot to count me! I would have opened the email, but still haven’t received one even though I’m signed up. I wonder how many me’s there are?

  3. MOOC-E says:

    Hey Adam,

    When did you sign up. If it was after around October 20th, you are queued up for the e-mail cycle starting at the end of November. It’ll follow much the same sequence as the October 15th group (https://mechanicalmooc.wordpress.com/category/seq15oct12/) so you can start working ahead. If you signed up before October 20th, e-mail mooc-e@p2pu.org and we’ll check our system. Also, take a peek at your spam filter if you haven’t already.

  4. josh2w says:

    I also wonder how many others like me and Adam didn’t receive the emails?
    I signed up October 15 and only received one email.

    From: The Machine

    Sent: Monday, October 15, 2012 7:15 PM
    Subject: Hello

    Thanks for signing up

  5. MOOC-E says:

    Josh– we are having a few issues around the cutoff for the second round. If you are in for the second round, you’ll begin receiving “getting ready” e-mails soon, but please do e-mail mooc-e@p2pu.org if you’d like to be included in the first round so we know which e-mail to look for.

  6. Pingback: The (mis)measure of MOOC #mmooc | The Mechanical MOOC – A Gentle Introduction to Python

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s