Professors have access to the classes' data to learn how their students learn. We will facilitate experiments. For example, we intend this to be the best plaform for running A/B/N tests to measure the impact of different teaching methods on student outcomes
That they're giving data driven decision making such a prominent place is great. I would like to know more, though.
Having taught before at the University level I know it's a ton of work. It's even more work if you're developing parallel versions of a course for testing purposes. So the A/B feature had better be really easy to use.
There is also the ethical issue of giving students a worse performing variant of the class. The solution here is to use a bandit algorithm or an early stopping method to find the best variant as quickly as possible.
Finally, I think the Coursera model where everyone runs through the course on the same schedule is not optimal for experimentation. It doesn't allow for a tight feedback loop, because you have to wait N weeks before you can experiment with changes on the next cohort.
I thought this was pretty annoying, actually. I was used to Coursera already, but had to create new logins, etc. Then I was confused because Coursera listed (and still lists) the DB course as coming soon at the same time leading up to the Class2Go version. :-( I'm slightly disappointed by the muddled way it was carried out.
6 comments
[ 3.5 ms ] story [ 20.7 ms ] threadProfessors have access to the classes' data to learn how their students learn. We will facilitate experiments. For example, we intend this to be the best plaform for running A/B/N tests to measure the impact of different teaching methods on student outcomes
That they're giving data driven decision making such a prominent place is great. I would like to know more, though.
Having taught before at the University level I know it's a ton of work. It's even more work if you're developing parallel versions of a course for testing purposes. So the A/B feature had better be really easy to use.
There is also the ethical issue of giving students a worse performing variant of the class. The solution here is to use a bandit algorithm or an early stopping method to find the best variant as quickly as possible.
Finally, I think the Coursera model where everyone runs through the course on the same schedule is not optimal for experimentation. It doesn't allow for a tight feedback loop, because you have to wait N weeks before you can experiment with changes on the next cohort.
[1] http://news.ycombinator.com/item?id=4880112
Also of interest is the urls.py which shows you all the different aspects of the web application. https://github.com/Stanford-Online/class2go/blob/master/main...
Good stuff.
It would be nice if you could separate out the ops infrastructure from the actual application, to suit a wider variety of deployment scenarios.