How can they say they're the fastest growing when Popcorn Time nearly doubled the unique contributors (95-ish to 40ish) in nearly the same time? Seems rather disingenuous to include PopcornTime clearly showing it's much younger than CodeCombat, showing at the same age of being open-source it has nearly double the unique contributors, then claiming it's growing slower than CodeCombat. Don't get me wrong, what CodeCombat is doing is great and all the well wishes to them, but I don't think they're interpreting the data very fairly.
It's not the ideal metric, but it's one that is realistic to compute with limited time and resources. To analyze that statistic using a growth metric like "largest 30 day average contributor gain" across such a gigantic dataset, consisting of hundreds of millions of rows and tens of millions of unique repositories, requires resources and engineer time which few people possess. We certainly don't want to divert too much of either of those things away from development of CodeCombat just for a fun data science experiment!
So you narrowed the problem set down to one that was sufficiently effortless, then uploaded a blog post declaring yourself world champions. I'm not sure that makes anything better. "Fastest-growing open-source project, as far as we care" is a perfectly fine headline. What you've got now is disingenuous and baselessly self-congratulatory.
Popcorn Time is awesome, and if it continues (unlike the other Popcorn Time repository) it will eventually pass CodeCombat. However, it might also experience some event or plateau, like Pullup did, that will stop its growth.
A really interesting way to analyze the GitHub Timeline would be to rank repositories based on sustained average growth rate. We chose a fast, (sort of) cheap, and computationally easy way to run these statistics, but there are a myriad of other ways to process the same dataset, which we'd definitely be interested in seeing.
erm, and also the fact that MANY MANY open source projects are not on github? Linux, apache, android, some of the most influential projects are missing. This should be titled, "the fastest growing open source project by people who are not too important to be hosting on Github".
Actually, the Linux kernel, Android, and Apache are all on GitHub, and were analyzed (though the latter two are mirrors, so I guess not purely on GitHub.)
For projects like Linux, while it may have 3500+ contributors, it got that way over 23 years.
We also made the assumption that due to the social nature of GitHub, its meteoric growth, and the visibility that it brings open source projects, the fastest growing open source projects are on GitHub. We presume this assumption is correct, but don't have the data (or even know where we could get the data) to verify it.
I was pretty confident I knew that openstack was the fastest growing open source project. They didn't show up in the answers, so maybe the criteria was such that it was cut out?
Looking at OpenStack, it does look like an aggregate of all of the organization's repositories would be definitely the fastest growing. However, we looked at individual repositories, so that's why it's not on the graph. It would be interesting see the growth of the big open source organizations (GNU vs Apache vs OpenStack etc.)
OpenStack kinda has a single repo - every commit to the core projects triggers a submodule update in https://github.com/openstack/openstack - and there are LOTS of projects which aren't included in this. For example CI and Developer tooling etc.
> openstack$ git count -pm -n3
> 2014-03-01 2608
> 2014-02-01 2545
> 2014-01-01 2173
That looks OpenStack has 7326 commits in the previous 3 months.
> codecombat$ git count -pm -n3
> 2014-03-01 642
> 2014-02-01 426
> 2014-01-01 427
The same tool gives me 1495 commits in the previous 3 months for CodeCombat.
Replying to my own comment feels odd.. whatever ;)
If you're interested in numbers[1] shows the commits currently running through the OpenStack CI system at this very moment in time.. I'm counting 125 when I look at the page.
In the last 24 hours, OpenStack's CI system has peaked at just under 1000 jobs ran in an hour, an eyeballed average looks like to be 450 jobs every hour for the last 24 hours.
For build slaves over the same period, It looks like about 750 Jenkins slaves.
Oh and - OpenStack has a strict 1 change == 1 commit policy. So "Fix typo in previous commit" never happens .. The actually reduces the overall number of commits ;)
That's quite impressive! It is apparent our analysis of GitHub events didn't account for "super-repositories" like this, as this would be the fastest growing repository on GitHub by a large margin, dwarfing large projects like Linux or Gaia (unless Apache or Mozilla also had these super-repositories.) I'm not sure it's quite an apples to apples comparison though xD
On a somewhat related note, another interesting thing to note is that we found apparent discrepancies with GitHub's statistics for contributor counts.
I looked, and I couldn't find anything out of place with his profile. Our method for growth through unique contributors actually handles this case. We didn't have time to explore other properties of the GitHub data set which might throw our analysis off, but hopefully someone in the future will produce a nifty analysis which does account for them.
Navel gazing is the best way to convince yourself that things are going great and to give you that boost to keep working on the project. At some point you will probably look back at this and think it was nothing to brag about, but it doesn't matter because you're having fun and enjoying the feedback loop between you and your users -- this is a good sign.
18 comments
[ 2.6 ms ] story [ 30.7 ms ] threadIt's not the ideal metric, but it's one that is realistic to compute with limited time and resources. To analyze that statistic using a growth metric like "largest 30 day average contributor gain" across such a gigantic dataset, consisting of hundreds of millions of rows and tens of millions of unique repositories, requires resources and engineer time which few people possess. We certainly don't want to divert too much of either of those things away from development of CodeCombat just for a fun data science experiment!
Popcorn Time is awesome, and if it continues (unlike the other Popcorn Time repository) it will eventually pass CodeCombat. However, it might also experience some event or plateau, like Pullup did, that will stop its growth.
A really interesting way to analyze the GitHub Timeline would be to rank repositories based on sustained average growth rate. We chose a fast, (sort of) cheap, and computationally easy way to run these statistics, but there are a myriad of other ways to process the same dataset, which we'd definitely be interested in seeing.
For projects like Linux, while it may have 3500+ contributors, it got that way over 23 years.
We also made the assumption that due to the social nature of GitHub, its meteoric growth, and the visibility that it brings open source projects, the fastest growing open source projects are on GitHub. We presume this assumption is correct, but don't have the data (or even know where we could get the data) to verify it.
> openstack$ git count -pm -n3
> 2014-03-01 2608
> 2014-02-01 2545
> 2014-01-01 2173
That looks OpenStack has 7326 commits in the previous 3 months.
> codecombat$ git count -pm -n3
> 2014-03-01 642
> 2014-02-01 426
> 2014-01-01 427
The same tool gives me 1495 commits in the previous 3 months for CodeCombat.
(The tool I used is https://github.com/moskytw/git-count )
If you're interested in numbers[1] shows the commits currently running through the OpenStack CI system at this very moment in time.. I'm counting 125 when I look at the page.
In the last 24 hours, OpenStack's CI system has peaked at just under 1000 jobs ran in an hour, an eyeballed average looks like to be 450 jobs every hour for the last 24 hours.
For build slaves over the same period, It looks like about 750 Jenkins slaves.
Oh and - OpenStack has a strict 1 change == 1 commit policy. So "Fix typo in previous commit" never happens .. The actually reduces the overall number of commits ;)
[1]: http://status.openstack.org/zuul/
On a somewhat related note, another interesting thing to note is that we found apparent discrepancies with GitHub's statistics for contributor counts.
For instance, take a look at this PR: https://github.com/codecombat/codecombat/pull/372
It was clearly merged. Now take a look at the two possible usernames GitHub could have used to tally his contribution: QelioX (his GitHub username) and Esh2349, the username he used on the commit. https://github.com/codecombat/codecombat/commits?author=Esh2... https://github.com/codecombat/codecombat/commits?author=Qeli...
I looked, and I couldn't find anything out of place with his profile. Our method for growth through unique contributors actually handles this case. We didn't have time to explore other properties of the GitHub data set which might throw our analysis off, but hopefully someone in the future will produce a nifty analysis which does account for them.
You have 50 more days to build a working parser and help others learn to code in new languages playing CodeCombat