Yes, its verbosity is skewing the results. Same with PHP, C++, and Java. We can see that Ruby is the most popular language on Github here http://github.com/languages, with 28% of all projects being in Ruby. C, PHP, Java, and C++ rank 6%, 5%, 4%, and 4% respectively.
You can potentially have a sample of programs implemented in many different languages and compare the length of each program across languages. With enough programs you should be able to get a good enough sample as long as they are diverse enough.
Also, I wonder how those would be weighted if you counted projects that consist of >90% identical code as the same project (for example, how many of those Ruby projects are thinly veiled forks of Rails?), filtered out projects that are less than (say) 10k of source, etc. Maybe more Python projects are on bitbucket because of Merucial.
The 90% and 10k there are pretty arbitrary; just, those language stats need a lot more clarification before their meaning is clear.
I wonder if you could pull all the projects into one repo, repack and figure out it's unique objects using git. I always wondered if a global object store was possible or even a good idea for a site like github. I guess that's a different project though.
FWIW, finding duplicated code (particularly in a language-independent manner!) is a surprisingly difficult problem. You need to scan not just for large copy-and-pasted blobs, but code chunks that have just had constants changed, variable names adjusted, indentation style changes, etc. I've been working on a tool for this in my spare time, and I hope to have it out sometime this summer.
The C projects could also be more complete and mature. Everyone and their mother has a 50 line Ruby twitter-client/blog/wiki implementation up on Github. I would also argue that the presence of a Linux kernel mirror, Perl mirror, Android (including the Linux kernel, again), and large projects like them skews the byte vs projects percentages a great deal.
The top languages calculation on GitHub is done by lines of code per language. By that calculation, there are 4.5x more Ruby LOC than C LOC on Github, yet C is still has the most bytes.
I'm pretty sure that lines of Ruby code tend to be shorter than lines of C. Static typing annotations tend to add to line length more directly than line count.
However, in this case, yeah, verbosity is skewing things. Also, it should be noted that github numbers, like stack overflow, are still influenced by big initial communities with strong biases (Ruby, and C#, respectively).
cloc does a rough estimation of the terseness/verbosity of the almost 80 languages it supports. It'd be pretty easy to incorporate those scaling factors for this analysis.
I know that I personally have a Clojure project that has many more lines of JavaScript than Clojure just because I have a large JavaScript library checked in with my other code.
I wonder if there's a way to figure out unique lines of code, or unique files. jQuery really shouldn't be counted 5000 times, I would think.
My project shows up on github as Shell even though it's in C just because of all the autoconf/automake/libtool files. I imagine that's why Shell places so high on the list.
For example, my 267 line configure.in file produces a 12952 line configure script. And a 6 line Makefile.am gets you a 683 line Makefile.
33 comments
[ 4.2 ms ] story [ 40.6 ms ] threadAlso, I wonder how those would be weighted if you counted projects that consist of >90% identical code as the same project (for example, how many of those Ruby projects are thinly veiled forks of Rails?), filtered out projects that are less than (say) 10k of source, etc. Maybe more Python projects are on bitbucket because of Merucial.
The 90% and 10k there are pretty arbitrary; just, those language stats need a lot more clarification before their meaning is clear.
I'd like to see some language stats broken down by activity on the project.
how do you know that?
90% of all projects are junk.
http://github.com/blog/99-popular-languages
If your project includes both Ruby and Perl, Ruby and Perl each get a +1.
(just saying)
http://langpop.com
However, in this case, yeah, verbosity is skewing things. Also, it should be noted that github numbers, like stack overflow, are still influenced by big initial communities with strong biases (Ruby, and C#, respectively).
Still, though, interesting numbers, thanks!
On a side note - is anyone here following potion's development?
I wonder if there's a way to figure out unique lines of code, or unique files. jQuery really shouldn't be counted 5000 times, I would think.
For example, my 267 line configure.in file produces a 12952 line configure script. And a 6 line Makefile.am gets you a 683 line Makefile.
er, what?