Ugh, should've put a [pdf] warning in the title. Other than that, R can indeed be hot, but maybe just in the special 'flavor' Revolution Analytics developed, otherwise it's one of those languages you dread using but are forced to because so much scientific code/packages etc has been written on it, some with no equivalent in other languages.
"How Did a Statistical Programming Language Invented in New Zealand Become a Global Sensation?". Hmm, slightly condescending title—though I had no idea that it was invented by two academics at the University of Auckland (not "Aukland"!)
R has some spectacular libraries and a huge userbase among statisticians, so it is definitely a big actor. On the downside, the core team who manage it appear to be somewhat averse to making the language progress in meaningful ways. R is very slow, has difficulty handling large data, and is a source of many "wtf" moments (e.g. see "the R inferno"), even for advanced users.
I think the best thing that could happen would be a mass migration of R users towards Clojure, Julia or (more realistically) Python, and an attempt to reproduce in that language the key packages that are R's main appeal at the moment (ggplot2 mostly).
what advantages does this offer in comparison to matplotlib?
Are the points they describe (easy faceting, bulk application of aesthetic and visual parameters across categorical variables, pleasing default color palettes for categorical data, etc.) really that much of a pain point with matplotlib?
I used it for two or three projects but not really extensively and especially not for complex statistical plotting, thats why I ask.
The core goals of the project are to offer: interactive, easy-to-specify, powerful novel graphics, with good support for statistical plots, in a web browser. That's quite a handful of things to balance, but we are picking up the best ideas from already existing projects: matplotlib, chaco, ggplot, protovis, stencil. The project is still nacent but we have promising results so far. (For instance, here is an ipython notebook export with interactive ggplot-style faceted plots of a Pandas DataFrame: http://htmlpreview.github.com/?https://github.com/ContinuumI...)
This kind of composite plot is a one-liner in the grammar of graphics. It would be much more with Matplotlib or Chaco. There are also marvelous novel graphics that are possible with Protovis-style Marks & Glyphs: http://mbostock.github.com/protovis/ex/ Although these are technically possible with existing libraries, the amount of programming skill required to create them is outside the capability of most data analysts. Our goal with Bokeh is to make these all accessible to that larger audience.
To add to pwang's comment, to me the problem with matplotlib is that it makes an assumption that I believe to be untrue: the majority of data analysis tasks only need a pre-set list of plots. In my experience, most data analysis tasks need some novel combination of existing components, that may not have a specific name (e.g. pie chart, bar chart).
Agreed. Ideally, a new language/api will emerge that learns from the successes and failures of R. Something that is designed to be used as it is used presently, that incorporates Hadley Wickham's ideas, that has a less quirky OOP system, and that is more readable.
Perhaps currying/function compositions could be encouraged to avoid(hard, to(read(nested, functions))). However, being unexperienced with currying/function compositions, I have no idea if that would complicate the language...
I think the best thing that could happen would be the development of a competing R implementation. In my mind this would include the definition of a minimal R language implemented in something like Riposte (http://www.justintalbot.com/wp-content/uploads/2012/10/pact0...), and the majority of base code written either in R, or the sort of C++ that Rcpp makes very natural.
The best thing that can happen to R would be to rewrite the parser in Haskell and make R a domain specific language. Leave the stats/R language to statisticians who originally designed R, and bring in the Comp Sci crowd to actually make a meaningful speedup.
That's an unusual suggestion - I haven't heard anyone complain about R's parser before. It's also hard to argue that R is a DSL (at least in the spirit of http://en.wikipedia.org/wiki/Domain-specific_language). There are many features of R, features that make it particularly well suited to statistics and data analysis, that would be very difficult to reproduce in an embedded DSL (missing values are one particularly important example)
Interesting. First time I have heard of Riposte. It dousn't sound like a new language though, but rather a new runtime. Am I understanding incorrectly?
The biggest problem R faces is the corrosive attitude the language developers and maintainers have towards the R community. The R core team sometimes seems openly hostile to users; instead of Benevolent-Dictator-for-Life, it's more of a military junta that suppresses the opposition.
So, let's say you're interested in playing with R, where's the best place to start? I've got basic statistics background and some datasets I could play with.
If you are looking for a book, "The Art of R Programming" is (by far) my favorite R book.
It explains a lot of the general principles behind the language, so you can figure out details for yourself (in comparison to, for instance, the R Cookbook, which has hundreds more pages of details and code samples for specific use cases).
I like R, it's more fun to write in than, say, Matlab. But it is unbelievably slow and has very little support for web-based projects. I'm reading "Python for Data Analysis" right now, and I'm pretty sure the book is converting me. Pandas' DataFrame includes R's data.frame's functionality as a strict subset, and that's enough for me to switch. I still might need to use rpy2 for awhile for certain packages though.
I started using data.table [1] as a replacement for data.frame recently and it's hugely improved my productivity in R. On the web front, the Shiny [2] project has a emerged as a possibility for quickly building web frontends for analysis projects in R.
I think R is just as terrible as Matlab. Personally I prefer Mathematica, but I even stopped using that, it's just so painful to use with everything being lists.
Could you elaborate your dislike for mathematica and matlab?
In my university departement a lot of people are using matlab while I somehow prefer C/C++ as I still cannot get my head around it.
My use of Mathematica is usually reading some data (generated from a simulation, say) from a CSV file and then doing something with it and plotting it.
What I dislike is that simple things like aggregation, slicing and dicing the data is kind of pain when everything is a list and using Table[], Map[] etc. It often occurs to me that it'd be much quicker to express what I want with an SQL query. Also, with Mathematica's lists header information is lost, so I always have to remember which column in the list of lists is what. It's kind of a pain. Basically, whereas in SQL I can tell it what I want (declarative), in Mathematica I actually have to implement it (functional).
Then, in the plotting phase I'm constantly looking up Mathematica's plot options because I don't use it often enough to remember them. By default almost everything looks like crap in Mathematica, and you have to put in 10+ options to try to make it look good, but even so I'm usually dissatisfied with the result. Often I want something and then I find someone on a message board saying that it doesn't support that, and then he shares his 100 line program which gets Mathematica to do that. Then I usually give up. Getting programs from the Internet to work with Mathematica is a pain because Wolfram pushes out a new version every year and they keep changing small things which break backwards compatibility. Compare to the (much less powerful) Google Charts library, where everything looks much better and easier to comprehend by default.
One of my startup ideas is to write something like Mathematica, much simpler, runs on the web as a service, you can program it in Javascript, has embedded SQL, and it uses something like Google Chart for simple plotting, or you can pull in your favorite plotting library. Let's you share your work by sending around a link.
PS: I played around with R, but it seemed even more cryptic than Mathematica. Admittedly I didn't spend much time in it.
Actually I'm developing/maintaining something like this at my job, as part of the Data Services team. Similar to this, it uses Google Charts and is a node.js app. It will be open-sources in a couple of months. The goal is to avoid using Gooddata for as much stuff as possible, GD is too sluggish.
Matlab as a language is awful and here are a few examples why:
- You can't chain indexing with function calls. So you can't do myfun()[1]. This gets obnoxious really quickly. I hope you like useless intermediate variables.
- You can't have a N-D array with singleton trailing dimension. So if you want dynamically created array sizes you will have to create special cases for that.
- One external function per file.
Expect R is dominate as the SAS legacy begins to die. R is spreading across a lot on analytics departments in Universities, from Stats to Finance. The holdouts are the CS departments, who have rebranded much of statistics into data mining/machine learning and, smartly, use python.
Another important metric is the use of R in clinical studies. The FDA now accepts much research in R whereas before they did not. That is a slow moving change, and shows R will be around, at least in use by Pharma, for a while.
Python has some nice tools to interface with R when needed (rpy2 for example). On top of that, scipy/numpy/matplotlib provides a lot of scientific routines.
I dunno if R is that hot, but I think RStudio's Shiny (http://www.rstudio.com/shiny/) is on its way to making it hot. Best thing to come out in the R world in a long time. Besides being a great visualization tool for people who already use R, it might be something that makes programming easier to learn for kiddos (they can even learn some HTML + JS at the same time...) This almost brings R to a level where all the material taught in a freshman COMP101 class could be implemented in R (... as long as you don't even mention OOP)
55 comments
[ 5.0 ms ] story [ 114 ms ] threadhttp://en.wikipedia.org/wiki/Ernest_Rutherford
Are the points they describe (easy faceting, bulk application of aesthetic and visual parameters across categorical variables, pleasing default color palettes for categorical data, etc.) really that much of a pain point with matplotlib?
I used it for two or three projects but not really extensively and especially not for complex statistical plotting, thats why I ask.
"Although it is a Python library, its primary output backend is HTML5 Canvas."
The core goals of the project are to offer: interactive, easy-to-specify, powerful novel graphics, with good support for statistical plots, in a web browser. That's quite a handful of things to balance, but we are picking up the best ideas from already existing projects: matplotlib, chaco, ggplot, protovis, stencil. The project is still nacent but we have promising results so far. (For instance, here is an ipython notebook export with interactive ggplot-style faceted plots of a Pandas DataFrame: http://htmlpreview.github.com/?https://github.com/ContinuumI...)
This kind of composite plot is a one-liner in the grammar of graphics. It would be much more with Matplotlib or Chaco. There are also marvelous novel graphics that are possible with Protovis-style Marks & Glyphs: http://mbostock.github.com/protovis/ex/ Although these are technically possible with existing libraries, the amount of programming skill required to create them is outside the capability of most data analysts. Our goal with Bokeh is to make these all accessible to that larger audience.
Abstract: If you are using R and you think you're in hell, this is a map for you.
Perhaps currying/function compositions could be encouraged to avoid(hard, to(read(nested, functions))). However, being unexperienced with currying/function compositions, I have no idea if that would complicate the language...
See: http://docs.julialang.org/en/latest/packages/packagelist/?hi... and: http://docs.julialang.org/en/latest/packages/packagelist/?hi...
http://www.revolutionanalytics.com/why-revolution-r/whitepap...
Codeschool recently launched a free tutorial class for R thats interactive. They do a great job with intro level learning.
go to the R website - http://cran.r-project.org/
http://gallery.r-enthusiasts.com/
Examples of graphics with code, and some are very smart
It explains a lot of the general principles behind the language, so you can figure out details for yourself (in comparison to, for instance, the R Cookbook, which has hundreds more pages of details and code samples for specific use cases).
1. http://datatable.r-forge.r-project.org/
2. http://www.rstudio.com/shiny/
Then, in the plotting phase I'm constantly looking up Mathematica's plot options because I don't use it often enough to remember them. By default almost everything looks like crap in Mathematica, and you have to put in 10+ options to try to make it look good, but even so I'm usually dissatisfied with the result. Often I want something and then I find someone on a message board saying that it doesn't support that, and then he shares his 100 line program which gets Mathematica to do that. Then I usually give up. Getting programs from the Internet to work with Mathematica is a pain because Wolfram pushes out a new version every year and they keep changing small things which break backwards compatibility. Compare to the (much less powerful) Google Charts library, where everything looks much better and easier to comprehend by default.
One of my startup ideas is to write something like Mathematica, much simpler, runs on the web as a service, you can program it in Javascript, has embedded SQL, and it uses something like Google Chart for simple plotting, or you can pull in your favorite plotting library. Let's you share your work by sending around a link.
PS: I played around with R, but it seemed even more cryptic than Mathematica. Admittedly I didn't spend much time in it.
http://afterquery.appspot.com/help
Actually I'm developing/maintaining something like this at my job, as part of the Data Services team. Similar to this, it uses Google Charts and is a node.js app. It will be open-sources in a couple of months. The goal is to avoid using Gooddata for as much stuff as possible, GD is too sluggish.
- You can't chain indexing with function calls. So you can't do myfun()[1]. This gets obnoxious really quickly. I hope you like useless intermediate variables. - You can't have a N-D array with singleton trailing dimension. So if you want dynamically created array sizes you will have to create special cases for that. - One external function per file.