This text is sort of out of context. Do you have a link with surrounding emails?
To provide a little more context from what I know myself, Doug Bates is a long-time R contributor and package maintainer, but started switching to Julia for the reasons he gives here, among others, in mid 2012. He's a great hacker, not to mention brilliant statistician, and we're extremely lucky to have him in the Julia community.
The prompting email is: "What it the proper citation for the lme4 package and the Bates' book?
Also, can lme4 datasets (e.g., Pastes, ScotsSec, InstEval etc.) be
used for illustration in publications? Can the authors grant
permission or is the permission from the source needed?
Many thanks for the package and the book. When can I hold a
non-digital copy in my hands?
Thanks,
Russ"
Sadly, I don't think that adds much to the context. But yeah, Julia benefits greatly from this. It's still not at the point where I'd move (I'm more of an end-user than a developer) but it's got my curiosity.
"> Maintaining an Rcpp-based package on CRAN these days is a case of "no good deed shall go unpunished" and "the flogging will continue until morale improves". I am the maintainer of the RcppEigen package which apparently also makes me the maintainer of an Eigen port to Solaris. When compilers on Solaris report errors in code from Eigen I am supposed to fix them. This is difficult in that I don't have access to any computers running Solaris, which is a proprietary operating system as far as I can tell, and Eigen is a complex code base using what is called "template meta-programming" in C++. Making modifications to such code can be difficult. I can't claim to fully understand all the details in Eigen and in Rcpp. I am a user of these code bases, not a developer. The Eigen authors themselves don't test their code under Solaris because they don't have access to Solaris systems either and they don't regard Solaris as an important platform for numerical computing. The CRAN maintainers feel differently, which puts me in a box.
It's not just packages that use Rcpp that suffer from this problem. lubridate, which has extensive user tests, often fails on solaris because it does something different to every other platform. While this has uncovered a number of bugs and limitations in R's datetime support, the only feedback we get from CRAN is extremely negative.
For lubridate, in the absence of any help from a solaris expert (and no evidence that anyone on solaris uses lubridate), we have simply told CRAN that it does not work on solaris. We continue to argue that the CRAN policies only require that an R package need only work on two major platforms to be distributed via CRAN, and while the CRAN
maintainers continue to push back at us, they are bound by their own words.
(While a somewhat biased sample, we see very few solaris downloads from the Rstudio cran mirror: since Jan 1, there were 845 packages downloaded from solaris, 0.003% of the total)
> There are days when I am tempted to say, "okay, if RcppEigen is not suitable for CRAN then remove it" which would result in removal of all the packages that depend on it, including lme4. That may seem childish of me but I really don't know what else to do.
I have been tempted to do this too, and for RcppEigen more than an empty threat. Currently the complete reverse dependencies (Depends, Imports, LinkingTo and Suggests) of RcppEigen includes 3626 packages, almost three-quarters of CRAN. It would certainly create more work for CRAN can allowing RcppEigen to fail on Solaris.
> So I have reached the point of saying "goodbye" to R, Rcpp and lme4 and switching all of my development effort to Julia. I'm sorry but others are going to need to determine how to maintain lme4 to the satisfaction of the CRAN maintainers or whether there should be an alternative distribution mechanism for R packages.
This is a great loss for the R community. nlme was one of the first R packages that I used, and I still remember taking a SAS-based mixed models course, while reading "Mixed-Effects Models in S and S-PLUS" and doing all the homework in R. I certainly learned much more about
mixed models and data analysis from you and your book than I did from that class! Your patient and thoughtful responses to questions about R and statistics have helped me become a better statistician, and have helped many others solve their real scientific problems.
While I hope that one day we can lure you back to R with better ways of distributing packages, I wish you all the best in making great modelling software for julia. The julia community is truly lucky to have you!
I have had the pleasure of working with Doug on Julia in the last couple of years, and I fully agree that we are extremely fortunate to have Doug working on Julia. I have heard the sentiment you express from friends of mine too - they all talk about having learnt their R and statistics from Doug, through all his contributions on the mailing lists, software, documentation, email exchanges, etc.
6 comments
[ 8.4 ms ] story [ 24.9 ms ] threadTo provide a little more context from what I know myself, Doug Bates is a long-time R contributor and package maintainer, but started switching to Julia for the reasons he gives here, among others, in mid 2012. He's a great hacker, not to mention brilliant statistician, and we're extremely lucky to have him in the Julia community.
Many thanks for the package and the book. When can I hold a non-digital copy in my hands?
Thanks,
Russ"
Sadly, I don't think that adds much to the context. But yeah, Julia benefits greatly from this. It's still not at the point where I'd move (I'm more of an end-user than a developer) but it's got my curiosity.
"> Maintaining an Rcpp-based package on CRAN these days is a case of "no good deed shall go unpunished" and "the flogging will continue until morale improves". I am the maintainer of the RcppEigen package which apparently also makes me the maintainer of an Eigen port to Solaris. When compilers on Solaris report errors in code from Eigen I am supposed to fix them. This is difficult in that I don't have access to any computers running Solaris, which is a proprietary operating system as far as I can tell, and Eigen is a complex code base using what is called "template meta-programming" in C++. Making modifications to such code can be difficult. I can't claim to fully understand all the details in Eigen and in Rcpp. I am a user of these code bases, not a developer. The Eigen authors themselves don't test their code under Solaris because they don't have access to Solaris systems either and they don't regard Solaris as an important platform for numerical computing. The CRAN maintainers feel differently, which puts me in a box.
It's not just packages that use Rcpp that suffer from this problem. lubridate, which has extensive user tests, often fails on solaris because it does something different to every other platform. While this has uncovered a number of bugs and limitations in R's datetime support, the only feedback we get from CRAN is extremely negative.
For lubridate, in the absence of any help from a solaris expert (and no evidence that anyone on solaris uses lubridate), we have simply told CRAN that it does not work on solaris. We continue to argue that the CRAN policies only require that an R package need only work on two major platforms to be distributed via CRAN, and while the CRAN maintainers continue to push back at us, they are bound by their own words.
(While a somewhat biased sample, we see very few solaris downloads from the Rstudio cran mirror: since Jan 1, there were 845 packages downloaded from solaris, 0.003% of the total)
> There are days when I am tempted to say, "okay, if RcppEigen is not suitable for CRAN then remove it" which would result in removal of all the packages that depend on it, including lme4. That may seem childish of me but I really don't know what else to do.
I have been tempted to do this too, and for RcppEigen more than an empty threat. Currently the complete reverse dependencies (Depends, Imports, LinkingTo and Suggests) of RcppEigen includes 3626 packages, almost three-quarters of CRAN. It would certainly create more work for CRAN can allowing RcppEigen to fail on Solaris.
> So I have reached the point of saying "goodbye" to R, Rcpp and lme4 and switching all of my development effort to Julia. I'm sorry but others are going to need to determine how to maintain lme4 to the satisfaction of the CRAN maintainers or whether there should be an alternative distribution mechanism for R packages.
This is a great loss for the R community. nlme was one of the first R packages that I used, and I still remember taking a SAS-based mixed models course, while reading "Mixed-Effects Models in S and S-PLUS" and doing all the homework in R. I certainly learned much more about mixed models and data analysis from you and your book than I did from that class! Your patient and thoughtful responses to questions about R and statistics have helped me become a better statistician, and have helped many others solve their real scientific problems.
While I hope that one day we can lure you back to R with better ways of distributing packages, I wish you all the best in making great modelling software for julia. The julia community is truly lucky to have you!
Hadley"