Most of the remarks are deserved. The exception hierarchy is not consistent currently, but the language is still evolving a lot, including on a syntax level (julia 0.4 deprecates the short dict/array syntax, just to name one).
To be honest, I hope they continue to break stuff. There are still many areas where it feels that the syntax/language itself could be improved a lot (too many to list!). Some of the changes have a significant impact on the entire system (like multiple inheritance for abstract types). Most of the proposed discussion/changes I've seen improve the language in several ways, they make it more uniform, so I'm really looking forward to them!
The base lib is very minimal, and the quality of the contributed packages varies wildly. It's a new language... you might lament most of the same issues with Rust.
Being a long time lisper, for me Julia it's a lisp in disguise with an edge on performance. The compromises/choices done so far have a great sense of direction and balance. It's a good time to jump into the development before the language is set in stone and the warts cannot be changed anymore!
I would like to defend Rust here. For example, the article complains "The build is often broken or has failing tests". Rust build is basically never broken nor has any failing tests. Being a new language really is no execuse.
I personally never had a failing build, and I'm following the master branch. I cannot relate to the author for that specific point. Julia seems to be reasonably unit-tested to me for being such a young project. The testing framework is very simple, but does the job. I would say that also the complaint that @test is just an assert is mostly moot: it's actually the only thing you need the vast majority of the time. Other frameworks are maturing.
But if you consider the entire ecosystem (see http://pkg.julialang.org), many packages that you might rely on, especially for simple operations, might break.
Most of the breaking changes are developed in branches and merged when completed. Again, overall the source management is also following the best practices in this sense.
I didn't really want to get into too much detail because for me most of the arguments made are ok, but it's an ongoing development effort. When new ideas are proposed, I want them to be considered, not dismissed as "breaking changes".
One thing I definitely miss, especially with things being split into separate extension traits (e.g. iterators a few weeks ago) is a Hoogle-like type-based search. Because finding out how to perform operations, especially very generic ones, has become fairly frustrating.
Also the ability to easily search both the standard library and third-party libraries is probably going to be needed soon if nothing experimental is going to be available in the "standard track".
Maybe I'm off base, but in this day and age I can not imagine writing a new compiler and runtime without at least 100% branch coverage and automated fuzz testing and actually asking people to use it.
It is a gut wrenching feeling when code crashes in production and it is my compiler to blame. It seems so much more shameful to me then if a library or service of mine crashed, not that that isn't shameful as well.
You need to be loose enough that you'll actually write and release a compiler, imperfect as it is. Shame can be a powerful motivator but it can also be paralyzing.
As I've used more modern languages with better type systems I've moved away from high test coverage and towards writing code that simply can't go wrong (or at least, that is exceedingly limited in the ways it can go wrong); http://spin.atomicobject.com/2014/12/09/typed-language-tdd-p... is a well-written explanation of something very close to my current approach. So I wouldn't put the emphasis on branch coverage or fuzz testing that you do. But I absolutely agree that you should have a very high level of confidence in a compiler before you ask people to use it, substantially higher than the level of confidence required of a library or service.
I buy the problem with cultural issues around testing and possibly poor quality of the current implementation.
I do think the OP doesn't get exceptions, though. Catching and doing something with an exception should be very rare, and almost never happen in library code, unless you've been infected by Java-itis and feel compelled to wrap implentation exceptions at the API level.
Failures are a function of implementation. The stack trace includes the fact that the API was involved. Further wrapping is usually busywork. If action needs to be taken on failure, the code that must not fail should have an exception handler, which can deal with the failure if possible (necessarily failure modes it can enumerate), and retry or abort as required.
Wrapping is particularly pointless with more functional code. When code is idiomatically passed around, library exceptions are frequently from user code passed in, rather than something the library author could exhaustively list.
Julia's sweet spot seems to be anyone who needs to do statistical work who also has an existing C ABI based environment.
The ability to seamlessly plug in Julia to your production system via C code is a boon for those of us who have to use R in an "offline" capacity and then feed data to production systems.
The downside of alot of the Julia posts is that its not R. R, for better and worse, has many years of bug fixing and library design behind it.
I have yet to stump cran (http://cran.r-project.org/) when looking for a library and after 5+ years of using R I still am amazed that I can learn some new ability that ggplot can do.
My advice to people who ask what language to use is always, I think Julia is still a bit too early on to use unless you already know what code you want to write and are using a C based enviroment.
The reason is that most programmers just want to get one task done. They aren't an expert on the mathematical domain they are trying to use, they just want to plug their values into a library and get their results and maybe print them in a nice format. With Julia I still run into issues too often where you have to second guess the langugage or library and if you aren't fully sure that your math is right you tend to assume you are wrong:)
Otherwise use R. It just works and it will make googling for answers much easier.
EDIT replaced jvm with C ABI. Not sure how I made that big of a typo:(.
I'm not sure you're talking about Julia here. Or maybe it has some way of interfacing with the JVM that I'm not familiar with? I've thought Julia's interop story had more to do with being able to interface with C code easily via LLVM.
For the JVM, Clojure and Incanter seem like a better statistical bet. I heard they're moving toward a 2.0 release of Incanter too...
Even though there parent misspoke here, there's https://github.com/aviks/JavaCall.jl for interfacing with the JVM, though of course that's more the reverse.
R is actually a nice language (also with multiple dispatch!), but it's basically useless for any actual numerical work. Too slow. There have been efforts in the past to introduce a JIT compiler, but nothing has become mainline yet. The GC is also very poor.
The actual base system in R is also very small. The built-in vector types helped the language mitigate the issue due to the fact that it's very easy to build a C extension that does the actual computation by just taking the underlying memory pointer (it's actually easier to do than python and it maps much better). But it stops there. If it wasn't for the large swath of statistical packages, R wouldn't have much traction. Not to mention that the code quality of the typical CRAN package is quite low, coming from people with a background in statistics and very little CS knowledge.
The hope with Julia is being able to actually develop methods in Julia itself. I personally looking forward to replace python&c++ with Julia&Rust in the future.
> There have been efforts in the past to introduce a JIT compiler, but nothing has become mainline yet
It's been included in the default install of R since version 2.14. You can compile any function you want, and in some cases you can get a speedup of several times. Sure, you're not getting the speed of C, but it's there, it works, and in the future it will lead to much larger speedups.
It can, but it often doesn't, at least not a year or so ago (perhaps it got much better). I've seen the examples with great speedups but every time I've tried it on real code I never saw anything. You're much better off just using Rcpp for those sorts of cases.
Well, it's true, it is included in the default install. After unsuccessfully trying to locate the elusive 'compiler' package at CRAN, I came to the conclusion that the R compiler was an urban legend. Now I realise it was already installed all along from the start and all I had to do is 'library(compiler)'. Silly me.
If it wasn't for the large swath of statistical packages, R wouldn't have much traction.
I agree, but Community & Ecosystem are critical to any language's success. R's community is great and getting better [1].
You could make very similar criticisms about JavaScript, which is a lousy language that has accomplished amazing things thanks to third-party extensions and a very active development community.
In my experience Julia is quick with low level array oriented programming which doesn't allocate ("C in Julia"), but if the garbage collector is needed within loops or you use anonymous functions there then you'll suffer greatly.
Rust covers the lower-level much better. In Julia for example you currently don't have a nice way to modify C-like structs. You can offset memory directly, but it's a pointless exercise. Even if a record-like syntax is added, the dynamic nature of Julia might incur in different latencies, you have the GC (not optional), and the runtime is quite big.
Rust is filling the same niche I'm using c/c++ for, where I need absolute control of every detail: not necessarily for performance.
This is very fair criticism. Julia has yet to make the full transition from research project to production language. Having seen Julia evolve over the past 2+ years, I feel like stability and test coverage have improved a lot and I expect them to improve further.
I fully agree on the need for more testing. We've been trying to make sure that new functionality and all issues reported come with tests where possible and I actually think we've been making progress on that front (and especially compared to two years ago), but we need to do a lot more of this. For a significant number of the core contributors (myself included), Julia is probably the first big project we're working on and we're still learning proper release engineering and making software for a large number of users.
That said, I think we've been pretty good about keeping master functional recently and I don't remember the last time I pulled from master and had something break (I think the travis status analysis is misleading here, because it probably includes development branches as well as instances where the CI system itself is failing - either because travis is down, or because the servers for dependencies are down etc). I also don't think that we've ever left master broken for days at a time when there was an easy fix available (of course bugs that come up and you don't know how to fix can always happen).
On that topic though, I think we have room for improvement. I've been thinking about moving to a pre-commit CI system (you push to a branch and the CI system merges once everything's green), especially now that we have CI coverage for all 3 supported platforms. I'll bring that up on the mailing list.
So much for the Julia core. Packages are a different issue and I agree that we have a problem with package quality. That are a few very high quality packages (I'm thinking e.g. Color and Distributions) and a long tail of packages of varying quality. Iain has been doing great work on this front with http://pkg.julialang.org/pulse.html and making sure that packages keep up with changes in julia and at least pass their own test suite, but there's obviously a lot of work to be done there.
The good thing is though that since these packages are completely decoupled from Julia core, this can be easily parallelized. One of the advantages of Julia is that the core is pretty small (admittedly bigger now than it should be and we need to split some stuff out) and it's in general pretty easy to look at the code which hopefully makes it easy hack on packages. Making high quality packages requires a lot of time and work and since Julia is such a young language, the ecosystem is still immature.
If you have thoughts on what we can do to improve package quality or make things easier on package developers, please let us know, we'd love to hear them.
Finally, I also would like to ask to please, please file bug reports for things that don't work the way you expect them to or for any bugs you encounter. One of the things mentioned in the OP was the REPL rewrite (which I worked on). The original REPL was a messy readline based hack, which really needed to be replaced. Admittedly, readline has a lot of features and I originally only implemented the ones that were part of my workflow, but I fully expected people to file issues for any features they were missing (and they did) and I and others quickly implemented lots of features (including quite a number not present in the original rewrite). Of the REPL issues currently open, I can see only one that may have worked with the original REPL (emacs keybindings). In any case, if you're missing features in the REPL please file an issue.
I'll end with my general disclaimer that while Julia has a lot to offer, it is not yet a polished system and I don't yet recommend using it if that's the experience you want.
The package manager in Julia is great, but the process of making a package needs much better documentation.
It looks like Bindeps is the intended way of building packages that have external dependencies, but last time I checked the "documentation" for bindeps was another Julia package called Cairo, which apparently did a good job.
The other problem is that when Googling around to find the standard way of putting together packages, almost all the results that come back are list posts or things from the issue tracker which tell you the old way of doing things (which no longer works).
Things like how to build things differently for Windows, Linux, OSX, Unix, how to deal with paths, how to use the OpenSUSE binary production system, etc., all need documentation, with examples.
Also needed is documentation on linking against external libraries that have their own dependencies. Where should libraries be canonically installed. How does the linker resolve dependencies (this is especially difficult to work out on Windows).
Including all the .h files for Julia's own shared library dependencies in the repo would also be useful, along with documentation of which versions of external libraries Julia currently uses.
I also find the documentation on how to peg to a commit or tag difficult to understand, as is the documentation on the path to becoming an official Julia package, with explicit examples of what needs to be done at each step.
Finally, I find it annoying that you can do Pkg.update("Mypkg") and Pkg.build("Mypkg") but not Pkg.test("Mypkg") (or is it the other way around, I always forget). One should be able to selectively update, build and test any particular package, without having to do so for every package you currently have installed.
Some of my comments might be out-of-date by now. I'm simply trying to recall some of the issues I had with building a package when I last put effort into that part of my development cycle.
You cannot do Pkg.update("MyPackage", but you are able to do Pkg.build("MyPackage") and Pkg.test("MyPackage").
Pkg.update() is currently designed to update the local METADATA snapshot and then update all packages. You can also update one package at a time manually with Pkg.checkout.
From personal experience as a software engineer at Google, precommit with a force override fall back is great. I worked here before we had it and after we had it, and it makes things way better. master should stay clean, and the pre-commit approach gets you pretty far down that road.
It also helps to have a culture of quickly rolling back breaking changes when they do slip through.
I second the move to a precommit based system. Getting a rock solid build is low hanging fruit for most projects, And getting to that point is usually just straightforward engineering. That said, this is a young project, and probably just needs more developer bandwidth to fix these minor glitches.
The problem with Travis failures is not that people aren't running the tests, it's that Julia depends on a lot of complex, finicky software [1], much of which is unusually sensitive to platform-specific differences and even differences between specific CPUs and VMs. This makes keeping builds green across the board pretty challenging. But anyone who has tried to build a fully working numerical stack like the one Julia ships with, can attest that it's non-trivial and changes that work on one machine may break things elsewhere.
In contrast, getting R up and running is very easy. I can easily set up multiple R versions on my machine. If I want to emulate a library environment on another machine, I just need to copy the folder over to that machine.
Are you talking about compiling R from source, including all dependencies and high-performance BLAS, LAPACK, FFT, ARPACK, etc. libraries? If not, you are comparing apples to oranges.
Yes. You can get multiple versions easily on the same machine and run them from that location. It's also easy to point rstudio to that instance. All the libraries install in the same location as well.
R ships with the reference BLAS which is 4x slower than OpenBLAS even on modestly sized problems. If you value portability over performance, I guess that's an ok choice, but we want to make sure that Julia ships with the best open source BLAS available. If that means putting up with the occasional broken builds on Travis, I think it's worth it. OpenBLAS is just one example – R seems to have consistently chosen portability over performance for their libraries. That's a viable choice, but it's not the one we've made.
This isn't quite true. I know of several R packages that I've had to build from source on Linux and I'd be worried about installing on windows (Combinations and Cranvas come to mind).
Generally the packages that aim for higher performance are more difficult to install, which makes sense.
edit: and if, god forbid, cran kicks off one of the packages you're using or an upstream package for a package you're using, then you've got to track it down and install it on your own. (tikzDevice is the one I'm thinking of.)
I understand, but having so many dependencies is probably all the more reason why you need very good test coverage and a build/deploy that keeps the trunk clean, so nothing gets released without all three (or however many) platforms all build clean.
Not to armchair quarterback, I know it is a hard thing to get there and you guys are all busy. Just my 2 cents.
Personally I hope the the julia testing package that gets adopted most widely is QuickCheck.jl. That form of testing deserves wider adoption than it gets.
I really like Julia despite the criticisms. I think the problem is that it's still a young project that needs more core developers to help maintain it.
I find bugs and unintuitive behavior a lot when using Julia. Often it requires me searching the mailing list or Stack Overflow to figure out "idiomatic" Julia code when this really should be in the documentation (who knew that a matrix operation A += B allocates new memory for A each time?) And I think global variables should just be removed altogether, as many problems as they cause. I commonly find myself thinking "There's no way that someone new to Julia would know you're supposed to do it this way", whereas with Python or even C, that's not the case.
But I think the language has a lot of promise, and the language design is really slick and well put together: powerful macros, a nice type system, multiple dispatch, clean code when you don't need performance but lots of additional decorations you can add when you do (devectorization, @parallel, @inbounds, @simd, type annotations).
Remember that Julia is only at dev version 0.4. The author of the article complains about breaking backwards compatibility on an almost daily basis by removing or changing things. At this early stage, I see absolutely nothing wrong with this. I'm tired of languages that have unintuitive and quirky (stupid) ways of doing things just in order to not break existing code. C++, for instance, is abhorrent to me. How many pages is the standard??
At the moment, Julia has a steep learning curve if you want high-performance code. But once you know what you're doing, it's easy to get it running at the level of C. Plus, features like viewing the AST, or the LLVM and native assembly representation of the code are handy for debugging.
> And I think global variables should just be removed altogether, as many problems as they cause.
Hi, I'm curious what your thinking is?
Is it because global variables make testing hard?
Is it because global variables are can not be optimized well?
Is it because of bugs in julia related to global variables?
I would like to use a language that doesn't have static variables, global variables, or static functions to make writing tests easier, but that makes it really hard to interface with C and the JVM. I often need functions like write() and System.out.print.
> Is it because global variables are can not be optimized well? Is it because of bugs in Julia related to global variables?
Mainly this; it's incredibly easy to accidentally use global variables and swamp your whole program. Think about defining some constant "n" in the global space and forgetting to put "const" in front of it. Then any for loops that use "for i = 1:n" will reallocate n on every iteration. If it's a nested loop, you'll get huge accidental memory allocations (this particular problem [not mine] is one on Stack Overflow that took a while to debug).
It's the same reason garbage collection is used; it's not strictly necessary but it prevents a lot of accidental bugs and makes programs easier to reason about. I think it would be the same with removing global variables.
* Of course, you still want function names to be global, so perhaps define a namespace that you can dump immutable functions or constants into. Or if there's good reasons not to remove global variables, at least make them all const by default and add a "mutable" keyword (and maybe swap that in the REPL). This would at least remove most of the accidental bugs.
It's personal preference and very minor thing but I don't really like end keyword to end a block. Those don't save space over braces (well at least not over K&R style braces) and are less readable than them (eyes naturally focus on words so the code with a lot of ends is more difficult to read). Why not just indentation or if that's too troublesome standard {} braces ?
Discussion about such things is usually quickly dismissed here but I wonder how other people react to syntax things like that.
It's quickly dismissed probably because that, along with 1 indexing (instead of 0 offsetting) are such minor negative decisions compared to all the extremely apt decisions made in the design of Julia. Those two are both controversial with minor pros and cons, but larger scale core design issues like multiple dispatch, integrated multidimensional arrays, a packaging system based off of github, the ability to immediately generate llvm or native code etc. far outweigh the cosmetic issues people might not be as comfortable with.
I agree but I also believe that cosmetic issues help with adoption and sparking enthusiasm for a language. It's silly but a lot of people are like that (I am as well). Beautiful cars, apartments, smartphones... in all of them core design is way more important but people still go for looks at least to some extent.
I think that those two features in particular are more polarizing than anything. I don't think there is a wide consensus as to which is better even though most programmers have an opinion. Many programmers would agree (I think) that Julia is cosmetically beautiful in a lot of other ways however.
Yeah, I think it's one of the those where you're not going to be able to please everyone, so don't even waste time on it. Personally I like the "end" statements and the 1-based indexing better than the alternatives, but that's just one preference among many. If syntax is important, you're going to alienate many potential users no matter what decision you choose.
Guy Steele once quipped (paraphrasing) that the most challenging thing about programming language design is the small number of paired brace characters in ASCII. Wasting one of these pairs for blocks the way C does is criminal. This forced C++ to co-opt < and > for templates, with all the parsing horror that ensued. Think how much nicer it would be if C++ template parameters had their own legitimate brace pair – like say { }.
Having significant indentation like Python is another option, but one I've never cared much for – it makes cutting and pasting code a bit of a nightmare, among other things.
This is probably mostly a UI/IDE question (although lack of object method call notation doesn't help), but what's stopping me now from using Julia is discoverability of functions. I'm just too used to typing obj.<tab> and going through a list of functions in something like IPython. In Julia, it seems like you have to dig through docs to find functions that you need.
Thanks for the pointer!
There is a big difference between having an easy tab completion vs running `methodswith` every time you want to call a method. But the fact that it's possible to do right now makes me hopeful, that it should be a built-in function of REPL and IJulia some time in the future.
While I appreciate the convenience of this, I think you may be misunderstanding how Julia does method dispatch. Rather than dispatching purely on the type of their first argument (as in Python) or into a class hierarchy/namespace based on that argument, then by parameter types (ala Java) Julia methods are fully qualified over every parameter type.
Sadly, that makes it hard to ask the question "what methods can I call on an object of this type?" because the answer might include functions with that type in any parameter position. In many cases that wouldn't be a meaningful answer - as in the case of, say, a numeric type which could be used for all kinds of indexing, iteration, and offset methods involving collections, IO handles, etc.
You might benefit a bit more from using the 'apropos' builtin to lookup functions based on simple keyword matching. Given a type name it lists methods that take or emit objects of that type (which may or may not be exactly what you want, but it's a starting point at least).
This is kind of a good reason to not touch Julia (from the second link):
> You cannot compare Julia with a project that has Google in the background. Its clear that they have a "more clear" development model and more documentation. Some goes for Rust. Julia is from and for researchers.
It's true that Julia doesn't have big corporate backing, but it is certainly not only a research language. Julia is being actively used in a growing number of industry applications, not just in academia (finance, aerospace, etc.).
You can say Julia is for everyone, but it's not informative. You could just as well (and just as uninformatively) say that Erlang is for everyone, or Rust is for everyone, or Haskell is for everyone. Each programming language has strengths and weaknesses that are the reason you use the language.
Julia's strength so far is fast numerical algorithms, and that's what most people seem to use it for.
I'm not sure if by researchers, you mean "CS researchers". I do materials science work and have used Julia on Titan (#2 supercomputer in the world). It's becoming very relevant in the HPC space. It has SSH-based cluster capabilities even though it doesn't support MPI (although I'm not using it for millions of CPU hours... yet). I've introduced it to other colleagues in my field and many of them are exploring it as well.
I feel like people are perhaps a bit overly negative about the state of Julia's packages. Compared to other early-stage languages, I'd say our package ecosystem is vibrant and full of fantastic packages. Naturally perhaps they are more math/scientific focussed, so if you are looking for cutting-edge packages for web dev you won't find them yet (although there are packages!).
See http://pkg.julialang.org/pulse.html, for example. We have over 470 packages in total that are registered, and on Julia 0.3 we have over 300 packages with tests that pass - and we run the tests in all registered packages every night.
Some of my favorite packages (that I didn't make, of course :D) would include
68 comments
[ 3.5 ms ] story [ 120 ms ] threadTo be honest, I hope they continue to break stuff. There are still many areas where it feels that the syntax/language itself could be improved a lot (too many to list!). Some of the changes have a significant impact on the entire system (like multiple inheritance for abstract types). Most of the proposed discussion/changes I've seen improve the language in several ways, they make it more uniform, so I'm really looking forward to them!
The base lib is very minimal, and the quality of the contributed packages varies wildly. It's a new language... you might lament most of the same issues with Rust.
Being a long time lisper, for me Julia it's a lisp in disguise with an edge on performance. The compromises/choices done so far have a great sense of direction and balance. It's a good time to jump into the development before the language is set in stone and the warts cannot be changed anymore!
This is not rocket science. You can do it too. Go read http://graydon2.dreamwidth.org/1597.html written by Rust's original author.
But if you consider the entire ecosystem (see http://pkg.julialang.org), many packages that you might rely on, especially for simple operations, might break.
Most of the breaking changes are developed in branches and merged when completed. Again, overall the source management is also following the best practices in this sense.
I didn't really want to get into too much detail because for me most of the arguments made are ok, but it's an ongoing development effort. When new ideas are proposed, I want them to be considered, not dismissed as "breaking changes".
[0] http://words.steveklabnik.com/rusts-documentation-is-about-t...
Also the ability to easily search both the standard library and third-party libraries is probably going to be needed soon if nothing experimental is going to be available in the "standard track".
It is a gut wrenching feeling when code crashes in production and it is my compiler to blame. It seems so much more shameful to me then if a library or service of mine crashed, not that that isn't shameful as well.
Do I need to loosen up?
As I've used more modern languages with better type systems I've moved away from high test coverage and towards writing code that simply can't go wrong (or at least, that is exceedingly limited in the ways it can go wrong); http://spin.atomicobject.com/2014/12/09/typed-language-tdd-p... is a well-written explanation of something very close to my current approach. So I wouldn't put the emphasis on branch coverage or fuzz testing that you do. But I absolutely agree that you should have a very high level of confidence in a compiler before you ask people to use it, substantially higher than the level of confidence required of a library or service.
I do think the OP doesn't get exceptions, though. Catching and doing something with an exception should be very rare, and almost never happen in library code, unless you've been infected by Java-itis and feel compelled to wrap implentation exceptions at the API level.
Failures are a function of implementation. The stack trace includes the fact that the API was involved. Further wrapping is usually busywork. If action needs to be taken on failure, the code that must not fail should have an exception handler, which can deal with the failure if possible (necessarily failure modes it can enumerate), and retry or abort as required.
Wrapping is particularly pointless with more functional code. When code is idiomatically passed around, library exceptions are frequently from user code passed in, rather than something the library author could exhaustively list.
Julia's sweet spot seems to be anyone who needs to do statistical work who also has an existing C ABI based environment.
The ability to seamlessly plug in Julia to your production system via C code is a boon for those of us who have to use R in an "offline" capacity and then feed data to production systems.
The downside of alot of the Julia posts is that its not R. R, for better and worse, has many years of bug fixing and library design behind it.
I have yet to stump cran (http://cran.r-project.org/) when looking for a library and after 5+ years of using R I still am amazed that I can learn some new ability that ggplot can do.
My advice to people who ask what language to use is always, I think Julia is still a bit too early on to use unless you already know what code you want to write and are using a C based enviroment.
The reason is that most programmers just want to get one task done. They aren't an expert on the mathematical domain they are trying to use, they just want to plug their values into a library and get their results and maybe print them in a nice format. With Julia I still run into issues too often where you have to second guess the langugage or library and if you aren't fully sure that your math is right you tend to assume you are wrong:)
Otherwise use R. It just works and it will make googling for answers much easier.
EDIT replaced jvm with C ABI. Not sure how I made that big of a typo:(.
For the JVM, Clojure and Incanter seem like a better statistical bet. I heard they're moving toward a 2.0 release of Incanter too...
R is actually a nice language (also with multiple dispatch!), but it's basically useless for any actual numerical work. Too slow. There have been efforts in the past to introduce a JIT compiler, but nothing has become mainline yet. The GC is also very poor.
The actual base system in R is also very small. The built-in vector types helped the language mitigate the issue due to the fact that it's very easy to build a C extension that does the actual computation by just taking the underlying memory pointer (it's actually easier to do than python and it maps much better). But it stops there. If it wasn't for the large swath of statistical packages, R wouldn't have much traction. Not to mention that the code quality of the typical CRAN package is quite low, coming from people with a background in statistics and very little CS knowledge.
The hope with Julia is being able to actually develop methods in Julia itself. I personally looking forward to replace python&c++ with Julia&Rust in the future.
It's been included in the default install of R since version 2.14. You can compile any function you want, and in some cases you can get a speedup of several times. Sure, you're not getting the speed of C, but it's there, it works, and in the future it will lead to much larger speedups.
Some info and timings are here: http://www.rinfinance.com/agenda/2014/talk/LukeTierney.pdf
The experimental version of the compiler can achieve speedups in the neighborhood of 20x.
[1] See page 7 of the presentation I linked above.
Not really. That's an example of one R's rough edges. I did the same thing.
I agree, but Community & Ecosystem are critical to any language's success. R's community is great and getting better [1].
You could make very similar criticisms about JavaScript, which is a lousy language that has accomplished amazing things thanks to third-party extensions and a very active development community.
[1] http://www.tiobe.com/index.php/content/paperinfo/tpci/index....
Rust is filling the same niche I'm using c/c++ for, where I need absolute control of every detail: not necessarily for performance.
I fully agree on the need for more testing. We've been trying to make sure that new functionality and all issues reported come with tests where possible and I actually think we've been making progress on that front (and especially compared to two years ago), but we need to do a lot more of this. For a significant number of the core contributors (myself included), Julia is probably the first big project we're working on and we're still learning proper release engineering and making software for a large number of users.
That said, I think we've been pretty good about keeping master functional recently and I don't remember the last time I pulled from master and had something break (I think the travis status analysis is misleading here, because it probably includes development branches as well as instances where the CI system itself is failing - either because travis is down, or because the servers for dependencies are down etc). I also don't think that we've ever left master broken for days at a time when there was an easy fix available (of course bugs that come up and you don't know how to fix can always happen).
On that topic though, I think we have room for improvement. I've been thinking about moving to a pre-commit CI system (you push to a branch and the CI system merges once everything's green), especially now that we have CI coverage for all 3 supported platforms. I'll bring that up on the mailing list.
So much for the Julia core. Packages are a different issue and I agree that we have a problem with package quality. That are a few very high quality packages (I'm thinking e.g. Color and Distributions) and a long tail of packages of varying quality. Iain has been doing great work on this front with http://pkg.julialang.org/pulse.html and making sure that packages keep up with changes in julia and at least pass their own test suite, but there's obviously a lot of work to be done there.
The good thing is though that since these packages are completely decoupled from Julia core, this can be easily parallelized. One of the advantages of Julia is that the core is pretty small (admittedly bigger now than it should be and we need to split some stuff out) and it's in general pretty easy to look at the code which hopefully makes it easy hack on packages. Making high quality packages requires a lot of time and work and since Julia is such a young language, the ecosystem is still immature.
If you have thoughts on what we can do to improve package quality or make things easier on package developers, please let us know, we'd love to hear them.
Finally, I also would like to ask to please, please file bug reports for things that don't work the way you expect them to or for any bugs you encounter. One of the things mentioned in the OP was the REPL rewrite (which I worked on). The original REPL was a messy readline based hack, which really needed to be replaced. Admittedly, readline has a lot of features and I originally only implemented the ones that were part of my workflow, but I fully expected people to file issues for any features they were missing (and they did) and I and others quickly implemented lots of features (including quite a number not present in the original rewrite). Of the REPL issues currently open, I can see only one that may have worked with the original REPL (emacs keybindings). In any case, if you're missing features in the REPL please file an issue.
I'll end with my general disclaimer that while Julia has a lot to offer, it is not yet a polished system and I don't yet recommend using it if that's the experience you want.
Sorry for making this so long...
It looks like Bindeps is the intended way of building packages that have external dependencies, but last time I checked the "documentation" for bindeps was another Julia package called Cairo, which apparently did a good job.
The other problem is that when Googling around to find the standard way of putting together packages, almost all the results that come back are list posts or things from the issue tracker which tell you the old way of doing things (which no longer works).
Things like how to build things differently for Windows, Linux, OSX, Unix, how to deal with paths, how to use the OpenSUSE binary production system, etc., all need documentation, with examples.
Also needed is documentation on linking against external libraries that have their own dependencies. Where should libraries be canonically installed. How does the linker resolve dependencies (this is especially difficult to work out on Windows).
Including all the .h files for Julia's own shared library dependencies in the repo would also be useful, along with documentation of which versions of external libraries Julia currently uses.
I also find the documentation on how to peg to a commit or tag difficult to understand, as is the documentation on the path to becoming an official Julia package, with explicit examples of what needs to be done at each step.
Finally, I find it annoying that you can do Pkg.update("Mypkg") and Pkg.build("Mypkg") but not Pkg.test("Mypkg") (or is it the other way around, I always forget). One should be able to selectively update, build and test any particular package, without having to do so for every package you currently have installed.
Some of my comments might be out-of-date by now. I'm simply trying to recall some of the issues I had with building a package when I last put effort into that part of my development cycle.
Pkg.update() is currently designed to update the local METADATA snapshot and then update all packages. You can also update one package at a time manually with Pkg.checkout.
It also helps to have a culture of quickly rolling back breaking changes when they do slip through.
[1] https://github.com/JuliaLang/julia#required-build-tools-and-...
Generally the packages that aim for higher performance are more difficult to install, which makes sense.
Combinations: http://www.omegahat.org/Combinations/
Cranvas: https://github.com/ggobi/cranvas
edit: and if, god forbid, cran kicks off one of the packages you're using or an upstream package for a package you're using, then you've got to track it down and install it on your own. (tikzDevice is the one I'm thinking of.)
Not to armchair quarterback, I know it is a hard thing to get there and you guys are all busy. Just my 2 cents.
(And I think Julia is bloody brilliant, fwiw)
I find bugs and unintuitive behavior a lot when using Julia. Often it requires me searching the mailing list or Stack Overflow to figure out "idiomatic" Julia code when this really should be in the documentation (who knew that a matrix operation A += B allocates new memory for A each time?) And I think global variables should just be removed altogether, as many problems as they cause. I commonly find myself thinking "There's no way that someone new to Julia would know you're supposed to do it this way", whereas with Python or even C, that's not the case.
But I think the language has a lot of promise, and the language design is really slick and well put together: powerful macros, a nice type system, multiple dispatch, clean code when you don't need performance but lots of additional decorations you can add when you do (devectorization, @parallel, @inbounds, @simd, type annotations).
Remember that Julia is only at dev version 0.4. The author of the article complains about breaking backwards compatibility on an almost daily basis by removing or changing things. At this early stage, I see absolutely nothing wrong with this. I'm tired of languages that have unintuitive and quirky (stupid) ways of doing things just in order to not break existing code. C++, for instance, is abhorrent to me. How many pages is the standard??
At the moment, Julia has a steep learning curve if you want high-performance code. But once you know what you're doing, it's easy to get it running at the level of C. Plus, features like viewing the AST, or the LLVM and native assembly representation of the code are handy for debugging.
Hi, I'm curious what your thinking is? Is it because global variables make testing hard? Is it because global variables are can not be optimized well? Is it because of bugs in julia related to global variables?
I would like to use a language that doesn't have static variables, global variables, or static functions to make writing tests easier, but that makes it really hard to interface with C and the JVM. I often need functions like write() and System.out.print.
Mainly this; it's incredibly easy to accidentally use global variables and swamp your whole program. Think about defining some constant "n" in the global space and forgetting to put "const" in front of it. Then any for loops that use "for i = 1:n" will reallocate n on every iteration. If it's a nested loop, you'll get huge accidental memory allocations (this particular problem [not mine] is one on Stack Overflow that took a while to debug).
It's the same reason garbage collection is used; it's not strictly necessary but it prevents a lot of accidental bugs and makes programs easier to reason about. I think it would be the same with removing global variables.
* Of course, you still want function names to be global, so perhaps define a namespace that you can dump immutable functions or constants into. Or if there's good reasons not to remove global variables, at least make them all const by default and add a "mutable" keyword (and maybe swap that in the REPL). This would at least remove most of the accidental bugs.
Having significant indentation like Python is another option, but one I've never cared much for – it makes cutting and pasting code a bit of a nightmare, among other things.
Sadly, that makes it hard to ask the question "what methods can I call on an object of this type?" because the answer might include functions with that type in any parameter position. In many cases that wouldn't be a meaningful answer - as in the case of, say, a numeric type which could be used for all kinds of indexing, iteration, and offset methods involving collections, IO handles, etc.
You might benefit a bit more from using the 'apropos' builtin to lookup functions based on simple keyword matching. Given a type name it lists methods that take or emit objects of that type (which may or may not be exactly what you want, but it's a starting point at least).
https://groups.google.com/forum/#!topic/julia-users/GyH8nhEx...
My response (which is too long to include here) is this:
https://groups.google.com/d/msg/julia-users/GyH8nhExY9I/0Bzn...
> You cannot compare Julia with a project that has Google in the background. Its clear that they have a "more clear" development model and more documentation. Some goes for Rust. Julia is from and for researchers.
> Julia is from and for researchers.
I think the actual criteria is being ["a greedy, unreasonable, demanding programmer"](http://julialang.org/blog/2012/02/why-we-created-julia/) i.e. everyone.
Julia's strength so far is fast numerical algorithms, and that's what most people seem to use it for.
If installation issues are daunting, you may find juliabox.org a gentler way of getting your colleagues started.
See http://pkg.julialang.org/pulse.html, for example. We have over 470 packages in total that are registered, and on Julia 0.3 we have over 300 packages with tests that pass - and we run the tests in all registered packages every night.
Some of my favorite packages (that I didn't make, of course :D) would include
https://github.com/JuliaStats/Distributions.jl
https://github.com/JuliaStats/StatsBase.jl
https://github.com/pluskid/Mocha.jl (deep learning)
https://github.com/stevengj/PyCall.jl
The JuliaOpt stack of optimization packages (http://juliaopt.org)
and then you get fun new ones like https://github.com/anthonyclays/RomanNumerals.jl