I like Julia, but their multi-machine stuff needs a jolt.
I spent some time with HDFS.jl and trying to re-do spawn to launch tasks onto a HADOOP/YARN cluster, but couldn't really get @everywhere includes to work.
But oherwise compared say R, which runs on a cluster, the SSH based communication methods of Julia kinda made it difficult to run it anywhere beyond small data land.
It would kick-ass to use on a cluster, if only it could do that & distribute keys on a total order partitioner like radix-sort.
1 comment
[ 4.3 ms ] story [ 12.4 ms ] threadI spent some time with HDFS.jl and trying to re-do spawn to launch tasks onto a HADOOP/YARN cluster, but couldn't really get @everywhere includes to work.
But oherwise compared say R, which runs on a cluster, the SSH based communication methods of Julia kinda made it difficult to run it anywhere beyond small data land.
It would kick-ass to use on a cluster, if only it could do that & distribute keys on a total order partitioner like radix-sort.