The implementations for Rust and Julia are different in a way that means the comparison is not really apples-to-apples.
The Julia implementation replaces the input list with a Julia object, which is of course trivial for Julia to iterate over.
The Rust implementation is asked to convert the list-of-list-of-floats back to Python types, and then back into Rust types. Of course converting 100M floats back and forth is going to take a non-trivial amount of time.
The equivalent in Rust would be to wrap the return value in a PyObject that contains the Rust `Vec<Vec<f64>>`, and the Rust code could then just iterate that directly. It would even be made safe without the GIL as you'd implement your own mutex on the Rust side if you wanted it to be mutable.
If you do this then it runs in about 1.2s on my laptop.
It's also repeatedly `push`-ing elements into the `Vec`, which will likely lead to repeated allocation and copying. In a case like this were you pretty clearly know the size beforehand you should crate a Vec of the appropriate size with `Vec::reserve`.
You left out the explanation from your quote. The full is:
> Rust is not that fast beacuse it needs to copy data; using Pyo3 objects would probably lead to similar results as cython, but with an added library.
"It needs to copy data", because it's converting Python objects into Rust objects and back again. As the full quote states, it could be written in a different way, although that would make the code look really weird.
In this benchmark there is also no normal C++ involved. There is only Cython using some C++ ints, but operating on Python lists. And that's not handled by writing C++ code that operates on Python structures, but writing Python code in Cython in a "C++ mode".
> although that would make the code look really weird.
Not it wouldn't?
Using Pyo3 is arguably the most idiomatic way of interfacing Rust with python. The "added library" cost mentioned as an excuse not to use it is literally the 1 second it takes me to write `cargo add pyo3` .
They didn't do this because it would make their claim that "Julia is the better language for extending Python" kind of moot.
They started from the conclusion that they wanted to have, and worked their way backwards towards implementation and data that would support it.
I'll defer to your knowledge on that. I generally don't work with Python. I've only looked into interfacing C# with Rust and it looked crazy, so that's where I'm coming from.
But yes, it's pretty clear that the results are more a consequence of what's in the source files than the languages used (if we can even say that e.g. C++ was used at all).
C usually comes out a bit worse because it gives even less information to the optimizer.
And thanks to the GNU/LLVM monoculture most languages should perform roughly the same if they have them as a target e.g. I have found that D makes its it ridiculously easy to write highly specialized code that is both readable and visible to the optimizer (I will be blogging about that part later), but I'm sure I could force the same asm out from C++ or Rust etc.
I feel like the #1 downside of Python for the last few years is that you cannot take advantage of multiple cores of a CPU easily. Especially when you think it is heavily used in data analysis. We use Python for data analysis as well, and for 95% of operations we are doing, numpy is fast enough that we don't have any complaints. But sometimes, we do wish to be able to take advantage of all the cores in our CPUs, especially now that we can easily get an 8 core CPU for a reasonable price.
There is multiprocessing module but you cannot share memory between processes. I guess the best options are either writing a C extension or using Numba. Writing C extensions require either distributing binary packages or C compiler to be present on the target computer which is not always ideal. So is using Numba the best solution currently? I tried it a bit in the past but the errors I got was a bit hard to interpret compared to regular Python errors.
I wish the multithreading module had support for native threads. Is there any PEPs trying to bring easy multi-core support for Python?
Why not use the multiprocessing module? (if you can partition your problem space). I find memory sharing much more difficult to reliably implement an algorithm, and speed gain might be lower through cache conflicts.
A real hard requirement is going to be resource pools such as session linked connection pools with associated data. But there are just plenty of places where you don't want to be forced into a message passing architecture for a service and shared memory may be a better fit.
The thing is that the current ecosystem (numpy + scipy + pandas + PyCharm) fits our company structure perfectly (very few software engineers, mostly test engineers who are not very proficient coders). And we already have tens of thousands lines of code. So changing the whole ecosystem just because 5% of the problems are slow is too big of a jump for us. For now, it is easier for us to write a bit of C code for places where we absolutely need speed. But it would have been so convenient if we could just have native threads, so that we don't even need that 5% C code.
Do you know what you are actually stalled on? i.e. Memory or Instructions, if it's the latter you can probably eek out some more performance just by fiddling with compiler flags (you'd be surprised simultaneously how clever and how utterly braindead some compilers are if you play with them on Compiler Explorer for a bit)
Also, if you are running on Intel you might have some luck with Intel's profiling tools as, although I've never used them for python they are by far and away the best things in this area, they do support Python.
I don't know what details they give you in from Python code directly, but if you've used perf to profile code at a low level before, vTune is like a Tom & Jerry shotgun in comparison.
You don't have to replace the core python parts. We moved a decent amount of our data pre-processing into Scala, Golang, and Data Base queries and then just store the processed data for the python models to use to train/run (which ends up using the super efficient C/Fortran/Cuda code).
It’s process based, so communication overhead is heavy and the infrastructure is minimal - if you’re distributing jobs which are individually large it works fine, but it can’t really be used to replace multithreading in a shared memory model.
> I feel like the #1 downside of Python for the last few years is that you cannot take advantage of multiple cores of a CPU easily.
Also a big downside with JavaScript. Of course both Python and JS are high-level interpreted languages where high-performance use cases aren't the foremost priority.
> Of course both Python and JS are high-level interpreted languages where high-performance use cases aren't the foremost priority.
Modern JS engines are all primarily JIT-based, generally only interpreting code when it's faster than waiting on the JIT. JS JITs are quite good now, and in many cases will produce optimized C-equivalent compiled code from fairly naive JS.
Numpy operations release the GIL (usually at least) so you can use a threadpool and, indeed, share memory. Just try it and you may be pleasantly surprised.
Dask is great if you’re processing large amounts of data, and it recommends and supports threads for this reason.
I didn't know that. So let's say I have 4 regular Python threads calling the same function, and in this function, let's say I call numpy.add on the same array (but different parts of the array), then will it actually use different cores for these 4 different threads? I will try it out, if it works, then that's actually great and would be super easy.
Not sure there’s anything to prevent you from foot-shooting if you’re operating on the same array. I’d expect that this is only safe to do if, as you say, arrays are not overlapping. Let me know what you find.
I've been using JAX (https://jax.readthedocs.io/en/latest/) for scientific computing in general (in particular MCMC algorithms), as it's really fast. Even on a CPU you get massive speedups compared to numpy (can be up to 2 or 3 orders of magnitude faster in some cases).
The main selling point of the library is automatic differential and compilation to XLA, but I've been using it even when I don't need gradients, as it's really fast (due to compilation). I also really like the random number generator as it's very good for reproducibility.
I've played around with Julia in the past and really liked it, but in terms speed Jax has pretty much solved that problem for me
Have you tried numba+numpy? In my experience, it is much faster than Jax and can compile to cuda. It's not caveat free, but it also removes the hustle of labeling arrays as donated in Jax.
Have you been successful in implementing non-trivial computational code in numba/numpy? I've always found it starts to really break for anything which isn't really trivial, and the errors are mostly non-prescriptive and highly verbose.
I had the same experience about 2 years ago. Maybe it's changed since then? It was nice when you had a pure math function to write, but otherwise seemed to be unreliable, especially in multithreaded and multiprocess situations.
I still have some issues in nojit mode, but jit is most of the time fine. It's still a bit iffy with lists, but most of the time I can use numpy arrays.
TBF I am mainly using it for mostly pure path functions.
We have a very large portion of production code written in numba we’ve been running for about 3 years and I made a small contribution to the library. There are a lot of gotchas to numba when the codebase gets large but the benefits far outweigh the downsides. I highly recommend numba.
Edit: also note that a big part of the umap library is written in numba.
I just implemented both a CSV parser and an address standardizer in numba (both CPU and GPU) running in parallel feed through a message queue with a bunch of workers subprocs.
It takes a bit of getting used to but the performance gains on impressive. Basically, my bottlenecks shift from compute to i/o.
I think you have to balance it against writing in C/C++. Mentally, it is basically the same work as writing in C (you manage memory/you write complicated for-loops) but you have good array support with numpy. The primary advantage for me that everything stays in the python runtime environment. You just run the code without any extra steps.
...
What is missing from the timing type 'toy' benchmarks is an understanding that there is typically more than one bottleneck in a real problem and it is easy to choose the wrong one to optimize and get little gains.
After starting C (30 years ago now), spending a long time in C#, then switching to Python a few years ago, I think the unappreciated advantage of python is that I have to abandon all pretense of caring about speed and just get stuff working. It basically solves the pre-mature optimization problem for me by being a fast interpreted language rather than a slow compiled language.
> I think the unappreciated advantage of python is that I have to abandon all pretense of caring about speed and just get stuff working. It basically solves the pre-mature optimization problem for me
I feel the same way. With Python I just write the simplest algorithm that first comes to my mind, even though I know that it is not the most optimized way of doing things. But most of the time I am surprised that it works so fast that I realize I actually don't need to optimize it.
And being able to create and easily manipulate dictionaries and tuples also allows me to create efficient data structures very quickly.
I have a 25,000+ loc scientific codebase in Python, with lot of use in my specialized domain. I gave up on numba, because it's speedups vanish when transitioning to real problems. And secondly, it's bugs are not that intuitive.
I haven't tried numba but I've heard good things about it! Nice linked tutorial. If I understand correctly, you pass in jitted functions (using numba, and jax) into iminuit which does the optimisation?
With Jax you can write native for loops that can also be jitted (I imagine you can also do this in numba?); this can then be really fast. Though in that case you would have to write the optimisation algorithm yourself which is not always practical!
Another big speedup in Jax is due to vmap/pmap, which allow to vectorise/parallelise computation. For example you can build a massive gram matrix really quickly using vmap.
Another point: Jax can also run on GPU (like numba :) ) without having to rewrite anything.
> I haven't tried numba but I've heard good things about it! Nice linked tutorial. If I understand correctly, you pass in jitted functions (using numba, and jax) into iminuit which does the optimisation?
Yes, you can just pass the function and it runs the optimization, in severe cases, you can start by doing a grid `.scan` or `.simplex` (Nelder-Mead simplex method), then `migrad` to minimize, and `.hesse` for 1 sigma bound.
You can also provide a gradient function that, well computes the gradient instead of computing it numerically.
> I imagine you can also do this in numba
Yes! It compiles to native code.
> Another big speedup in Jax is due to vmap/pmap, which allow to vectorise/parallelise computation.
It's possible to compile some vmapped functions with numba too, if the link is any indication, you may see even greater speedup than just jax.grad
However, I do concur that jax's vmap is absolutely fantastic and I found it very useful on many occasions.
> We use Python for data analysis as well, and for 95% of operations we are doing, numpy is fast enough that we don't have any complaints. But sometimes, we do wish to be able to take advantage of all the cores in our CPUs, especially now that we can easily get an 8 core CPU for a reasonable price.
I understand the sentiment, but in this case I wonder if it is less a python issue and more an issue with numpy. I don't see any reason why numpy couldn't execute the last line of the following multi-threaded:
>>> import numpy as np
>>> a = np.arange(1000000000)
>>> b = 2 \* a
The same thing applies to matrix multiplication. I'm guessing numpy has chosen not to for good reason though.
edit: Actually it seems numpy is multi-threaded under certain circumstances:
Numpy will perform a parallelized calculation when using one of the underlying BLAS routines, if you have the appropriate libraries installed on your machine. So to take advantage of your cores, try to write the calculation as a call to a linear algebra function. Loops or expressions with arithmetic operators as in your example will run on one core. Or, just use Julia.
Distribute your data and run everything as dask.delayed and then compute only at the end.
Also check out legate.numpy from Nvidia which promises to be a drop in numpy replacement that will use all your CPU cores without any tweaks on your part.
What I do is write single-threaded programs that process only a section of data, and manage them with HTCondor. It's some extra overhead and complexity but this way you can easily scale to hundreds of machines without changing anything. You also get mature queuing and job management tools that work independently of your program. If your application is easy to paralellize and you might need more than one machine i highly reccommend this route.
Rust doesn’t need to copy the data. It’s trivial to pass e.g. Numpy arrays to Rust as slices via Cython (let alone originating in Cython!), modify them, and return them, or use them as input for a new returned struct.
Each of those repos has links to the corresponding Rust “shim” libraries that provide FFIs for dealing with the incoming data, constructing Rust data structures from it, and then transforming it back on the way out.
As a more general comment, using a GC language as the FFI target from a GC language is begging for difficult-if-not-impossible-to-debug crashes down the line.
To add, it is trivial to guarantee that Rust code is "zero-copy", so if this is something you care about, Rust allows your program to fail to compile if it tries to make a copy.
> As a more general comment, using a GC language as the FFI target from a GC language is begging for difficult-if-not-impossible-to-debug crashes down the line.
When I was interfacing D code with a part of Unreal Engine that I think is garbage collected, I actually just took the L and copied everything into buffers on their malloc when handing stuff off to the engine. It wasn't particularly hot code so the memcpys were worth the peace of mind I found, ugly as it was.
There is no problem at all interfacing Julia's C API to Rust. It's a shame that C++ and Rust are bad fits together, but this actually strengthens the argument for Julia as the extension language rather than Rust, since Julia interfaces easily to not just C but also the C++ code in which much of the world's best numerical algorithms are written, and has an unrivalled FFI to Python, while Rust's object system is a poor fit for either Python or C++.
> As a more general comment, using a GC language as the FFI target from a GC language is begging for difficult-if-not-impossible-to-debug crashes down the line.
<s>As for the FUD about interfacing JIT code to C/C++, this is a problem Julia was designed from the outset to tackle. Incidentally, Julia built on the excellent experience LuaJIT has had. A challenge: can you name a particular extension that would interface better from Rust than from Julia?</s>
Oops, I misunderstood the criticism you were hinting at but failing to justify, sorry. OK, Julia already does interface to Python and this interface sees widespread use. If your suspicion is right, then where are the horror stories from people who were bitten when deploying code that built on the interface?
> As a more general comment, using a GC language as the FFI target from a GC language is begging for difficult-if-not-impossible-to-debug crashes down the line.
Not true!
What you do is that you keep a registry for objects passed from the host vm to the foreign vm in which you register objects thus transferred. And you use a similar mechanism for objects passed from the foreign vm to the host vm. In CPython, you simply increment the refcount to prevent Python from collecting them prematurely.
This is how Java does it (via JNI) and how many gc:ed runtimes interact with other gc:ed runtimes. This is how you do it in Rust too since Rust can't tell how long an object passed to a foreign vm is supposed to last.
> This ... is how many gc:ed runtimes interact with other gc:ed runtimes.
It's not that easy. You'd need to register the object as a GC root as long as it's being managed by the foreign GC, and similarly ensure that objects in the foreign VM are properly "de-registered" when the local GC finalizes them. It's far from trivial, particularly when compared with other memory-management strategies.
Yes, that is what the scheme I described is doing. You'd need a similar mechanism in Rust too to prevent it from collecting objects prematurely. I think implementing this is fairly trivial. The difficult part of writing an FFI is handling closures: Java calling Python calling Java calling Python which throws an exception...
This is a solved problem: look at how the Lua FFI does it. Lua both is capable of exporting its own garbage collector to its counterparty and has a plug-in system for GC so that Lua can use the counterparty's GC. That allows you to either use Lua's GC for foreign objects or have the counterparty's GC manage Lua's objects.
It's funny the hostility to JIT that some fans of compiled languages have.
Given that it's via pyO3, you could even pass the numpy arrays using https://github.com/PyO3/rust-numpy and get ndarrays at the other side.
Same no copy, slightly more user friendly approach.
Further criticism of the actual approach - even if we didn't do zero copy, there's no preallocation for the vector despite the size being known upfront, and nested vectors are very slow by default.
> As a more general comment, using a GC language as the FFI target from a GC language is begging for difficult-if-not-impossible-to-debug crashes down the line.
Safe interop between two GC'd languages (Haskell and Java) was one motivation of Haskell's new -XLinearTypes extension
Generally, the new linear types are exciting but are nascent at the moment. But one extension to them (linear constraint [1]) seems to allow embed the equivalent of Rust's ownership in Hsakell using more primitive features in the type system (to my understanding..)
I honestly think people are severely underestimating what a massive impact this is currently having in increasing productivity of python devs who know a little C++.
I believe the toolchain of Rust is nicer in that you get a relatively small sized and self-contained rust library that can be easily distributed with a python package. Julia can't be easily bundled this way because you need to ship the entire runtime, with all the gubbins this entails.
On the other hand: having used pyo3 to integrate rust with python in the past, the biggest pain is simply to reconcile the dynamism of python with AOT compiled rust code. There's a lot of noise at the interface from the large amount of type checking to unpack the specific types of numpy arrays coming across. Do you want to have your rust code work with all sorts of integer bit lengths? That'll be a code path for each. If you have a number of input types, have fun coercing them all.
This can be alleviated with macros to a degree, but that just hides the problem really.
Julia is JIT compiled, which means that there are no problems with types being determined only at run time. In fact, this one fact makes integration at the code level much nicer with Julia.
So all in all it's a tradeoff really. I also found the tooling for Rust to be much more robust and stable (1 year ago admittedly).
I'm a massive Julia fanboy, but I would not extend Python with Julia if I could choose not to. Julia has a _massive_ runtime with a hello-world script consuming 150 MB of RAM, not to mention the dreaded startup-time.
It's better and easier to use Julia as your main top-level "glue" language and call Python/Rust/C from Julia. Julia is in many ways a better glue language than Python - better multithreading, easier calling into C/Rust, etc. Then, over time, to the extent it is practical, you can replace foreign code with Julia code - because Julia is fast enough that it actually makes sense to do so.
If you already have a large Python code base and can't switch the top-level language to Julia, I would just not use Julia for that project.
You seem to understand Julia well. Is there a reason there is no nexus plugin or way to mirror the julia repo for dev networks that dont have unrestricted access to the internet, or are even airgapped. I see multiple people asking this online, so it is a common enough problem, but it seems like people are saying the Julia approach makes it hard to support.
I'm sorry, I have no idea, but I'm sure someone else has insight.
Why would you want to mirror the Julia repo instead of just downloading already-compiled binaries? Do you want to maintain your own fork with your own set of patches?
Figuring the whole dependency tree without the package manager would be difficult and the build file sometimes has other side effects besides building for many packages. And downloading the binaries rather than letting the source go through nexus would be sidestepping most of the network security (and maybe legal) reasons people use nexus.
Frankly, Julia is a young language, so these things have yet to materialize. Also, for those that really do want it, the Julia package mirror is one of the few products that the company behind Julia sells as part of JuliaHub. It is inconvenient though that it doesn't fit in with the rest of the languages in Nexus.
I'm not sure honestly. We didn't end up buying it, because so few people use Julia in our org. Too much legacy matlab code and the professors in academia have yet to switch over.
It is possible to write a script that mirrors the registry, packages, and artifacts and set up a dev image that respects it. The script basically just needs to use Pkg and Base. However, artifacts end up taking too much space and then you have to decide how to host only a subset of package versions intelligently and propagate that change through at least the registry and artifact mirrors.
I went far down this path before infosec agreed to give github / highly respected cdn access.
> _massive_ runtime ... 150MB of RAM, not to mention the dreaded startup-time.
I understand how can this be inconvenient for me and you while scripting. But this is absolutely no problem for julia's currently main target market who runs multi-GB simulations.
Also it is not like they will be spawning a julia instance in a hot loop (THAT would be horrible). You write some julia code, and import it using smth like pyjulia at the beginning of the file.
I was a huge fan of Matlab way back. I wrote a hundred small Matlab programs for usage in the research department of the company I worked in. Doing data operations in Matlab was way more elegant than in say, Numpy which I tried later. Development was fast and ergonomics were good.
After using Ruby for years, returning to Matlab style code in Julia felt somewhat awkward. Instead of my_array.length you have length(my_array). In my personal preference the method call is just a nicer way of doing the same thing. The single-instruction-multiple-data or dot notation sometimes worked and sometimes didn't, so you had to resort to loops anyway.
Ruby and map or each have their verbosity but overall it feels a more robust "hammer" for general programming tasks.
On the other hand, Julia can be really dense and still easy to understand.
Other languages of course take some of these things even further. Maybe some day I will find an ergonomics and nicety first successor to both.
Plotting with two Y axes or generating histograms in Julia was also way harder than I remember it being in Matlab. Also manually having to load the file before every run to see the changes in action added a lot of overhead to the workflow.
The workflow I was used to in Matlab involved very frequent making changes to code, running of the code that usually made some plot. Which is one command in Matlab, and plotting was fast, in 2001 already, on Windows NT 4.
In Julia, you have to first load the modified file, then run it. Plotting takes a long time. One just can't get nearly as productive with it in 2021, compared to Matlab of 2001 vintage.
What language would I pick if I had to do some quick analysis from some tables downloaded from the internet? Probably Julia still. If I had free access to Matlab, I would probably use it though.
Those who are attached to the class-based OOP model used, for example, in Python will find the dot notation more natural. But Julia’s multiple dispatch is a superset of this, and unarguably more powerful and flexible. Python OOP leads to monstrosities like
It’s hard to remember what order to put things in, because the two methods follow opposite conventions. And that’s because this type of OO design has no obvious method of organization.
In the first case, I want to do something to a list: join it into a string. So, clearly, I need a list method? But no, I need to engage in some form of indirection; for some reason, I need to reach for a string method. Even if there is no string that I want to use as a delimiter. In that case, I need to use a string method on an empty string.
OK, I’ll play along. Now I want to take the string and split it into an array. Now that I’ve been educated, I know better than to try the sensible thing. Pre-enlightenment, I would have reached for a method applied to the thing that I wanted to transform. But now I know I should think backwards, and use a method applied to the delimiter. OOPs.
Note how they are both functions. The data that they operate on is the first argument, in both cases. The optional second argument is the obvious next most important thing, the delimiter. Other optional arguments come after that. There is nothing to remember, because it makes sense.
You'll have to decide if you find that convincing. I understand your point and kind of wish I hadn't read this thread because I'm more torn than before.
That’s fair Lee. I think I’d accidentally stumbled upon the logic for .join acting on the separator rather than the iterable, but I do agree it’s awkward unless you’re trying to figure out why it is (which is poor design), and certainly not how most people think, especially with the inconsistency of split not acting on the separator.
Named types that can be dispatched on are classes, see CLOS. Hell, even in Python, classes are pretty much the named types that you can dispatch on, although of course, in Python, you generally can't create different kinds of types that can't be dispatched on like types satisfying a predicate - at least to my knowledge you can't, so the difference is somewhat less tangible there.
In Julia they are not classes in the sense that the term is normally used in, for example, Python. Classes bind together data and methods, which is what leads to all the problems. In Julia functions are generic, and specialized to methods acting on arbitrary combinations of types; they are freed from the data.
Not in CLOS, from which Julia's mechanisms are derived (which leads to the question why not CLOS but Python of all things should be used as a source of "normal sense" of anything Julia-related). In CLOS, classes bind together data, generic functions name abstract operations, and methods represent specific code that deals with implementing a generic function for a particular combination of type arguments.
> which is what leads to all the problems
I don't think anyone is disputing that here. There's a reason why CLOS didn't do any of that.
> In Julia functions are generic, and specialized to methods acting on arbitrary combinations of types; they are freed from the data.
...yes, just like in CLOS (unsurprisingly, given Julia's heritage), and those types are effectively CLOS classes.
Well, it seems as if we were just at cross purposes due to terminology.
Although multiple dispatch in CL (and Perl) predates Julia, I was not aware that Julia’s design derived from it. Do you have a reference that traces this?
I find it virtually certain that it does derive from it. Considering that one of the authors of Julia wrote Julia's front-end in Lisp (https://github.com/JuliaLang/julia/blob/master/src/julia-par... and some other files in the same directory), it would have been astonishing for CLOS to not have major impact on the design. There's also some relevant statements in a paper on Julia's design (https://dl.acm.org/doi/10.1145/3276490) in the part on multiple dispatch in section 7. Related Work, where CLOS and its "algebraic cousin" Dylan are mentioned. I got the impression that Julia's object system is basically CLOS without quite a few of CLOS' complexities such as inheritance (which in CLOS necessitates some advanced extension facilities to cover some corner cases if method lookup doesn't do what you want it to do if you're attempting a highly complex application model). The nice effect of those feature removals was that in many cases monomorphization of call sites in emitted native code is possible, which is presumably the other reason for those feature removals: suddenly even primitive operations such as +, * etc. can be generics without incurring (most of the time) dispatch cost at runtime. That (primitive operations being generic functions) is not the case in CLOS, although that can also very well be attributed to backwards compatibility efforts in Common Lisp.
Interestingly enough, in Julia's documentation, the section "Noteworthy Differences from other Languages" (https://docs.julialang.org/en/v1/manual/noteworthy-differenc...) compares Julia to only several relevant languages, which are: Matlab, R, Python, C/C++, and...Common Lisp, of all things. I very strongly doubt that this is a coincidence.
Broadcasting and map are two different operations. If all the inouts have the same shape broadcast is equivalent to map, but in Julia you can also just use map.
The standard way to make a histogram in Julia is
histogram(data)
Using the latest version (1.6 - although 1.6.1 just came out) the time to first plot is just a few seconds. After that, plotting in the REPL is instantaneous.
I probably don’t understand what you’re getting at when you speak of making frequent changes to code. REPL-based development in Julia is excellent, and there are Pluto notebooks as well.
The way I would like to work is to have the repl open on the left hand side of the screen and code editor (like Sublime text, I'm sure many use Vim) on the right hand side.
I would run the code in repl (just using up-arrow and enter), get some plot, modify the code in the editor, save it, and rerun it repl.
Repl is used for connecting inputs to the program, not for editing. Often times you want to develop some small program that is part of a bigger program. So you run the big program up to the point it calls the small program. Or you run with some mock data. Then you have all your inputs for the small program, so you can start iterating it. Run - observe - edit - run - observe - edit.
In Matlab this worked because it detected if the file had changed, and reloaded it. You didn't specifically have to call load("my_function.m") before every call to it.
This means you can just lift code from the big program and iterate on it in repl.
Also, my Julia REPL is in a term buffer in vim, and I communicate with it (in both directions) from my editing buffer using the vim-sendtowindow plugin.
I use python in a scientific context, but have so far not written much extensions for python in any of the languages tested. I'm interested in some guidance which language a) is easy to integrate with python and b) has some good performance, but this benchmark lacks the details to come to any conclusion.
I tried to run the benchmark on my own computer, but the setup documentation was not enough for me to get the julia integration running. I haven't used julia before, so it might just be something very simple.
Similar I haven't used poetry much before, and the given documentation failed to install the necessary setuptools-rust for me. I could fix it on my own, but doesn't make me feel certain about the outcome of the benchmark.
The rust benchmark did not reproduce for me: "Rust (Pyo3) parallel after" has a 1.56 speedup for me, but a 2.6x slowdown for the author. Also I don't understand what the difference between "after" and "before" is, the code just calls the same code twice. Might be a JIT/Cache thing, but it's unclear to me. One sentence what before/after refers to would be very helpful.
Generally all measurements are only done once. Measuring at least thrice gives one at least a chance to detect an outlier and gives possibility for statistics, e.g. is a difference betwee the different cython annotations even meanginful?
The "C Cython (pure-python mode)" is reported faster then "C Cython (.pyx)". The Cython project itself says that using pyx files should be faster, so something strange is going on.
"Cython is fast, but none of these methods are able to release the GIL. " (A) this is not true (B) this seems to be mostly over single threaded performance, so why is that meaningful?
"Rust is not that fast beacuse it needs to copy data; using Pyo3 objects would probably lead to similar results as cython, but with an added library." The rust code already contains Pyo3, so an "added library" is not necessary as far as I understand.
I'd guess the performance stems more from conversions between different types then anything else. Maybe Julia (and the python-julia bridge) is particular smart about it and thus it's super easy to use, while pyo3 (and cython) needs some more work to interface with python. Even if that is true, I couldn't say it from the presented data.
With these caveats resolved I'd be interested in the benchmark, but without it I can't really say anything from it.
About two years ago (before I switched from Python to Julia), I was in the same boat as you. What I concluded was:
1) Calling into an actual static language like C or Rust is the best option. You get maximal performance and all the benefits of the static language. The downside is that you need to learn another language, and manage both languages in your project, including setup and compilation of the static language etc.
2) Cython is easiest for small-scale projects, since it integrates very well with Python, and you can learn it incrementally. But I found it annoying to work with - it felt like half a language that fell between Python and a proper static language. I ended up using Cython in the end, but I wasn't happy with it.
3) Numba looks interesting and promising. At least 2 years ago, it was too brittle and had too many situations where it didn't work or didn't give noticable speedups. I'm sure they improved it since then. I would definitely take a look.
You can always just learn Julia of course and have this entire problem of "my high-level language is too slow" completely disappear ;)
Their .pyx implementation leaves much to be desired. Among other problems, the hot loop uses an untyped python list. Also, they're indexing into the list instead of iterating over the list. In creating the list, they're using .append() instead of list comprehensions.
Fixing those minor issues cuts the runtime in half (see #3). Going for actual high-performance Cython (on my 12-core workstation) cuts the runtime by 20x (see #2).
py2many doesn't care which language is faster as long as the source language is annotated python3.
I don't know if this particular benchmark transpiles correctly or not, but it should be possible to achieve much speedup if you annotate your python code properly.
This is largely a test of cython, but why doesn't it use the cython features comparable to Julia's? Specifically, you can achieve parallelism with prange(..., nogil=True) and no-copy with cython.view.array.
cpdef float iterate_list(a_list):
cdef double count = 0
cdef int i, j
for i in range(len(a_list)):
internal_list = a_list[i]
for j in range(len(internal_list)):
count += internal_list[j]
print(count)
return count
To:
cpdef float iterate_list(list a_list):
cdef double count = 0
cdef double val = 0
cdef list ilist
for ilist in a_list:
for val in ilist:
count += val
print(count)
return count
Speeds up the iterate_list function an order of magnitude. On my PC:
In [9]: %timeit list_cy.iterate_list(a_list)
1000000.0007792843
1000000.0007792843
1000000.0007792843
1000000.0007792843
1000000.0007792843
1000000.0007792843
1000000.0007792843
1000000.0007792843
385 ms ± 6.15 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
In [10]: %timeit list_cyo.iterate_list(a_list)
1000000.0007792843
1000000.0007792843
1000000.0007792843
1000000.0007792843
1000000.0007792843
1000000.0007792843
1000000.0007792843
1000000.0007792843
2.71 s ± 182 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
(yeah, I kept the prints that the code has)
Where list_cy is the fixed code and list_cyo is the original code. Even then, iterating over a list of lists is _definitely not_ the optimal way you'd face a problem of this kind. Numpy arrays and memoryviews would be the correct tool to use.
OP has an open pull request on his repo[0] where someone made basically this same change (different names for the variables, but same idea).
According to OP, he tried this but it resulted in slower execution for him. I'm not sure he really followed what that PR says. In the case of the submitter, it says on their machine it gave a 2x speedup (smaller than my ~7x, but still significant).
If you carefully read OP's response in the PR, he says he tried static typing make_list in addition to iterate_list. Your comment elides the resource allocation step.
Like OP, I observed that static typing make_list yields little benefit. It's runtime is 3x the runtime of iterate_list. And that makes sense. I'm not sure why we'd expect Cython to speed up the allocation of large numbers of Python objects.
But OP didn't type iterate_list either. Perhaps he saw little benefit on one and assumed the same would happen in the other. Cython's annotation mode (cython -a) doesn't really help either here, showing both OPs and optimized versions with a strong yellow color, indicating that they would be about the same, but compiling and trying both definitely shows the improvement.
Another funny thing: making a pure python version that uses the sum() built-in results in code that runs 80% as fast as the cython version, but also gives lower error (e-8 versus e-4 error that gets accumulated when the numbers are summed up one by one).
1000% this. Multiple times I’ve encountered a Julia benchmark claiming to show its superiority in a task I routinely perform. And every time the pro Julia benchmark turned out to be total BS.
I wonder if it's more common for that language than other languages. Probably not many people have enough expertise in many different languages to compare fairly across them.
It's hard to say anything about your use-case without more information, but I will say that one thing is that the way people talk about julia often seems to give people a mistaken impression about how to attain it's performance claims.
Namely, realizing these claims requires learning the language, and actually taking advantage of it's strengths rather than just writing 'python in julia'.
It's very common for Python users to show up on the Discourse forum and complain that julia is slower than python and then show some code that's basically just Python code written in julia, including a huge proliferation of global variables, allocating huge amounts of temporary arrays, etc.
There's also a huge spectrum of 'benchmark quality' out there. E.g. the benchmarks this HN post features seem pretty shitty and are not measuring anything interesting or useful as far as I can tell.
Note that the OP is a Python user, not a Julia user. The Github profile is a bunch of Python packages and the Julia code wasn't even optimized (https://github.com/00sapo/cython_list_test/pull/5). If this test says anything, it at least would say that a inexperienced Python user could pick up Julia and do pretty well, even if the code they write isn't great. I think the right thing to do is just to help this guy learn to code for performance a bit better: it'll be better for him and would bring some positivity.
Even if it doesn't say that, bashing people who use Julia for a repository made by a Python user is a new level of HN trolling.
146 comments
[ 2.7 ms ] story [ 188 ms ] thread> Rust is not that fast because it needs to copy data;
I'm surprised.
Don't know much about Rust, but isn't it hailed as being competitive with C/C++ ?
The Julia implementation replaces the input list with a Julia object, which is of course trivial for Julia to iterate over.
The Rust implementation is asked to convert the list-of-list-of-floats back to Python types, and then back into Rust types. Of course converting 100M floats back and forth is going to take a non-trivial amount of time.
The equivalent in Rust would be to wrap the return value in a PyObject that contains the Rust `Vec<Vec<f64>>`, and the Rust code could then just iterate that directly. It would even be made safe without the GIL as you'd implement your own mutex on the Rust side if you wanted it to be mutable.
If you do this then it runs in about 1.2s on my laptop.
In general, rust is at least as fast as C or C++, but this seems to be a special case.
> Rust is not that fast beacuse it needs to copy data; using Pyo3 objects would probably lead to similar results as cython, but with an added library.
"It needs to copy data", because it's converting Python objects into Rust objects and back again. As the full quote states, it could be written in a different way, although that would make the code look really weird.
In this benchmark there is also no normal C++ involved. There is only Cython using some C++ ints, but operating on Python lists. And that's not handled by writing C++ code that operates on Python structures, but writing Python code in Cython in a "C++ mode".
Apples to oranges.
Not it wouldn't?
Using Pyo3 is arguably the most idiomatic way of interfacing Rust with python. The "added library" cost mentioned as an excuse not to use it is literally the 1 second it takes me to write `cargo add pyo3` .
They didn't do this because it would make their claim that "Julia is the better language for extending Python" kind of moot.
They started from the conclusion that they wanted to have, and worked their way backwards towards implementation and data that would support it.
I'll defer to your knowledge on that. I generally don't work with Python. I've only looked into interfacing C# with Rust and it looked crazy, so that's where I'm coming from.
But yes, it's pretty clear that the results are more a consequence of what's in the source files than the languages used (if we can even say that e.g. C++ was used at all).
AFAIK, there is no library that achieves a similar idiomatic interoperability between Rust and C#, like PyO3 for Python.
C usually comes out a bit worse because it gives even less information to the optimizer.
And thanks to the GNU/LLVM monoculture most languages should perform roughly the same if they have them as a target e.g. I have found that D makes its it ridiculously easy to write highly specialized code that is both readable and visible to the optimizer (I will be blogging about that part later), but I'm sure I could force the same asm out from C++ or Rust etc.
Although as mentioned in the benchmark, with py2o, rust will be a superb contender, especially considering:
- the toolchain is the easiest to setup
- you can embed asm if you really need this extra juice
There is multiprocessing module but you cannot share memory between processes. I guess the best options are either writing a C extension or using Numba. Writing C extensions require either distributing binary packages or C compiler to be present on the target computer which is not always ideal. So is using Numba the best solution currently? I tried it a bit in the past but the errors I got was a bit hard to interpret compared to regular Python errors.
I wish the multithreading module had support for native threads. Is there any PEPs trying to bring easy multi-core support for Python?
This can be an absolute sinkhole for performance, but if you have a large dataset you shouldn't be sharing lines that much one would hope.
Also, if you are running on Intel you might have some luck with Intel's profiling tools as, although I've never used them for python they are by far and away the best things in this area, they do support Python.
I don't know what details they give you in from Python code directly, but if you've used perf to profile code at a low level before, vTune is like a Tom & Jerry shotgun in comparison.
Never seen this before. Thanks!
Enjoy
The implementation may still be a bit buggy though, so use with caution! https://bugs.python.org/issue38119
Same thing in the R ecosystem, it's possible to use multiple cores with the parallel package but there are caveats too.
Also a big downside with JavaScript. Of course both Python and JS are high-level interpreted languages where high-performance use cases aren't the foremost priority.
Can't Node run multiple processes (or multiple workers, not sure of the proper terminology) on multiple cores? As discussed in this thread on StackOverflow: https://stackoverflow.com/questions/61893497/node-js-on-mult...
Of course, but you can also run multiple Python processes.
Modern JS engines are all primarily JIT-based, generally only interpreting code when it's faster than waiting on the JIT. JS JITs are quite good now, and in many cases will produce optimized C-equivalent compiled code from fairly naive JS.
Dask is great if you’re processing large amounts of data, and it recommends and supports threads for this reason.
Not sure there’s anything to prevent you from foot-shooting if you’re operating on the same array. I’d expect that this is only safe to do if, as you say, arrays are not overlapping. Let me know what you find.
The main selling point of the library is automatic differential and compilation to XLA, but I've been using it even when I don't need gradients, as it's really fast (due to compilation). I also really like the random number generator as it's very good for reproducibility.
I've played around with Julia in the past and really liked it, but in terms speed Jax has pretty much solved that problem for me
You may find this interesting https://github.com/scikit-hep/iminuit/blob/develop/tutorial/...
TBF I am mainly using it for mostly pure path functions.
Edit: also note that a big part of the umap library is written in numba.
It takes a bit of getting used to but the performance gains on impressive. Basically, my bottlenecks shift from compute to i/o.
I think you have to balance it against writing in C/C++. Mentally, it is basically the same work as writing in C (you manage memory/you write complicated for-loops) but you have good array support with numpy. The primary advantage for me that everything stays in the python runtime environment. You just run the code without any extra steps.
...
What is missing from the timing type 'toy' benchmarks is an understanding that there is typically more than one bottleneck in a real problem and it is easy to choose the wrong one to optimize and get little gains.
After starting C (30 years ago now), spending a long time in C#, then switching to Python a few years ago, I think the unappreciated advantage of python is that I have to abandon all pretense of caring about speed and just get stuff working. It basically solves the pre-mature optimization problem for me by being a fast interpreted language rather than a slow compiled language.
I feel the same way. With Python I just write the simplest algorithm that first comes to my mind, even though I know that it is not the most optimized way of doing things. But most of the time I am surprised that it works so fast that I realize I actually don't need to optimize it.
And being able to create and easily manipulate dictionaries and tuples also allows me to create efficient data structures very quickly.
With Jax you can write native for loops that can also be jitted (I imagine you can also do this in numba?); this can then be really fast. Though in that case you would have to write the optimisation algorithm yourself which is not always practical!
Another big speedup in Jax is due to vmap/pmap, which allow to vectorise/parallelise computation. For example you can build a massive gram matrix really quickly using vmap.
Another point: Jax can also run on GPU (like numba :) ) without having to rewrite anything.
Yes, you can just pass the function and it runs the optimization, in severe cases, you can start by doing a grid `.scan` or `.simplex` (Nelder-Mead simplex method), then `migrad` to minimize, and `.hesse` for 1 sigma bound.
You can also provide a gradient function that, well computes the gradient instead of computing it numerically.
> I imagine you can also do this in numba
Yes! It compiles to native code.
> Another big speedup in Jax is due to vmap/pmap, which allow to vectorise/parallelise computation.
It's possible to compile some vmapped functions with numba too, if the link is any indication, you may see even greater speedup than just jax.grad
However, I do concur that jax's vmap is absolutely fantastic and I found it very useful on many occasions.
I understand the sentiment, but in this case I wonder if it is less a python issue and more an issue with numpy. I don't see any reason why numpy couldn't execute the last line of the following multi-threaded:
The same thing applies to matrix multiplication. I'm guessing numpy has chosen not to for good reason though.edit: Actually it seems numpy is multi-threaded under certain circumstances:
https://stackoverflow.com/questions/16617973/why-isnt-numpy-...
Distribute your data and run everything as dask.delayed and then compute only at the end.
Also check out legate.numpy from Nvidia which promises to be a drop in numpy replacement that will use all your CPU cores without any tweaks on your part.
https://github.com/nv-legate/legate.numpy
https://github.com/urschrei/simplification
https://github.com/urschrei/lonlat_bng
https://github.com/urschrei/pypolyline
Each of those repos has links to the corresponding Rust “shim” libraries that provide FFIs for dealing with the incoming data, constructing Rust data structures from it, and then transforming it back on the way out.
As a more general comment, using a GC language as the FFI target from a GC language is begging for difficult-if-not-impossible-to-debug crashes down the line.
https://play.rust-lang.org/?version=stable&mode=debug&editio...
Because the data structure LotsOfData doesn't implement the Clone trait, attempting to create a copy will fail at compile-time.
https://run.dlang.io/is/jOXfJz is a D equivalent
When I was interfacing D code with a part of Unreal Engine that I think is garbage collected, I actually just took the L and copied everything into buffers on their malloc when handing stuff off to the engine. It wasn't particularly hot code so the memcpys were worth the peace of mind I found, ugly as it was.
> As a more general comment, using a GC language as the FFI target from a GC language is begging for difficult-if-not-impossible-to-debug crashes down the line.
<s>As for the FUD about interfacing JIT code to C/C++, this is a problem Julia was designed from the outset to tackle. Incidentally, Julia built on the excellent experience LuaJIT has had. A challenge: can you name a particular extension that would interface better from Rust than from Julia?</s>
Oops, I misunderstood the criticism you were hinting at but failing to justify, sorry. OK, Julia already does interface to Python and this interface sees widespread use. If your suspicion is right, then where are the horror stories from people who were bitten when deploying code that built on the interface?
Nobody claimed there was a problem?
> As for the FUD about interfacing JIT code to C/C++
I don't know what you're talking about. Sorry!
Not true!
What you do is that you keep a registry for objects passed from the host vm to the foreign vm in which you register objects thus transferred. And you use a similar mechanism for objects passed from the foreign vm to the host vm. In CPython, you simply increment the refcount to prevent Python from collecting them prematurely.
This is how Java does it (via JNI) and how many gc:ed runtimes interact with other gc:ed runtimes. This is how you do it in Rust too since Rust can't tell how long an object passed to a foreign vm is supposed to last.
It's not that easy. You'd need to register the object as a GC root as long as it's being managed by the foreign GC, and similarly ensure that objects in the foreign VM are properly "de-registered" when the local GC finalizes them. It's far from trivial, particularly when compared with other memory-management strategies.
It's funny the hostility to JIT that some fans of compiled languages have.
Same no copy, slightly more user friendly approach.
Further criticism of the actual approach - even if we didn't do zero copy, there's no preallocation for the vector despite the size being known upfront, and nested vectors are very slow by default.
So you could speed up the entire thing by passing it to ndarray, and then running a single call to sum over the 2D array you'd find at the other end. (https://docs.rs/ndarray/0.15.1/ndarray/struct.ArrayBase.html...)
Safe interop between two GC'd languages (Haskell and Java) was one motivation of Haskell's new -XLinearTypes extension
https://www.tweag.io/blog/2020-02-06-safe-inline-java/
Generally, the new linear types are exciting but are nascent at the moment. But one extension to them (linear constraint [1]) seems to allow embed the equivalent of Rust's ownership in Hsakell using more primitive features in the type system (to my understanding..)
[1] https://arxiv.org/pdf/2103.06127.pdf
I honestly think people are severely underestimating what a massive impact this is currently having in increasing productivity of python devs who know a little C++.
On the other hand: having used pyo3 to integrate rust with python in the past, the biggest pain is simply to reconcile the dynamism of python with AOT compiled rust code. There's a lot of noise at the interface from the large amount of type checking to unpack the specific types of numpy arrays coming across. Do you want to have your rust code work with all sorts of integer bit lengths? That'll be a code path for each. If you have a number of input types, have fun coercing them all.
This can be alleviated with macros to a degree, but that just hides the problem really.
Julia is JIT compiled, which means that there are no problems with types being determined only at run time. In fact, this one fact makes integration at the code level much nicer with Julia.
So all in all it's a tradeoff really. I also found the tooling for Rust to be much more robust and stable (1 year ago admittedly).
It's better and easier to use Julia as your main top-level "glue" language and call Python/Rust/C from Julia. Julia is in many ways a better glue language than Python - better multithreading, easier calling into C/Rust, etc. Then, over time, to the extent it is practical, you can replace foreign code with Julia code - because Julia is fast enough that it actually makes sense to do so.
If you already have a large Python code base and can't switch the top-level language to Julia, I would just not use Julia for that project.
Why would you want to mirror the Julia repo instead of just downloading already-compiled binaries? Do you want to maintain your own fork with your own set of patches?
I went far down this path before infosec agreed to give github / highly respected cdn access.
I understand how can this be inconvenient for me and you while scripting. But this is absolutely no problem for julia's currently main target market who runs multi-GB simulations.
Also it is not like they will be spawning a julia instance in a hot loop (THAT would be horrible). You write some julia code, and import it using smth like pyjulia at the beginning of the file.
After using Ruby for years, returning to Matlab style code in Julia felt somewhat awkward. Instead of my_array.length you have length(my_array). In my personal preference the method call is just a nicer way of doing the same thing. The single-instruction-multiple-data or dot notation sometimes worked and sometimes didn't, so you had to resort to loops anyway.
Ruby and map or each have their verbosity but overall it feels a more robust "hammer" for general programming tasks. On the other hand, Julia can be really dense and still easy to understand.Other languages of course take some of these things even further. Maybe some day I will find an ergonomics and nicety first successor to both.
Plotting with two Y axes or generating histograms in Julia was also way harder than I remember it being in Matlab. Also manually having to load the file before every run to see the changes in action added a lot of overhead to the workflow.
The workflow I was used to in Matlab involved very frequent making changes to code, running of the code that usually made some plot. Which is one command in Matlab, and plotting was fast, in 2001 already, on Windows NT 4. In Julia, you have to first load the modified file, then run it. Plotting takes a long time. One just can't get nearly as productive with it in 2021, compared to Matlab of 2001 vintage.
What language would I pick if I had to do some quick analysis from some tables downloaded from the internet? Probably Julia still. If I had free access to Matlab, I would probably use it though.
Well...they're both method calls, aren't they? (So they're both the nicer way?)
I read the first as the concat symbol applied to an iterable leads to string concated by the concat symbol.
I read the second as an iterable broken by split symbol leads to an iterable of the chunks.
In the first case, I want to do something to a list: join it into a string. So, clearly, I need a list method? But no, I need to engage in some form of indirection; for some reason, I need to reach for a string method. Even if there is no string that I want to use as a delimiter. In that case, I need to use a string method on an empty string.
OK, I’ll play along. Now I want to take the string and split it into an array. Now that I’ve been educated, I know better than to try the sensible thing. Pre-enlightenment, I would have reached for a method applied to the thing that I wanted to transform. But now I know I should think backwards, and use a method applied to the delimiter. OOPs.
You'll have to decide if you find that convincing. I understand your point and kind of wish I hadn't read this thread because I'm more torn than before.
https://arstechnica.com/science/2020/10/the-unreasonable-eff...
Not in CLOS, from which Julia's mechanisms are derived (which leads to the question why not CLOS but Python of all things should be used as a source of "normal sense" of anything Julia-related). In CLOS, classes bind together data, generic functions name abstract operations, and methods represent specific code that deals with implementing a generic function for a particular combination of type arguments.
> which is what leads to all the problems
I don't think anyone is disputing that here. There's a reason why CLOS didn't do any of that.
> In Julia functions are generic, and specialized to methods acting on arbitrary combinations of types; they are freed from the data.
...yes, just like in CLOS (unsurprisingly, given Julia's heritage), and those types are effectively CLOS classes.
Although multiple dispatch in CL (and Perl) predates Julia, I was not aware that Julia’s design derived from it. Do you have a reference that traces this?
Interestingly enough, in Julia's documentation, the section "Noteworthy Differences from other Languages" (https://docs.julialang.org/en/v1/manual/noteworthy-differenc...) compares Julia to only several relevant languages, which are: Matlab, R, Python, C/C++, and...Common Lisp, of all things. I very strongly doubt that this is a coincidence.
This could have easily been avoided if `join` would have been a method of the list object, not the string object. But we're stuck with this now.
Maybe it would be better as a standalone function. But then you'd either have to import it or it would pollute the global namespace.
Other OOP models offer different solutions.
`map(el->el^2, a)`
and inspired by Ruby
``` map(a) do element log(element) end ```
The latter being syntax sugar for the former.
Using the latest version (1.6 - although 1.6.1 just came out) the time to first plot is just a few seconds. After that, plotting in the REPL is instantaneous.
I probably don’t understand what you’re getting at when you speak of making frequent changes to code. REPL-based development in Julia is excellent, and there are Pluto notebooks as well.
I would run the code in repl (just using up-arrow and enter), get some plot, modify the code in the editor, save it, and rerun it repl.
Repl is used for connecting inputs to the program, not for editing. Often times you want to develop some small program that is part of a bigger program. So you run the big program up to the point it calls the small program. Or you run with some mock data. Then you have all your inputs for the small program, so you can start iterating it. Run - observe - edit - run - observe - edit.
In Matlab this worked because it detected if the file had changed, and reloaded it. You didn't specifically have to call load("my_function.m") before every call to it. This means you can just lift code from the big program and iterate on it in repl.
https://docs.julialang.org/en/v1/manual/workflow-tips/#Revis...
I tried to run the benchmark on my own computer, but the setup documentation was not enough for me to get the julia integration running. I haven't used julia before, so it might just be something very simple.
Similar I haven't used poetry much before, and the given documentation failed to install the necessary setuptools-rust for me. I could fix it on my own, but doesn't make me feel certain about the outcome of the benchmark.
The rust benchmark did not reproduce for me: "Rust (Pyo3) parallel after" has a 1.56 speedup for me, but a 2.6x slowdown for the author. Also I don't understand what the difference between "after" and "before" is, the code just calls the same code twice. Might be a JIT/Cache thing, but it's unclear to me. One sentence what before/after refers to would be very helpful.
Generally all measurements are only done once. Measuring at least thrice gives one at least a chance to detect an outlier and gives possibility for statistics, e.g. is a difference betwee the different cython annotations even meanginful?
The "C Cython (pure-python mode)" is reported faster then "C Cython (.pyx)". The Cython project itself says that using pyx files should be faster, so something strange is going on.
"Cython is fast, but none of these methods are able to release the GIL. " (A) this is not true (B) this seems to be mostly over single threaded performance, so why is that meaningful?
"Rust is not that fast beacuse it needs to copy data; using Pyo3 objects would probably lead to similar results as cython, but with an added library." The rust code already contains Pyo3, so an "added library" is not necessary as far as I understand.
I'd guess the performance stems more from conversions between different types then anything else. Maybe Julia (and the python-julia bridge) is particular smart about it and thus it's super easy to use, while pyo3 (and cython) needs some more work to interface with python. Even if that is true, I couldn't say it from the presented data.
With these caveats resolved I'd be interested in the benchmark, but without it I can't really say anything from it.
1) Calling into an actual static language like C or Rust is the best option. You get maximal performance and all the benefits of the static language. The downside is that you need to learn another language, and manage both languages in your project, including setup and compilation of the static language etc.
2) Cython is easiest for small-scale projects, since it integrates very well with Python, and you can learn it incrementally. But I found it annoying to work with - it felt like half a language that fell between Python and a proper static language. I ended up using Cython in the end, but I wasn't happy with it.
3) Numba looks interesting and promising. At least 2 years ago, it was too brittle and had too many situations where it didn't work or didn't give noticable speedups. I'm sure they improved it since then. I would definitely take a look.
You can always just learn Julia of course and have this entire problem of "my high-level language is too slow" completely disappear ;)
Fixing those minor issues cuts the runtime in half (see #3). Going for actual high-performance Cython (on my 12-core workstation) cuts the runtime by 20x (see #2).
https://github.com/00sapo/cython_list_test/pull/2 https://github.com/00sapo/cython_list_test/pull/3
I don't know if this particular benchmark transpiles correctly or not, but it should be possible to achieve much speedup if you annotate your python code properly.
https://github.com/adsharma/py2many
Looking for help with open issues.
produces
https://paste.ubuntu.com/p/vwxYkppRCw/
https://github.com/00sapo/cython_list_test/pull/2
Where list_cy is the fixed code and list_cyo is the original code. Even then, iterating over a list of lists is _definitely not_ the optimal way you'd face a problem of this kind. Numpy arrays and memoryviews would be the correct tool to use.
Will Julia advocates every use honest benchmarks to make their language look good? I doubt it.
According to OP, he tried this but it resulted in slower execution for him. I'm not sure he really followed what that PR says. In the case of the submitter, it says on their machine it gave a 2x speedup (smaller than my ~7x, but still significant).
[0] https://github.com/00sapo/cython_list_test/pull/3
Like OP, I observed that static typing make_list yields little benefit. It's runtime is 3x the runtime of iterate_list. And that makes sense. I'm not sure why we'd expect Cython to speed up the allocation of large numbers of Python objects.
Another funny thing: making a pure python version that uses the sum() built-in results in code that runs 80% as fast as the cython version, but also gives lower error (e-8 versus e-4 error that gets accumulated when the numbers are summed up one by one).
Namely, realizing these claims requires learning the language, and actually taking advantage of it's strengths rather than just writing 'python in julia'.
It's very common for Python users to show up on the Discourse forum and complain that julia is slower than python and then show some code that's basically just Python code written in julia, including a huge proliferation of global variables, allocating huge amounts of temporary arrays, etc.
There's also a huge spectrum of 'benchmark quality' out there. E.g. the benchmarks this HN post features seem pretty shitty and are not measuring anything interesting or useful as far as I can tell.
Even if it doesn't say that, bashing people who use Julia for a repository made by a Python user is a new level of HN trolling.