I have been using evcxr for the last 2-3 weeks and I'm loving it. There are two things that bother me: (a) fancy UI components seem to only work with Python and C++ kernel (not theoretically, but with available packages), (b) while you can redefine values and functions when you're iterating, you cannot do so with structs (yes, big structs go to a local package that I depend on, but sometimes I have ad-hoc utility structs).
A `display_data` Jupyter kernel message includes a `data` key with a dict value: "The data dict contains key/value pairs, where the keys are MIME types and the values are the raw data of the representation in that format."https://jupyter-client.readthedocs.io/en/latest/messaging.ht...
Rust hold interesting promise for data analysis and visualization at scale, especially as someone who hits use cases where Python/R start to bottleneck. The ecosystem is not there yet, though.
not to sound like a broken record, but Julia has the performance while also having dynamic memory management and an ecosystem close to Python/R (depending on the area).
Rust is nice when you want something lightweight and fast, since Julia has a pretty heavyweight runtime.
Yes, absolutely. Version 1.5 brought significantly faster precompilation and version 1.6 (currently in beta) brings some precompilation speed improvements on top of that as well as multithreaded precompilation, so things have improved a ton in that regard.
We're expecting more benefits down the pipe as well with better AOT compilation support so you don't have to redo compilation so much between sessions.
Compilation and precompilation latency have been the two most consistent focuses of the developer team over the past couple years.
Version 1.6 (beta) is significantly faster (5-10x on my machine) than 1.5, largely due to code invalidations and parallelism. And 1.5 is faster (in both runtime and compiletime) than earlier versions.
I also imagine future iterations will focus on AOT options as well (we already have some in the form of PackageCompiler.jl),
If anyone's interested in a collection of scientific programming/machine learning libraries written in Rust, we maintain a collection of them across different domains at https://www.arewelearningyet.com/.
That's an impressive amount of hacks under the hood! I thought maybe there would be an interpreter involved, but it really does invoke rustc and then loads the code with dynamic linking.
Despite my aversion to these 'live-coding' environments for compiled languages, I'm actually interested to see what can be managed with Rust. The dynamic linking they are using sort of reminds me of rust-analyzer which quickly re-parses it's tree at every save. I don't code in a way that normally requires interpretative feedback but quality tools like evcxr might contribute to changing my habits.
This is amazing! But I still posit there are better UI paradigms than Jupyter (and REPL). Working in Jupyter is like having one hand tied behind my back.
Not OP, but I like to avoid Jupyter where I can too. I like using a REPL though.
For very interactive, experimental work I like having code in one pane and a REPL in another separate one - so that you can highlight and send snippets of a file to the REPL with a keystroke or two, and the code and results aren't mixed together.
RStudio or Vim Slime[1] would be my favourite examples of this kind of approach.
For more "static" literate programming, I prefer the way RMarkdown and Sweave work to Jupyter Notebooks (though I don't even use R anymore). Both have their strengths and weaknesses of course.
I added jupyter notebook support to my nsi[1] crate via evcxr.
It allows you to do a 3D (offline) render with 3Delight (on AWS, if you want) – directly into a notebook.
Here is an example notebook[2].
I haven't managed to update the image dynamically while it's rendering (there's no support for that in evcxr). If anyone knows how to do this, I'd be keen to add that too.
I really loved the Rails console for developing web apps. Write a few lines on the REPL, then past them in an app/model file.
Knowing Rust (and other strong typed langs) I understand that it is not so easy to implement this workflow for these languages. But looking at this project I kind of have new hopes. With the strong languages it should even be possible to have intellisense-like completion/suggestion available on the REPL!
Would it be, eventually, possible use Evcxr for this purpose?
I used this about a year ago for a small astrophysics simulation+plotting etc, together with futhark for some GPU code. It wasn't perfect (some crashes and weird behaviour), but worked remarkably well. I imagine it's even better today.
It was really useful to have easy acces to the simulation code and datastructures from the notebook where I made the plots, and use the same language for both. Glad to see the project still going strong.
29 comments
[ 0.22 ms ] story [ 70.3 ms ] threadDo any of the other non-Python Jupyter kernels have examples of working fancy UI components? https://github.com/jupyter/jupyter/wiki/Jupyter-kernels
Jupyter kernels implement the Jupyter kernel message spec. Introspection, Completion: https://jupyter-client.readthedocs.io/en/latest/messaging.ht...
Debugging (w/ DAP: Debug Adapter Protocol) https://jupyter-client.readthedocs.io/en/latest/messaging.ht...
A `display_data` Jupyter kernel message includes a `data` key with a dict value: "The data dict contains key/value pairs, where the keys are MIME types and the values are the raw data of the representation in that format." https://jupyter-client.readthedocs.io/en/latest/messaging.ht...
This looks like it does something with MIME bundles: https://github.com/jupyter-xeus/xeus-cling/blob/00b1fa69d17b...
ipython.display: https://github.com/ipython/ipython/blob/master/IPython/displ...
ipython.core.display: https://github.com/ipython/ipython/blob/master/IPython/core/...
ipython.lib.display: https://github.com/ipython/ipython/blob/master/IPython/lib/d...
You can also run Jupyter kernels in a shell with jupyter/jupyter_console:
Rust hold interesting promise for data analysis and visualization at scale, especially as someone who hits use cases where Python/R start to bottleneck. The ecosystem is not there yet, though.
Rust is nice when you want something lightweight and fast, since Julia has a pretty heavyweight runtime.
We're expecting more benefits down the pipe as well with better AOT compilation support so you don't have to redo compilation so much between sessions.
Compilation and precompilation latency have been the two most consistent focuses of the developer team over the past couple years.
I also imagine future iterations will focus on AOT options as well (we already have some in the form of PackageCompiler.jl),
The data science ecosystem isn’t quite there yet, but you can still do some things easily if you want to.
If you’re interested, you may want to check out my book on the topic https://datacrayon.com/shop/product/data-analysis-with-rust-...
That's an impressive amount of hacks under the hood! I thought maybe there would be an interpreter involved, but it really does invoke rustc and then loads the code with dynamic linking.
For very interactive, experimental work I like having code in one pane and a REPL in another separate one - so that you can highlight and send snippets of a file to the REPL with a keystroke or two, and the code and results aren't mixed together.
RStudio or Vim Slime[1] would be my favourite examples of this kind of approach.
For more "static" literate programming, I prefer the way RMarkdown and Sweave work to Jupyter Notebooks (though I don't even use R anymore). Both have their strengths and weaknesses of course.
[1] https://github.com/jpalardy/vim-slime
It allows you to do a 3D (offline) render with 3Delight (on AWS, if you want) – directly into a notebook.
Here is an example notebook[2].
I haven't managed to update the image dynamically while it's rendering (there's no support for that in evcxr). If anyone knows how to do this, I'd be keen to add that too.
[1] https://crates.io/crates/nsi
[2] https://github.com/virtualritz/nsi/blob/master/examples/jupy...
If you streaming in blocks on the server side and compositing them into a final render, in this case there are a couple hacks.
1) just use a timer and reload the image every x seconds
2) do a hanging read on and endpoint, return updated image. This has the nice quality of the server being able to decide the framerate.
I was wondering if sending it as an MNG with loop=1 would work.
https://ipython.readthedocs.io/en/stable/config/integrating....
You might be able to do everything in there with a little JS.
Knowing Rust (and other strong typed langs) I understand that it is not so easy to implement this workflow for these languages. But looking at this project I kind of have new hopes. With the strong languages it should even be possible to have intellisense-like completion/suggestion available on the REPL!
Would it be, eventually, possible use Evcxr for this purpose?
It was really useful to have easy acces to the simulation code and datastructures from the notebook where I made the plots, and use the same language for both. Glad to see the project still going strong.