Even with data flow extension (also like ipyflow [0] ) I am still struggling with the execution model of notebooks in general. I often still see people defining functions and classes in notebooks to somehow handle prototyping loops.
I would love to see DAGs like in SSA form of compilers, that also supports loop operators. However, IMHO also the notebook interface needs to adjust for that (cell indentation ?). However, the strength of notebooks rather shows in document authoring like quarto, which IMHO mostly contradicts more complex controll flow.
> You have to be very disciplined to make a Jupyter notebook that is actually reproducible
This seems not necessarily very hard to me? All you have to do is keep yourself honest by actually trying to reproduce the results of the notebook when you're done:
1. Copy the notebook
2. Run from first cell in the copy
3. Check that the results are the same
4. If not the same, debug and repeat
What makes it hard is when the feedback loop is slow because the data is big. But not all data is big!
Another thing that might make it hard is if your execution is so chaotic that debugging is impossible because what you did and what you think you did bear no resemblance. But personally I wouldn't define rising above that state as incredible discipline. For people who suffer from that issue, I think the best help would be a command history similar to that provided by RStudio.
All that said, Marimo seems great and I agree notebooks are dangerous if their results are trusted equally as fully explicit processing pipelines.
Personally, I had good experience with marimos so far. Reactive execution, variable deduplication, clear business logic vs UI elements logic separation that is forced on you is good. It retrains ppl to write slightly better structured Python code which is a win in my eyes.
There are a lot of these tools to somehow "fix the reproducibility crisis of notebooks".
Yet from my experience, you quickly learn to "restart kernel and run all" before sharing to make sure everything is good.
All but the most novice users get caught by the "out of order cells" trap, and those will
1) not use anything that adds complexity, because by definition they are novices
2) fall in any other trap on their way because anyway that's how you learn
Thus, IMHO, these flow tools are only seen as useful by _real devs with savior syndrome_, pushing dev solution to exploratory research users, and that will never catch on.
I've been using marimo since January pretty heavily, I absolutely love it and would recommend it to anyone.
I run it with uv and --sandboxed which makes it much easy to share notebooks with teammates and not have to worry about limiting dependencies. Any issues I've had were were Python libraries themselves (specifically graphviz).
I really like how much easier it is to reason about interactive components vs Jupyter. The mo.ui.altair_chart method has got me to migrate off of matplotlib because charts can be fully integrated – as you can see in the demo being able to lasso data points or scrub a chart and analyze specific time periods is awesome.
One thing which I don't like about reactive notebooks is that you have to be much more mindful of expensive and long running calculations. There are feature to help, like adding a run button, but often I end up just disabling auto-run which does reduce the value of the reactive flow. For those use cases I don't find myself using marimo over Jupyter.
I think the entire marimo team deserves a shoutout, the quality of the software is excellent, they've moved very quickly, and they have been very receptive to issues and feature suggestions.
As an outsider to the whole notebook ecosystem, I am absolutely gobsmacked that the representation of the notebook makes it possible to have out-of-date intermediate results. Haven't they been around for like 10+ years?
This is one of those things that is blindingly obvious to people in adjacent sectors of the industry, but maybe there just hasn't been enough cross-pollination of ideas in that direction (or in either direction).
For as long as there have been Jupyter notebooks, there have been people saying "Wait, what??" But there have been way more people using them, and finding them pretty effective despite their inherent lack of soundness.
I personally fall into both camps--both horrified at the execution model but also, it's not actually caused problems for me as it's second nature to restart the kernel when changing upstream cells.
That said, marimo feels like a significant step forward and I'm rooting for them to succeed.
11 comments
[ 3.0 ms ] story [ 34.2 ms ] threadI would love to see DAGs like in SSA form of compilers, that also supports loop operators. However, IMHO also the notebook interface needs to adjust for that (cell indentation ?). However, the strength of notebooks rather shows in document authoring like quarto, which IMHO mostly contradicts more complex controll flow.
[0] https://github.com/ipyflow/ipyflow
This seems not necessarily very hard to me? All you have to do is keep yourself honest by actually trying to reproduce the results of the notebook when you're done:
1. Copy the notebook
2. Run from first cell in the copy
3. Check that the results are the same
4. If not the same, debug and repeat
What makes it hard is when the feedback loop is slow because the data is big. But not all data is big!
Another thing that might make it hard is if your execution is so chaotic that debugging is impossible because what you did and what you think you did bear no resemblance. But personally I wouldn't define rising above that state as incredible discipline. For people who suffer from that issue, I think the best help would be a command history similar to that provided by RStudio.
All that said, Marimo seems great and I agree notebooks are dangerous if their results are trusted equally as fully explicit processing pipelines.
Yet from my experience, you quickly learn to "restart kernel and run all" before sharing to make sure everything is good.
All but the most novice users get caught by the "out of order cells" trap, and those will
1) not use anything that adds complexity, because by definition they are novices
2) fall in any other trap on their way because anyway that's how you learn
Thus, IMHO, these flow tools are only seen as useful by _real devs with savior syndrome_, pushing dev solution to exploratory research users, and that will never catch on.
I run it with uv and --sandboxed which makes it much easy to share notebooks with teammates and not have to worry about limiting dependencies. Any issues I've had were were Python libraries themselves (specifically graphviz).
I really like how much easier it is to reason about interactive components vs Jupyter. The mo.ui.altair_chart method has got me to migrate off of matplotlib because charts can be fully integrated – as you can see in the demo being able to lasso data points or scrub a chart and analyze specific time periods is awesome.
One thing which I don't like about reactive notebooks is that you have to be much more mindful of expensive and long running calculations. There are feature to help, like adding a run button, but often I end up just disabling auto-run which does reduce the value of the reactive flow. For those use cases I don't find myself using marimo over Jupyter.
I think the entire marimo team deserves a shoutout, the quality of the software is excellent, they've moved very quickly, and they have been very receptive to issues and feature suggestions.
This is one of those things that is blindingly obvious to people in adjacent sectors of the industry, but maybe there just hasn't been enough cross-pollination of ideas in that direction (or in either direction).
I personally fall into both camps--both horrified at the execution model but also, it's not actually caused problems for me as it's second nature to restart the kernel when changing upstream cells.
That said, marimo feels like a significant step forward and I'm rooting for them to succeed.