I assume it works much better in a live presentation, where he is talking while running the slides. A 10-20 page deck or 3 page paper would have communicated this more readily as a stand-alone.
They only work in that capacity if you already watch a lot of memes, so they effectively act as a shibboleth (in-group out-group separator) for separating people who spend a lot of time on reddit from those that don't.
They still help a lot to keep the audience alerted as long as they are funny or surprising without meta information. Adding "Confused Nick Young" to underline, well, how confusing some features are, will still be understood perfectly well by those who see it the first time. On the other hand, memes like "Scumbag Steve" [2] are indeed problematic, because there is no way to understand the point the author wants to make without prior knowledge.
I see how it can be annoying; for me it's only mildly entertaining. However from a structural point of view I have the feeling that it's it's fine if only slightly obscured (how I am, what I don't like about notebooks point 1, example, point 2,...). At least at the end there's some summary slides that states his main points/gripes (how you could win me over)
They aren't fun, don't have anything to do with technical content and just put me off, specially if the presenter thinks s/he could have a 2nd job as stand up comedian.
Most public presentations / speeches contain a few jokes. They don't have much to do with the technical content either, but they help people pay attention and ease in the speech.
It's not like we don't have enough boring powerpoint presentations already. It's also not like those going into the trouble of compiling the slides and sharing their knowledge owe us anything...
Well, others don't compartmentalize things so tightly.
Most would welcome one or more jokes in a developer speech. So much so, that it's common advice for any kind of public speaking and presentation to add a few jokes to lighten the mood (you can find thousands of articles, books, and public speaking training sessions advising about this).
Some of the best technical speakers add humor in their presentations (often lots of it, e.g. Raymond Hettinger).
Agree with most of the things in this presentation. BUT.
I generally use Jupyter Notebooks for these things:
- Play around with some libs and charts
- Draw charts for my papers or whatever
- In short: draft stuff
And, for serious stuff, of course, python in text files with modules, tests (pytest) etc.
Had a project once where the "super helpful notebook" ended up being nicknamed "The Wall of Madness". Tons of weird out of order errors, #DONT RUN BELOW THIS LINE WITHOUT TALKING TO JIM, etc.
When I used notebooks the first time my mind automatically assumed the cells would work as a sort of data-flow variables and auto-update when a previous one changed.
Maybe a "this are potentially out of date" marker on out of order cells would help with the problem.
I'll be honest and admit that I have barely used a Notebook, but I can see they have their place. They are very popular with scientists, where they are writing experimental code, much of which will be thrown away. For stuff that needs to be used in production then they seem like the wrong tool.
I had actually thought that they would be great for learning until I read this.
(I am a bit disappointed that the presentation doens't actually explain the reason for out of sequence execution, which seems to be one of the central complaints).
Fascinating presentation! Joel is such a meme factory :-) Has anyone done a "rejected memes" section at the end of a technical talk before?
On topic: I applaud the effort to set things straight, but I'm afraid that ship has sailed. Hacky "Kaggle notebook solutions" are now Data Science. That's what the term evolved to mean, and all the rest of the impedance highlighted in this presentation follows.
If your work involves creating well-designed, well-factored, tested, reusable ML software that is meant to be integrated, picked apart, extended and applied over time in practice (as opposed to be submitted once to an oracle in order to claim "SOTA"), you better come up with another term. And obviously Notebooks are not a good fit, beyond as a tool for documentation & reporting (which Joel correctly calls out as genuinely useful).
-They're excellent for beginner tutorials that explain an abstract concept or a library's use cases
-They're very well suited for academic peer-review. It's basically a way to say: here's what we obtained, here are the exact steps we did to obtain it, you can obtain it as well if you do exactly as we did. You'd think this reproducibility requirement would be common sense but in practice you're usually thankful if the data is available, or the software is released and works.
When I first found notebooks I was amazed at their utility. The ability to quickly display algorithms and processes to a wider (potentially layman) audience is amazing.
To be able to have a scratchpad and play around with techniques while keeping a sort-of record is also amazing.
I see it like I see excel, it's a fantastic tool for data exploration and some visualisation. It's not something that should be used in any final workflow or production system.
Like excel, it can be badly mis-used but they unlock ways of working that simply weren't possible before it.
The criticisms about hidden state are fair, I think it would be better if previous step data was more explicitly wrapped in the following cells so you could choose to use the "wrapped package" of the previous data or choose to use a re-evaluating version.
I think it works best when most cells have few side-effects, even if that means repeating previous calculations.
Ironic this pops up today when a colleague has this scenario happen...
BA: We found this duplicate field error in our database, do you know about this.
DS: I told you about this 2 years ago.
BA: Oh so you have a script to produce all the duplicate records?
DS: No I did this 2 years ago I don't know what version of Python or any of the libraries I used for the notebook.
In my experience a lot of data scientists like this work alone on business problems - if they were in a more collaborative environment they'd probably run into the problems that Joel is discussing a lot more and be more open to build libraries.
Hmm, agreed. But before you share a notebook, restart kernel and rerun all cells, fix all errors, and then share... Solves many of his gripes.
I use notebooks all the time, but I put repetitive code in a module, try do put all functions at the start etc. It is much easier to share data science experiments, but indeed notebooks can be used and presented in horrible ways.
So... now we write a presentation on how we can horribly abuse VSCode + an iPython console and call it even?
Although I am a huge fan of Jupyter, Joel made some serious points there.
First, we should think about embedding used package versions in our notebook to enhance usability.
I love notebooks, but I agree with you. I like to use notebooks to instrument and debug modular models. Top down design, which is what I was taught years ago when I was learning C/C++ seems harmful when paired with out of order execution.
I agree. Found the "I know a few things" spiel distasteful, and the general tone to be patronizing and rude. Attitudes likes this are negatively impactful and points for or against notebooks can be made without the toxicity.
It's mostly a critical mass problem, as far as I see it. If everyone had a different more positive tone, the one bitter one would stick out (and mostly people wouldn't even think about choosing that tone in the first place). But since that's what a lot of people do, that becomes the energy-minimizing approach to take. I suppose the best thing one can do is to have the courage to go against the stream ("vote with your feet") and find a way to make your presentation optimistic and happy (e.g. Julia Evans), despite the fact that you know there will be bitter people who judge you as cringy.
How is this bitterly angry? It's light-hearted, almost irreverent, witty, respectful to the community and understanding of the fact that it's being critical and might not be well-received, so makes efforts to mitigate that.
Jupyter notebooks were fun to use for a bit, then I hit the inevitable wall of "ok, now let's turn this into a real, properly built script, but now everything is breaking for inexplicable reasons".
Notebooks are fine for early stage experimenting, but if you've got to the point where you start up you are relying on a notebook for anything, or your workflow consists of "start up notebook, run cells until I get to the one I'm working on" then you need to stop, and build it into something proper.
Experimentation is fine, but it should not come at the cost of writing things properly when the time comes, and they are not an excuse for not knowing some good software engineering. Serious alarm bells go off in my head when I read tweets like "Data science code doesn't need to follow the rules of good software engineering".
Edit: that remote notebook - is that honestly not one of the most terrifying things you've seen? That runs so counter to almost every good bit of software design and engineering practice about clarity, maintainability, good practice, security, etc that I can think of. It's the very definition of indecipherable, inscrutable hidden state and unknown side effects.
I wrote a loader for this. And there are tools to split cells for compiling. I wrote a tool to split up cells too and compile in order. Pretty easy, all the code is in notebooks.
I don't even like it for initial experimentation and exploration. Much prefer the command line repl (the jupyter/ipython one). Faster feedback and easier navigation then you can just export the history when you're ready to capture and make permanent some workflow. It still requires going through and picking out the important bits, but reading the history like a story of commands is generally enough to pick out the important parts.
The only thing I would use notebooks for is demonstration / teaching.
If you work with image processing, notebooks are very handy as they can display images directly. Also to display tables nicely with pandas, or for any data visualization actually.
That's my problem. I agree that notebooks create a troublesome dev situation, but "%matplotlib inline" and stuff like the pivot table plugin keep me crawling back.
I mean Jupyter doesn't really do that either, right? You can't edit most function definitions. Only ones that you've defined right there in the notebook, and only because you can run the cell that defines them again, overwriting the function.
But if you do use libraries instead of just a huge mess of notebooks, you're stuck if you want to change the code in a well-supported way. You can ask your notebook to monkey-patch the code (which is an even bigger mess), or you can use an unreliable magic extension, "%autoreload".
One-off "experiments" become production technology all the time. They address an immediate need at the cost of tomorrow. This hasn't ever changed. Hyperbolic discounting is real.
When you're in a job that requires running experiments, analyzing results, and summarizing findings, all within an hour, shortcuts are required.
I think that analysts/scientists using python and Jupyter notebooks are doing IT a favor. If your job is to refactor/re-engineer python notebooks and corresponding workflows, you've either forgotten how much worse it could be with excel+vba or you fortunately arrived after trench warfare.
The code written by a couple of "data scientists" I was working with is the worst code I have ever seen. They don't care, they just want to have an experimental results. The problem starts when their experimental "code" needs to be used on production or they are asked to describe how it works. Why cannot we just get good programmers and train them as data scientists?
I work in data science (came in via a maths background) but I agree 1000%. My software engineering is pretty self taught, but I make an effort to follow best coding practices at all times.
That said, I think it goes both ways: people from the sciences tend to be cavalier coders, and people from software background tend to be cavalier about the underlying mathematics.
Seems to me that the solution needs to be a stronger culture of both increased scientific and software engineering rigor.
So there is hope for you :) You care, you will learn it. There is nothing wrong in not knowing something.
My main point is that too many people in data science don't care at all. They don't care about the repeatable results, about the code quality, even about units (they can even use `mb`, `Mb` and `MB` for megabytes in the same document).
I hope companies will learn that it's really important that if you have code it should be a good quality code, not a randomly gathered set of lines.
Why cannot we just get good programmers and train them as data scientists?
For the same reason we can't just get good programmers and train then in biology or chemistry or structural engineering. Sure they exist, as do data scientists that are really good programmers, it's just that they're more rare and in very high demand.
Often much easier to find a domain expert and a programmer and have the programmer rework the code done by the domain expert. In fact that used to be my job for a while (working with physicists), and it was actually quite fun.
sounds like this is a process issue! EG, why data scientists and software engineers are both needed, and how they can work together to produce quality code which will produce quality data analysis
> Often much easier to find a domain expert and a programmer and have the programmer rework the code done by the domain expert. In fact that used to be my job for a while (working with physicists), and it was actually quite fun.
It's basically what people are now calling 'Research Software Engineers'
I've worked with a few people who are software engineers -> data scientists. They were great at bringing good coding / database practices into the team. That said, their lack of formal statistics training was definitely a problem from time to time, and they seemed to show as much disregard for it as data scientists have for engineering practices (at least the ones talked about on this thread). This can be equally damaging.
Dealing with code (including tests and versioning) is on the same level as knowing the basic math notation. Should be embarrassing to not to apply the practices.
The only problem is that these good coding practices aren't that exact, and tend to go on and on all the way to infinity.
I agree, many (most?) data scientists spend most of their time coding, it's ridiculous how little time they're willing to hone this skill.
Data science practices are similarly inexact - a lot of good decision making comes from experience, knowing when to apply each tool to a specific problem, when to just throw in a hack etc.
I only use notebooks in R (via knitr). Unless you choose to cache specific blocks by flagging it, it re-runs the whole notebook from scratch every time... So personally I haven't had any of these problems before. Further, the rendering is static, so you can't really choose to run things out of order either.
If you use notebooks inside R-Studio (like I imagine 99% would do) you click ctr+enter inside a code block and that block and only that block is executed, and below, the output of that block is rendered.
I absolutely love Jupyter as a computer science student who uses it a lot to fiddle with new concepts, create visualizations, or write markdown reports with annotated code. I do agree with some points, though.
The problems with state felt sort of overstated (heehee), but it obviously is an issue. I always wonder why the menu option Kernel -> Restart & Run All is not a first class citizen, with a big red button at the top of the window. It's my main way of interacting with the notebook for anything that takes under a minute to run. Running cells one by one throughout the netbook isn't very useful. You either want to rerun the cell you're working on, or clear the state just in case and run everything.
Module versioning is another thing that should be a no-brainer.
I also wish the notebooks diffed better when it comes to SCM, without all the JSON artifacts.
Proper integration with existing Python tools/IDEs would be wonderful.
Also yeah, kernel sharing is one of the most horrific ideas I've heard recently.
> I also wish the notebooks diffed better when it comes to SCM, without all the JSON artifacts.
This is one of my biggest qualms as well. I built a DevOps tool [1] that uses Notebooks & not being able to diff and review Notebooks was a pain. I have decided to solve it with a GitHub marketplace app. Wrote more about it here: https://medium.freecodecamp.org/how-to-handle-version-contro...
I love the notebook concept, it is the best way to show to other people pieces of working code and their results.
The jupyter implementation of notebooks has two serious problems, however:
1. The notebook source is not natively stored as a text file that you can easily edit with a text editor. You are forced to use the javascript interface.
2. The editor component has a lot of strange quirks to the point of being unusable.
Notebooks were not created as a way to implement and organize software. You can do that with code files.
Jupyter Notebook is a presentation software, for demonstrating something to yourself or others. There is hardly anything comparable to build interactive demonstrations. You'd have to implement a (multi-paged?) GUI application, or a web application. Just plain html output may cut it for certain use cases, but still more painful.
And I've done more stupid stuff than that. For example I have a demo recording and showing EEG data. And a snake game implemented as a widget with Bokeh output. And live-update graphs from FlightGear. Lately I've started experimenting with controlling my 3D printer from the notebook.
Exactly - it's a classic case of a tool being used for a different purpose ("IDE") than its original intention ("presentation software") and getting criticized for not holding up to misguided expectations.
His criticism is valid but the main lesson is to use the appropriate tool for the job. He himself uses VSCode + iPython, which is a better tool more generally for the desired use case of code development.
Yeah notebooks are fantastic tool to show your works to others.
But the problem OP staes is that notebooks encourages unreproducible demonstrations. Markdown + code cells makes data scientists to think a single notebook file is enough to upload to GitHub. Missing dependancy information and etc as OP states make others very hard to reproduce the notebooks. You can see tons of GitHub repos that have notebooks only and impossible to reproduce out of the box.
> Notebooks were not created as a way to implement and organize software. You can do that with code files.
It doesn't have to be an either/or. The granddaddy of the modern Notebook, Literate Programming [0] _was_ about implementing and organizing software. Just because today's Notebooks like Jupyter aren't currently sufficient to implement and organize software doesn't mean that they cannot be (again) in the future.
A lot of the problems pointed out in the slides are solvable. The DevOps of Notebooks is stuff we can absolutely sink our teeth into as an industry. We can make good Notebook formats that source control well. (Some folks have filters already for Jupyter.) We can make better bridges to (incrementally, per user interest) move Notebooks into source control, CI, testing, etc. Jupyter should already have some idea of the environment it is running in, it could certainly build things like requirements.txt or even full Docker containers. We can build beyond the single cell or single Notebook page and ask deeper questions about how do we organize Notebooks, how do we organize software in Notebooks, how do we interoperate with maybe some code that has a strong narrative to live in a Notebook alongside code that doesn't have a strong narrative or doesn't need one (or wasn't written with one in mind and is legacy code in the project). It could be great to take an existing Python codebase and say "this feature is best explained in a Notebook" and just build it that way. Similarly it could be great to say "this Notebook I found is already a great module, I'm going to build a more traditional app around it" and getting the Notebook's own help in bootstrapping that effort.
We certainly have the technology and the opportunities to do interesting software development in/with/alongside Notebooks and Notebook-like tools. The questions are certainly more ones of what are our priorities? Knuth argued in the 80s that all software development was best when embedded in a human-focused narrative. I'm not that extreme myself, but I certainly see some great opportunities for pragmatic middle grounds where you can mix-and-match as opportunity/interest/need warrants. Having Notebooks as a tool in software development _can_ make us better software developers. It's cool to have a lot of tools in your toolbelt so you can pick and choose the best ones for the jobs at hand.
Some of this is interactive development vs static development. A lot of the IDE features and stuff you get in "static" development, you could imagine getting into notebooks one day. The thing I don't know any good way to deal with is cached state during development.
If I have to get data that takes 15 minutes to query, another 10 to preprocess, and then maybe a few more steps before it's ready to put into my algorithm and start poking around, notebooks shine. In an IDE you could (and I often do) use have cached datasets partway through the process saved to disk, as mentioned in OP. But that's a hack too. And it still takes a minute or so to load and process anyways if cached.
So the options are
1) Shitty practices, but I can write try new changes on your data immediately.
2) Better practices, wrapped in caching hacks, but I have to wait minutes between every single change.
I feel like I haven’t really dealt with this out of order execution madness this guy and some others are talking about here, usually I end up with like 2-5 big cells that I use, typically just a “init everything cell” a “run simulation/training cell”, and then some plotting cells and other “utility cells” that I use to poke at things.
I’ve been moving functionality to modules when I can too which helps minimize the amount of code actually in the notebooks, and I also will break up code into different notebooks (occasionally saving/loading specific variables between kernels) when it makes sense to. Maybe all this is helping a lot, have you all needed notebooks with dozens of cells as this presentation mentions?
I love notebooks! They are an excellent tool, if used judiciously.
They are really practical for situations where you want to play around (er even outright work with) principally not with code, but that code's output.
You see, I give lots of trainings.
Notebooks offer me an excellent way to mix commands, their output, and explanations into a single document with little effort.
I'm able to show my students exactly what happens (including the literal messages), going step-by-step.
They are wonderful to create exercises.
However, for my use case, the notebook is the output (perhaps rendered as PDF).
Rules I've adopted for my own training notebooks:
* the first lines are to print the versions of all things I'm using, e.g. "git --version" for git trainings
* I use "restart&run all" frequently
* obviously, notebooks are version-controlled, including their output
* before checking in, prove that "restart&run all" provides exactly the desired result
Having said all that, I'd never use a notebook to write actual programs. It feels weirdly impractical, to the point that I was wondering of the presentation was actually presenting reality, or a strawman (I'm not doubting the veracity of the description, I just had a hard time accepting it as real).
111 comments
[ 3.5 ms ] story [ 171 ms ] threadAlso they are lighthearted and fun and life is short so why not have some fun even when being serious?
------
They aren't fun, don't have anything to do with technical content and just put me off, specially if the presenter thinks s/he could have a 2nd job as stand up comedian.
That's subjective.
>don't have anything to do with technical content
Most public presentations / speeches contain a few jokes. They don't have much to do with the technical content either, but they help people pay attention and ease in the speech.
It's not like we don't have enough boring powerpoint presentations already. It's also not like those going into the trouble of compiling the slides and sharing their knowledge owe us anything...
Most would welcome one or more jokes in a developer speech. So much so, that it's common advice for any kind of public speaking and presentation to add a few jokes to lighten the mood (you can find thousands of articles, books, and public speaking training sessions advising about this).
Some of the best technical speakers add humor in their presentations (often lots of it, e.g. Raymond Hettinger).
And, for serious stuff, of course, python in text files with modules, tests (pytest) etc.
Maybe a "this are potentially out of date" marker on out of order cells would help with the problem.
I had actually thought that they would be great for learning until I read this.
(I am a bit disappointed that the presentation doens't actually explain the reason for out of sequence execution, which seems to be one of the central complaints).
On topic: I applaud the effort to set things straight, but I'm afraid that ship has sailed. Hacky "Kaggle notebook solutions" are now Data Science. That's what the term evolved to mean, and all the rest of the impedance highlighted in this presentation follows.
If your work involves creating well-designed, well-factored, tested, reusable ML software that is meant to be integrated, picked apart, extended and applied over time in practice (as opposed to be submitted once to an oracle in order to claim "SOTA"), you better come up with another term. And obviously Notebooks are not a good fit, beyond as a tool for documentation & reporting (which Joel correctly calls out as genuinely useful).
I'll add positives notes, though:
-They're excellent for beginner tutorials that explain an abstract concept or a library's use cases
-They're very well suited for academic peer-review. It's basically a way to say: here's what we obtained, here are the exact steps we did to obtain it, you can obtain it as well if you do exactly as we did. You'd think this reproducibility requirement would be common sense but in practice you're usually thankful if the data is available, or the software is released and works.
To be able to have a scratchpad and play around with techniques while keeping a sort-of record is also amazing.
I see it like I see excel, it's a fantastic tool for data exploration and some visualisation. It's not something that should be used in any final workflow or production system.
Like excel, it can be badly mis-used but they unlock ways of working that simply weren't possible before it.
The criticisms about hidden state are fair, I think it would be better if previous step data was more explicitly wrapped in the following cells so you could choose to use the "wrapped package" of the previous data or choose to use a re-evaluating version.
I think it works best when most cells have few side-effects, even if that means repeating previous calculations.
BA: We found this duplicate field error in our database, do you know about this.
DS: I told you about this 2 years ago.
BA: Oh so you have a script to produce all the duplicate records?
DS: No I did this 2 years ago I don't know what version of Python or any of the libraries I used for the notebook.
In my experience a lot of data scientists like this work alone on business problems - if they were in a more collaborative environment they'd probably run into the problems that Joel is discussing a lot more and be more open to build libraries.
I use notebooks all the time, but I put repetitive code in a module, try do put all functions at the start etc. It is much easier to share data science experiments, but indeed notebooks can be used and presented in horrible ways.
So... now we write a presentation on how we can horribly abuse VSCode + an iPython console and call it even?
And if he hadn't included it, all the comments would be "yeah, and who is he to talk".
Jupyter notebooks were fun to use for a bit, then I hit the inevitable wall of "ok, now let's turn this into a real, properly built script, but now everything is breaking for inexplicable reasons".
Notebooks are fine for early stage experimenting, but if you've got to the point where you start up you are relying on a notebook for anything, or your workflow consists of "start up notebook, run cells until I get to the one I'm working on" then you need to stop, and build it into something proper.
Experimentation is fine, but it should not come at the cost of writing things properly when the time comes, and they are not an excuse for not knowing some good software engineering. Serious alarm bells go off in my head when I read tweets like "Data science code doesn't need to follow the rules of good software engineering".
Edit: that remote notebook - is that honestly not one of the most terrifying things you've seen? That runs so counter to almost every good bit of software design and engineering practice about clarity, maintainability, good practice, security, etc that I can think of. It's the very definition of indecipherable, inscrutable hidden state and unknown side effects.
My JavaScript version I just put a simple wrapper around my print-results call to check if it is in module mode or notebook mode.
The only thing I would use notebooks for is demonstration / teaching.
But if you do use libraries instead of just a huge mess of notebooks, you're stuck if you want to change the code in a well-supported way. You can ask your notebook to monkey-patch the code (which is an even bigger mess), or you can use an unreliable magic extension, "%autoreload".
When you're in a job that requires running experiments, analyzing results, and summarizing findings, all within an hour, shortcuts are required.
I think that analysts/scientists using python and Jupyter notebooks are doing IT a favor. If your job is to refactor/re-engineer python notebooks and corresponding workflows, you've either forgotten how much worse it could be with excel+vba or you fortunately arrived after trench warfare.
That said, I think it goes both ways: people from the sciences tend to be cavalier coders, and people from software background tend to be cavalier about the underlying mathematics.
Seems to me that the solution needs to be a stronger culture of both increased scientific and software engineering rigor.
My main point is that too many people in data science don't care at all. They don't care about the repeatable results, about the code quality, even about units (they can even use `mb`, `Mb` and `MB` for megabytes in the same document).
I hope companies will learn that it's really important that if you have code it should be a good quality code, not a randomly gathered set of lines.
For the same reason we can't just get good programmers and train then in biology or chemistry or structural engineering. Sure they exist, as do data scientists that are really good programmers, it's just that they're more rare and in very high demand.
Often much easier to find a domain expert and a programmer and have the programmer rework the code done by the domain expert. In fact that used to be my job for a while (working with physicists), and it was actually quite fun.
It's basically what people are now calling 'Research Software Engineers'
The only problem is that these good coding practices aren't that exact, and tend to go on and on all the way to infinity.
Data science practices are similarly inexact - a lot of good decision making comes from experience, knowing when to apply each tool to a specific problem, when to just throw in a hack etc.
Emacs + ESS all the way .
The problems with state felt sort of overstated (heehee), but it obviously is an issue. I always wonder why the menu option Kernel -> Restart & Run All is not a first class citizen, with a big red button at the top of the window. It's my main way of interacting with the notebook for anything that takes under a minute to run. Running cells one by one throughout the netbook isn't very useful. You either want to rerun the cell you're working on, or clear the state just in case and run everything.
Module versioning is another thing that should be a no-brainer.
I also wish the notebooks diffed better when it comes to SCM, without all the JSON artifacts.
Proper integration with existing Python tools/IDEs would be wonderful.
Also yeah, kernel sharing is one of the most horrific ideas I've heard recently.
This is one of my biggest qualms as well. I built a DevOps tool [1] that uses Notebooks & not being able to diff and review Notebooks was a pain. I have decided to solve it with a GitHub marketplace app. Wrote more about it here: https://medium.freecodecamp.org/how-to-handle-version-contro...
[1] https://nurtch.com
[0] https://www.nurtch.com/
The jupyter implementation of notebooks has two serious problems, however:
1. The notebook source is not natively stored as a text file that you can easily edit with a text editor. You are forced to use the javascript interface.
2. The editor component has a lot of strange quirks to the point of being unusable.
Jupyter Notebook is a presentation software, for demonstrating something to yourself or others. There is hardly anything comparable to build interactive demonstrations. You'd have to implement a (multi-paged?) GUI application, or a web application. Just plain html output may cut it for certain use cases, but still more painful.
And I've done more stupid stuff than that. For example I have a demo recording and showing EEG data. And a snake game implemented as a widget with Bokeh output. And live-update graphs from FlightGear. Lately I've started experimenting with controlling my 3D printer from the notebook.
His criticism is valid but the main lesson is to use the appropriate tool for the job. He himself uses VSCode + iPython, which is a better tool more generally for the desired use case of code development.
But the problem OP staes is that notebooks encourages unreproducible demonstrations. Markdown + code cells makes data scientists to think a single notebook file is enough to upload to GitHub. Missing dependancy information and etc as OP states make others very hard to reproduce the notebooks. You can see tons of GitHub repos that have notebooks only and impossible to reproduce out of the box.
It doesn't have to be an either/or. The granddaddy of the modern Notebook, Literate Programming [0] _was_ about implementing and organizing software. Just because today's Notebooks like Jupyter aren't currently sufficient to implement and organize software doesn't mean that they cannot be (again) in the future.
A lot of the problems pointed out in the slides are solvable. The DevOps of Notebooks is stuff we can absolutely sink our teeth into as an industry. We can make good Notebook formats that source control well. (Some folks have filters already for Jupyter.) We can make better bridges to (incrementally, per user interest) move Notebooks into source control, CI, testing, etc. Jupyter should already have some idea of the environment it is running in, it could certainly build things like requirements.txt or even full Docker containers. We can build beyond the single cell or single Notebook page and ask deeper questions about how do we organize Notebooks, how do we organize software in Notebooks, how do we interoperate with maybe some code that has a strong narrative to live in a Notebook alongside code that doesn't have a strong narrative or doesn't need one (or wasn't written with one in mind and is legacy code in the project). It could be great to take an existing Python codebase and say "this feature is best explained in a Notebook" and just build it that way. Similarly it could be great to say "this Notebook I found is already a great module, I'm going to build a more traditional app around it" and getting the Notebook's own help in bootstrapping that effort.
We certainly have the technology and the opportunities to do interesting software development in/with/alongside Notebooks and Notebook-like tools. The questions are certainly more ones of what are our priorities? Knuth argued in the 80s that all software development was best when embedded in a human-focused narrative. I'm not that extreme myself, but I certainly see some great opportunities for pragmatic middle grounds where you can mix-and-match as opportunity/interest/need warrants. Having Notebooks as a tool in software development _can_ make us better software developers. It's cool to have a lot of tools in your toolbelt so you can pick and choose the best ones for the jobs at hand.
[0] https://en.wikipedia.org/wiki/Literate_programming
If I have to get data that takes 15 minutes to query, another 10 to preprocess, and then maybe a few more steps before it's ready to put into my algorithm and start poking around, notebooks shine. In an IDE you could (and I often do) use have cached datasets partway through the process saved to disk, as mentioned in OP. But that's a hack too. And it still takes a minute or so to load and process anyways if cached.
So the options are
1) Shitty practices, but I can write try new changes on your data immediately.
2) Better practices, wrapped in caching hacks, but I have to wait minutes between every single change.
I’ve been moving functionality to modules when I can too which helps minimize the amount of code actually in the notebooks, and I also will break up code into different notebooks (occasionally saving/loading specific variables between kernels) when it makes sense to. Maybe all this is helping a lot, have you all needed notebooks with dozens of cells as this presentation mentions?
They are really practical for situations where you want to play around (er even outright work with) principally not with code, but that code's output.
You see, I give lots of trainings.
Notebooks offer me an excellent way to mix commands, their output, and explanations into a single document with little effort. I'm able to show my students exactly what happens (including the literal messages), going step-by-step.
They are wonderful to create exercises.
However, for my use case, the notebook is the output (perhaps rendered as PDF).
Rules I've adopted for my own training notebooks:
Having said all that, I'd never use a notebook to write actual programs. It feels weirdly impractical, to the point that I was wondering of the presentation was actually presenting reality, or a strawman (I'm not doubting the veracity of the description, I just had a hard time accepting it as real).The tool is just a detail. My point was rather that they are very much "code", and deserve to be treated as carefully as any other.
My notebooks also get refactored on occasion.