We do something similar for our teaching materials. All lessons are in Jupyter notebooks, as well as cells with inline unit tests, marked with the special comment `# HIDE CELL`.
A build script then runs the full notebook along with tests, and strips out cells (or pieces of a cell) which contain these magic comments.
Admittedly this only captures a piece of what's discussed in the article. Perhaps in the future will construct our videos directly from the notebooks as well, with other special comments containing the relevant pieces of script as well as where the camera should be centered. Then we can generate the notebooks and the visuals automatically, and just send an email to the voiceover person to re-record any script changes.
My slides and books are all "executable". I write them in restructured text and have tooling to either execute the code or build the output (in pdf, epub, mobi, slides).
(I do a lot of corporate training in DS/ML and Python)
I've got an example of doing this in Jupyter. The repo is here [0]
I generate the repo using a combination of Jupytext to convert the completed notebooks (with the labs done) to python scripts and code to remove the homework/labs from the Python script and then go back to notebooks. (Python/text is easier than working with JSON/notebook format.)
When I teach Python I don't use Jupyter. I use unittest code. The labs are contained in the tests and if the students follow directions, the tests will pass and won't complain. They can use any editor they want, and then just run the tests to validate.
I've thought about building tooling (in Python) to automate code creation in both editors (for Python) and Jupyter (for data science) like you suggest. I have a repo sitting around my computer where I started automating the browser but ... there is not enough time in the day.
I've even thought about teaching via Twitter (this is kind of wierd and might make a fun conference gimmick). But I also have tooling to take md-like content consisting of prose, images, and code, and automate posting to Twitter. Then I could hit a button and instead of going to the next slide, I post the next tweet.
You might consider R Markdown and knitr, which have supported Python and many other languages besides R for a long time, but the templates are stored as plain text rather than the terrible .ipynb format, and render to both html and pdf and support LaTeX.
The idea is that for didactic resources (be it documents, tutorials or even small pieces of software), your prose and code are both included in a MarkDown file and fenced code blocks are used to designate creation, altering and deletion of files. Other files that get created when "running" the MarkRight document are artifacts (the source code, the compiled executables, graphics etc.), but the document is a single source of truth.
I recommend still keeping those in version control for online viewing in GitHub. I have not built anything cool with MarkRight yet, but I think the idea has potential. Curious what others think.
We built a C++ project-based course [0] that uses exactly the same ideas.
The core of our approach is that every code snippet is extracted from a fully working codebase and every code iteration is tracked in git.
We unit-test each commit, which means that every tutorial step works. Also, making a change in an early code iteration requires a proper git rebase, with the change propagating through history without conflicts.
We also annotate the code to extract snippets, and then run scripts to autogenerate git diffs for those snippets. The scripts also output the unannotated version of the codebase, which we share with the course participants.
We are thinking about writing a blog post about this if there's interest!
Semi related... the 3d sculpting software ZBrush used to have interactive demos within the software powered by its own scripting language. Very helpful. Placed the help exactly where it was needed.
I’m building https://codeamigo.dev which only works with JS/TS atm but the goal is to support many more languages. Love the idea of executable tutorials written by the community.
I've been using Jupyter Book [0] at work to put together a cookbook for a market data analysis package, and am really liking it so far. I'm not a big fan of Jupyter's GUI editors for notebooks (and all the usual issues with .ipynb's under version control), but Jupytext is great and works quite well with Jupyter Book.
24 comments
[ 3.1 ms ] story [ 58.8 ms ] threadA build script then runs the full notebook along with tests, and strips out cells (or pieces of a cell) which contain these magic comments.
Admittedly this only captures a piece of what's discussed in the article. Perhaps in the future will construct our videos directly from the notebooks as well, with other special comments containing the relevant pieces of script as well as where the camera should be centered. Then we can generate the notebooks and the visuals automatically, and just send an email to the voiceover person to re-record any script changes.
Very nice ideas in this article, thanks!
Are the Jupyter notebooks that you refer to available publicly?
One piece of my work in this area: https://glench.github.io/fuzzyset.js/ui/
My slides and books are all "executable". I write them in restructured text and have tooling to either execute the code or build the output (in pdf, epub, mobi, slides).
I've got an example of doing this in Jupyter. The repo is here [0]
I generate the repo using a combination of Jupytext to convert the completed notebooks (with the labs done) to python scripts and code to remove the homework/labs from the Python script and then go back to notebooks. (Python/text is easier than working with JSON/notebook format.)
When I teach Python I don't use Jupyter. I use unittest code. The labs are contained in the tests and if the students follow directions, the tests will pass and won't complain. They can use any editor they want, and then just run the tests to validate.
I've thought about building tooling (in Python) to automate code creation in both editors (for Python) and Jupyter (for data science) like you suggest. I have a repo sitting around my computer where I started automating the browser but ... there is not enough time in the day.
I've even thought about teaching via Twitter (this is kind of wierd and might make a fun conference gimmick). But I also have tooling to take md-like content consisting of prose, images, and code, and automate posting to Twitter. Then I could hit a button and instead of going to the next slide, I post the next tweet.
Would be curious to see what you build.
0 - https://resources.oreilly.com/binderhub/beginning-supervised...
The developer's book shows how to do it: https://bookdown.org/yihui/rmarkdown-cookbook/eng-python.htm...
The idea is that for didactic resources (be it documents, tutorials or even small pieces of software), your prose and code are both included in a MarkDown file and fenced code blocks are used to designate creation, altering and deletion of files. Other files that get created when "running" the MarkRight document are artifacts (the source code, the compiled executables, graphics etc.), but the document is a single source of truth.
Examples: https://github.com/TomasHubelbauer/markright/edit/master/exa...
I recommend still keeping those in version control for online viewing in GitHub. I have not built anything cool with MarkRight yet, but I think the idea has potential. Curious what others think.
But so do the OP and many of the other projects being posted here.
0: https://rmarkdown.rstudio.com/
1: https://mirror.las.iastate.edu/tex-archive/info/knuth/webman...
The core of our approach is that every code snippet is extracted from a fully working codebase and every code iteration is tracked in git.
We unit-test each commit, which means that every tutorial step works. Also, making a change in an early code iteration requires a proper git rebase, with the change propagating through history without conflicts.
We also annotate the code to extract snippets, and then run scripts to autogenerate git diffs for those snippets. The scripts also output the unannotated version of the codebase, which we share with the course participants.
We are thinking about writing a blog post about this if there's interest!
0: https://learncppthroughprojects.com
0: https://jupyterbook.org/intro.html