JupyterLab is ready for users (blog.jupyter.org)

As a single piece of anecdata, this is exactly what prevented me from using the free licenses I had through my university to learn these programs. After my experiences with Stata, seeing the non-affiliated prices and having to do my work in a computer lab instead of on my own laptop, I didn't want to be tied to software that I might not have access to in the future, and that I couldn't share with other people.

The one time I made a handsome demo in Mathematica, I realized that there was no straightforward way to share it, so I gave up and redid it in R.

I'm grateful I never spent time learning those programs. I still miss Mathematica now and then, but free software is the only way to go for me nowadays.

I share your sentiment, but in truth, Mathematica's symbolic algebra capabilities are above and beyond what everything else offers by a wide margin.

That's all nice but you go to real-world space/robotics/drones etc. shops and they all use Matlab or similar for their scientific computation and validation and treat Jupyter stack as a toy. Similar to telling a photographer to use Linux because there is GIMP and that should be enough for them.

Mathematica/Python/MATLAB user. Mathematica and MATLAB do both have great documentation and intuitive examples shipped with their product, however, document your own code using their system is surprisingly hard (compared to Sphinx). Jupyter learnt the idea of the notebook from Mathematica (and some other notebook based math software) but the killer point is that it extends itself to more and more general purpose programming language with Python, R, Julia and even C/C++ kernels. MATLAB still popular in the field of control engineering, communication partly because of its SIMULINK simulation environment. I have been looking for a Python alternative but the closest I have found so far is Modelica. In my workflow, Mathematica is used to derive the mathematical formula, Matlab for simulation and finally software is written and distributed as Python package.

A number of years ago I knew a postdoc who bought a student copy of Mathematica at the university bookstore so he could use it as part of a short-term collaboration without dropping a non-trivial chunk of his grants on a full license. Shortly after he installed it, the department chair got a personal email from Stephen Wolfram asking what someone who wasn't a graduate student was doing with a student license.

This supported the upward trend of interest in open source and especially ipython around the labs.

Yours is a good question, and I can't really provide a good answer. Firstly, I suppose I wish for something more akin to an IDE with a proper list of variables, functions, classes, & cetera; tabs; and as the screenshots of this JupyterLab beta release show, a capability for displaying different data types in different self-contained windows that are not in-lines to the notebook document.

I probably expressed myself very poorly.

I agree here. Being able to write code in the mathematical rendering is such a beautiful feature of Mathematica that Jupyter is missing. I am not sure how it's outclassed as a notebook because of this. As an IDE? Sure, Mathematica's notebook isn't a good IDE at all. But as a notebook for writing mathematical statements, it's fantastic. It's the one thing I need in order to go to an only Julia workflow.

I also like Mathematica, and I think the curated data is amazing for impromptu explorations. I guess there are more and more sources of data available online for Jupyter to query, however, the Mathematica API and access to this curated data is effortless.

That's more a restriction of Python, though. It's important to remember that the Jypter Notebook (and now, JupyterLab) is language agnostic. The Julia language, for example, is very well integrated into Jupyter (which, in fact, is a backronym for Julia-Python-R), and Julia is homoiconic.

At http://www.mathematica25.com/ you can see that the notebook front-end was in Mathematica in its first release in 1988.

One of the images from 1987 says "The Mathematica front end begins to take shape…" and "(Theo Gray invents cells and groups … and other things still seen today…)".

Perhaps a dumb question, but do those languages tend to have wrappers for common Python libraries like Pandas?

Oh, I thought this was only for python :)

Nice! That seems like a great way to go about using it.. I'll have to give it a shot for my next project :)

I do the same thing with Emacs + Elpy, I love interactive programming!

I'm still not sold personally—it seems like the in-memory persistence is only useful for the intermediate case where my data is slow enough to generate/obtain that I don't want to run the code every time to do so, but fast enough that I don't mind running it every time I launch the editor. Most of the data I have that's worth caching due to speed is worth caching to disk. Combined with unpredictable side effects of variables persisting whilst I'm actively hacking on the code and implicit in-memory persistence is pretty off-putting.

A recent workflow I've had for a data analysis project is to have each stage of data processing in a separate function, with all the functions called in order from an " if __name__ == '__main__'" block, with all but the function I'm presently working on commented out. Each function returns nothing, but saves its data to an HDF5 file. Other functions read the inputs they need from the HDF5 file and write their outputs to the same file, and if I want a fresh run I just delete the file, uncomment everything in the '__main__' block and run again.

The functions also save output plots to subfolders.

This is compatible with version control, and caching on disk rather than just in memory.

The biggest downside compared to Jupyter notebooks is lack of interactivity in the saved plots (I can make interactive plots pop up of course but they're all in separate windows all at once so it's less clear which part of the code each plot came from), and lack of LaTeX in code comments - I still will have external LaTeX documents explaining what algorithm I'm using somewhere.

So for now, the downsides of notebooks with respect to version control, data caching and extra state that I have to remember in order to not hit subtle bugs in my code as I hack on it, seem to outweigh the upsides.

Maybe what I would like is an editor that renders LaTeX in comments, and which embeds arbitrary plot windows at given points in the code, but without any data persistence, and without the embedded plots actually being saved anywhere - your file is still a normal Python file and it's just the editor rendering things that way based on magic comments or something.

Or maybe I should just write a decorator that renders a function's docstring as LaTeX and embeds any matplotlib windows produced into one scrolling document with the sections named after the decorated functions. Decorator could take an argument telling it whether to include the full source of the function, the comments of which it could also render as LaTeX. Then you have input code compatible with your favourite text editor and version control, and an output document which optionally includes the code.

I'm using Python for lab automation, so my data comes fresh from an experiment. I've found that keeping stuff in memory is extremely convenient until the kernel shuts down for some reason (e.g., I inadvertently kill it while forgetting that I've left a notebook open).

Still, I love having my data collection scripts documented right there with the subsequent analysis. So, I've disciplined myself to handle experimental data in one of two ways:

* For "small" data, format it as a Python thing (list, dict, whatever is appropriate), and paste it into the next cell as an input. I haven't found a way to do this automatically, and I'm careful not to make things too automatic lest I run a cell and over-write old data.

* For "big" data, dump it to a file. I just turn the system time into a filename, to avoid over-writing an old file.

I don't think I've come up with the last word, on using Jupyter as a self data collecting lab notebook, nor am I yet 100% certain that it's even a good idea. This is a work in progress, but much better than anything else I've ever tried. For complicated experiments, I still create stand alone Python programs to control things.

This is the no. 1 reason I use notebooks. I recently worked on a Python library for an undocumented API that returned broken, non-semantic HTML. Counting spans, parsing inline styles - that kind of hell. I honestly don't think I could've done it without Jupyter.

This makes sense, it seems like one of its intended purposes is as a great pedagogical tool which I think would work well.

Thanks for sharing... I'm always excited to see examples of RStudio notebooks in the wild.

I wrote an article a few months ago on the differences between R Notebooks and Jupyter Notebooks (and why, IMO, R Notebooks are better): http://minimaxir.com/2017/06/r-notebooks/

Better than PowerPoint to management in some cases!

It sounds like that is essentially the same thing, yes. It still won't be as good as org-mode, though. With org, when you enter some Python code you press a key and you're editing that block with the best editor for Python (emacs) and when you edit bash code you're using the best editor for bash (emacs). With this, you have to use the same basic editor for everything.

I did try it, but it seemed to break "undo" unfortunately. That's something that will make me stop using a package immediately. I also tried ob-ipython for org-mode but found it not ready unfortunately. I think I will have to do what you suggest and just minimise the amount of code written in the notebook itself.

Thank you! I've been looking for this and didn't even realize. ob-julia is awful; really hope this can be made to work better. Also several obscure languages I've wanted to include snippets of in org that actually have Jupyter kernels...

> Version control is too important to mess with.

Version control is too important to be left to the content-blind tools we typically use for it. In a perfect world, there'd be a core version control engine with content-specific plug-ins.

The notebook server have "contents managers" which decide how notebooks get stored. It is perfectly possible to write what you request, and some users have done it: https://github.com/aaren/notedown here without the outputs but it's easy enough to add.

The other possibility is to export a notebook as an actual files and folder tree: https://github.com/takluyver/nbexplode so rich object (png, svg... etc) are independently editable.

It though can be challenging to have work well because of different filesystems.

You can even go further and tell the server to store nothing on disk but in a database, postgres for example :https://github.com/quantopian/pgcontents

Yeah, you can connect your GPU for machine learning, but that seems to be it.

very little is likely the answer

> There's also an official Google drive extension for jupyterlab worth checking out.

As the readme states, the realtime API it relies on has been deprecated.

Jupyter supports lots of languages (including R).

RStudio is probably still nicer for working R, but I haven't done any serious analysis in R for a while now.

What does "SKU" mean in this context?

But is it better? Worse? Will them thrive together?

Nope! Thanks for the catch!

Yeah, output right in the notebook is important. I'm wanting them to be able to do something like this in a cell:

    for ship in myShip.radarScan():
        if ship.is_enemy:
            display([ship.type, ship.range, ship.bearing])

And then have that cells results always showing the current data as they play the game.

I'm teaching someone to code right now on the Raspberry Pi, and JupyterLab is working far better for us than IDLE or a separate text file and terminal. We use the workflow described in http://jupyterlab.readthedocs.io/en/stable/user/documents_ke.... We open up our python file and a console side-by-side, and pressing shift-enter in the file sends a line or selection to a console. We can easily investigate it in the console outside of the file too.

You can just "pip install jupyterlab" on a Raspberry Pi and it works great!

These criticisms could be valid for Jupyter Notebook, but remember that JupyterLab is what's under discussion in this post. JuptyerLab is much more of an IDE-style than Notebook, and it includes very capable terminal and text editor components.

Although I'm new to Jupyter Lab, I just tried it and it does feel like it could be a very approachable IDE for educational use.

> Now, If you are teaching programming in it self, I think the terminal + text editor is important since that is a step towards how things are done irl.

Jupyter (and similar) Notebooks are a way things are done "in real life"; they aren't just for entertainment purposes; and JupyterLab blends that into an IDE and the IDE with notebooks approach is also a way things have been done "in real life"; both with third-party tools incorporating Jupyter (and earlier IPython) notebooks, and with other IDE's incorporating notebook-style interaction.

Which isn't to say that there aren't arguments for teaching program outside of the notebook environment rather than within it (though the notebook is kind of a super REPL, and REPLs seem to me to be great tools for teaching), but that not using notebooks is better because it is how things are done "in real life" is just taking a highly selective view of real life.

Isn't that what the map on the wall is for?

Thanks! This looks like exactly what I was looking for.

Also, it should be noted that, since it just proxies a single user notebook server, JupyterHub doesn’t support any interaction, sharing, or other collaboration between users (though one could implement some of that by mounting an NFS share inside notebook containers). Still, I’ve found it quite useful for teaching. IIRC, there’s also a project underway to bring collaboration features to Jupyterhub.

But what are jupyter notebooks? Had to Google it to find out. Still not sure what it does.

The fact that the GUI is written in JS is my point. The reports of Python's death... you heard it here first.

As someone new to not just Jupyter (and for someone coming from the Microsoft c# world, mind = blown, many kudos)....do you (or anyone else reading) happen to know off the top of your head a particularly well done, information dense (not dumbed down, showing what you can do, but skipping a hand holding explanation, one can go get that elsewhere) notebook that demonstrates a broad range of Pandas & Jupyter capabilities?