> When I say production machine learning, I’m referring to machine learning that manifests as a product feature. For example, Uber’s ETA prediction, or Gmail’s Smart Compose.
You can bet that prod services from companies you heard of are running on something more analogous to versioned docker images. Not a yaml file which says, 'Go run whatever predict.py is in the current folder.'
The moment one of your dependencies breaks your code, or snookers your performance, there will be a lot of head scratching going on.
The cortex tool mentioned looks really useful to get a service running out of a trained model. Though I didn't really understand what the article is trying to get at. Storing your deployment configuration in yaml and json files is pretty much the standard.
Perhaps for young people it's actually necessary to mention JSON and YAML as you people tend to read the news rather than history or best practises textbooks?
Many hyped development and administration practices make me snarky, too, but let's not take it out on random (young) people on the Internet. There's a great deal of history and best practices that are being ignored not just by ADHD juniors, but also by team leads and managers of every seniority level. Lots of people out there spend decades in the industry, build solid careers based on playing the right office politics cards and being friends with the right people, and miraculously manage not to learn almost anything.
(Edit: also, clearly, I don't read the news -- is YAML being "superseded"? By what now??)
Ha! The parent might be right about the young people though I'd think the reason for reinventing things is a lot more people writing code and preferring different styles. That is a positive!
I don't think YAML/JSON are being superseded, but I'd really love for something like Cue (https://cuelang.org/) to become the standard for storing configuration.
i hope by something that isn't the kitchen sink... i actually prefer XML and i hate XML.
started looking at https://dhall-lang.org/# which compiles to json/yaml, is seriously strongly typed and explicitly not Turing-complete both as a design goal and current reality.
I read "young people" as "young in their career people". It's truly exasperating to see the wheel reinvented again, and again, and again, and again (I felt like typing that a lot more but I'll stop there :)). There are so many subtle lessions in this field, will there ever be any hope of recording and communicating them? Seems not likely.
The title is seriously misleading. They aren't doing their ML engineering in yaml. If you look at the snipper the article, you can see that their code is in flat .py files. The config is in yaml (which is also how everyone else uses it). It's like someone saying that they do their ML in a dockerfile.
The linked site (towardsdatascience) is kind of like medium blogs. High variability in quality with a lot of self-promoters and the occasional diamond in the rough. But a ton of rough.
It is a Medium blog. Its literally a Medium page, they just rely on other people's contributions which they have editors look over and then aggregate rather than produce their own. I suppose the editing process may help with quality some, but it really shouldn't be seen as much more than a typical blog.
Its quite simple to develop a model, bundle it up for deployment and deploy. Nobody cares about your fancy YAML based containerized deployment and monitoring setup, everyone has that. The challenge comes in when you have a continuous cycle of data ingestion to model optimization, training, evaluation and deployment. Pretty much everybody has huge amount of code duplication in there. It also comes from the fact that ml researchers are barely capable of programming a light switch, like how are you ever gonna put the horrible trash of code they ducked tape together from medium posts into a production environment. Hopeless.
I don't think it's simple to deploy scaleable predictions - that's why model hosting solutions like SageMaker's and GCP's AI Platform exist, and there's no need for people to be re-implementing model deployment/monitering.
For .ipynb notebooks, I highly recommend using nbstripout [0] to strip the Jupiter output before committing the notebooks to the repository (thus making the diffs sane).
You can also set it up as a 'filter', so it automatically runs before any git operations, whether it's add, commit, diff or an interactive rebase.
You can also use ReviewNB [1] that is literally built for Jupyter notebook diffs. You can see notebook visual diffs for any commit or pull request on GitHub. For pull requests, you can also write comments on a notebook cell (emulating the typical code review experience for Jupyter notebooks)
Disclaimer: I built ReviewNB for Jupyter notebook code reviews on GitHub.
So, every month or two I see another article tut-tutting people for putting notebooks into production, and I'm curious, who is actually doing this? I've never seen such a thing in the wild, and I'm genuinely (morbidly?) curious what it would look like in practice.
We (Netflix) do a ton of prototyping/exploration in notebooks like everyone else. We run many ETL pipelines in production as templated notebooks. When something fails, you can just open a notebook to see the input and the output, which is handy.
We don't deploy or execute ML models in production as notebooks. We have many other solutions for that use case. In particular, check out https://metaflow.org
I just completed a project using notebooks in production. Granted, it was as a stepping stone to a more traditional application, but for 3 months, our production system was readily executing notebooks and consuming their output.
Papermill (https://papermill.readthedocs.io/en/latest/) made it extremely easy, to the point where I question the real value of moving away from this model. But software engineers practically hiss when you mention Jupyter because it's too different from the rest of their tooling.
If you use Databricks, it's pretty easy to schedule a Databricks notebook to run. AWS EMR also has notebook support, I haven't used it, but it might be possible to schedule those too. You would still have to grant permission to the Databricks role to write to a production location in S3.
I really dislike this practice. Code that runs in production should be code reviewed and there should be some monitoring in place to make sure the job is working correctly
The topic of production notebooks often shows up. I’ve seen tools like papermill and dagster being used for notebook prod, just like in Netflix.
I concluded that using notebooks for prod is always a tech decision, often influence by a tradeoff: risk for premature optimization (writing scripts early on in the project that may only be used once) and underengineering (using non-maintainable and clunky code to support mission-critical workloads): https://ljvmiranda921.github.io/notebook/2020/03/16/jupyter-...
Reason 1 (Your pipeline should be reproducible) for avoiding jupyter doesn't make any sense. That's the whole point of jupyter. Out of order execution can happen, but you can just as easily restart the kernel and run all... I can only imagine this is a problem for someone who doesn't understand the tool they're using.
Reasons 2 and 3 for avoiding jupyter are more justified but easy enough to work around between jupytext and papermill.
30 comments
[ 2.7 ms ] story [ 66.6 ms ] threadYou can bet that prod services from companies you heard of are running on something more analogous to versioned docker images. Not a yaml file which says, 'Go run whatever predict.py is in the current folder.'
The moment one of your dependencies breaks your code, or snookers your performance, there will be a lot of head scratching going on.
(Edit: also, clearly, I don't read the news -- is YAML being "superseded"? By what now??)
I don't think YAML/JSON are being superseded, but I'd really love for something like Cue (https://cuelang.org/) to become the standard for storing configuration.
i hope by something that isn't the kitchen sink... i actually prefer XML and i hate XML.
started looking at https://dhall-lang.org/# which compiles to json/yaml, is seriously strongly typed and explicitly not Turing-complete both as a design goal and current reality.
You can also set it up as a 'filter', so it automatically runs before any git operations, whether it's add, commit, diff or an interactive rebase.
[0] https://github.com/kynan/nbstripout
You can also use ReviewNB [1] that is literally built for Jupyter notebook diffs. You can see notebook visual diffs for any commit or pull request on GitHub. For pull requests, you can also write comments on a notebook cell (emulating the typical code review experience for Jupyter notebooks)
Disclaimer: I built ReviewNB for Jupyter notebook code reviews on GitHub.
[1] https://www.reviewnb.com/
[0] https://netflixtechblog.com/notebook-innovation-591ee3221233
We don't deploy or execute ML models in production as notebooks. We have many other solutions for that use case. In particular, check out https://metaflow.org
Have you never seen a script in production?
Papermill (https://papermill.readthedocs.io/en/latest/) made it extremely easy, to the point where I question the real value of moving away from this model. But software engineers practically hiss when you mention Jupyter because it's too different from the rest of their tooling.
I really dislike this practice. Code that runs in production should be code reviewed and there should be some monitoring in place to make sure the job is working correctly
The topic of production notebooks often shows up. I’ve seen tools like papermill and dagster being used for notebook prod, just like in Netflix.
I concluded that using notebooks for prod is always a tech decision, often influence by a tradeoff: risk for premature optimization (writing scripts early on in the project that may only be used once) and underengineering (using non-maintainable and clunky code to support mission-critical workloads): https://ljvmiranda921.github.io/notebook/2020/03/16/jupyter-...
Reasons 2 and 3 for avoiding jupyter are more justified but easy enough to work around between jupytext and papermill.