I would agree, most data scientists are not required to deploy their own models, probably only for those in startups or small companies. I also introduce how Docker can be used for development. I think it is a big enough trend in CS that any data scientist should know some basics, similar to data science from the command line.
It seems like the baseline of DS skill sets is all over the place, depending on where you started off.
I would have thought most DS wouldn't be interested in learning docker just to deploy. But then again, I could be wrong. What made you write the post? Was it because you saw lots of DS wanting to deploy, or something else?
i'm not a data scientist, but i have several on my team. In my experience,(as your question suggests) the interest is quite low, particularly given how much knowledge is required to do so that would otherwise be out of scope
building the model and packaging it for deployment are two completely different sets of tasks, are they not?
much more useful, again in my experience, is a tool like vagrant, which allows a production environment to be provisioned onto a VM; said VM is then run from the data scientist's desktop (eg, so they can do their work with the same versions of the same tools available in prod)
I agree this is a big gap in the curricula; "engineering chops" (managing servers, environments, etc.) is what I've seen be the biggest force multiplier for data scientists (esp. at non-huge orgs), not how good at stats they are or how fancy a model they can build.
Which orgs have you seen this as the biggest force multiplier?
From the sibling comments in this thread, it seems like there are some agree, and others don't. I was curious if there's a segment that does, and who they might be.
Biased sample for sure, but me and my friends/colleagues in the midwest at companies with DS teams of <10 people, often <5. Companies/teams far too small to separate the builders from the shippers.
I also practice data science from a more engineering angle than many which further colors my opinions. Building models is fun, but I also enjoy deployment and maintenance, so I don't _want_ to hand that part off.
At my last job, my (Data Science) team all deployed production code including our models. We were using Scala, which made it fairly easy since we could essentially just build fat JARs and upload them.
In contrast, I've found trying to do the same thing with Python nightmarish - simple things like having Pandas .20 on one machine and .21 on another will result in silent changes in how some metrics are calculated.
Anaconda mostly, but not entirely, solves that problem with its superior environment management (significantly better than pip IME.) But given that it's imperfect, I'll probably look more into Docker soon.
How did you manage the dependency versioning problem? Did you set up automated tests for that? Or did you just make sure everyone installed the same version with a requirements.txt file?
In theory, pip lets you do this with pip freeze, but I found that pip often broke on some finicky libraries, while Anaconda failed much less frequently (though still has occasional issues.) The main drawback is the same OS requirement – this makes it harder to e.g. develop on a mac and then launch a pipeline on an Ubuntu machine in AWS.
Anaconda has two environment-copying tools, one is environment.yml (which is analogous to pip's requirements.txt), and the other is the "explicit" spec file, which actually points to specific files for specific packages. I've found environment.yml to fail with finicky libraries, same as requirements.txt, while the explicit spec file almost always works.
Many data scientists want to operationalize their models. If you don't work at a hip startup with a killer dev ops team, then you are likely at a big enterprise shop where relying on anyone else to handle your infrastructure for you is a great way to have 90% of your work never see the light of day.
I use Docker daily for my needs. It ensures that I can easily prototype a scalable set of services on my local machine, and then deploy it to a multi-node environment on AWS, Gcloud, or our on-prem infrastructure. Good luck developing a Redis-backed ML app without Docker when it comes time for deployment. You'll get it done, but it'll be a lot slower and more difficult than if you had used Docker or containers of some sort.
Huh, that's surprising. I would have thought that working at a big enterprise shop, you'd have enough dev ops engineers to help you deploy your model, and it's the small startups that can't afford to do that.
I set up Kubernetes to deploy Docker containers, and it took a little while to wrap my head around it. What's the painful part of doing the deploy with Docker like you've been doing?
deployment is not the only reason. distributed ml (not deepmind scale but we're still talking about several instances, usually for something like hyperparameter tuning) is fairly common in practice. without docker, id waste most of my time provisioning and getting things to work - even if it's just for myself!
I'd think that Pachyderm [0] would be mentioned in the article, but it wasn't. It uses Docker under the hood for the data pipelines. I've always wanted an excuse to do some data science...
Thanks for the plug wyattjoh, I'm one of the founders of Pachyderm so I'd like to expand on what you said just a bit to clarify.
Pachyderm does use Docker under the hood, but we don't obfuscate it away, so Data Scientists get the full power of Docker (and most of the power of Kubernetes) at their fingertips. This means you can easily grab prefabricated environments such as Jupyter or Tensorflow container images and deploy them directly. Docker is so good for packaging environments we didn't want to conceal it.
On the other hand, we felt the data orchestration capabilities of Docker were pretty lacking for Data Science use cases, so that's where we've focused our energy. Volumes are a good basic tool for getting data to your code, but they're a pretty blunt instrument. How do you split data up to parallelize over it? How do you make sure you've got the right version of data? How do you schedule new computations when data becomes available? Those are some of the use cases we solve with our distributed file system PFS.
+1 for pachyderm; my team and I are using it to build backtesting platforms for financial modeling, and it's been really nice to use. Also great in combination with Gitlab's built-in CI/CD stuff. Product aside, we've appreciated the team's availability on both Slack and Github too.
I spent a lot of time learning docker and k8s, thinking it would be really useful. Interest from the rest of the team - is nearly zero. Actual VMs work great, I'm thinking its not really necessary except for if you need scalable/serverless applications.
As someone dabbling in ML for the first time, setting up the correct Python env has been a nightmare just to run some sample code. I am familiar with Docker and would love if this got more mainstream adoption.
Fighting with dependencies sucks when you aren’t intimately familiar with the ecosystem: python, node and other js, and even go.
I haven't worked in ML, but I've used Python extensively for scientific computing. Setting up the environment used to be a headache, but Anaconda solved that issue. Are ML tools not available in Anaconda?
I know you said "dabbling" but it seems like Anaconda is recommended by nearly every guide to ML with Python. What did you stumble across that recommended something else?
Check out Yhat. They were a Ycombinator alumni, and they offer that service. They were purchased by an enterprise software company called Alteryx, due entirely to the technical excellence of their product.
What you need is the conda package manager (from anaconda python), in my opinion. Docker is very powerful, but is useful for other purposes (e.g deploying code) than simply grabbing a bunch of python modules and running analyses in a notebook.
I'm asking as someone who is mid level Python & DS experienced and out of real curiosity (not a "mine is better than yours" thought): Could you elaborate what in your opinion makes anaconda superior? I am frequently doing data munging & low level ML things and - so far - I am more happy with the pip side of things as a combination of virtualenv, autoenv [1], pip tools [2], pyup [3] and the rest of the eco system.
Standard procedure, takes two minutes and I have a Jupyter notebook up, smartly automated base processes for dependency management, full control of environment variables and can deploy/ integrate this in any other Python setup (if I were to port IPython code to pure Python).
Is this more a thing of each his own or am I missing a crucial advantage of Anaconda?
Ease of use. I run outreach events and when you have sporadic internet access, being able to give new programmers an exe to install Python with a load of useful packages is a godsend. Also, it takes a lot of the pain out of setting up a Python environment on Windows when you need funny packages that require compilation (e.g OpenCV).
I've been to enough software carpentry talks where people nod and smile, and then promptly ignore the "how to be a responsible developer" advice. They don't care. They just want to run Python and plot some stuff or classify some data without worrying about multiple CLI tools. People seem to accept Anaconda because it comes across as a monolith and it's easier to understand for new programmers than virtualenvs and worrying about pinning pip installs.
Have a look at pipenv - [https://docs.pipenv.org/], it provides deterministic builds and sane package management for python and is the officially recommended Python packaging tool now. If dependency management is your major problem, Docker might be an overkill.
P.S. -- Even if you are on Windows, whole SciPy ecosystem can be easily installed with pipenv, no need of conda for that now.
This looks pretty, but I've never personally had trouble with `pip freeze > requirements.txt`
What benefits does pipenv have beyond storing the hashes as well? In what way is it more sane? For example, does it get around the issues with, e.g., matplotlib being installed in a virtualenv?
Pipenv is only as good as the wheels that get installed. And the wheels are only as good as the assumption that your runtime/deployment environment matches the buildtime environment of the person who made those wheels.
Oftentimes with ML and compute-intensive workloads, performance is an aspect of correctness. It is nontrivial to get the right accelerated libraries, and the right GPU libraries linking against the right Numpy version for the particular notebook you're running.
If you are doing work just for yourself, or if you have total control over the deployment environment (and your collaborator's environments), then you might be able to get away with just using pip for these things.
Agreed :) when linking with native libraries conda definitely helps a lot. I was merely trying to point to the fact that after the recent release of SciPy we finally have binary wheels for Windows for the whole SciPy ecosystem.
Does anybody have some good patterns for bundling dependencies for analytics in Docker Containers, but then handling the execution of the analytics on Spark clusters? There seems to be various permutations of this notion, but I've heard that most of them have various issues or don't work quite as expected.
Personally, I don't think sealing dependencies to certain versions is the best idea, while Docker can be a very valuable addition to a data scientist's toolbox. Reproducibility, just like integration, deployment, and delivery, should be a continuous process.
For data scientists who write workflows as R Markdown documents and want to containerize them, you might want to check out our R package liftr: https://liftr.me/.
I would recommend looking at Singularity as well: http://singularity.lbl.gov/. Docker is not supported in the HPC world but Singularity is showing rapid adoption
It really depends on what you are trying to do with it, for example we have around 15 different integration test configurations that run every night for which VMs may better suited as we want to test installation and deployments automatically for 3 distributions (6 different versions ex. Ubuntu 14.04, ubuntu 16.04, centos 6, centos 7 etc..) and the last 2 windows server. The good thing is that they reproduce the customer environment.
But they have large down sides as well which slow us down. They are a pain to maintain as they are somewhat undocumented (you make a poc for 1 and management always wants more without improvments), a lot of edge cases cause issues which are tough to reproduce (locally sometimes impossible and waste a lot of time) and it takes them a while to start, run, etc.
This is not too tragic for nightly tests as we get the results in the morning but for tests which are started every hour, you do not want to wait that long to verify your changes work/didnt break anything. You can do these in stages, where you create different images based on the result of a previous job (run basic tests that cover base functionality that should always work, then run more in depth tests, then run performance tests at the end to ensure no significant degradation was introduced, etc..) and send out notifications asap in case of failure. The Dockerfile is essentially the documentation as you can see what is installed/configured. You can run everything locally just as it would in a k8 env. which for some reason every one always struggles with.
I am sure there are also edge cases with Docker that are a pain as well but the other selling points show it may be the right direction. You just havevto find use cases and evaluate them.
I'm not sure what that has to do with Singularity?
I know that the HPC clusters I've used in the past few years have all supported Singularity, but none have supported Docker (aside from our small lab cluster). Many HPC admins are (understandably) hesitant to allow non-admins access to start Docker containers (requiring root), but Singularity has no such user permission issues -- and it's faster than initializing a full VM to run a job. I don't expect that to change so long as starting a container requires root-effective permissions.
I suspect that many data scientists will be in a similar situation w.r.t HPC clusters (except for those that are using custom clouds like Seven Bridges).
Sorry for the mix up, I was replying to rb808's post using the HN app on my tablet, no idea what happened :(.
Regarding HPC, from what I remember they usually have old kernels which are running (2.6) for compatibility reasons where Docker usually is not supported (unless it is backported like in RHEL).
SageMaker has three parts. Hosted notebooks, similar to case #2. API access to ML algorithms which are "optimized" and deployment by providing API access to trained models.
56 comments
[ 3.3 ms ] story [ 112 ms ] threadI would have thought most DS wouldn't be interested in learning docker just to deploy. But then again, I could be wrong. What made you write the post? Was it because you saw lots of DS wanting to deploy, or something else?
building the model and packaging it for deployment are two completely different sets of tasks, are they not?
much more useful, again in my experience, is a tool like vagrant, which allows a production environment to be provisioned onto a VM; said VM is then run from the data scientist's desktop (eg, so they can do their work with the same versions of the same tools available in prod)
From the sibling comments in this thread, it seems like there are some agree, and others don't. I was curious if there's a segment that does, and who they might be.
I also practice data science from a more engineering angle than many which further colors my opinions. Building models is fun, but I also enjoy deployment and maintenance, so I don't _want_ to hand that part off.
In contrast, I've found trying to do the same thing with Python nightmarish - simple things like having Pandas .20 on one machine and .21 on another will result in silent changes in how some metrics are calculated.
Anaconda mostly, but not entirely, solves that problem with its superior environment management (significantly better than pip IME.) But given that it's imperfect, I'll probably look more into Docker soon.
In theory, pip lets you do this with pip freeze, but I found that pip often broke on some finicky libraries, while Anaconda failed much less frequently (though still has occasional issues.) The main drawback is the same OS requirement – this makes it harder to e.g. develop on a mac and then launch a pipeline on an Ubuntu machine in AWS.
I use Docker daily for my needs. It ensures that I can easily prototype a scalable set of services on my local machine, and then deploy it to a multi-node environment on AWS, Gcloud, or our on-prem infrastructure. Good luck developing a Redis-backed ML app without Docker when it comes time for deployment. You'll get it done, but it'll be a lot slower and more difficult than if you had used Docker or containers of some sort.
Edit: Make sure to check out Yhat. https://www.yhat.com/
I set up Kubernetes to deploy Docker containers, and it took a little while to wrap my head around it. What's the painful part of doing the deploy with Docker like you've been doing?
[0]: https://www.pachyderm.io/
[0]: http://singularity.lbl.gov/
Pachyderm does use Docker under the hood, but we don't obfuscate it away, so Data Scientists get the full power of Docker (and most of the power of Kubernetes) at their fingertips. This means you can easily grab prefabricated environments such as Jupyter or Tensorflow container images and deploy them directly. Docker is so good for packaging environments we didn't want to conceal it.
On the other hand, we felt the data orchestration capabilities of Docker were pretty lacking for Data Science use cases, so that's where we've focused our energy. Volumes are a good basic tool for getting data to your code, but they're a pretty blunt instrument. How do you split data up to parallelize over it? How do you make sure you've got the right version of data? How do you schedule new computations when data becomes available? Those are some of the use cases we solve with our distributed file system PFS.
Fighting with dependencies sucks when you aren’t intimately familiar with the ecosystem: python, node and other js, and even go.
What if a data scientist could deploy models right from browser?
Standard procedure, takes two minutes and I have a Jupyter notebook up, smartly automated base processes for dependency management, full control of environment variables and can deploy/ integrate this in any other Python setup (if I were to port IPython code to pure Python).
Is this more a thing of each his own or am I missing a crucial advantage of Anaconda?
[1] https://github.com/kennethreitz/autoenv [2] https://github.com/jazzband/pip-tools [3] https://github.com/pyupio/pyup
I've been to enough software carpentry talks where people nod and smile, and then promptly ignore the "how to be a responsible developer" advice. They don't care. They just want to run Python and plot some stuff or classify some data without worrying about multiple CLI tools. People seem to accept Anaconda because it comes across as a monolith and it's easier to understand for new programmers than virtualenvs and worrying about pinning pip installs.
That said, I don't use Anaconda on Mac or Linux.
P.S. -- Even if you are on Windows, whole SciPy ecosystem can be easily installed with pipenv, no need of conda for that now.
What benefits does pipenv have beyond storing the hashes as well? In what way is it more sane? For example, does it get around the issues with, e.g., matplotlib being installed in a virtualenv?
If you are happy with your pip and venv workflow you can happily ignore pipenv, I personally prefer it because:
- It combines all the features of pip and venv with a better UI(in my opinion).
- I can specify my environments in more detail using Pipfile [https://docs.pipenv.org/advanced/#specifying-basically-anyth...]
- It is integrated with pyenv [https://docs.pipenv.org/advanced/#automatic-python-installat...]
- The promise of deterministic builds
Oftentimes with ML and compute-intensive workloads, performance is an aspect of correctness. It is nontrivial to get the right accelerated libraries, and the right GPU libraries linking against the right Numpy version for the particular notebook you're running.
If you are doing work just for yourself, or if you have total control over the deployment environment (and your collaborator's environments), then you might be able to get away with just using pip for these things.
For data scientists who write workflows as R Markdown documents and want to containerize them, you might want to check out our R package liftr: https://liftr.me/.
But they have large down sides as well which slow us down. They are a pain to maintain as they are somewhat undocumented (you make a poc for 1 and management always wants more without improvments), a lot of edge cases cause issues which are tough to reproduce (locally sometimes impossible and waste a lot of time) and it takes them a while to start, run, etc.
This is not too tragic for nightly tests as we get the results in the morning but for tests which are started every hour, you do not want to wait that long to verify your changes work/didnt break anything. You can do these in stages, where you create different images based on the result of a previous job (run basic tests that cover base functionality that should always work, then run more in depth tests, then run performance tests at the end to ensure no significant degradation was introduced, etc..) and send out notifications asap in case of failure. The Dockerfile is essentially the documentation as you can see what is installed/configured. You can run everything locally just as it would in a k8 env. which for some reason every one always struggles with.
I am sure there are also edge cases with Docker that are a pain as well but the other selling points show it may be the right direction. You just havevto find use cases and evaluate them.
I know that the HPC clusters I've used in the past few years have all supported Singularity, but none have supported Docker (aside from our small lab cluster). Many HPC admins are (understandably) hesitant to allow non-admins access to start Docker containers (requiring root), but Singularity has no such user permission issues -- and it's faster than initializing a full VM to run a job. I don't expect that to change so long as starting a container requires root-effective permissions.
I suspect that many data scientists will be in a similar situation w.r.t HPC clusters (except for those that are using custom clouds like Seven Bridges).
Regarding HPC, from what I remember they usually have old kernels which are running (2.6) for compatibility reasons where Docker usually is not supported (unless it is backported like in RHEL).
The Skymind Intelligence Layer helps data scientists operationalize their models on-prem and in the public cloud, and uses Docker to do that.
https://skymind.readme.io/v1.0.1/docs/quickstart
https://skymind.readme.io/v1.0.1/docs/docker-image