Launch HN: Slai (YC W22) – Build ML models quickly and deploy them as apps
ML applications are increasingly built by software engineers rather than data scientists, but getting ML into a product is still a pain. You have to set up local environments, manage servers, build CI/CD pipelines, self-host open-source tools. Many engineers just want to leverage ML for their products without doing any of that. Slai takes care of all of it, so you can focus on your own work.
Slai is opinionated: we are specifically for software developers who want to build models into products. We cover the entire ML lifecycle, all the way from initial exploration and prototyping to deploying your model as a REST API. Our sandboxes contain all the code, dataset, dependencies, and application logic needed for your model to run.
We needed this product ourselves. A year ago, Luke was working as a robotics engineer, working on a computationally intensive problem on a robot arm (force vector estimation). He started writing an algorithm, but realized a neural network could solve the problem faster and more accurately. Many people had solved this before, so it wasn’t difficult to find an example neural net and get the model trained. You’d think that would be the hard part—but actually the hard part was getting the model available via a REST API. It didn’t seem sensible to write a Flask app and spin up an EC2 instance just to serve up this little ML microservice. The whole thing was unnecessarily cumbersome.
After researching various MLOps tools, we started to notice a pattern—most are designed for data scientists doing experimentation, rather than software engineers who want to solve a specific problem using ML. We set out to build an ML tool that is designed for developers and organized around SWE best practices. That means leaving notebooks entirely behind, even though they're still the preferred form factor for data exploration and analysis. We've made the bet that a normal IDE with some "Jupyter-lite" functionality (e.g. splitting code into cells that can be run independently) is a fair trade-off for software engineers who want easy and fast product development.
Our browser-based IDE uses a project structure with five components: (1) a training section, for model training scripts, (2) a handler, for pre- and post-processing logic for the model and API schema, (3) a test file, for writing unit tests, (4) dependencies, which are interactively installed Python libraries, and (5) datasets used for model training. By modularizing the project in this way, we ensure that ML apps are functional end-to-end (if we didn't do this, you can imagine a scenario where a data scientist hands off a model to a software engineer for deployment, who's then forced to understand how to create an API around the model, and how to parse a funky ML tensor output into a JSON field). Models can be trained on CPUs or GPUs, and deployed to our fully-managed backend for invoking via a REST API.
Each browser-based IDE instance (“sandbox”) contains all the source code, libraries, and data needed for an ML application. When a user lands on a sandbox, we remotely spin up a Docker container and execute all runtime actions in the remote environment. When a model is deployed, we ship that container onto our inference cluster, where it’s available to call via a REST API.
Customers have so far used Slai to categorize bills and invoices for a fintech app; recognize gestures from MYO armband movement data; detect anomalies in electrocardiograms; and recommend content in a news feed based on previous content a user has liked/saved.
If you’d like to try it, here are three projects you can play with:
<...
55 comments
[ 2.2 ms ] story [ 115 ms ] threadAre you planning any git or IDE integration? Most of the magic here seems to happens in the backend with easier training, scheduling, and inference. Could this be enabled locally so devs iterate in an environment that's more comfortable to them?
Yeah we've been thinking about this quite a bit. We've explored a couple of options here - I think our first pass is going to be a way to synchronize an external git repository with a sandbox. Would love to hear your thoughts here on what kind of workflow might make the most sense.
I think long term we'll also add VSCode integration through an extension, but that might be a few months out.
Overall I like the idea and I agree with you, either the tools are too focused on Data Scientists or there are a lot of DevOps involved to get things started
I work on the field so I have some questions:
- Are there any plans to connect the project into a git repo?
- Is there any option for me to pass trained binaries to your product? For example I have a beast of a machine and I can easily train things locally, but I'd like to host the inference with you guys
- Do you intend to allow automated testing and linting?
Right now, you can upload a trained binary to a sandbox and return it from the train function, and then use it for inference. So it's a bit manual at this point, but we're planning on improving that workflow shortly.
We built linting and testing into the sandbox, but testing is currently triggered manually - we're planning on building both into our CI/CD system (scheduled training)
Secondly, I ran the train -> test cycle and I didn't see any error metrics. Is the idea that if we were spinning up our own we would be outputting these ourselves? Or would we have trained up the model somewhere else and we would transfer it to SLAI to do a final test and then package it?
- User is working with a pre-trained model that already went through extensive testing during training. In this case our test utilities are useful as e2e tests. Once you integrate the model into your handler, you can specify a bunch of test cases to be sure your API is going to behave as expected (like a unit test).
- User wants to train the model on our platform - they can add error metrics directly in their training script and prevent the model from being saved if any error metric exceeds a certain threshold. They can then additionally use the test.py script to run tests against the model + handler.
So when a user trains a model you guys startup a docker container with everything in it. You guys bind the container's ports to the host and add it to some key value store that a reverse proxy references. Is that correct?
Sorry, I'm just really curious. It's a really interesting project. Do you guys have anything open source?
We've haven't open-sourced any of it yet, but there are definitely a few components of our system that we'll open source once we feel they're stable enough.
What benefit would Kubernetes bring to this architecture? You can create and destroy docker container using the api.
What do you guys use Kubernetes for?
Internally, each pod is just running a docker image. You could probably throw something together with docker/the docker API - but in our case we needed a bit more control.
Thanks for the helpful responses!
I sent you an email regarding a possible internship opportunity. Are you guys open to interns?
You can think of us as being a store of useful ML based microservices, and not just a library of pre-trained models.
https://huggingface.co/autonlp
Looking forward to seeing your success, good luck.
now Sagemaker allows u to download ur running code and docker (https://docs.aws.amazon.com/sagemaker/latest/dg/data-wrangle...) . Also allows u to simulate local running - https://github.com/aws/sagemaker-tensorflow-training-toolkit
rather than anything else, this is basically just a way to calm worries about lock-in. Google ML resisted this for a long time, but even they had to finally do it - https://cloud.google.com/automl-tables/docs/model-export
are you planning something similar ?
However, this is probably a few months out since we're currently focused on startups/developers that don't have that requirement.
Two things that happened to me:
1) I wasn't able to install `slai` using Pip and PyPi. I ended up downloading the source tarball from https://pypi.org/project/slai/#files and installing locally.
2) I am following the example for how to "Integrate" my model using Python under the "Metrics" tab. However, the call to `model = slai.model("foobarbaz")` is failing. It looks like the regex check for `MODEL_ROUTE_URI` from line 21 in `model.py` doesn't like my custom email address :(. For example, the following model endpoint isn't valid according to the regex: "s@slai.io/foo-bar-baz/initial" (My custom email is very similar to `s@slai.io`). I'll post the regex below.
`MODEL_ROUTE_URI = r"([A-Za-z0-9]+[\._]?[A-Za-z0-9]+[@]\w+[.]\w{2,3})/([a-zA-Z0-9\-\_]+)/?([a-zA-Z0-9\-\_]*)"`
Just wanted to let you know! Looking forward to experimenting with this more.
Weird that you weren't able to install the slai sdk via pip, we just released a new version of the SDK this morning, unsure if that's related. I'll take a look into that this afternoon.
Thanks for trying it out!
Here are my unordered thoughts.
So it seems a lot like an improved colab with a deployment stage. Which sounds good to me, it will be much more expensive than colab though.
I like the pitch of SWEs doing ML instead of pitching towards Data Scientists. As Data Scientist turned SWE I still miss the Jupyter like cell based execution. (you said it exists but I couldn't find it.)
In general I'm quite sceptical when it comes to online IDEs. However, for Text and Image based models it might be enough (since you don't need tooo much code).
There might be a valid niche between colab on the one side and building it yourself with AWS cli on the other.
I wonder though, in your target market it really isn't such a big deal to spin up a rest api, there are no lambdas with GPU though (but this should be a matter of time). Or use something like AWS batch for remote training. It will come down to: Is it more convenient to code in your IDE and you handle Lambda, Batch Docker and CD. Or do you code in your own IDE and you have to handle this stuff yourself.
Wish you all the best!
When it comes to SWEs doing ML, our goal to bring together a bunch of apps that get people most of the way to something they can bring into production.
I appreciate your skepticism of online IDEs, and it’s unlikely we’ll ever completely replace notebooks or whatever IDEs people prefer hacking on locally. Instead, we’d like to take a hybrid approach, in which our online IDE is sufficient for making minor tweaks to a model after its in production, but the brunt of development can still happen locally, in which changes would be pushed to Git and synced with our online IDE.
It’s true that SWEs can deploy their own APIs, but that feels like an unnecessary annoyance to take for granted. At a high level you’re just setting up an API, but really you’re also going to setup some versioning system and a Docker file and monitoring, and all of that adds up to a lot of cognitive overhead.
BTW - the Jupyter-like cell execution can be turned on by clicking the “Interactive Mode” button on the bottom right corner of the IDE.
We chose to do it this way to ensure that the binary you upload is properly tracked in our versioning system, and that it can be integrated into your handler.
It’s fairly painful to productionize a model on Sagemaker — they make you think about a lot of things and fit into AWS primitives. Besides the code for the model, we don’t force users to think about anything. Our focus is helping engineers get models into production, not reading documentation.
Using our tool, you can fork a model and deploy it to production right away — there’s no time spent battling AWS primitives. We’re focused on developer experience above everything else - which means we enforce sandboxes on our platform to be consistent and reproducible.
I looked a bit more at your service, since I am migrating one of our text classification models anyways right now. I decided against using it but maybe my reasoning could still be helpful. (and I see a lot of potential so I want to help)
What I am using instead is a combination of AWS Batch and Colab. My reasoning:
# Local Development
Yes it is true that ML code can be quite well separated from the rest. But then there is data. So the extract and load step. I know you have bindings to for example Postgres but I wouldn't trust you with my DB.
And always moving over files could be done (we keep a backup anyways) but it would be more of a hassle. Also even for the actual ML code it is nice to have it in an actual IDe with a good debugger etc. I prefer to code your ML code locally and then just package it and send it away to be trained.
Also, yes there is a cost to set up the infrastructure but I prefer to solve that with code generation/templates and libraries (to send your docker to Lambda for inference and Batch/Colab for training). It is a cost that is paid once and then never again.
# Price
Your gpu instance costs 1 dollar an hour which is about 3 times as much as a p2.xlarge spot instance (which I assume is the closest one). Colab of course is 10 bucks a month / free. This is ignoring AWS credits for now. (would also be good to know which exact GPU you provide)
You can basically combine this with "Scheduled Training", and it'll retrain your model on a schedule - pulling in the new data. This is V1 and does not yet handle more complex, event based retraining. This is something we know is crucial for tons of use cases and we're planning on adding it in the coming months.
Happy to chat and get your feedback on what kind of event sources you might want to trigger the model to retrain on new data.