Ask HN: What ML platform are you using?

What do you want to train and why?

I think Colab is very popular since it's free. Should be perfect for a beginner who doesn't want to spend money. I don't think there's a lot of lock-in so just try it and see. There are bigger questions to worry about, like should you use TensorFlow (no) or PyTorch (yes) or JAX (maybe). That's much harder to change later.

Honestly, I've found that most ML tooling is overly complicated for most ML projects.

I use a paperspace VM + Parsec for personal ML projects. Whenever I've done the math an hourly rate on a standard VM w/GPU is better than purchasing a local machine and the complexity of a workflow management tool for ML just isn't worth it unless you are collaborating across many researchers. As an added bonus, you can re-use these VMs for any hobby gaming you might do.

The majority of ML methods train quickly on a single large modern GPU for typical academic datasets. The scaling beyond 1 GPU or 1 host leads to big model research. While big models are a hot field, this is where you would need large institutional support to do anything interesting. A model isn't big unless it's > 30 GB these days :)

Even in a typical industrial setting, you'll find the majority of scientists using various python scripts to train and preprocess data on a single server. Data wrangling is the main component which requires large compute clusters.

We use Kubeflow at our shop. If you use a managed K8s offering, it’s quite simple to manage, and of course you can deploy all your other stuff alongside it using the same tool stack.

Colab (Pro+?) should be enough until you decide to spend 500$/mo.

PyPer built on PyTorch

I like Colab for the most part since I'm biased towards Python, but being centered around Jupyter notebooks does have its shortcomings. Also, despite being a service offered by Google, I prefer PyTorch over Tensorflow.

For smaller projects, I generally find a Towhee pipeline (https://towhee.io/pipelines) that I then fine-tune on my 3080.

> I am also not opposed to configuring environments myself rather than using managed solutions

vast.ai has pretty low prices and gives you remote ssh into a GPU instance that you then have root on (albeit containerized).

Having a local GPU is effectively a requirement for doing "development" work (e.g. getting an architecture and/or codebase to the point where you would even be able to start training). Unfortunately, getting your own GPU is just absurdly expensive these days and probably not worth it. In the meantime, colab/kaggle/paperspace can be _okay_ as dev environments. Unfortunately, renting compute on vast.ai all day just to do occasional dev work gets expensive pretty quickly.

For something in-between vast.ai and AWS, datacrunch.io has slightly higher prices, with remote SSH into a server and a few more "niceties" that you get with a traditional cloud such as CPU instances and the ability to use those to pre-load data onto disk.

If and when you are able to get a GPU - just make sure to get nvidia as they have a stranglehold over the industry. The RTX cards are great - I've been doing tons of multimodal work on an RTX 2070 I bought pre-pandemic for around 350$. It only has 8 GiB of vram but is actually quite similar to a server-style V100 otherwise. I assume it probably costs like 2000$ these days.

If you're interested in the realm of running inference/training on giant models (say GPT-J 20B), you may find yourself in lack of VRAM. Using libraries like deepspeed, you can split the work across multiple GPU's. I highly recommend investing time to learning multi-GPU libraries or framework-provided features like pytorch's distributed data parallel as the size of models becomes a limiting factor very quickly in the case of transformers. A sibling comment mentions that you will need institutional support for training such models. This may be true, unfortunately. All I will say is that if you are even mildly competent, the demand for that type of work is increasing a lot lately.

Oh and yes, there is a new-ish site called replicate that I have been using to allow people to run inference on models that I've trained https://replicate.com/ without needing to be a coder. A lot of people use colab for this but that platform is annoying to support in practice.

I have been using Google Colab and AWS for 'big' personal projects while I use my PC to train light models. Colab is neat because it's free and I like the style of their notebooks. However, you cannot always find available GPU and I wouldn't recommend free Colab for anything else than learning and experimenting with ML. I have been using AWS Ubuntu server set up with PyTorch and had OK experience, but you need to be careful about pricing and set up policies, as well as remember to turn off your machines when you're not working on them if you don't want your credit card to blow up. In the future, I might give Google Colab Pro a try, but most of the work now I do on the company's server.

Anecdote: When I was taking the 'Computing For Data Science' class, we had a task to learn to use AWS tools like SageMaker, NLP bot or DeepRacer and present it in the class. The professor was also new to the whole AWS ecosystem. He opened many instances and left them running for a week which ended up taking 1000$ from bank account.. (Moral of the story: don't use aws with the card where all your money is)

Personally I stick with the classic SMLNJ

You can run on-demand training jobs on GCP vertex ai training. I'm not sure the price point but it's pretty useful both for development and training.

Are you just interested in the training part and managing the trained models, or you'd actually like to productionize the models and serve them at scale?

A lot of end-to-end platforms are available nowadays that try to cover the entire lifecycle of a model from data prep, ETL, to training, serving, monitoring, operating. However, I found none of them really robust enough to cover all these cases perfectly, so I resorted to using different pieces from different vendors combined with my own stuff to make the entire platform suit my needs. This is still not perfect, though, and I think there's a lot of room for improvement in the space to enable really easy to use and scalable MLOps.

Still some of the tools I found to be ok: TensorFlow TFX, Kubeflow (to some extent - ops are a nightmare), Feast, MLFlow, GCP Vertex and AWS Sagemaker can get some work done, too.

Get a decent NVIDIA GPU. Then install PyTorch and off you go. I advise to make all your own tooling, as you likely have a specific use-case, and so your tooling can be tailored to that. Most ML tools are very generic, or so simply you might as well do them yourself. The advantage if having your own box, is that (1) you'll learn some systems skills building it and (2) since you invested in it, you should feel obligated to use it! Good luck.

I put together a linux box with a 2080ti a few years ago and have been using it consistently for personal ml research ever since. Ive found it well worth the investment and learned that the ease with which I can jump into hacking on a project is key, which is why this works so well for me. I can just ssh in at any time and start experimenting with models. Even if its not technically economical when you do the math, the ease, reliability and fact that I know my models arent being billed per hour helps encourage me to experiment often, which is kwy to learning.

As for software, I do everything with jax and tensorboard for viewing experiments. Jax is a phenomenal library for personal ml learning as its extremely flexible and has relatively low level composable abstractions.

Thr best option for you is- Gradient Paperspace and Colab. Both are free and managed.

Learn Machine Learning first. Do not spend time on managing infra for ML while you are learning ML. Focus on learning ML first.

You can make decent cutting edge models and SOTA classic models just with free options. I am saying this because I have done this.

I suggest that you get Colab Pro after that.

AWS burns a hole on your pocket, and you should not spend money on that now. Although, AWS SageMaker is pretty tension-free experience.

I personally use GCP. I like the tooling around it to be the most convenient.

I suggest you learn the basics first. Learn classic ML, CNNs, RNNs, LSTM, Transformers, learn the necessary Maths, and even GANs if you are inclined.

If done in the right way, it will take you a 5/6 months to 18/20 months, depending on your time commitment, your current levels of grasp on programming and Math.

Do not rush or hurry.

When you reach that point, you can think of spending serious money for Deep Learning projects.

A few months back, I have gotten into TPUs, and these are fantastic. And GCP is my only option for these. I have only used TPUs for learning and personal project and never for work. I intend to keep it that way for a while.

I recommend to use Colab for learning because so many research papers publish their own examples as a Colab link nowadays, so you'll have plenty of stuff to try out and explore.

For the actual deployment in production, the only thing that's really affordable is if you send your own GPU workstations to a colocation hosting company. But that's a lot of work.

I created something that lets you get free GPU on VS Code with Google Colab with just 1-click. Have a look at https://github.com/DerekChia/colab-vscode

This is my default go-to as a poor man ML setup, with environment and dependencies set up automatically via bash script on start up.

For learning (and for development on most projects, until it actually comes time to train the real model) the K80 or whatever the lower tier is on colab is fine as a gpu.

The problem with colab IMO is that if it's your main platform, you'll be pushed to use notebooks for everything which is not really a good practice. Whatever you use, I'd suggest focusing on building a real train.py script (I'm assuming you'll be using python) that takes command line arguments for the hyperparameters. Don't get sloppy and just have things run as a bunch of cells.

If you are learning, my unsolicited advice is don't use built in datasets, make sure you can write datasets / dataloaders yourself so you understand what is going on and can adapt to your own work. All the stock examples using built in mnist or whatever gloss over the most important parts of setting up the data

I have been working last 1.5 years on my master thesis and I my setup evolved Colab -> Kaggle -> Azure ML.

Colab you is great for diving into examples that are already premade for colab.

Kaggle is better in my opinion in dataset handling, you can import public or upload your dataset with ease. They give you 30+ gpu hours for a week with ability to train your models in background. This can’t be done in Colab.

ML Azure platform is next level when you can pay for it. I’ve got credits from school. You can start experiments from python sdk with your own configurations, setup python environments, upload datasets, etc.

BTW this question probably suits a poll.

https://news.ycombinator.com/newpoll

Working with a large well known tech company, with surprisingly basic/non-existent ML until only very recently.

Using Redshift to do a lot of the heavy lifting and initial data preparation, then SageMaker for hosting models and scoring, and Tableau for dashboards.

While you can do training within SageMaker, we have a cluster of EC2 instances using H2O libraries (xgboost) to train, then wrap the resulting model as a docker image and deploy it to ECR and link to a SageMaker endpoint.

Clunky and very much human-in-the-loop for training and deployment, but you can't run before you can crawl in this space.

colab or rapidminer.

both of them work great for scratch projects.

I haven't experienced any real issues with GPU availability on Colab, I suggest that you just go ahead and use it and wait with the premature optimization until you actually hit a wall and need it.

For general advice focused on beginners and ESPECIALLY practical, cheap and efficient methods and hacks to do DL, I recommend searching in https://www.fast.ai/ and their forums https://forums.fast.ai/

I'll try to search inside fast.ai if there is a more specific link to give. I know that one of their chief pieces of advice has been to use Colab and take advantage of the 300$ free credit you get (per credit card) when signing up to Google Cloud, which you can use for DL.

Disclaimer - I'm one of the creators of DagsHub, we created the platform especially to help people like you with the difficulties of managing things like data and model versioning, experiment tracking, labeling, etc. we'd love to have you onboard, and thanks for reading until the end :)

Try it out on a local Linux machine first, if you have one. There are plenty of ML techniques outside of neural networks which train perfectly well on a CPU, so I'd start there.

Look at some kind of AutoML framework like AutoGluon, then dive deeper on the components it uses once you've got through the initial setup process. AutoGluon will let you train some basic models with all the data cleaning and normalisation steps handled for you.

> I am very much a beginner in the space of machine learning

While the (precious and useful) advice around seem to cover mostly the bigger infrastructures, please note that

you can effectively do an important slice of machine learning work (study, personal research) with just a battery-efficiency-level CPU (not GPU), in the order of minutes, on a battery. That comes before going to "Big Data".

And there are lightweight tools: I am current enamoured with Genann («minimal, well-tested open-source library implementing feedfordward artificial neural networks (ANN) in C», by Lewis Van Winkle), a single C file of 400 lines compiling to a 40kb object, yet well sufficient to solve a number of the problems you may meet.

https://codeplea.com/genann // https://github.com/codeplea/genann

After all, is it a good idea to use tools that automate process optimization while you are learning the deal? Only partially. You should build - in general and even metaphorically - the legitimacy of your Python ops on a good C ground.

And: note that you can also build ANNs in R (and other math or stats environments). If needed or comfortable...

Also note - reminder - that the MIT lessons of Prof. Patrick Winston for the Artificial Intelligence course (classical AI with a few lessons on ANNs) are freely available. That covers the grounds before a climb into the newer techniques.