Launch HN: FloydHub (YC W17) – Heroku for Deep Learning
10 months ago, I was working at Microsoft and doing a lot of deep learning (DL) there. While the DL community is terrific, I was often frustrated by how difficult it was to get started and build upon others’ work. For example, running any popular Github project often started with an exercise in dependency hell. As I untangled these for myself, I wrote up some notes on setting up popular DL frameworks, which unexpectedly started trending on HN after someone posted it there (https://news.ycombinator.com/item?id=11697571). That's when I realized that engineering was a huge bottleneck in deep learning and a problem worth solving after all.
I’ve since quit my job and have been working fulltime for the last 9 months on building FloydHub to make deep learning easier. Our goal is to let the data scientists focus on the science, while we handle the engineering grunt work (provisioning and scaling infra, running reproducible experiments, enabling sharing and collaboration, supporting DL frameworks with zero setup, shipping trained models to production easily, etc.) Lots of interesting challenges - happy to talk about them!
We have a lot of work ahead, but we’re excited to share with you what we have so far! Looking forward to your feedback.
88 comments
[ 3.1 ms ] story [ 185 ms ] threadFWIW, I worked as a data scientist in Bing for 6 years and haven't seen/heard any other data scientist use it internally. We ended up building our own GPU clusters and going through the regular drill.
Which also makes me curious about FloydHub's infrastructure. Any gory details?
The DL community is awesome in its openness and contributions. Our goal with FloydHub, in contrast to the ML APIs, is to provide the tools for data scientists to effectively leverage this. We want to solve the engineering hurdles that come in the way of doing some cool science.
GPUs instances are really expensive. One of the biggest challenges at the moment is around reducing this cost. Eg. Spot Instances and Spot Blocks. Still some challenges to be solved there.
We also want Floyd to be an end-to-end solution for building, training and deploying deep learning models. In that vein, we are also investing in adding support for Tensorflow serving but it has been a rough ride so far. Getting a generic solution that can host any Tensorflow model has not been straightforward.
Here’s an anecdotal story about how awesome AEther was (been a long time, so a little fuzzy on details): In 2011, Harry Shum was the VP of the Bing division at Microsoft. It was the early days of Bing (~10% market share, ~$2bn annual loss, etc.) - we had good talent, but were lagging behind Google in tech. In one of our all-hands meetings, Harry jokingly announced that if we beat Google in our core relevance metric (called NDCG), he’d take the entire Bing team, approx. 300 people strong, for a fully paid trip to Las Vegas.
Sure enough, a year later, Bing did beat Google in our core relevance metric (http://www.insideris.com/microsoft-bing-beats-google-in-the-...) and all 300 of us went to Vegas for a weekend as promised. (Spoiler: Google did eventually beat Bing back later)
The success and rapid acceleration in relevance gains was attributed in large parts to the introduction of a new tool called AEther (in addition to improving ML tech and hiring top talent). AEther was an experimentation platform for building and running data workflows. It allowed data scientists to build complex workflows and experiment in a massively parallel fashion, while abstracting away all the engineering concerns. I used it a ton on a daily basis and loved it. The AEther team claimed that it increased the experimentation productivity of researchers and engineers by almost 100X. Even now, when I ask ex-Bing data scientists working at other companies about what they miss the most from their time at Microsoft, AEther is almost always in the top 3 answers.
Having seen how awesome AEther was from the inside, one of our goals is to bring its benefits to the rest of the world as well. However, having talked to a few individual data scientists and researchers over the last month, their preference seems to be CLI over GUI (while bigger companies like it much better). May be its one of those things you have to get used to, or may be our implementation is clunky. So we’re making the GUI an enterprise only feature for now, while we continue to help individual data scientists through our CLI.
Everything else looks really slick!
But http://docs.floydhub.com/guides/jupyter/ says "IMPORTANT: Floyd does not save your Jupyter notebooks after you stop the floyd job. So you need to download any relevant notebooks by selecting File > Download As menu from the Jupyter notebook."
Which is it?
The docs have been updated to reflect this. Thanks for pointing this out.
So, Flow + Droid => Floyd. Also, a hat tip to one of my favorite bands :)
The jupyter jobs look neat, but I assume they are charged continuous time? Would be cool if somehow that only ended up charged for compute time, but I understand that would be difficult.
Are these instances guaranteed to be in a given region, for if I wanted to route more complex debug output / intermediate files to S3?
Just an observation :).
We are only using p2.xlarge (1 GPU) for now. Driving down the cost is really important to us. We use reserved instances, spot fleets, etc. to be at <50% of AWS pricing. Lots of interesting challenges to be solved there wrt. effective scheduling and fully utilizing resources.
We’ve been thinking about our own infrastructure. It would really drive down the cost, but obviously, comes with its own challenges :)
We were initially calling ourselves Heroku + Github for DL, but realized that was too confusing, haha.
Also, awesome 100 hours offering -- Looking forward to using FloydHub for deep learning!
[0] https://research.googleblog.com/2016/09/announcing-youtube-8... [1] https://www.kaggle.com/c/youtube8m
Re: Kaggle, we haven’t had a chance yet but that sounds like a great idea.
That said, it’s hard to say how things are going to turn out. This is my first startup after working in the corporate for 6 years and it’s been amazing so far. Learning a lot, and excited for what’s ahead! :)
It'll be interesting to see where we decide to go down different paths, or how we take different approaches to things.
As you say, there's plenty of room out there. And we're all competing with Amazon, Google, Microsoft, etc. anyway. :-)
And oh, about the time I forgot to turn off our GPU instance for a couple of weeks… racked up a nice bill...
Git is an apt analogy. The search space of hyperparameters is usually fairly large for most DL algorithms, so a good amount of experimentation is required to tune them. Things can start to get haywire without end-to-end version control of code, data, parameters, results, environments, etc. Definitely one of the core problems we solve.
It felt like a Heroku moment for me. They have the potential to do to Tensorflow what Heroku did to Rails. Super simple deploy!
Obviously their vision is much broader (with an entire eco-system/'hub', reproducibility, etc.), but to me atleast the first part is super useful and exciting!
One advantage they have is that GPUs are INSANELY expensive on the cloud - they can actually make it cheaper for everyone with clever binpacking and proper termination.
My advice is that in the initial stage, they should partner with all the Moocs to ensure that all deep learning students are using Floyd. It's cheaper, faster and the students can focus on the science. And they provide, 100 free hours!
Disclosure: I've known the guys for quite a while.
Here's our instructions for the Self Driving Car Engineer nanodegree program: https://github.com/floydhub/CarND-Term1-Starter-Kit. Happy to do the same for your class as well! How can I reach you? Feel free to to mail us directly: founders@floydhub.com.
We've also reached out to folks at Udacity to see if we can offer any official support for the courses.
Either way, I often find myself choosing backward compatibility and stability over innovation and polish and choose to learn vim and bash instead of replacing the silver bullet every year.
Shameless plug, I am working on a platform similar to FloydHub, but for frontend engineers [0]. The problem is a real one.
[0] https://pipez.io
Pipez sounds really useful, good luck!
A couple of years ago I worked for a local cloud server provider, as a backend developer. Some of the work I did moved the company into deploying VPS instances using OpenStack. Our backend code was mainly PHP-based; so we used OpenCloud for the purpose - extending it where needed (when we started it didn't support all we needed; I extended things in such a way so that when we did need to upgrade OpenCloud, it would gracefully work without breaking anything - it was a gamble that I didn't know if it would really work - just had a hunch - 9 months in we upgraded, and it all worked perfectly).
Anyhow - at that time, seeing what we had available for servers and such (we were competing somewhat with DO) - I suggested we add support for GPU instances and maybe pivot toward an ML offering of some sort. Not gut our bread-n-butter, but offer up some kind of ML package for those that needed or wanted it.
I was shot down by management as it being too "pie in the sky" - not even demand or something like that. To be honest, I'm not even sure they understood what I was trying to convey, so maybe part of the problem was mine as well.
The company was eventually sold and I moved on, but seeing now how these kinds of services are in demand, I sometimes wonder on what "could've been". Ever since taking my first MOOC in ML (Ng's ML Class in 2011) - I've tried to interest employers in applying what (little) I know on the subject. I'm not an expert, but I'd love to apply my learning (on top of the 25+ years of software dev experience I already have). Today, I'm in the middle of the Udacity Nanodegree MOOC - I doubt much of that will transfer for my current employer, but maybe the general knowledge I'm getting of TensorFlow and Keras, among other bits, might help in the future.
I think, though, that FloydHub might be fun to play around with for future personal ML/DL projects as time goes on; I look forward to trying it out someday soon!
Good luck with your Udacity class! If you want to play around, take a look at our guide for Neural Style Transfer: http://docs.floydhub.com/guides/style_transfer/
My first time deploying to Heroku in the summer of 2011 was a magical moment after spending a rather frustrating day futzing with a bunch of more barebones providers like Linode, AWS, Engine Yard, and few others I can't even remember at this point. I cannot stress how magical it was -- I like to think I'm a competent enough developer but I wasted a long time gluing things together and never quite getting it right for some reason. Before calling it quits for the day I decided to try that Heroku thing I had been reading and within 20 minutes I was up and running. That was impressive.
To note, I've been a happy paying customer of theirs since then, so they've made a decent chunk of change off my business over the last 6 years. If you can capture companies and projects in the infant stages, you can grow with them for quite a long time before it, if ever, becomes economical to roll your own infrastructure.
Currently working in a small startup and our data scientists (just a team of 6) are frequently fighting over the gpus in the office for time and online services atm are overpriced for our small budget.
One small piece of feedback. If you open your landing page from mobile, e.g. My iPhone 7, the header height is changing all the time due to the text changing dynamically. It get hard to scroll past it and keep reading.
Cheers