Is Machine Learning useful despite the hype?

I've been in Analytics + Data Science (most of that in customer facing roles) for close to a decade now, and I'm one of the first people to say there's a ton of hype in AI/ML. However, the real issue with the hype is unrealistic expectations and BS marketing, not the actual usefulness of ML/AI. This includes Data Science tutorials which are basically - 1) Here is some clean and ready data. 2)Pick the best model. 3)??? 4) $$$/Profit!! That doesn't match the real world at all.

Between myself other people I've worked with, I've seen ML/AI helping address business problems in Healthcare, Industrial Manufacturing and Operations, Energy (e.g. Oil&Gas), Consumer Goods, and Finance. These types of projects tend to succeed when they are part of a larger initiative that includes some organizational change. A lot of "let's drop ML/AI into our existing business process without changing anything" tends to fail.

As for needing an ML/AI specialist versus SWE with some basic knowledge of an ML toolkit, I think you need both. Not every business problem is a software engineering problem.

>I've noticed lately lots of discussions about AI and ML being just hype. Examples given in those discussions of where ML fails is in driverless cars and the fact that it cost too much to train.

This is mainly in traditional media and specialized blogs. Their job is to get content out there. Of all that is written, what is useful to me is in homeopathic concentration. The most popular blogs are targeted for people getting into machine learning. We actually spend time with candidates breaking the news to them à out what the job is. They tend to think training the model is the most valuable/hard part.

>Despite all of this is there a big enough problem space where Machine Learning is still the solution?

We've been delivering such solutions for enterprise for the past seven years. The catch is that to deliver this value, you need to get many things right. To get these things right, you need to have gotten the chance to do it before. To get the chance to do it, you need to get in front of the right people. Most content and solutions concentrate on training models, which in our experience delivering projects in the real world, is the most trivial thing compared to other parts of the whole. Training is like building a concept car: compared to getting the actual car to market to actual drivers, this is a walk in the park.

The majority of the work isn't news worthy. I mean, who would write about taking clients by their hand, communication, problem definition, empathy, building infrastructure, managing expectation, defining jobs to be done, measuring real world metrics [not talking about F1 scores], having long conversations about ethics, dealing with data dumps, moral support for your people, making your counterparts successful... These are topics that are common and not newsworthy.

>Do we need Machine Learning experts or we're fine with software engineers who use Machine Learning techniques in their toolkit?

Not mutually exclusive. These are different roles who do different tasks and have different outputs. One of the problems is precisely in interfaces between these roles.

I write a bit about these things:

- https://twitter.com/jugurthahadjar/status/130810318863405056...

- https://news.ycombinator.com/item?id=24326473

- https://news.ycombinator.com/item?id=24463187

- https://news.ycombinator.com/item?id=24325090

I interviewed for ML positions at my company. I'd imagine it would be more difficult as an external hire.

It seems to me that if you actually want to do interesting stuff like design the models, then they want a PhD or extensive ML experience (I only have a masters in info sci with no ML experience). I could have gotten into a position for cleaning data, helping to train the model, or integration with other systems. I really don't want to be cleaning data or other boring stuff like that, so I didn't take it.

A lot of hype. Agreed. But there are a lot of uses.

The first usage that came to my mind is healthcare. AI can help doctors to diagnose more correctly e.g. lung cancer from x-ray. Doctor may get a few errors because of his mood or his bad attention at the very day but AI will make more correct prediction because of thousands of examples it has already seen and its clear and not biased mind. And in this sphere each life is at stake.

So yes. It is very helpful as I see it.

Machine learning is being used in India to help close the gap of institutional voids for basic skill training in the health care industry. Pretty much creating a pool of skilled laborers to government funded entities

Depends on the problem space. For clinical drug trials you need people who can do statistics well. For some ML problems you just need engineers who can properly tune and deploy a DNN or forest.

Then there is a large world in between, where you need statistical and ML intuition that comes from years of training and study.