HN Explorer

Ask HN: Is data science and ML a bubble / scam

5 points by elisharobinson ↗ HN
before you tell me about chat bots, voice traslations, machine vision and reinforcement learning. I would like to clarify , i am not an expert in ML or AI i have no production code with AI or ML. My question is how does ML/data science fit in as a normal part of SDLC.

i would imagine the test scope is essentially infinite since we handle probabilistic states instead of deterministic states. And how would you identify a bug and reproduce it or an even more significant problem of how would you even identify the scope of values which are not allowed in a NxN dimensional vector matrix, and as i understand it the tolerance of error is marginal in customer facing applications as recomendation systems, voice translation, etc even a 95% accuracy is good enough to ship but how about medical applications and self driving where something like 3~4 sigma is needed.

i dont subscribe to the adversarial argument that the solution to a black box is another black box or that we achived 3 sigma because the adversary we designed says so. How many business's are aware of the fact that ML/AI SDLC has these fundamental difference's from reqular old web,system and embeded SDLC and the supposed ROI from being a replacement to manual work can be lost by just a few false positives in a business setting.

TLDR what is the SDLC to handle state explosion in ML / AI systems design . I dont see many compeling arguments as of yet.

6 comments

[ 3.2 ms ] story [ 25.9 ms ] thread
tlb
If you can write code to do a job, you probably should. ML makes sense for places where you can't write the code. We know many applications where this is the case: people tried for decades to write good natural language translation and image recognition, but ML does much better.

A good overview of how to build real systems with ML is at https://medium.com/@karpathy/software-2-0-a64152b37c35

elisharobinson
i read the article you mentioned and found this tit bit quite interesting.

" In the 2.0 stack, the programming is done by accumulating, massaging and cleaning datasets. "

there is large scope for error/issues/politics in each of the mentioned steps. but i think it helped me frame my argument more succinctly AI/ML is not software 2.0 because it is not software. all software needs a ietf RFC :)

minimaxir
Thinking about data science/ML in the scope of the software development lifecycle isn't the right way to think about the industry. For non-AI products, data sci is a complement to the traditional processes, providing more-informed decisions.

Even for AI-products, the development is still iterative, e.g. improving the product after real world testing/feedback.

rossdavidh
It is not a bubble, in the sense that once it is bound to abruptly "pop". It is also not a scam, in the sense that there is no value.

It _IS_, however, on the Gartner hype cycle (look it up in Google Images if you don't know it), and way to the left of the "plateau of productivity" where technologies like Java and python and smartphones and solar panels are. It is probably near the "Peak of Inflated Expectations", headed for the "Trough of Disillusionment".

Which does not make it a bubble or a scam. Just overhyped. Part of the learning curve is figuring out where it is useful and appropriate, and that is mostly in areas where decades of conventional, non-ML efforts have not paid off.

elisharobinson
its an interesting argument you are proposing. but i feel AI/ML is not a normal tech trend because they are non deterministic systems. A normal tech trend would hit "plateau" after the "magic" or mystery fades away .

But non deterministic systems there is a good chance your walking of a on a misty mountaintop with a very high chance of irreparable damage where the cornerstone of agile quick fix build patch is not possible. or you stumble upon the a stairway to heaven where the possibilities and potential for growth seem limit less.

rossdavidh
One of the most fundamental lessons one has to learn with AI/ML, is that it will never be 100%. Of course, that is true with conventional programming as well, but when a failure is found that particular bug is fixed, and one has the illusion that the problem which was just fixed was the last one, and _now_ it is 100% reliable.

But, with ML, it will never, ever be 100%. In some potential applications, this is ok, or in any case the alternatives are 100% _not_ working so it's the best there is. But in other potential applications, this is a deal-breaker, actually or politically.

Part of finding the "plateau of productivity", is that organizations learn lessons like this, about where you can and cannot use it effectively.