Isn't this just the same problem IBM has had for...well...most of my adult life?
I had a contract gig at IBM Advanced Technology in Boca for about 18 months in early 2001-2002. Talk about missing the boat...
I was brought in to prop up a soon-to-be-failed project for the Japanese government...basically a "Napster for Tokyo" that would allow paid-for-play C2C song sharing for customers of "the Big 5" record companies.
I asked simple questions that no one could answer...why would people pay for content when it was so easily available via other means? you are using DRM how??? really? you need a special player to play the music?
Why would anyone do that?
I stayed for a few simple reasons...the fat consultant check I cashed every Friday. Exposure to some outstanding engineers and coders where I got to learn from true talent. The great strip club on A1A next to my rental in Lauderdale-by-the-Sea.
But reading over this article reminds me that IBM is just too big to get out of its own way, and has been for the longest time.
Getting a nice check while doing nothing actually productive is soul crushing. Good for the short term but it's a Chinese water torture in the long run.
Two years ago at their vegas conference they had a coffee shop that used AI to recommend coffee types. I thought "boy, they don't understand this technology".
Many years ago, when AI was expert systems and "neural networks" were fringe, the main demo for one of the public expert system leaders was the Wine Advisor. You'd tell it what you were going to eat and it would recommend a wine.
Being rules based isn't necessarily a bad thing or disingenuous. I develop healthcare AI products (ML/DL researcher) and we actually aim to be able to translate our models into a rules based engine (find a strong signal, interpret/understand model well enough to translate/embed into a rules engine, look for a new signal in our models, rinse + repeat). We end up deploying a mix of rules based and true ML based models into production but it may not be immediately obvious to the end user which type of model they are using.
I didn't mean it as being disingenuous - that's precisely the value that was sold and if you could do the proper "knowledge engineering", it worked well. It's just interesting to me having seen the previous turn of the AI hype wheel, how much is being repeated.
Another interesting thing was the transition from special purpose hardware - Lisp machines - to C code on commodity platforms. A contrast from today's ML moving in the other direction.
That's fair. Google's recent paper on predicting patient deaths is another good example of this (logistic regression + good feature engineering performed just as well as their deep learning models, and the logistic regression has the added benefit of being significantly more interpretable and as a result, actionable).
It'll be interesting to see when specialized ML focused silicon will become readily available. Right now I find ML libraries that are able to run on blended architectures (any combination of CPU and GPU's) much more exciting/impactful than TPU's. The ability to deploy on just about any cluster a customer may have available is huge.
From my experiences (currently work with several Fortune 100 health insurers/benefits managers, and have previously worked for another large insurer, a major academic medical center, and a large pharma company), healthcare organizations tend to be rather cloud adverse (most of our contracts very explicitly forbid us from using any form of 3rd party cloud computing). So while I agree that much of the heavy lifting will shift to the cloud (or already has), I expect health analytics will continue to favor on-premises solutions (GPU’s still tend to be pretty rare compared to CPU based clusters but are slowly becoming more common).
The likes of INTERNIST, CADUCEUS, and MYCIN have been around and provably accurate starting in the late 70s through the mid-80s. MYCIN even arguably sparked the 1st AI boom. But there were ethical issues with computer-aided diagnosis that I'm not sure have been solved/overcome.
Perhaps the current startup generation can get past them with Zuckerberg, Kalanick and Holmes as role models. :)
It's funny how much complex "AI" really comes down to If and Switch statements. "Utility AI" is popular for videogame AI right now - it's weighted switches.
Diagnosing vibrations is all the rage right now, it's just rebranded under "predictive maintenance". The Industrial Internet of Things crowd is all hyped up about it.
Nearly everything about working in modern corporations is terrible, so a good paycheck and not-too-stressful working conditions are close to the best that most of us can hope for.
Of course, there are those who make their way into the small clique of people at any tech company that get to do impactful, fulfilling work. But any given person is unlikely to be one of them, and if you want to be one of them you usually have to kill yourself working crazy hours first. (And probably afterwards too.)
What about getting a nice check while thinking you're doing something actually productive which ends up, in the final analysis, being meaningless? Alternatively, what about doing something that seems soul crushing and pointless but is actually productive?
Well yes, that's because they immediately tried to differentiate themselves with fun stuff like Microchannel Architecture to try and make their machines proprietary again.
I stayed for a few simple reasons...the fat consultant check I cashed every Friday. Exposure to some outstanding engineers and coders where I got to learn from true talent. The great strip club on A1A next to my rental in Lauderdale-by-the-Sea.
Heh... I did some work for IBM at the office of Cypress Creek Road back in that same time range. My fondest memory of the entire experience is eating at the Calypso Restaurant[1], a great Jamaican / Caribbean Islands place nearby.
I'd almost go back to Fort Lauderdale and work for IBM again (if they even still have a presence there) just for the Jamaican Jerk chicken from Calypso.
I heard this joke years before working at Kaleida Labs (a joint venture of Apple and IBM), but it rang true while I was living the joke:
Q: What do you get when you cross Apple and IBM?
A: IBM.
While I was working at Kaleida, I gave a wild ScriptX demo to Lou Gerstner using a bouncing eyeball to navigate a map of interactive rooms. After the demo, he complained that "The eyeball was a little too right-brained for me." I was all "Dag nab it, I should have used the other eyeball!!!"
"Offering managers didn’t have technical backgrounds and sometimes came up with ideas for new products that were simply impossible."
Sounds like they drank their own kool-aid, e.g., "Products That Enhance and Amplify Human Expertise," rather than understand the actual limitations and possibilities of ML. And it seems to me that they're still doing it with this nonsense about a human-level "AI" debating stack.
The oversell seems a real shame in light of how much good can be done with EMR and machine learning / NLP.
That is my biggest gripe with this as well. We certainly don't want an another AI winter, especially not in health, where I'm hoping (perhaps too optimistically) that it will allow better and cheaper care.
That's surely part of the problem, but the catalyst is the marketing strategy that is used to brainwash the employees. In essence; sell the experience, not the product.
This works well for IBM generally (the products are shit) but especially well for Watson because it's extremely easy to sell AI without getting bogged down in details. You want to identify brain tumors? We'll just teach Watson to do it.
Whilst IBM research might be able to pull it off, it'll never get to market because there is nobody capable of making good products at IBM anymore.
> Whilst IBM research might be able to pull it off, it'll never get to market because there is nobody capable of making good products at IBM anymore.
As an ex-IBMer this is so true and so frustrating at the same time. Engineers are thrashed about on a nearly sprintly basis by PM's with short attention spans and no understanding of how disruptive their continuously changing requirements are.
It doesn't help that IBM consistently puts the cart before the horse is even born and pivots multiple teams all at the same time such that nothing you build upon is stable or consistent. Working there was maddening.
Sounds like the way non-tech companies operate. Pretty damning that IBM can't understand why leadership of a tech company should have a tech background.
> This works well for IBM generally (the products are shit) but especially well for Watson because it's extremely easy to sell AI without getting bogged down in details.
The cynic in me says that every use of the term AI in any capacity is to sell experience and not functionality. When was the last time you used a product billed as 'AI' and thought 'wow, this is a huge game changer'? Siri is cool, but it's ultimately not super useful. Google translate is incredible, but it can only do what it can do because of the absolutely mind-boggling amount of training data that google can access. Most disciplines have the problem of not enough data, despite what 'big-data' folks say. In contrast, humans can extrapolate and make reliable predictions about the future based on really small sample sizes. We can pick up a new skill or recognize a new pattern with a high degree of accuracy really effing fast compared to a computer. This gives humans an enormous advantage. If IBM and anyone else in this space were really focused on delivering excellent real-world results, step 0 is building out world-class data integration and search tools (which we still actually suck at, weirdly.)
In fact, I would say apart from maybe self-driving cars, almost all of the biggest gains from machine learning are in unsexy, hidden backend problems, like automatically rectifying disparate data, optimizing resource utilization, flagging difficult-to-articulate events or triggers in a stream of data too large for human evaluation, machine translation, and other “unsexy” things.
Product interfaces usually offer simple features to users and the value proposition is easy to see. Effective use of machine learning is well hidden upstream in a bunch of unsexy preprocessing or heavy lifting to get to the interface. Not something you’d ever need to emphasize in marketing, except maybe at tech meetups or in recruiting materials, but not to the end consumer.
It just makes pop references to AI-powered products more egregious.
I use & depend upon plenty of products that are built upon AI - GMail spam filtering & categorized inbox, Google image search, YouTube & Netflix recommendations, cheque OCR at my ATM, predictive keyboards on my phone, Amazon's "people also buy with this product" feature, Google translate, computer opponents in games that I play, and all of the signals that feed into Google Search.
The irony is that not one of these bills itself as AI. It's just "a product that works", and the company that produces it is happy to keep the details secret and let users enjoy the product. So you may be right that the term "AI" itself is pure salesmanship. When it starts to work it ceases to be AI.
Also - humans only look like we're fast at picking up new domains because we apply a helluva lot of transfer learning, and most "new" domains aren't actually that different from our previous experiences. Drop a human in an environment where their sensory input is truly novel - say, a sensory deprivation tank where all visual & auditory stimulation is random noise - and they will literally go insane. I've got a 5-month-old and a project where I'm attempting to use AI to parse webpages, and I will bet you that I can teach my computer to read the web before I can teach my kid to do so.
None of the things you mentioned are even close to AI. They’re applied statistics, and they mostly use techniques we’ve known about for decades but have only now found a use case because computing and storage is cheap enough to make them viable.
The recommendation, translation, & image classification algorithms are all done with deep-learning; that's considered AI now.
There was a time, not all that long ago, when SVMs, Bayesian networks, and perceptrons were considered AI. That's behind the spam filters, predictive keyboards, and most of the search signals.
There was a time, a bit longer ago, when beam search and A* were considered AI. That's behind the game opponents.
As the linked Wikipedia article says, "AI is whatever we don't know how to do yet." There will be a time (rapidly approaching) where deep learning and robotics are common knowledge among skilled software engineers, and we won't consider them AI either. We'll find something else to call AI then, maybe consciousness or creativity or something.
This is my point: the term AI has always been BS. It was BS when beam search was AI, it was BS when expert systems were AI, and it is equally as BS when applied to neural networks. It comes to the same thing: the 'AI' tools we use are increasingly good function approximators. That's it. It's still reaching the moon by building successively taller ladders.
As much as I look into what’s being done with deep learning, I see they’re all stuck there on the level of associations. Curve fitting. That sounds like sacrilege, to say that all the impressive achievements of deep learning amount to just fitting a curve to data. From the point of view of the mathematical hierarchy, no matter how skillfully you manipulate the data and what you read into the data when you manipulate it, it’s still a curve-fitting exercise, albeit complex and nontrivial.
And
I left the arena to pursue a more challenging task: reasoning with cause and effect. Many of my AI colleagues are still occupied with uncertainty. There are circles of research that continue to work on diagnosis without worrying about the causal aspects of the problem.
the 'AI' tools we use are increasingly good function approximators
Nothing in the definition of AI says that AI has to work the same way the human brain does... and as far as that goes, we're probably not 100% sure that, in the end, the brain is anything more than a really good function approximator and some applied statistics.
I would say the canonical definition of AI, to the extent that there is one, is roughly something like "making computers do things that previously only humans could do". If people think "AI is bullshit" I'd say it's because they're applying their own definition to the term, where there definition imposes much more stringent requirements.
This is an interesting comment - where would you draw the line between AI and applied statistics? A lot of AI which happens to be ML (not saying there is non-ML AI, just that a significant chunk of AI being practiced today is ML) also happens to be applied statistics. Or have statistical interpretations.
Also, because something has been around for decades does not make it not AI. For ex the cheque OCR mentioned probably runs off (or can feasibly run off) of a neural network. I think the parent's comment holds well - not sure about the last line though ...
The line is clear: everything today branded "AI" is just applied statistics. AI is a buzzword. I don't know what the definition of intelligence is, but I have a feeling it doesn't rest anywhere near concepts like function approximation, and that's all even the most sophisticated "AIs" at Google or Facebook or Apple boil down to.
What was not clear from your earlier comment, and is now, is that when you say AI you don't mean AI as is practiced by most of academia and the industry but the vision of Artificial General Intelligence (AGI). If so, yes, that's a good point to make. However, it is debatable whether the path of statistical learning wont lead to AGI, or is not how our brains function, or the truth partly does comprise of statistical learning and part of something else. The Norvig-Chomsky debate is an example of the arguments on both sides.
I didn't make an earlier comment. You're replying to my one and only comment.
> when you say AI you don't mean AI as is practiced by most of academia and the industry but the vision of Artificial General Intelligence (AGI).
What I actually mean is people practicing what they call "AI" in academia and the industry have co-opted the name to make what they do sound more interesting. First it was called "statistics". Then it was called "pattern matching". Then it was called "machine learning". Now it's called "AI". But it hasn't changed meaningfully through any iteration of these labels.
I don't really think that characterization is fair, for example GANs, there is no data set of correct input output pairs for the function that is learned.
If you can definitely a problem rigorously, you've essentially defined a function. So "function approximation" is basically "general problem solving approximation".
Something that actually learns on its own and is not completely stumped when it encounters something new but actually learns. When it recognizes failure it should go and start learning by itself, i.e. try to get more data and analyze that and do its own trial and error - so that it actually grows in capabilities (on its own).
>The irony is that not one of these bills itself as AI. It's just "a product that works"
I think you are on to something, put differently:
If you need to use the term "AI" to enhance the marketability of the product it is probably because the product sucks.
And employees. Google's embrace of the term "AI" isn't because they need help developing or selling AI-powered products, it's to encourage all the kids to go into computer science and all the existing developers to learn TensorFlow. They can then pick off the best of them as potential employees without having to train them up themselves.
I would argue that all of your examples have failed to be anything even remotely resembling AI, just data crunching to fit most use cases. I don't use GMail but I do regularly use Google image search, Translate, YouTube, Netflix and predictive typing via SwitfKey. And IMHO they all suck horribly (SwiftKey still sucks pretty bad after 8 years of learning from me). Google Translate is getting better and I have recently started using first-pass Google Translate before correcting the mistakes... instead of everything by hand. YT/Netflix Recommendations are always bullshit. I wish there was a way to say "never show me anything like this ever again" because I often feel like 90% of the recommendations make absolutely no sense. Sometimes I think that someone else must be logged into my account clicking on things just to mess with my recommendations. I usually spend a minimum of 30 minutes searching, often giving up out of frustration (and I always have an IMDB tab open to check details because all of the IMDB rating plugins for Firefox stop/ped working). Maybe I'm an edge case living outside the U.S.? Are their algorithms only tuned for English-speaking countries?
The most creative, intelligent and least frustrating "AI" I've ever encountered was in some games, such as Dota2 or many years ago F.E.A.R. They were frustrating but only due to unpredictability, even after hundreds of hours of playtime. YouTube and NetFlix AI after hundreds/thousands of hours invested are also very unpredictable and frustrating, but that's the opposite experience I am looking for in those situations.
Completely have to agree. YouTube has so much content, far more than Spotify, Vimeo and everybody else in the space, which is why I use it. But the recommendations are an offense. YT is only good at 'recommending' stuff I already watched or listened to. What's the point?
Translate can be useful at times...like once a year when I want to comprehend a Japanese website, usually I close the tab after 2 minutes.
I used GMail for many years and still do to some degree but I'm moving to a different mail provider. GMail's spam filter is great!
Not sure, since 2 years it became acceptable to make no difference between ML and AI. ML appears smart because of bizillions of training samples and I feel very impressed when I hear of that. But yeah, at the end of the day it doesn't have exactly the biggest impact on me... ;)
> humans can extrapolate and make reliable predictions about the future based on really small sample sizes
You severely underestimate the bandwidth of your eyes and ears and other senses, and the volume of your brain's memory (despite it's uber-loosy compression). That's terabytes a day probably, if not big data than I dunno what is. Yeah, 99% of it is thrown away at passing through the first few hundreds of layers of your neural networks, but they still know what to throw away...
To get a digital computer on "equal" terms with the zillions of hacky optimizations your semi-analog brain uses you need a shitton of raw power and data volume ("if you don't know what to throw away of the input data, you need to just sift through all/more of it") to compensate for the fact that you don't have N million years of evolution to devise similar hacky optimizations.
Also, humans work as a "network of agents", that's also recurrent (aka "culture"). Current sub-human-level AI agents are far from any sort of reliable interop.
My guess is that we'll get human level performance levels at AGI tasks when we learn to build swarms of AI-agents that cooperate well and "model each other", and few people are working on this... Heck, when it happens it will probably be an "accident" of some IoT optimizations thing, like, "oops, the worldwide network of XYZ industrial monitoring agents just reached sentience and human level intelligence bc it was the only way it could solve the energy-efficiency requirement goals it was tasked to optimize for" :)
> We can pick up a new skill or recognize a new pattern with a high degree of accuracy really effing fast compared to a computer.
Evolution by natural selection is the OG genetic algorithm, and it's been "running" on billions of organisms in parallel for hundreds of millions of years. The intuition that we take for granted such as the abstract concept of a shape is all hard-coded in our brains from trial and error.
Agree. Consultancies rarely sell products or solutions. Instead they sell project management by making it sound like you are a more safe bet than a smaller product studio who actually do make products work. Its a real shame but i mostly blame the zero mistake KPI culture primarily fueled by how managers on the client side are promoted.
Over the past 10 years I’ve been surprised when anyone smart would join IBM. I understand why people hung on, but why join that dying ship? Now I can’t think of any still there.
Who says IBM is dying? They aren’t anymore (since a long time) at the bleeding edge of research, but they still have a solid consulting and integration business. They also offer decent salaries and good opportunities for sales-oriented technical people.
It wouldn’t be my first choice of employer, however I can see how some people would enjoy working there.
Dying may be extreme. Perhaps it’s safer to say that they are in the mode of returning capital to investors rather than growing.
I’m at a large customer of theirs and they are bleeding the customer for every dollar as they get phased out. Very low caliber of services professionals too.
It's double whammy for the Watson stuff, because not only did IBM lose track with AI, they also had that whole cloud thing whoosh by them. So not only do you have marketing telling outrageous lies about the abilities of their AI systems, they're also exaggerating the cloud angle where Watson is a cornerstone.
Yep. But if your marketing and product management are all focused on selling AI instead of IR, then they're not really working toward finding a way to deliver the IR value they have to the people who need it.
I'd actually like to give Watson a spin for an IR problem I'm looking at, but, thanks to their hype machine being set to overdrive, they've got the thing priced in the "The Bold Leaders of the Future Creating a Bright New Tomorrow Full of People in Glasses Staring Wistfully Toward the Right Edge of the Photograph, While Blue Curvy Streaks Wave Through the Background and Random Zeroes and Ones Float Around Their Heads" tier. Sadly, I've only got a "businesses solving business problems" sized budget.
Jeopardy is an information retrieval problem in a game show format, with the minor twist that the query is phrased as a declarative sentence and the response is phrased as an interrogative one.
AI is not simply things that a computer can't do yet. But I think most of people who aren't currently trying to sell a piece of software would expect AI to include some things that you don't need to do to play Jeopardy. I'd want to see general-purpose pattern recognition, for example.
There's a pretty good podcast interview with Eugene Dubossarsky that has relevant discussion about issues with management and data science in general. https://www.datafuturology.com/podcast/1
Here are a few of my notes (my words not the interviewee's):
- in order to use data science, you have to have creative people thinking about data on the front end
- they don't have to be data scientists, but they need to be creative and want data to support decisions and iteration via feedback loops
- that creativity and desire will lead to "doing good data science"
- management on the receiving end of data science output must be intelligent in terms of synthesizing many inputs and have a strong desire to puzzle through the implications. If management is asking the data science to actually make the decisions - the situation is broken
- data science must be done with provisions for decision support and feedback loops; this is the output that is helping drive the business.
- Lack of desire for decision support and feedback loops leads to "fancy pets" and management using data science as a means to brag about what they are doing; but the data science might not being doing anything to drive the business meaningfully.
- data science that attempts to actually make decisions vs providing decision support is likely in the category of "commodity data science". Corollary : non-commodity data science is the kind that supports decisions in executing higher-level business strategy. Strategy at that level has rather unique attributes and is embedded in unique circumstances for a particular business. This requires a good data scientist to help tackle.
BTW - in listening to that podcast I found a lot of parallels with database design.
whenever I'm asked to design a database for an early-stage system (I work in early stage tech ventures), I ask the following:
- what are the questions that this database should answer for you? How are those questions supporting your business goals 3,6,12 months out? (I'm trying to get to the business requirements here)
- who will be asking those questions (I'm trying to put together some user personas in my head)
- how frequently will they be asking these questions? corollary: how often will historical data be needed? (I'm thinking hot vs cold and complexity of retrieval, minimally required performance)
- how much data to we anticipate is needed to answer the questions (this is really tricky in new ventures - often the answer is more data than what will actually occur in practice in the first year)?
- finally, what systems & tools are people using to ask the questions and be notified of events? (I'm thinking about interfacing, apis)
its all an attempt to stay very focused on the questions and business drivers and the people who use the answers.
This is a great summary. I think his guideposts are helpful for most decision support initiatives, whether you're using data to try and support decisions, or reaching out to humans.
We run prediction markets inside companies and find that if we don't establish a good lifecycle of asking forecasting questions, having people respond with probabilities, then decision makers REACTING to those probabilities in some way (whether they agree with them or not, just acknowledge their existence) the likelihood of the project failing is far higher.
following is a text snippet from the Article . I do not know which part of the following is AI , it is pure "data analysis" writing few SQL queries . This is the Biggest problem of IBM, they bill these kind of things as AI .
> A clinic could use the system to search its patient records and find, for example, all the men over age 45 who were overdue for a colonoscopy, and then use an autocall to remind them to schedule the dreaded appointment.
Being directly involved in the execution of the example you gave, I believe ML would be unnecessary and error-prone.
Even if successful, a system which could "interpret" a health record (such as a freetext note) using anything other than properly codified data would set the health industry back a decade. Moving doctors away from freetexting their notes is the only way to advance the industry.
It will be a generational shift. You won't get current doctors (over 40) to change their ways, ever.
My previous doctor (who was probably mid-50s) didn't use email or any kind of secure electronic messaging system. Everything had to be faxed to him.
My new doctor who is younger uses all kinds of digital tools including a voice recorder with a pre-trained text-to-speech engine that understands medical terminology and codifies the transcription based on keywords.
So it's not entirely getting away from freetext but at least it's extracting some structured data from it automatically.
Other reports seemed to indicate that Watson (and maybe all AI) requires a lot of very careful, slow, and somewhat arduous data entry and testing to get good results.
I can imagine someone who doesn't know at IBM selling a product:
"Hey we will solve all these problems like magic!"
Then IBM comes back:
"Hey do you have all this data in a specific format and a ton of time to enter and test it and maybe we'll get back to you???"
That's a big loss of trust there with the customer if you come back with that.
It seems like these are products where a lot of caveats needs to be made clear to customers and a real careful technical partnership formed with them to succeed long term. You have to bring the customer along for the ride and exploration and keep them excited for a long time it sounds to make it work.
Are the articles at ieee.org usually technically competent?
To see one of their articles with the common press confusion of mixing different definitions and interpretations of AI (correct or incorrect) doesn’t help build confidence.
For example, what was used to play Jeopardy vs. approaches being taken to improve cancer treatment, are just so different, it seems almost disingenuous to throw it all haphazardly into one conceptual bucket.
The article does IBM a disservice in some ways. They come off looking bad overall but some of the failed projects mentioned like MD Anderson, failed for reasons beyond any control they had, other than recognizing some obvious red flags earlier and detaching their name and participation from it.
On the other hand I believe the article lets them offf the hook to easily when they bring out the old trope they’ve been using for years, which is encapsulated here:
“IBM Watson has great AI” [one engineer said] “It’s like having great shoes, but not knowing how to walk—they have to figure out how to use it.”
It doesn’t make sense to say, xyz is great we just have to figure out how to use it, as a stand alone argument. It’s nonsensical unless you mention something about the seeming implied untapped potential, specific innovations, novel approach, or whatever makes it great.
I’m not familiar with all their IP so maybe there are some great things, you just don’t get to claim that and get off the hook so many times in the press without providing at least some detail or reference point.
IMO, this is a bigger issue in the industry. People are hyping up ML/AI to the point where the actual application is either impossible or extremely difficult. Just look at how many people are so fearful of AI/ML taking away jobs and replacing humans. Anyone in the field knows that AI/ML can take away jobs but it is more of the low-end jobs and everything happens gradually rather than immediately. AI/ML is being more as a tool to enhance human productivity rather than as a direct replacement for entire occupations unless the job is very basic to begin with.
The article only mentions the engineers being laid off, not the managers who botched it, nor the executives who hired with incompetence. The reward structure of such a corporate environment would seem counter-intuitive to success.
Who makes money by pointing out flaws at IBM? Their stock on NASDAQ is 139.31 as of Jun 25 12:58 PM ET. Is there a large enough short position to be worth buying a news article?
It only reveals IBM's problem with AI if you were at any point under the impression that it was something more than marketing for them and watch actual market signals.
The audience that gobbles up their ad campaign during the Masters that touted their "block chain" logistics probably wouldn't even notice that they had layoffs at Watson health.
It seems to me that Watson is basically just IBM's version of AWS/GCE services (at least the non infra ones). But it gets thrown around as a buzzword so often. The marketing makes it look like there's a single AI codebase that can be accessed through a bunch of APIs, but I would be very surprised if that was actually the case.
It's not the case. A year or two ago some of my friends who work at IBM couldn't tell me themselves what Watson really is. As of today I understand Watson as everything from IBM that can be related to AI, I.e. it includes IoT as well because the data could be used by AI.
tl;dr: it's a good search engine and a disparate set of machine learning tools. Sales is promising Hollywood AI, but the reality is that it takes a sizable project team to build anything worthwhile.
Scanning past commentary, it seems that startups are eating their lunch (more nimble, dedicated to customer space). I'll add that half the machine learning battle is getting access to data, so hyping the brand makes sense strategically.
Both, sort of. There is a "thing" called Watson, which is related to the Watson that played Jeopardy. But "Watson" is also a brand which lumps in stuff that has absolutely nothing to do with the "old" Watson.
To illustrate a bit.. "Watson Health" is (or was) made up of a ton of people and technologies who came into IBM as the result of several acquisitions: Truven, Phytel, Explorys, etc. In many cases, they repackaged stuff from those vendors, gave it a "Watson name" and shipped it. And some of this stuff was literally no more sophisticated than linear regression / logistic regression, etc.
"IBM Watson Marketing Automation" being another I was made painfully aware of recently, which was the result of the Silverpop acquisition at least in part.
This sounds like a case where the AI is used in a "marketing" way to make people interested in a product, and then there isn't really much AI involved, and the people developing the AI have a struggle to prove that it's relevant to the business.
I wonder if DeepMind at Google has a similar problem. It is certainly getting a lot of headlines, but there are plenty of other AI groups within Google that do business-relevant things like improve search or ad matching or make Google Home's voice recognition work. I would not be surprised if in the long run DeepMind becomes a group that performed a neat stunt with Go, but kind of fades in practical relevance, like Watson with Jeopardy.
> This sounds like a case where the AI is used in a "marketing" way to make people interested in a product, and then there isn't really much AI involved, and the people developing the AI have a struggle to prove that it's relevant to the business.
It's pretty common in many industries to have products to showcase your chops while providing zero real world value and zero sales.
I've always viewed DeepMind as more of a skunk works program and less as a profit driven enterprise. DeepMind exists primarily to push the limits of what can be done when you put group of leading researchers together in a room, provide them with nearly limitless resources, and simply tell them to "go". I expect some of that effort to eventually trickle down into Google's consumer products (maybe a healthcare focused version of AutoML https://cloud.google.com/automl/). Google has already done a lot of work on the HIPPA side of things (https://cloud.google.com/security/compliance/hipaa/)
The health care landscape is strewn with the wreckage of software companies who thought the latest shiny software doodad could cause a "disruption" like it had in so many other industries.
People who know that healthcare is different try to warn them. They don't listen. Instead they charge in with people who have no experience in the field.
From the article:
After the acquisition, IBM management started the process known internally as “bluewashing,” in which an acquired company’s branding and operations are brought into alignment with IBM’s way of doing things. During this bluewashing, “everything stopped,” the first Phytel engineer says, and the workers were told not to focus on improving their existing product for current clients. “People were sitting around doing nothing for almost a year,” the second engineer says.
This stuff is the friggen worst if you ask me. i'm so done with the AI hype train and IBM is the worst. Linked article mentions a 'watsons law' (similar to moore's law etc). If you ask me, it is more likely for watsons law to be that all commercial BigCo 'AI' offerings will burn thru hundreds of millions and ultimately fail rather then the intended meaning.
"Phytel’s contribution was analytics paired with an automated patient communication system. A clinic could use the system to search its patient records and find, for example, all the men over age 45 who were overdue for a colonoscopy, and then use an autocall to remind them to schedule the dreaded appointment"
This shit isnt AI it's literally a database query and then some 3rd party library to send a text message or a phone call.
Watson's law: as the complexity of technology and business processes increases, the amount of time it takes for people to recognize and acknowledge that the emperor in not in fact wearing clothes increases in proportion to the profitability of the lie being sold.
IBM was simply an AI hype train grifter. AI is still valuable (although it's not advancing as quickly now as it was in, say, 2015), it's just that you have to dig deep to make sure that the "AI" being used is actually AI and that it's being used in an appropriate way.
AI as a research field generates plenty of useful applied techniques that make real products possible. The problem, which Watson is ground zero of, is calling AI itself a product -- like something you can just mix in and make business processes better.
AI seems more and more, in communication to the general public, to mean "computer program that does something useful, that hasn't yet been commonly called a computer program."
A little case history - there were two illuminating threads ~10 months back where several current & former IBM employees commented on the growing disconnect between the reality vs. marketing of Watson - looks like vindicated by today's news:
I was happily surprised the other day when the IBM debater was presented and there was no mention of it being Watson anywhere on their website. They really need to stop with this personification.
"layoff" being a euphemism for "mass firing of people", that has been around so long it doesn't even seem like a euphemism any more. As you said, it's a long list.
The problem at IBM is not technological; it's managerial.
For a long while now, IBM has been treating "AI" as a product that can be managed, packaged, and sold by "general" business managers -- think MBA-types with only a superficial, qualitative grasp of deep learning and AI. Doing that with rapidly evolving technology is a sure-fire recipe for failure.
Most such MBA-types today are ill-equipped to manage, package, and sell "AI." They're roughly in the same position as English or History majors who are asked, say, to manage, package, and sell a new kind of quantum-computing technology without knowing or understanding much about quantum physics. The technology is moving faster than their ability to keep up.
IBM's mismanagement is a shame, because the system they showcased nearly a decade ago -- the one that competed and won in Jeopardy -- was state-of-the-art at the time.
MBAs vs. undergrads with liberal arts degrees is a whole other discussion. An argument can definitely be made for a liberal arts skill-set in terms of adaptability and capacity to quickly learn other skills/subjects. Not sure if that encompasses something as complicated or esoteric as the technicalities of AI/ML, though.
To be honest, I would rather work for someone who just got an undergrad liberal arts degree, who is aware they don't know technology. They can learn, and there's plenty of stuff about communicating with (and for) customers that I don't know or don't do well, so it can work well. Which is all theoretically true of an MBA too, but...
It's the entire company, not just these Watson groups. They have a history of buying companies or taking over I/T departments and re-badging employee's only to ax nearly all of them once the knowledge is transferred to some 3rd world country workforce.
Stories like this are not a surprise, it's IBM's way of doing business. Maybe Watson will learn HR and just fire everybody from middle mgmt on up..
erikpukinskis, airstrike, jeffjose: I did not criticize MBA's in general!
My comment mentioned specifically "MBA-types with only a superficial, qualitative grasp of deep learning and AI."
MBAs who understand what they're managing (and who know what they don't know) are not in that group. And BTW, I suspect most MBAs who read HN are not in that group either :-)
Watson seems to be hyped as powering the entire world but I have yet to see a real project using it in any capacity. I haven't even heard a cohesive description of what Watson even is, beyond surmising that it's a suite of AI-like services, although any APIs seem to be hidden within the broken IBM cloud interface, perhaps on purpose.
> They couldn’t decide on a roadmap,” says the second engineer. “We pivoted so many times.”
> Both Phytel engineers say the offering managers didn’t have technical backgrounds and sometimes came up with ideas for new products that were simply impossible.
The death knell of all (potentially) good products. I don't know why this is so often the case. All software companies need engineers involved in product development decisions. Period. It's not optional.
Facebook who was smart about this. They hired or retrained technical people to fill many business roles in marketing, product development, project management, etc.
I'm not sure why technical people are restricted to merely being the builders in these companies. Lots of other companies recruit internally from people familiar with the end product and train them in other business areas.
> these potential customers weren’t impressed. Instead they asked for something resembling Phytel’s old system.
So they simply imagined a new product without interviewing potential customers beforehand on what they actually want? They spent years merging databases of two big systems, pivoted multiple times, to find out there wasn't a market for it in the first place?
Why aren't the 'offering management' people getting fired?
IBM's organizational structure is book-ended with great talent. The engineers, developers, and even front-line managers are really fantastic, and the people at the very top are pretty good.
In the middle, there are 100 layers of middle managers that completely cock everything up, and the really sad part is that they have enough say to really cause damage. One of my first proper white collar technical jobs with them was an L2 support job for this network performance monitoring suite for huge networks... mostly large, national ISPs and the like. The job required maybe a just-post-jr-level sys-admin knowlege of networks and UNIX systems while also having smooth customer service skills. Definitely a great step up from my previous lower-mid-level IT jobs and call center work.
I had three(3) managers. Three! I had a technical manager, a non-technical manager, and my actual manager, who was the head of the department.
At the highest levels, the management was talking about switching everybody's workstation over to Linux. Everybody from admin assistants to developers to managers was supposed to be moved off of Windows at some point in the relatively near future. I was psyched— I hated windows, and the product I supported ran on Solaris, so not having to deal with the extremely primitive (at the time) tools like Cygwin to get some UNIX functionality on my machine was great. They seemed to be positioning themselves to sell the consulting for other large companies to do the same thing.
Though we got no word of this internally— I only knew from what I had read in articles— I found the internal workstation disk image on the intranet and eagerly installed it. It was pretty smooth! I was excited! As I was getting my tools set up, I noticed that it didn't have the internal bug/ticket tracking clients installed, so I cruised on over to their intranet page... hmmm, nothing listed for Linux. After hours of searching, I found some internal discussion showing that, months earlier, the department that writes that software unilaterally decided that they were discontinuing their initiative to port those applications to Linux. While there was an extremely limited CLI to these tools, critical functionality was literally impossible without the GUI app. Without the ability for anybody on their Linux workstations to interact with tickets or bug reports, the Linux initiative was pretty much dead-in-the-water for most technical people and their managers.
Perfect example of just how badly their forest of middle managers completely messes up great executive initiatives that the bottom of the food chain really wants to embrace.
(I might have gotten some of the details wrong. It was 13 or 14 years ago and I drank a lot back then.)
IBM Watson is bad for the "AI" community because when non-experts see IBM repeatedly fail they assume the whole field is nonsense. Hopefully IBM will be more cautious in what they claim to be possible. Hype and deceit do not belong in healthcare.
I found out that Tesla's autopilot technology can't even really detect stationary objects. Albeit this is hard technically as it's actually a point of reference physics problem but the writing on the wall is clear: marketing departments are overpromising and no one is really delivering big on AI, not even Google as a lot of other companies have already matched their efforts (Microsoft, Waze, etc.). IBM? Give me a break.
I think self-driving cars are possible, and will probably become a reality. However I don't think it will be a generalized AI driving it. I'm thinking more along the lines of a lot of subsystems, one of which is visual object detection with deep learning (the one thing deep learning has shown it beats everything else), all the other subsystems from the sensors up to the driving behavior will have to be Properly Engineered(TM).
The problems with Watson does not mean that this is a problem with AI. If history is any guide IBM fails but eventually succeeds during most technological revolutions (PC, ecommerce etc)
They can't just build a system, and leave it for the customers to do all the hard work of integrating various systems.
Integration is the key and should be seen as product in itself.
If IBM stops milking the cow like the services, asking the customers to integrate, It will die a miserable death, and unfortunately a premature one.
It should be pluggable, like set of pick and choose building blocks of interfaces, that only should take domain expertise and custom specifications as input from the client (Which ever domain.) Until then AI only will become ambitious Sunk cost.
Look I think everybody needs to cut IBM some slack here. Integrating technology with customer needs is hard, and it takes a while to get good at it, to get a process for reconciling with product managers want with what developers can actually make. Once IBM has been in this technology game for a little while, I'm sure they'll get the hang of it.
People aren't faulting IBM for tackling hard systems-integration problems.
People are faulting IBM for over-promising, under-delivering, using misleading advertising, and internally seeming to have foolish management practices.
263 comments
[ 3.6 ms ] story [ 247 ms ] threadI had a contract gig at IBM Advanced Technology in Boca for about 18 months in early 2001-2002. Talk about missing the boat...
I was brought in to prop up a soon-to-be-failed project for the Japanese government...basically a "Napster for Tokyo" that would allow paid-for-play C2C song sharing for customers of "the Big 5" record companies.
I asked simple questions that no one could answer...why would people pay for content when it was so easily available via other means? you are using DRM how??? really? you need a special player to play the music?
Why would anyone do that?
I stayed for a few simple reasons...the fat consultant check I cashed every Friday. Exposure to some outstanding engineers and coders where I got to learn from true talent. The great strip club on A1A next to my rental in Lauderdale-by-the-Sea.
But reading over this article reminds me that IBM is just too big to get out of its own way, and has been for the longest time.
[edits]
Given all the expectation of its product and the importance of ML/AI, it makes sense this was impossible.
Two years ago at their vegas conference they had a coffee shop that used AI to recommend coffee types. I thought "boy, they don't understand this technology".
I worked on another one for this company called Vibration Advisor which diagnosed odd noises in GM cars.
Another interesting thing was the transition from special purpose hardware - Lisp machines - to C code on commodity platforms. A contrast from today's ML moving in the other direction.
It'll be interesting to see when specialized ML focused silicon will become readily available. Right now I find ML libraries that are able to run on blended architectures (any combination of CPU and GPU's) much more exciting/impactful than TPU's. The ability to deploy on just about any cluster a customer may have available is huge.
Are we close to it being technically feasible , leaving aside regulation and interpersonal qualities doctors bring to the table ?
The likes of INTERNIST, CADUCEUS, and MYCIN have been around and provably accurate starting in the late 70s through the mid-80s. MYCIN even arguably sparked the 1st AI boom. But there were ethical issues with computer-aided diagnosis that I'm not sure have been solved/overcome.
Perhaps the current startup generation can get past them with Zuckerberg, Kalanick and Holmes as role models. :)
Of course, there are those who make their way into the small clique of people at any tech company that get to do impactful, fulfilling work. But any given person is unlikely to be one of them, and if you want to be one of them you usually have to kill yourself working crazy hours first. (And probably afterwards too.)
Brilliantly put
Funny thing is, when it did get out of its own way, what did we get? The IBM PC
Heh... I did some work for IBM at the office of Cypress Creek Road back in that same time range. My fondest memory of the entire experience is eating at the Calypso Restaurant[1], a great Jamaican / Caribbean Islands place nearby.
I'd almost go back to Fort Lauderdale and work for IBM again (if they even still have a presence there) just for the Jamaican Jerk chicken from Calypso.
[1]: http://www.calypsorestaurant.com/
Q: What do you get when you cross Apple and IBM?
A: IBM.
While I was working at Kaleida, I gave a wild ScriptX demo to Lou Gerstner using a bouncing eyeball to navigate a map of interactive rooms. After the demo, he complained that "The eyeball was a little too right-brained for me." I was all "Dag nab it, I should have used the other eyeball!!!"
https://medium.com/@donhopkins/1995-apple-world-wide-develop...
"The last thing IBM needs right now is a vision." -Lou Gerstner
https://en.wikipedia.org/wiki/Louis_V._Gerstner_Jr.
Sounds like they drank their own kool-aid, e.g., "Products That Enhance and Amplify Human Expertise," rather than understand the actual limitations and possibilities of ML. And it seems to me that they're still doing it with this nonsense about a human-level "AI" debating stack.
The oversell seems a real shame in light of how much good can be done with EMR and machine learning / NLP.
This works well for IBM generally (the products are shit) but especially well for Watson because it's extremely easy to sell AI without getting bogged down in details. You want to identify brain tumors? We'll just teach Watson to do it.
Whilst IBM research might be able to pull it off, it'll never get to market because there is nobody capable of making good products at IBM anymore.
As an ex-IBMer this is so true and so frustrating at the same time. Engineers are thrashed about on a nearly sprintly basis by PM's with short attention spans and no understanding of how disruptive their continuously changing requirements are.
It doesn't help that IBM consistently puts the cart before the horse is even born and pivots multiple teams all at the same time such that nothing you build upon is stable or consistent. Working there was maddening.
The cynic in me says that every use of the term AI in any capacity is to sell experience and not functionality. When was the last time you used a product billed as 'AI' and thought 'wow, this is a huge game changer'? Siri is cool, but it's ultimately not super useful. Google translate is incredible, but it can only do what it can do because of the absolutely mind-boggling amount of training data that google can access. Most disciplines have the problem of not enough data, despite what 'big-data' folks say. In contrast, humans can extrapolate and make reliable predictions about the future based on really small sample sizes. We can pick up a new skill or recognize a new pattern with a high degree of accuracy really effing fast compared to a computer. This gives humans an enormous advantage. If IBM and anyone else in this space were really focused on delivering excellent real-world results, step 0 is building out world-class data integration and search tools (which we still actually suck at, weirdly.)
Product interfaces usually offer simple features to users and the value proposition is easy to see. Effective use of machine learning is well hidden upstream in a bunch of unsexy preprocessing or heavy lifting to get to the interface. Not something you’d ever need to emphasize in marketing, except maybe at tech meetups or in recruiting materials, but not to the end consumer.
It just makes pop references to AI-powered products more egregious.
The irony is that not one of these bills itself as AI. It's just "a product that works", and the company that produces it is happy to keep the details secret and let users enjoy the product. So you may be right that the term "AI" itself is pure salesmanship. When it starts to work it ceases to be AI.
https://en.wikipedia.org/wiki/AI_effect
Also - humans only look like we're fast at picking up new domains because we apply a helluva lot of transfer learning, and most "new" domains aren't actually that different from our previous experiences. Drop a human in an environment where their sensory input is truly novel - say, a sensory deprivation tank where all visual & auditory stimulation is random noise - and they will literally go insane. I've got a 5-month-old and a project where I'm attempting to use AI to parse webpages, and I will bet you that I can teach my computer to read the web before I can teach my kid to do so.
There was a time, not all that long ago, when SVMs, Bayesian networks, and perceptrons were considered AI. That's behind the spam filters, predictive keyboards, and most of the search signals.
There was a time, a bit longer ago, when beam search and A* were considered AI. That's behind the game opponents.
As the linked Wikipedia article says, "AI is whatever we don't know how to do yet." There will be a time (rapidly approaching) where deep learning and robotics are common knowledge among skilled software engineers, and we won't consider them AI either. We'll find something else to call AI then, maybe consciousness or creativity or something.
As much as I look into what’s being done with deep learning, I see they’re all stuck there on the level of associations. Curve fitting. That sounds like sacrilege, to say that all the impressive achievements of deep learning amount to just fitting a curve to data. From the point of view of the mathematical hierarchy, no matter how skillfully you manipulate the data and what you read into the data when you manipulate it, it’s still a curve-fitting exercise, albeit complex and nontrivial.
And
I left the arena to pursue a more challenging task: reasoning with cause and effect. Many of my AI colleagues are still occupied with uncertainty. There are circles of research that continue to work on diagnosis without worrying about the causal aspects of the problem.
I don't see how this follows from this:
the 'AI' tools we use are increasingly good function approximators
Nothing in the definition of AI says that AI has to work the same way the human brain does... and as far as that goes, we're probably not 100% sure that, in the end, the brain is anything more than a really good function approximator and some applied statistics.
I would say the canonical definition of AI, to the extent that there is one, is roughly something like "making computers do things that previously only humans could do". If people think "AI is bullshit" I'd say it's because they're applying their own definition to the term, where there definition imposes much more stringent requirements.
Yeah, maybe the marketing people said that..
Also, because something has been around for decades does not make it not AI. For ex the cheque OCR mentioned probably runs off (or can feasibly run off) of a neural network. I think the parent's comment holds well - not sure about the last line though ...
> when you say AI you don't mean AI as is practiced by most of academia and the industry but the vision of Artificial General Intelligence (AGI).
What I actually mean is people practicing what they call "AI" in academia and the industry have co-opted the name to make what they do sound more interesting. First it was called "statistics". Then it was called "pattern matching". Then it was called "machine learning". Now it's called "AI". But it hasn't changed meaningfully through any iteration of these labels.
To your point, I agree. They hype around the area has evolved much faster than the area itself.
then what is 'close to AI'?
FWIW, "AI" as a field has been around since the 1950's. So calling something "AI" in no way implies that the techniques are especially new.
I think you are on to something, put differently: If you need to use the term "AI" to enhance the marketability of the product it is probably because the product sucks.
YouTube recommendations aren't great, they have a short memory and my feed is rarely diverse, it just shows a bunch of whatever I just watched.
The most creative, intelligent and least frustrating "AI" I've ever encountered was in some games, such as Dota2 or many years ago F.E.A.R. They were frustrating but only due to unpredictability, even after hundreds of hours of playtime. YouTube and NetFlix AI after hundreds/thousands of hours invested are also very unpredictable and frustrating, but that's the opposite experience I am looking for in those situations.
Translate can be useful at times...like once a year when I want to comprehend a Japanese website, usually I close the tab after 2 minutes.
I used GMail for many years and still do to some degree but I'm moving to a different mail provider. GMail's spam filter is great!
Not sure, since 2 years it became acceptable to make no difference between ML and AI. ML appears smart because of bizillions of training samples and I feel very impressed when I hear of that. But yeah, at the end of the day it doesn't have exactly the biggest impact on me... ;)
You severely underestimate the bandwidth of your eyes and ears and other senses, and the volume of your brain's memory (despite it's uber-loosy compression). That's terabytes a day probably, if not big data than I dunno what is. Yeah, 99% of it is thrown away at passing through the first few hundreds of layers of your neural networks, but they still know what to throw away...
To get a digital computer on "equal" terms with the zillions of hacky optimizations your semi-analog brain uses you need a shitton of raw power and data volume ("if you don't know what to throw away of the input data, you need to just sift through all/more of it") to compensate for the fact that you don't have N million years of evolution to devise similar hacky optimizations.
Also, humans work as a "network of agents", that's also recurrent (aka "culture"). Current sub-human-level AI agents are far from any sort of reliable interop.
My guess is that we'll get human level performance levels at AGI tasks when we learn to build swarms of AI-agents that cooperate well and "model each other", and few people are working on this... Heck, when it happens it will probably be an "accident" of some IoT optimizations thing, like, "oops, the worldwide network of XYZ industrial monitoring agents just reached sentience and human level intelligence bc it was the only way it could solve the energy-efficiency requirement goals it was tasked to optimize for" :)
Sample size and record size are two different things.
This is so common there’s a term for it https://en.m.wikipedia.org/wiki/Moravec%27s_paradox
Evolution by natural selection is the OG genetic algorithm, and it's been "running" on billions of organisms in parallel for hundreds of millions of years. The intuition that we take for granted such as the abstract concept of a shape is all hard-coded in our brains from trial and error.
No. Siri is shit.
I’m at a large customer of theirs and they are bleeding the customer for every dollar as they get phased out. Very low caliber of services professionals too.
Watson is a $@#% amazing information retrieval system. Information retrieval is only a small part of what people think of when they think, "AI".
Which I imagine still has lots of value on it's own for large, complicated data sets.
I'd actually like to give Watson a spin for an IR problem I'm looking at, but, thanks to their hype machine being set to overdrive, they've got the thing priced in the "The Bold Leaders of the Future Creating a Bright New Tomorrow Full of People in Glasses Staring Wistfully Toward the Right Edge of the Photograph, While Blue Curvy Streaks Wave Through the Background and Random Zeroes and Ones Float Around Their Heads" tier. Sadly, I've only got a "businesses solving business problems" sized budget.
AI is not simply things that a computer can't do yet. But I think most of people who aren't currently trying to sell a piece of software would expect AI to include some things that you don't need to do to play Jeopardy. I'd want to see general-purpose pattern recognition, for example.
Here are a few of my notes (my words not the interviewee's):
- in order to use data science, you have to have creative people thinking about data on the front end
- they don't have to be data scientists, but they need to be creative and want data to support decisions and iteration via feedback loops
- that creativity and desire will lead to "doing good data science"
- management on the receiving end of data science output must be intelligent in terms of synthesizing many inputs and have a strong desire to puzzle through the implications. If management is asking the data science to actually make the decisions - the situation is broken
- data science must be done with provisions for decision support and feedback loops; this is the output that is helping drive the business.
- Lack of desire for decision support and feedback loops leads to "fancy pets" and management using data science as a means to brag about what they are doing; but the data science might not being doing anything to drive the business meaningfully.
- data science that attempts to actually make decisions vs providing decision support is likely in the category of "commodity data science". Corollary : non-commodity data science is the kind that supports decisions in executing higher-level business strategy. Strategy at that level has rather unique attributes and is embedded in unique circumstances for a particular business. This requires a good data scientist to help tackle.
(hope this is useful)
(edit typos,grammar)
whenever I'm asked to design a database for an early-stage system (I work in early stage tech ventures), I ask the following:
- what are the questions that this database should answer for you? How are those questions supporting your business goals 3,6,12 months out? (I'm trying to get to the business requirements here)
- who will be asking those questions (I'm trying to put together some user personas in my head)
- how frequently will they be asking these questions? corollary: how often will historical data be needed? (I'm thinking hot vs cold and complexity of retrieval, minimally required performance)
- how much data to we anticipate is needed to answer the questions (this is really tricky in new ventures - often the answer is more data than what will actually occur in practice in the first year)?
- finally, what systems & tools are people using to ask the questions and be notified of events? (I'm thinking about interfacing, apis)
its all an attempt to stay very focused on the questions and business drivers and the people who use the answers.
We run prediction markets inside companies and find that if we don't establish a good lifecycle of asking forecasting questions, having people respond with probabilities, then decision makers REACTING to those probabilities in some way (whether they agree with them or not, just acknowledge their existence) the likelihood of the project failing is far higher.
how many of those managers will now be able to get even a higher paying job because they have manager of Watson AI project on their resume?
http://www.mikadosoftware.com/articles/ibmadverts
> A clinic could use the system to search its patient records and find, for example, all the men over age 45 who were overdue for a colonoscopy, and then use an autocall to remind them to schedule the dreaded appointment.
Maybe this was a terrible example, and the author didn't grasp a good example of legitimately non-schematic data points?
Even if successful, a system which could "interpret" a health record (such as a freetext note) using anything other than properly codified data would set the health industry back a decade. Moving doctors away from freetexting their notes is the only way to advance the industry.
Doing so could have incredible utility for sharing data across various clinics/hospitals/pharmacies/etc.
My previous doctor (who was probably mid-50s) didn't use email or any kind of secure electronic messaging system. Everything had to be faxed to him.
My new doctor who is younger uses all kinds of digital tools including a voice recorder with a pre-trained text-to-speech engine that understands medical terminology and codifies the transcription based on keywords.
So it's not entirely getting away from freetext but at least it's extracting some structured data from it automatically.
I can imagine someone who doesn't know at IBM selling a product:
"Hey we will solve all these problems like magic!"
Then IBM comes back:
"Hey do you have all this data in a specific format and a ton of time to enter and test it and maybe we'll get back to you???"
That's a big loss of trust there with the customer if you come back with that.
It seems like these are products where a lot of caveats needs to be made clear to customers and a real careful technical partnership formed with them to succeed long term. You have to bring the customer along for the ride and exploration and keep them excited for a long time it sounds to make it work.
To see one of their articles with the common press confusion of mixing different definitions and interpretations of AI (correct or incorrect) doesn’t help build confidence.
For example, what was used to play Jeopardy vs. approaches being taken to improve cancer treatment, are just so different, it seems almost disingenuous to throw it all haphazardly into one conceptual bucket.
The article does IBM a disservice in some ways. They come off looking bad overall but some of the failed projects mentioned like MD Anderson, failed for reasons beyond any control they had, other than recognizing some obvious red flags earlier and detaching their name and participation from it.
On the other hand I believe the article lets them offf the hook to easily when they bring out the old trope they’ve been using for years, which is encapsulated here:
“IBM Watson has great AI” [one engineer said] “It’s like having great shoes, but not knowing how to walk—they have to figure out how to use it.”
It doesn’t make sense to say, xyz is great we just have to figure out how to use it, as a stand alone argument. It’s nonsensical unless you mention something about the seeming implied untapped potential, specific innovations, novel approach, or whatever makes it great.
I’m not familiar with all their IP so maybe there are some great things, you just don’t get to claim that and get off the hook so many times in the press without providing at least some detail or reference point.
Once scientists in biology and healthcare get on board like they are in linguistics and computer vision I'd expect things to pick up.
The audience that gobbles up their ad campaign during the Masters that touted their "block chain" logistics probably wouldn't even notice that they had layoffs at Watson health.
It seems to me that Watson is basically just IBM's version of AWS/GCE services (at least the non infra ones). But it gets thrown around as a buzzword so often. The marketing makes it look like there's a single AI codebase that can be accessed through a bunch of APIs, but I would be very surprised if that was actually the case.
https://news.ycombinator.com/item?id=15456211
It's reasonably in depth and appears sincere.
tl;dr: it's a good search engine and a disparate set of machine learning tools. Sales is promising Hollywood AI, but the reality is that it takes a sizable project team to build anything worthwhile.
Scanning past commentary, it seems that startups are eating their lunch (more nimble, dedicated to customer space). I'll add that half the machine learning battle is getting access to data, so hyping the brand makes sense strategically.
Both, sort of. There is a "thing" called Watson, which is related to the Watson that played Jeopardy. But "Watson" is also a brand which lumps in stuff that has absolutely nothing to do with the "old" Watson.
To illustrate a bit.. "Watson Health" is (or was) made up of a ton of people and technologies who came into IBM as the result of several acquisitions: Truven, Phytel, Explorys, etc. In many cases, they repackaged stuff from those vendors, gave it a "Watson name" and shipped it. And some of this stuff was literally no more sophisticated than linear regression / logistic regression, etc.
I wonder if DeepMind at Google has a similar problem. It is certainly getting a lot of headlines, but there are plenty of other AI groups within Google that do business-relevant things like improve search or ad matching or make Google Home's voice recognition work. I would not be surprised if in the long run DeepMind becomes a group that performed a neat stunt with Go, but kind of fades in practical relevance, like Watson with Jeopardy.
It's pretty common in many industries to have products to showcase your chops while providing zero real world value and zero sales.
People who know that healthcare is different try to warn them. They don't listen. Instead they charge in with people who have no experience in the field.
From the article:
After the acquisition, IBM management started the process known internally as “bluewashing,” in which an acquired company’s branding and operations are brought into alignment with IBM’s way of doing things. During this bluewashing, “everything stopped,” the first Phytel engineer says, and the workers were told not to focus on improving their existing product for current clients. “People were sitting around doing nothing for almost a year,” the second engineer says.
"Phytel’s contribution was analytics paired with an automated patient communication system. A clinic could use the system to search its patient records and find, for example, all the men over age 45 who were overdue for a colonoscopy, and then use an autocall to remind them to schedule the dreaded appointment"
This shit isnt AI it's literally a database query and then some 3rd party library to send a text message or a phone call.
That’s already called Musk’s Law.
(1) https://news.ycombinator.com/item?id=14979642
(2) https://news.ycombinator.com/item?id=14766793
> IBM Watson Health has initiated a significant RA across multiple offices.
For a long while now, IBM has been treating "AI" as a product that can be managed, packaged, and sold by "general" business managers -- think MBA-types with only a superficial, qualitative grasp of deep learning and AI. Doing that with rapidly evolving technology is a sure-fire recipe for failure.
Most such MBA-types today are ill-equipped to manage, package, and sell "AI." They're roughly in the same position as English or History majors who are asked, say, to manage, package, and sell a new kind of quantum-computing technology without knowing or understanding much about quantum physics. The technology is moving faster than their ability to keep up.
IBM's mismanagement is a shame, because the system they showcased nearly a decade ago -- the one that competed and won in Jeopardy -- was state-of-the-art at the time.
Stories like this are not a surprise, it's IBM's way of doing business. Maybe Watson will learn HR and just fire everybody from middle mgmt on up..
My comment mentioned specifically "MBA-types with only a superficial, qualitative grasp of deep learning and AI."
MBAs who understand what they're managing (and who know what they don't know) are not in that group. And BTW, I suspect most MBAs who read HN are not in that group either :-)
> Both Phytel engineers say the offering managers didn’t have technical backgrounds and sometimes came up with ideas for new products that were simply impossible.
The death knell of all (potentially) good products. I don't know why this is so often the case. All software companies need engineers involved in product development decisions. Period. It's not optional.
Facebook who was smart about this. They hired or retrained technical people to fill many business roles in marketing, product development, project management, etc.
I'm not sure why technical people are restricted to merely being the builders in these companies. Lots of other companies recruit internally from people familiar with the end product and train them in other business areas.
> these potential customers weren’t impressed. Instead they asked for something resembling Phytel’s old system.
So they simply imagined a new product without interviewing potential customers beforehand on what they actually want? They spent years merging databases of two big systems, pivoted multiple times, to find out there wasn't a market for it in the first place?
Why aren't the 'offering management' people getting fired?
In the middle, there are 100 layers of middle managers that completely cock everything up, and the really sad part is that they have enough say to really cause damage. One of my first proper white collar technical jobs with them was an L2 support job for this network performance monitoring suite for huge networks... mostly large, national ISPs and the like. The job required maybe a just-post-jr-level sys-admin knowlege of networks and UNIX systems while also having smooth customer service skills. Definitely a great step up from my previous lower-mid-level IT jobs and call center work.
I had three(3) managers. Three! I had a technical manager, a non-technical manager, and my actual manager, who was the head of the department.
At the highest levels, the management was talking about switching everybody's workstation over to Linux. Everybody from admin assistants to developers to managers was supposed to be moved off of Windows at some point in the relatively near future. I was psyched— I hated windows, and the product I supported ran on Solaris, so not having to deal with the extremely primitive (at the time) tools like Cygwin to get some UNIX functionality on my machine was great. They seemed to be positioning themselves to sell the consulting for other large companies to do the same thing.
Though we got no word of this internally— I only knew from what I had read in articles— I found the internal workstation disk image on the intranet and eagerly installed it. It was pretty smooth! I was excited! As I was getting my tools set up, I noticed that it didn't have the internal bug/ticket tracking clients installed, so I cruised on over to their intranet page... hmmm, nothing listed for Linux. After hours of searching, I found some internal discussion showing that, months earlier, the department that writes that software unilaterally decided that they were discontinuing their initiative to port those applications to Linux. While there was an extremely limited CLI to these tools, critical functionality was literally impossible without the GUI app. Without the ability for anybody on their Linux workstations to interact with tickets or bug reports, the Linux initiative was pretty much dead-in-the-water for most technical people and their managers.
Perfect example of just how badly their forest of middle managers completely messes up great executive initiatives that the bottom of the food chain really wants to embrace.
(I might have gotten some of the details wrong. It was 13 or 14 years ago and I drank a lot back then.)
This may be possible this time round, because we’ll have a very good record of who said what and when via the web.
Without any kind of accountability, history will continue to repeat itself.
How about hyperbole.com, where you can google academic researchers and industrial leaders and pull up quotes from them, dated and fact-checked.
I’m sure you must be able to train a deep net to do this. They can do anything.
Self-driving cars have killed pedestrians, Watson isn't doing all they wanted... is this the dawn of a new winter?
Anyways, may I interest you in cheap VR headset?
In particular, I'm not sure what sets comprise the numerator and denominator of "most technological revolutions".
If IBM stops milking the cow like the services, asking the customers to integrate, It will die a miserable death, and unfortunately a premature one.
It should be pluggable, like set of pick and choose building blocks of interfaces, that only should take domain expertise and custom specifications as input from the client (Which ever domain.) Until then AI only will become ambitious Sunk cost.
People are faulting IBM for over-promising, under-delivering, using misleading advertising, and internally seeming to have foolish management practices.
> Once IBM has been in this technology game for a little while, I'm sure they'll get the hang of it.
I'd assume the parent is being sarcastic