In our application, one key operation is that the user is required to classify a line item based on the text description. There's a huge code list of possible classifications, and the user has to pick one that is the most correct.
This is definitely a task that registers consciously. And, while most of the time it's fairly easy for trained users, there are often cases which require extra thought or research.
For example, a T-shirt of mostly cotton vs a T-shirt of mostly synthetic fibers should be classified differently. How would you know based on the description "Small Womens V neck Short"?
Right, so the user would have to do further research like contacting their customer and ask.
Of course, an AI/ML system that could reliably classify a majority, and reliably classify the rest as "unknown", would be interesting. Not sure how close we are to that though.
You might just as well ask why that miraculous cure for baldness is so useless. You let some used-car salesman talk you into believing that it actually works, but it doesn't — no matter how much you want it to.
In my opinion, most of the issues leading about AI "failing" in traditional organizations are due to the following:
(1) Inflated expectations from higher/middle management which trickle down the organization. AI is seen as a high-profile case which has to lead to success (and a larger budget next year for my dept.)
(2) Data quality issues. The data itself has issues, but the key issue is lack of metadata and dispersed sources. Lack of historical labels (or them being stuck in Excel or on paper) is part of this as well. Big data without any labels is mostly useless, contrary to expectations
(3) Most AI or ML projects are not about ML. In fact, they're mostly about automation or rethinking an internal or customer-facing process. In many cases, such projects could be solved much better without a predictive component at all, or by simply sourcing a 1 cent per call API. AI is somehow seen as necessary, however, without which our CX can never be improved. ("We need a chatbot" vs. "No, you just need to think about your process flow")
(4) Deployment issues and no clean ways to measure ROI leads to projects being in development indefinitely without someone daring to stop them early. This is also related to orgs starting 30 projects in parallel (2m lead times with one to two data scientists for each), which end up all doing kind of the same preprocessing and all lead to kind of the same propensity model. No one dares to invest in long-term deeply-impacting projects as "we want to go for the low hanging fruit first"
I pretty much agree with all of your points, but I also think there may be a more fundamental issue at play here. ML doesn't actually "understand" things - it can do very sophisticated and accurate pattern matching without actually "knowing" the logic of the patterns it's matching.
This in turn means that it may fail catastrophically when faced with adversarial examples or with examples that are drawn from a different distribution than that of the training set.
How is that different from a typical human given a routine boring role? Specifically in regards to adversarial input, humans are often the weakest link in terms of process security.
Optical illusions indeed highlight limitations in human perception. However, the dress illusion seems to me far less of a problem than mistaking noise for an object.
More relevant however is that we humans can understand that we're faced with an optical illusion and we can make adjustments accordingly. We have formed the concept of an "optical illusion" and we just place "The dress" in that category. A machine needs to be specifically trained on adversarial examples in order to be able to predict them. Once you come up with a different class of adversarial examples it will continue to fail to detect them. There is no understanding there, just more and more refined pattern matching.
Does a machine that can match any pattern actually "understand"? I would say no. But these are already philosophical considerations :D
> More relevant however is that we humans can understand that we're faced with an optical illusion and we can make adjustments accordingly.
Broadly speaking, yes. At the same time that's not what happened in 2015. It produced so much polarizing content with people deeply entrenched in their believes. They might have recognized it as an optical illusion, but they refused to make adjustments.
> A machine needs to be specifically trained on adversarial examples in order to be able to predict them. Once you come up with a different class of adversarial examples it will continue to fail to detect them. There is no understanding there, just more and more refined pattern matching.
Moving away from image recognition examples, isn't that exactly what happens with humans predicting whether an email is a phishing attempt? I remember reading here on Hacker News this week about phishing tests at GitLab. It had a lot of comments about tests and training employees to spot adversarial emails. Some companies are more successful than the others. It is a complicated problem; otherwise we would have solved it already. But it's the same principle because phishers come up with different ways of tricking people. And some people will fail to detect them.
I would say that there are indeed many examples of things that are hard to categorize for humans. Sometimes there isn't even a way to categorize things perfectly (there may be a fuzzy boundary between categories). It is for example really hard to train people to figure out if a certain stock is going to be profitable or not - there are many other such examples.
This doesn't mean that the kind of thinking that goes on in the human mind is the same as the pattern-matching that goes on in an ANN (for example). Think about how ppl learn to talk. It's not like we expose infants to the wikipedia corpus and then test them on it repeatedly until they learn. There are structures in the brain that have a capacity for language - not a specific language, but language as an abstract thing. These structures are not the same as a pre-trained model.
The truth is I don't know enough about cognitive science to properly express what I'm thinking, but I'm pretty sure it's not just pattern matching :D
But what about if a person sees a cat but accidentally presses the dog button because they were distracted?
(To your point, though, I agree that machines can make strange errors, raising trust issues. My experience is that ML is useful in cases like recommendations or search results where a person can interact with predictions rather than being a complete replacement)
There is no button involved. If you look at a cat (that is within your field of vision in good lighting etc.) you will understand it's a cat. Without mistake. Most certainly you won't mistake it for a square full of static, or for a car or for smth else.
This is one of the key points in Melanie Mitchell's book "Artificial Intelligence – A Guide for Thinking Humans". In the process of showing this, she gives really good explanations of how current AI/ML systems work. Really worth a read in my opinion.
I think of the current neural net based approaches as more "artificial instinct" than "artificial intelligence". The goal of the older, expert system AI paradigm was to create software that could reason about a problem, and it therefore produced systems that could answer the question "why?". ANN systems cannot answer that question. Both approaches seem useful to me, but for different problem domains.
My background is in Standards for engineering data. I'm trying to push the idea that cleaning up a business' data before trying to apply ML to it may give better results.
That really only solves part of the problem. Clean up your data. Clean up your processes. Then decide which processes can actually be improved in a meaningful way by ML. I've gone down the ML road twice now with companies that didn't do the second part and both times we had to kill the projects because the processes were so fuzzy and full of gotchas that no ML model could ever hope to be a net positive addition.
As one of the other comments points out, technology can only change productivity when you change the process, sometimes radically. And that may mean restructuring the business.
The article is just vaguely complaining about undefined problems which makes it very difficult to defend or argue against. Are we talking about ML? Optimization problems? Operational research in general? Automation? And which tasks in which businesses? There's tonnes of useful stuff in each of those categories and vaguely saying that it's all useless doesn't get you anywhere.
> why can't it read a PDF document and transform it into a machine-readable format?
> why can't I get a computer to translate my colleague's financial spreadsheet into the format my SAP software wants?
Because you probably expect it to be 100% or maybe 99.999% accurate, and we can't do that. Imagine "AI" translating someones financial spreadsheet into a different format and dropping a zero somewhere. Oops.. but your test set accuracy is 99.8984%. Still not good enough. Just getting 1 thing wrong breaks everything. This is fundamentally different from clicking on image search and ignoring the false positives.
Exactly this. The unreasonable expectations of what AI can do - whilst it can speed things up significantly, majority of cases, you can never fully automate everything with AI.
Why exactly do you need an AI that reads a PDF ? Unless you’re dealing with ancient data wouldn’t it be easier to have whatever that’s generating the PDF return machine readable data ?
This suggests to me that lot of office jobs will be lost just by modernizing systems and making them spit out JSON
You generally do not control the process that generates the PDF or other unstructured data.
Consider invoicing, you may have hundreds of distinct sources, but none of them individually would warrant automation.
It is a problem of standardization. Any standard complex enough to cover all business cases would be too expensive to implement, or you could not get all stakeholders to adopt it, and so on. Accountants would not want to make themselves redundant, obviously.
there's probably a better tradeoff than a format that only gives you "print this character at x,y coordinate".
Websites have been able to produce "ready to print" html for a while now, and html (even with a lot of restriction) would be hundred times better than PDFs.
don't know why you got downvoted. The widespread use of PDF is a disease that needs to be eradicated... the only problem is we don't know (yet) with what we should replace it, but it's definitely amazing we have to hope for ML to be able to understand a format we, humans, decided to have machine produce.
> Why exactly do you need an AI that reads a PDF ?
Much of the world is full of PDFs that are bad scans. Upside down, sideways, green, or maybe in French because why not.
>making them spit out JSON
This doesn’t just require fixing your business but every other biz you interact with. Modernize your own stuff isn’t enough. A bit like offices still list fax numbers
If school grades are indicative, even a 90% accuracy is commendable for human driven tasks. The big difference is that when a human double checks their own work, they may not make the same mistake twice. Or even better, when a second person checks the work.
Mistakes are tolerable, but you need to be able to recover from them somehow, and recovery from an AI mistake seems to be something that people like to pretend is unnecessary or impossible.
We currently live in a world where failures are tolerable but seem adamant that moving to an AI driven world means that failures are no longer tolerable. We still haven't lost the ability to make mistakes (because we still do many things manually) but we really seem keen on losing it.
School grades are testing something very different from data entry. We know as a species that our learning process is slow and error prone, so doing decently at that is good, and we don’t need to have 100% knowledge in a school subject. Meanwhile, yes, if my bank is copying account values, or even just account ledgers with history, I want them to have 100% accuracy. Not 99.99999% or 99.9% or 99%, let alone 90%.
Are you training the two AIs on the same data set? If so, won't they be likely to make the same sorts of errors rather than making different errors and thus providing an effective check on each other?
The requirement for 100% accuracy is close, but not quite correct.
Even in many highly critical human endeavors, there are many errors.
The key to success is not absolute error-free perfection, it is no critical errors in components that are severe enough to kill the project.
Every rocket launch has some issues, but the successful ones have issues where it doesn't explode or land in the wrong orbit.
In the spreadsheet example, dropping a critical zero will cause damage akin to the rocket explosion. But dropping an "O" in a label field is utterly trivial.
Humans understand the distinction, constantly make such judgements and focus on the critical areas in their moment-to-moment work and embed it in their work processes. These constant criticality judgements are not just binary, but refined scaled, and serve to apply resources where needed.
The AI systems do not have such a judgement layer, and apply the same degree of inaccuracy to every part of their domain. So, absolute 100% accuracy is required, as errors are no less likely in the critical components.
In my opinion, it's because business operations isn't that complicated and people don't know what AI is.
By "not that complicated", I mean a decent CRM system to track information about the organization is approaching peak operational efficiency for most businesses. Most inefficiency I see after that is people/political problems.
By "people don't know what AI is", I mean that business owners are unable to describe their business problem as a supervised learning problem. If you can formulate your business problem as a supervised learning problem, then you can probably solve it with AI (which, yes, is really just a marketing term for supervised ML).
But most business problems are really "order taking" or "production/delivery" or "moving things through a funnel" problems and thus AI isn't the solution, CRM or CRUD apps are the solution.
"AI is a 'brain' that you plop down in the organization and it does 'unsupervised learning' to automate your business and delight your clients with optimized outcomes." - As explained to me
I now see Google sales dragging these people out to 'AI meetings', which are about Dialog Flow. (Meeting pitch email gets around, business manager reply all: 'WTF is this?') Later they summarized the meeting, as they understood it: "You just need to drag and drop the CRM on the 'Brain' and then you can book haircut appointments and open a bank account. We were like um, we develop and support software, they were like - just drag and drop you knowledge management API on it too." Uh... what KM system?
The complicated part is that in small and midmarket business, their knowledge is tribal and processes are folklore. There are processes that are followed though they don't exist, and business rules violated as they aren't aware.
Meanwhile, no one in sales puts useful information in a CRM system if they can avoid it, the least effort principle on administrivia is important if you are ever going to hit your ever growing number.
Finally, the management must still be spooging for SPOGs, I sill see marketing for them as some sort of nirvana. Meanwhile, they have been collecting data randomly about everything, all over the place, excel documents, FTP servers, and of course databases. Here, no one in IS/IT will admit: The servers were here when they started their job and they have no idea what is on them or if they are even needed anymore. They certainly aren't mentioning that to the incoming CIO, who gets focused on improving efficiency as per his executive mandate. So he/she gets to work with cloud migrations (more hilarity ensues)
Because "Artificial Intelligence" is a label forever applied to the effort of replicating some human cognitive ability on machines. A well-known lament goes something like: "once it's possible, it's no longer AI".
Business is about exploiting what exists. This is why the buzzword is "innovation", not "invention". Incremental improvements, not qualitative jumps. So nothing will ever be really considered "Artificial Intelligence" once it is boring enough for business.
Scheduling algorithms are incredibly useful for business. There was a time when this was considered AI, but that was the time when they didn't work well enough to be useful.
Example: my wife is an admin in a school office, and a ludicrous amount of her and her colleagues' time is spent on replicating data entry between a multiplicity of different incompatible systems. The Rolls Royce / engineer's solution to this would be to provide APIs for all these disparate systems and have some orchestration propagating the data between them, except of course that's never going to be remotely practical; instead, dumbly spoofing the typing that the workers do into the existing UIs is a far more tractable approach. My (admittedly not 1st person based) experience of these things is that they currently still require significant technical input in the form of programming and "training", but this fruit has got to be hanging a lot lower than any ML-based approach.
RPA can be a dangerous band aid. It often uses screen scraping or similar brittle interfaces that are known to change. Or, it doesn't know about certain error conditions, etc.
Also, if it's been running for months before it breaks, the humans that used to do the work are gone, or have forgotten how to do it.
I've always found rpa a very curious thing. In some corners you have people really hyping it and how much value it can bring and everything e.g ui path and other products. But I can never imagine it to be a very good solution, the thing must be extraordinarily fragile and rule based clicking automation is a real pain to put together.
At my work we are leaning heavily on RPA to automate away drudgery and ultimately reduce expendature. However it has been immensely frustrating, prone to errors and garnered endless suspicion. The experience has been that bots written by the service desk staff doing the job function better and are much more under our teams governance, which the official "automation" teams within our org are painful to deal with due to their lack of availability and not having first hand knowledge of the things being automated. I think the RPA approach requires dedicated people developing and monitoring the bots who have an active part in the process being automated. The difference in approach determines the result.
I've experienced the same. Centralized RPA teams tend to, for example, do web scraping when they could easily use an existing REST API. Because they either don't know it exists, or don't have that skill set.
Similarly, seen things like using a email as a trigger, when the source application has configurable web hooks.
Feels like there's an RPA culture of sorts to assume the things being automated only have human based interfaces.
The work an RPA does is obvious, you can watch what it does.
Your first statement is a trusism, any solution can be a dangerous band aid, if applied incorrectly. RPAs solve real problems now, really the only correct measure.
Of the RPAs I have raised, I always watched them work, they just prevent mistakes. I would suggest taking a screen recording and verbally annotating it for posterity. Furthermore, nothing says your RPA has to run open-loop, one can put in checks to ensure that it hasn't gone off the rails.
Good advice - carefully observing the process, annotating successful (& failing) processes and installing post-hoc checks seem like sound practices. Do you have other such practices to recommend? Can you say anything about what your production and testing stack look like (esp the versions that work better)?
Agreed. Taking large amounts of admin workers from low productivity to moderate productivity (and RPA can easily boost the productivity of these kinds of tasks, if not the whole job by 200%) has a much bigger effect than hyperoptimising workflows that were already highly optimised.
Strange they didn't mention RPA in the article. In fact I think that most of their examples could be addressed with advanced RPA.
I also believe that the leading edge RPA systems do take advantage of some real AI techniques. And that more AI will be deployed in RPA as time goes on.
I'm at a hospital where someone at long last got authorized to try ML on the clinical database.
The ethical committee required that prior to using any data, you have to make a static copy in another database. Their argument is
1. They don't want excel files flying around (which will happen regardless) and
2. To perform any analysis, you "obviously" have to have "structured data", which "obviously" means that you have to extract a csv from the base system (MongoDB) and put that into a RDBMS (redcap).
Why are the ethics committees allowed to make any decisions?
When I was on one I would by default let everything through unless I came across something where I thought I could help improve the experimental design. My colleagues on the other hand would go through them like little emperors and cause grief for the applicant just because they could.
I'm trying to imagine the kind of org that uses MongoDB and has someone on the ethical board with enough technical knowledge to understand the difference between sql and nosql and at the same time believe that nosql can't hold structured data, and that data for an analysis that probably won't have more than some megabytes must be put in a specific-purpose database instead of being consumed as CSV, a format this same person appears to be familiar with.
I mean, what a weird combination of tech (un)savviness
I’ve been working in the “real world business processes that companies are trying to AI-ify” realm for quite a while now. Pharma, cyber security, oil and gas production, etc.
This article doesn’t mention a really, really straightforward factor for why AI hasn’t invaded these domains despite billions of dollars being dumped into them.
An automated process only has to be wrong once to compel human operators to double or triple check every other result it gives. This immediately destroys the upside as now you’re 1) doing the process manually anyway and 2) fighting the automated system in order to do so.
99% isn’t good enough for truly critical applications, especially when you don’t know for sure that it’s actually 99%; there’s no way to detect which 1% might be wrong; there’s no real path to 100%; and critically: there’s no one to hold responsible for getting it wrong.
In those sorts of domains the best pitch for AI is as a failsafe. The model and the human probably make different errors, often a human will make mistakes due to simple inattention. This lets you substitute the model for some other process controls that you'd need to maintain 100% accuracy, e.g. instead of having the work reviewed by a second person, you can have model + person.
In lots of business contexts, probably most, reducing variance is much more valuable than reducing mean expenses. Variance can halt downstream production, so the loss can be some huge amount of opportunity. And variance propagates through a supply chain, so your customers will hate variance in your output, as it may mean they have to ship the variance forward, which their customers hate. Plus if you allow your supply chain to get away with inconsistency, they can start to rob you with lower average quality and it will take you time to notice.
If a company has been bothering to do some process manually and they haven't outsourced it to the cheapest humans possible, then they care more about low variance than low cost. Pitching these businesses a solution that lowers cost at the expense of unknown high variance is very unattractive. Instead, you want to tell them "I can reduce your variance even further! Here's what that would cost".
> instead of having the work reviewed by a second person, you can have model + person
But the argument still applies then - if only one incident occurs where a catastrophic mistake was not spotted by the model failsafe, and if later investigations show that the mistake could've been easily spotted by a second human, a human will be installed as a failsafe for the failsafe.
I think the general problem is the following: if a human makes a grand mistake, it can usually be attributed to a temporal lack of care, or just random bad luck, or happened because the person was having a bad day, or... it is also generally understood that making a grand mistake will be such a shock for the person responsible that this person will most likely never make the same mistake again. On the other hand, if a machine makes a grand mistake, the intuition of the general public, trained by centuries of experience with techology, is that this machine will make the same mistake again and again and again, when prompted with the same input. If the model is not designed to learn from mistakes in production, this will of course actually be true.
> But the argument still applies then [...] a human will be installed as a failsafe for the failsafe.
Surely this will depend on the cost of failure, and the cost of the human failsafe.
Spellcheck in an email client helps prevent the minor embarrassment of typos and spelling errors in emails, and few emails are so consequential that it's worth having them carefully manually vetted.
Of course, this is why I wrote "catastrophic" above. I also would not call an email spellchecker a business use case. Regarding spellcheckers per se: I have worked with book publishing companies in the last few years, and I can assure you all major book publishers employ real humans for the final spellchecking before a book goes into print.
But the parent's point is that "the model and the human probably make different errors".
That is frequently true. Take the job of a lifeguard for instance. A single mistake can be catastrophic, and yet we know that people have trouble staying completely focused for hours on end.
AIs have no trouble staying focused and they can be trained to spot drowing swimmers pretty well even on a crowded beach.
Having a human lifeguard plus an AI that alerts the lifeguard when it spots something suspicious could lead to better outcomes than employing two lifeguards.
This is very dependent on the false positive rate. Similar to the examples above about false negatives, if the AI gives false positives the lifeguards will stop paying attention to its feedback.
> Having a human lifeguard plus an AI that alerts the lifeguard when it spots something suspicious could lead to better outcomes than employing two lifeguards.
I disagree. I think you'll see the same thing as with drivers falling asleep while Tesla autopilot is running. The lifeguard will let the computer do all the monitoring since a low false negative rate combined with a low incidence rate means that most bodyguards will experience the computer being 100% reliable for weeks or months at a time. In fact it's not unreasonable that if the lifeguard sees someone drowning but the computer doesn't register it as such the lifeguard may question their own judgement based on experience.
Huh? In my experience, people who make a mistake tend to make the same mistake again and again. Whereas, when a machine makes that mistake, somebody fixes the bug and it doesn't happen again. If the machine can teach itself, then so much the better, but "has someone supporting it" is a sufficient form of "learning from itself in production" to address the concern you raise. In the industry where I've been working for the last past several years (finance), when something goes wrong, the first instinct everyone has is to bump up the prioritization of automating whatever human task was responsible for the human error. And the automation is pretty much only bounded by the amount that the company is willing to spend on programmers and its ability to find good programmers who are willing to work for it and its insistence on maintaining backwards compatibility with bad legacy systems.
> Whereas, when a machine makes that mistake, somebody fixes the bug and it doesn't happen again.
Well, that is kind of the problem with AI. How do you fix a "bug" (if you can even call it that) in a model you cannot fully understand? Do you re-train it on the catastrophic mistake and somehow give it more weight? How can you be sure that this won't lead to any problems where previously there were none? How do you explain to a customer that your model now doesn't make mistake A anymore, but now mistakes B and C frequently occur? The only safe bet is to write some auxiliary code, which first uses the AI as a black box, and afterwards explicitly checks the result for this particular mistake. If this happens again, and again, and again, you need a human to maintain and extend this auxiliary code and also adjust it to changes in the underlying model, at which point I am quite certain just using a person of average intelligence instead of AI will be cheaper, more reliable and more flexible.
Except that in most practical scenarios, most models make basic mistakes that no human would ever make. In most cases, people have a very low tolerance for errors that humans would not make.
Huh, N26, a major online bank in Europe is famous for some of its customers getting their accounts blocked every time they tweak their ML model and yet is doing great financially dispite the shitstorm it generates each time.
It's not like Google blocking your email or YouTube account, we're taking about your friggin bank account here.
I don't know how they're still in business and growing with such a process in place.
I’ve found that financial institutions have a disproportionately large appetite for this. This is frankly because they view their own mission criticality as somewhat low — lower even than their customers perceive it.
Undoing bad automation in finance often means reversing a very cheap edit on some database and pissing off a customer who’s so powerless that it won’t affect your business anyway.
Yeah but usually the bank just buys package solution with some customization for accessing the data & reporting, so they don't have to keep up the army of very narrowly specialized folks catching up with ever changing laws etc.
Do you do any work with uncertainty estimation with neural networks? There are many different ways to estimate uncertainty depending on your application, such as ensemble-training or dropout during testing to produce ranges of predictions that you can then use to get boundable error measurements.
That sort of misses the point. The uncertainty estimation is, itself, dependent on and a direct result of the quality of the training data you give it and that training data's representation of reality.
And even if the inputs are great, it's hard to understand what we can do with a 99% confidence prediction. The next great unlock that society is waiting for in the AI realm applies to industries where failure is not an easily acceptable outcome.
Take self-driving cars for example. Even if we can objectively prove that current AI models can drive on current roads in current conditions with a lower fatality rate than humans (this is debatable anyway, but let's assume) - what do we do when it knows it's not confident? If we assume the driver hasn't been paying attention during the ride so far, and a scenario comes up that the AI is uncertain about...the human now has likely mere seconds (at most) to capture their surroundings, analyze the risk, and take corrective action. If we assume the driver has been paying attention during the ride, then what was the point of the AI? Moreover: what if it thought it was confident, but it was still wrong. Who do we blame? How do we mitigate future instances of it? What were once societal problems we could blame on a fallible humans are now obscure technology problems we can't introspect. That's a scary place for a lot of people to be.
Basically, we've reached the level of AI where it can be used as a backup to humans and protect us from royally fucking something up. But the next big unlock will come when it can be the primary actor. It's hard to imagine how we'll get there without the AI actually understanding what it's processing in some real way.
I agree, the question of how to deal with high uncertainties is definitely task dependent. In your example of a self-driving car the answer is not clear. However, for many other tasks its entirely reasonable to have the AI make decisions and then pop it out to human review upon high uncertainty, or just not act on it. In tasks like this AI can be the primary task-doer with the human as the backup.
The problem with this is that NN are in general, overconfident in their predictions, even when they are wrong. This is a well known problem in the AI/ML literature. Using ensembles of overconfident predictors is not the same as getting an unbiased estimate of the uncertainty.
Yes, definitely. Though there has been some interesting papers recently on this using different methods to approximate bayesian posteriors in NN's. One of the most recent ones (Mi et al., 2019; https://arxiv.org/abs/1910.04858) that benchmarks a few different methods -- infer-dropout, infer-transformation, and infer-noise -- are all promising for different applications and neural network models (black, 'grey', 'white' box).
Humans are hardly ever 99% accurate instantaneously on classification problems either. Humans have the ability to know when they should collect more information though, and can often perform some sort of hedging action when they are unsure.
But isn't also the case that humans understand the consequences and the depth of multi-variable decisions?
For example, Amazon Seller Central, Google, Youtube, can "outsource" their customer service to AI, because they are pretty much the only players, so customers have to suck it and deal with the frustration of a terrible experience by not getting help and not talking with a human.
With any other business if they get automated replies that don't solve a customer issue and they are unable to talk with a human, 99% of them say "fuck it" and go somewhere else.
This is one small realm of the relationship of AI and businesses. Then you have employees, suppliers, supply chains, finances, internal processes, so many multi-variable, fragile and nuanced systems that I doubt if they're not developed in-house, they'll probably do more damages than solve problems.
This is interesting to reason about because it's may even be true that the human and AI error rates could be the same. In fact the human error rate might even we worse, but the kinds of errors and their impact makes a big difference.
I can only speculate, but while it's true humans can fail when following a process, it's also true they can sometimes spot a potential failure even then there is no process to prevent it. They can come up with new processes, or ways to improve existing ones. They can also account for their actions, and managers can account for the activities of their team. All of this builds trust that the process can be improved in ways that can be well understood.
People are also scalable. You can implement controls like four-eyes on changes and critical metrics, so you're less exposed to one person's idiosyncrasies. It's hard to do that with AI.
I agree. Humans are interactive agents, they have broader knowledge about the world, and they can (to a degree) diagnose their performance and compensate.
Think of a DL powered, visually guided robot vs. a human on a production line.
One task might be to do QC inspection at the end of the line. Suppose something gets on the camera lens. In general, the DL system will keep chugging along and the accuracy will degrade. The human will notice this and clean his glasses.
If he sees something ambiguous as it passes on the line, he might give it a bit more attention or adjust the angle he's looking at it from. If he sees a series of the same anomalies, he will notice a pattern. Perhaps one of the machines up the line from him has started to introduce a new type of defect.
Suppose in assembly a worker has a sore muscle. He might adapt his motions to compensate, slow down, go to the doctor, take some pain pills, or take a day off. Unless programmed or trained to detect this, a robot will keep driving its failing motor harder until it breaks.
> 99% isn’t good enough for truly critical applications, especially when you don’t know for sure that it’s actually 99%; there’s no way to detect which 1% might be wrong; there’s no real path to 100%; and critically: there’s no one to hold responsible for getting it wrong.
Pharmacy - adding 1% more of a chemical compound to a pill could be lethal, but not otherwise detected for 3-12 months (pills sit on shelves for a long time). Figuring out why that happened will be very difficult.
Manufacturing - molding plastic parts requires very specific mixes of chemicals. Too much of one and it becomes brittle under certain temperatures, too little and it warps down under the warranty period. So if you manufacture the mold of a baby car seat, and you add or subtract 1% of a chemical, that car seat could break / shatter when it's involved in a collision. Terribly large lawsuits would occur because some C-level someone adopted a process to save headcount and reduce oversight.
Business processes are a lot like software - the first 80% takes 20% of the time and the remaining 20% takes 80% of the time.
AI for pharma seems to be targeted at things like drug discovery, where yes every output from the algorithm has to be carefully validated. But that's already the case for human-generated ideas. So I don't think your argument makes sense in this context.
Well it’s a bit simplistic to say that it’s used “for drug discovery.” A particular application might be doing NLP on e.g. clinicians’ notes or on scientific journals.
There are plenty of places where human validation shouldn’t be necessary, but is.
Why dump billions of dollars then? Nowhere else to spend it? Effective marketing?[1] Is no one asking this question?
"... and critically: there's no one to hold responsible for getting it wrong."
Could this be part of "AI"'s appeal? A dream of absolving businesses and individuals from accountability.[2]
1. "What's more, artificial research teams lack an awareness of the specific business processes and tasks that could be automated in the first place. Researchers would need to develop an intuition of the business processes involved. We haven't seen this happen in too many areas."
2. Including the ones who designed the "AI" system.
> Why dump billions of dollars then? Nowhere else to spend it? Effective marketing? Is no one asking this question?
Because whoever does achieve the next unlock - should it happen - will receive an unimaginably large windfall. This is the classic intent of venture capital. In fact, I'd suggest that AI is actually one industry where VC is doing what it does best: taking extremely risky bets with a large potential upside.
> Could this be part of "AI"'s appeal? A dream of absolving businesses and individuals from accountability.
Presently, this seems to be one of its large detractors. If I have an employee do something stupid, I can say that an employee did something stupid. People might wonder why they were allowed to do that stupid thing, and what we're going to do to prevent it from happening again, but the explanation of the source is satisfactory. We're fallible, and we understand the fallibility of others (generally speaking).
AI is not that at all. If my automation does something stupid, I still have all the blame, and yet I have nowhere else to pass it off to. "We don't understand why our AI did this really stupid thing" is, frankly, not a satisfying response (nor should it be). Businesses employing AI certainly are not absolved of any form of accountability, and are arguably exposed to more of it (since they're not able to pass the blame on to another fallible human, and have to take direct accountability of a system they built but don't fully understand).
> 99% isn’t good enough for truly critical applications, especially when you don’t know for sure that it’s actually 99%; there’s no way to detect which 1% might be wrong; there’s no real path to 100%; and critically: there’s no one to hold responsible for getting it wrong.
AI also exposes the possibility of systemic error where humans would be stochastic.
A human might only identify the right number of rentable units from a spreadsheet (to pick an example from this article) 97% of the time when an AI might do it 99% of the time, but even the same human will have a different 3% error on each day. The consequences of failure are more limited and more dilute.
On the other hand, the AI may work perfectly right up until a holding company redesigns their data tables for the 100th time, whereupon it misreads every financial report with much more concentrated ill effect.
When humans are wrong, the business’ “ego” can be saved by blaming the employee who made the call, sometimes firing them. But the process goes on with the same error rate.
But when software makes the wrong call, it feels like the business itself has done the wrong thing. With no way to externalize the blame for the decision, the blame gets placed on the decision to use ml in the first place.
This.
This is strictly more accurate than the above comment.
It has nothing to do with accuracy or the repetitiveness of the mistake. It has to do with managers preferring to make sure that they can somehow avoid being personally blamed for their subordinate's mistakes, because most companies are designed to absolve everyone at the top and blame everyone at the bottom.
I've worked in an industry (prop trading) where the costs of mistakes were real, and it is the most heavily automated industry there is. I've also worked in another part of the finance industry (hedge funds) where mistakes are paid for in other people's money, and it's super obvious that managers there are way more interested in passing off blame than fixing problems relative to how interested they are in doing these things in prop trading, and there is correspondingly less automation.
It's not about people in the population at large believing that machines are more prone to repeating mistakes than humans. Look at how many people are excited about the concept of self driving cars. It's about managers trying to convince themselves that this is true, so they can go on being unethical.
> It has to do with managers preferring to make sure that they can somehow avoid being personally blamed for their subordinate's mistakes
You can't fire a crappy AI and replace it with one with better judgement. If it does something weird or bad or inexplicable, you can't ask why it did that and tell it to how to be better in the future. Blame is a tool that organizations and people; you can't blame AI because you mostly can't teach AI.
>An automated process only has to be wrong once to compel human operators to double or triple check every other result it gives. This immediately destroys the upside as now you’re 1) doing the process manually anyway and 2) fighting the automated system in order to do so.
I think this is the core of the problem. 99% isn't good enough. Even 99.9% isn't good enough when we are talking acceptable accepted error margins. Even if humans make more mistakes than the AI, telling our customer it was a human error is much easier for our customers to accept than telling them it was a program error without our threshold tolerance.
We see this with self driving cars. People's reactions to the machines is that the machines have to be far better than humans before humans will be okay with the risks involved. This also holds for financial aspects. Imagine your grocery store telling you that there is a X% chance of being double charged for an item and that is within acceptable error tolerance. Even if X is lower than the rate that human grocers accidentally double charge will people be okay with that as the planned error rate or will they demand perfection?
Even if you balance the double charging with a chance to get an item for free (or negative price, acting as a credit), humans will still complain when they get double charged and stay silent when they get the bonus. But of course there's rich people who don't even care they were double charged for something like groceries.
Most of these issues aren't about how many 9's of reliability there are, but whether or not some person is accountable. AI is not itself accountable, only the person in charge of it.
Many companies already give gift cards or whatever when a human mistake happens as a form of customer service, and there's not enough compensation around AI error rates to make them palatable.
>Even if humans make more mistakes than the AI, telling our customer it was a human error is much easier for our customers to accept than telling them it was a program error without our threshold tolerance.
The veracity of this statement greatly depends on whose mistake you are saying it was.
List of things by how hard they are to say (easiest to hardest):
Apologies, I didn't communicate what I meant clearly. I meant that our customers would accept us blaming human error easier than they would accept us saying it is by design.
This is well put, and I've worked places where this has happened to customers. Our current domain is in augmenting humans with AI, which avoids the problem you're describing because it doesn't take humans out of the loop at all, but just makes them more efficient at what they were already doing by cutting out a lot of work. So jobs that would take an operator 30 minutes now take 10 because the AI has done most of the usual rote work.
From what I've seen of attempts to apply AI to the cyber security domain, a big impediment tends to be the lack of cohesion between the AI experts and the domain experts. There's often decades of work on the domain of study, and it's important that you use this information to inform your algorithms/ML models. I once worked with some ML researchers on trying to apply machine learning to some netflow data we had. These researchers were quite good at their field, but didn't know much about netflow data analysis. I remember them being all excited in a meeting because their models found some pattern in the data. It turns out they had discovered the difference between high ports and low ports...
I wish I could upvote this twice. This is the exact problem preventing widespread adoption of "AI" throughout most businesses. A model that is 99% accurate is not good enough for most mundane business tasks because they will fail in ways that no human ever would. Additionally, when you consider how much it costs upfront to hire a team of engineers and data scientists, build a training dataset, develop the code, maintain it etc, it quickly becomes clear that except for the most costly processes internally there is no way that AI is going to be cheaper than hiring a bunch of people in India by the hour. Not to mention that the people in India can be retrained to do something in hours that would take your super specialized team of engineers months to reproduce with code.
Let's also not forget that most business processes are bespoke to each corporation. Finding a single process that can be successfully targeted across multiple companies is hard. Some people like AWS and Google are trying with Textract and some of these other AI-as-a-service products, but they're not having a lot of success. They still fail all the time.
> why can't it read a PDF document and transform it into a machine-readable format?
It can definitely do that, but you might not like the cost/benefit analysis, depending on how many such documents you want to process. The costs are coming down steadily though as the tech improves. If you need to do millions of such documents, yeah a model will probably be worth it. But if you need to do a few hundred you probably should just do them manually.
The thing is, reality has surprisingly high resolution. When you give out a task like this to a person, they will likely come back to you for clarification about how you want to deal with some of the examples. Your initial requirements will be underspecified, or incorrect, in some details. When you are dealing with a person, these minor adjustments are pretty inconsequential, and so you don't really notice it happening. The worker might also have enough context to guess what you want and not ask, and just tell you the summary when they deliver the work.
If you're training a model, you need to work through all these annoying details about what you want, just as you would when you're creating any other sort of program. This adds some overhead, and places a lower bound on how many examples you'll need to have annotated -- you'll always need enough examples of annotation to actually specify your requirements, including various corner-cases. You need enough contact with the data to realise which of your initial expectations about the task were wrong.
So there will always be a lower scaling limit, where the automation isn't worthwhile for some small volume of work. The threshold is getting lower but there will always be a trade-off.
> The thing is, reality has surprisingly high resolution. When you give out a task like this to a person, they will likely come back to you for clarification about how you want to deal with some of the examples.
What could that even look like? In the extreme case this is like asking, "Why do I have to write programs, can't the computer just do what I tell it?". Writing the program is telling it what you want. In supervised machine learning, annotating the data is, instead.
If you tell me, "Go to ebay and get me a list of the prices of washing machines", that sounds simple, but then I'm faced with some washer/dryer combo or some hand-crank contraption. These are things you didn't think of. I can either ask you, or take a guess and hand you something that needs to be cleaned up later.
If I'm instead training a model, I need to encounter these tricky examples in order to ask you for a policy on them. If I'm collecting an unbiased sample of the training data, this could take a very long time. If there's some sampling strategy maybe it's faster, but there's still a minimum number of examples we need to think about, no matter what.
A better question would be why AI works great for some business (e.g. Netflix, AirBnB, Uber, Waze, Amazon) yet fails miserably for other (JC Penney, Sears).
In my view, the older companies are trying to strap on AI on top of a traditional dataset, which never collected any useful signals.
The new companies designed their entire business concepts around data, and collected what's needed from the get-go.
Sears may have a 100 years worth of useless data.
AirBnB has about 13, but so much more informative.
Amazon applies A/B testing all the time - would anyone at Sears even know what it is?
A secondary issue with business data is that vast majority of the features are categorical, for example: vendor id, client id, shipper id, etc. These usually get hot-one encoded, and you end up with hundreds of features where there's no meaningful distance metric. Random Forest and XGB are about the only that produce somewhat rational models, but in reality, they are good because they approximate reverse engineering of business process.
And lastly, the hype far outweighs the possibilities, at least until the business are ready to re-engineer the processes, if it's not too late.
AI works for online businesses where you have millions going to a single interface and thus any testing is on a random selection of the population.
You couldn't do that as well with different stores simply because malls have different demographics (meaning any conclusions could be noise) and the costs of shuffling where things are located is high so you can't just test 50 different setups and see what works.
Sears has been around for about a 100 years, and they have invented the catalog sales model - they were the Amazon of that era. I don't blame them for not doing things like A/B testing a 100 years ago, but 10 or 20? They were asleep at the wheel. That's when Walmart took off like a space rocket.
Essentially the same offering to the same demographic. Yet Sears collapsed.
There were other factors at play in the downfall of Sears than just failure to take advantage of their positions at the time. Sears, in a sense, became a victim of its own success. In the heyday of parasite capitalism, it became more profitable for a small group of bad actors to make Sears fail.
Xerox is a more apt example of the 'missed opportunity' narrative.
The big tech companies all deal with this issue as well. One of the big problems when I started at Google in 2009 was that an experiment would show a mild negative effect on click-throughs when what was actually happening is that it was a mild positive effect for users but broke logging on IE6, hence resulting in a 0 CTR for that population. They solved this by building a system that automatically sliced results by population, alerted immediately if any one population was a serious outlier, and displayed sliced results on the experiment dashboard.
The big old-line brick & mortar chains just didn't think it worthwhile to build this sort of granularity into their systems, and are paying the price for it. I suspect that many executives who grew up in the 50s-70s think in terms of "Is this change good or bad?" vs. "Why is this change good or bad?" (Note that brick & mortar retailers who have embraced extensive data operations - notably Walmart, Target, and Safeway - are doing great. It's the Sears & JC Penneys of this world that are failing.)
The problem with Sears is not that they have bad technical leadership. Their problem is they were purchased by a vulture who attempted to extract all of the wealth from the company, in a short period of time, for personal gain.
Your post makes some really salient points, and then misses the head of the nail in my experience. Sure, most of the information Sears historically collected is probably junk for supervised learning models. That isn't what makes this hard. High cardinality categorical features aren't what makes this hard. Re-engineering the business processes aren't what makes this hard.
What makes this hard, is that for all of these companies, machine learning models are essentially used in place of heuristics of varying degrees of complexity. The models are being used to incrementally improve heuristics that in some cases are tuned quite well. Couple that with the issue that a machine learning model is only one piece of an actual product improvement for these websites/companies (ie: now you have a prediction, what are you gonna do with it to effect the product?), and all of the sudden you have actual incremental improvements completely misaligned with moonshot expectations.
We've had plenty of rankers and recommenders for decades, the current wave of deep learning variants are improvements, but they're incremental from a high level. If your business wasn't successful/is failing using simple heuristics (sears, jcp, etc), a machine learning model isn't going to magically correct that.
The obvious answer to me is that the hardware is only available to handful of players and the libraries aren't mature yet. PyTorch has been around for about 4 years; that isn't enough time for a lot of people to have gotten comfortable with it.
The people who have access to people with software and hardware have found a lot of uses for the tech - I assume AI basically is Google Image Search.
I'm surprised to hear this organisation is successfully doing ML speech to text. Is it running 100% of volume in production? Or is it more of a pilot type thing? I know of a French multinational bank that just tried for 2 years to get a ML speech transcription up and running, for transcribing conversations with customers, but due to unreliable results, recently put the project on ice. Their experience was much along the lines of everything discussed in this comment thread.
I think the expectation of "drop some AI in" and do XYZ job is off in the same way that "we're going to drop a humanoid robot in" and do XYZ job is off.
Where I've seen it used well is as a piece in a larger system of automation. In the healthcare case, it's doing a first pass at transcribing an audio dictation so that a transcriptionist can then start with a 90%+ accurate document.
This is tough, their role shifts some (more editor/correctionist than true transcriber) and not everyone makes that transition well, but the end result is 2x+ efficiency gains.
The problem goes much deeper than researchers not having enough PDFs. If you look at where machine learning is successful it’s usually processing spatially related data. The pixels in a photograph have a spacial component where points near each other are more related than points far apart. Same goes for audio, and even text. Words in a paragraph that are close to each other are usually more related than words far apart.
A spreadsheet has very little or no spacial component for a neural network to learn, and the location of an important number in a pdf probably has little to do with the number’s significance. Without a spacial component to do pattern recognition a lot of the recent advancements in machine learning like transformers or convolution get thrown out the window.
There are some machine learning problems that can be solved with more or better data, but I don’t think PDF to JSON is one of them.
IMO machine learning has nothing to do with learning. It might be able to find out some patterns in the data set which can be very useful to recognize something like license plates etc. But unfortunately finding out patterns does not mean understanding them which is the fundamental of learning. Then the result is quite obvious and simple: you don't know what you don't know and you cannot transform something you don't understand.
I forget the name of that silly robot in the picture, Ginger or something. Designed to take food orders and possibly deliver them to tables, they get dragged out to bank branches and need to be supervised by employees the whole time. It is bad enough that they could only provide the most trivial of information (worse than web search), IT also struggled to keep the things connected to WiFi.
Multi-billion dollar corps 'doing AI'.
Because at this stage of the game, "artificial intelligence" is still an oxymoron .
It's really just a database developed through trial and error (aka "training") that we "hope" contains enough differentiating data points to produce a reasonable, weighted "best guess".
People try to apply AI to high-risk problems that smart people can't solve. When AI is applied to lower risk probelms that are usually easy for people to solve, we seem to get great results (i.e. recommendation engines).
I seem to emit some kind of anti-AI field - voice recognition works about 40% of the time so is basically useless, recommendation algorithms seem to recommend stuff I have already watched or seem completely random and as for the mechanisms that select adverts - Youtube seems to be taking the approach of showing me the same adverts again and again and again (and yes, I do click thumbs down on them) until I passionately hate the products being advertised to me (because I watch videos about cars does not mean I want to watch the same BMW 8-series advert for a few months).
The recommendation algorithms seem to be just the most simplistic type of pigeon holing. Think Netflix, just because I watched a European political thriller last week, it is assumed I now want to watch this genre for eternity.
We should keep in mind that companies and sectors set a different goal for the recommendations. Others prefer to show recommendations on similar items, others from what similar users have purchased/viewed/listened, others on highly-profitable items, others focus on discoverability of new items, etc. And as a result people view recommendations differently based on their personal taste and purpose.
And of course, we should always compare the recommendations with the usual baselines. e.g. i sometimes hate youtube recommendations, but what if the baseline was videos that are trending or are watched by users in my country? I would hate them more.
It's much easier to build a rec-engine that uses user data to make recommendations than it is to design one that analyzes intrinsic properties of items to build recommendations. Think how Spotify recommends music based on what other people who listen to this song like. This favors popular music. They could build an engine that analyzes musical characteristics to make recommendations, which would eliminate the popularity bias, but introduce others.
Actually Spotify does more than collaborative filtering. Here’s a superb blog post on using convolutional filters on the spectrograms to build content-based recommendations: https://benanne.github.io/2014/08/05/spotify-cnns.html
A more pertinent example may be an early warning system, common in finance, fraud, and ops, which flags when a statistic is an outlier of some type. These flags then get followed up on by designated folks.
Not as sexy as a general recommendation engine, but useful nonetheless.
Because the very concept of "business" is supposed to be boring and not very intelligent (which doesn't mean it doesn't require knowledge in its field, it's just not... alive).
I am an AI researcher but and I would love to investigate useful systems for business, but I have no idea about the business processes that this article mentions.
Many real world business processes assume a certain knowledge of the world and the relationships between entities, and not just a limited set of data points about the task at hand. Such kind of knowledge is not yet incorporated into today's ML systems.
One thing I haven't seen explicitly mentioned is the (probably) intrinsic limitations of AI/ML in classifying/predicting human behavior. For a number of years I have worked on fraud prevention tasks where the goal was to take in some information about a payment and decide whether it was fraudulent or not.
Even though what we were doing was primitive and could have benefited from a lot more ML, I suspect that even then the best that you could have gotten would have been some sort of anomaly detection system that can catch a good share of the kind of fraud that you have seen in the past, but will never be very good at detecting an intrinsic change in fraudster behavior.
On top of this, especially when dealing with humans, you often are expected to be able to explain why a certain decision was made. Setting payments aside, think of predictive policing or sentencing decisions. In those cases ML is essentially guaranteed to build in all sorts of biases regarding somebody's race, gender, place of living etc.
This reminds me of Data Robot laying off people "beacause of covid" right after finishing a 300 million dollar round. This AI companies have lied about their valuation for awhile and it's catching up to them.
Ironically enough, after data robot did a layoff they also completed a large acquisition and hired more executives.
389 comments
[ 3.3 ms ] story [ 298 ms ] threadIn our application, one key operation is that the user is required to classify a line item based on the text description. There's a huge code list of possible classifications, and the user has to pick one that is the most correct.
This is definitely a task that registers consciously. And, while most of the time it's fairly easy for trained users, there are often cases which require extra thought or research.
For example, a T-shirt of mostly cotton vs a T-shirt of mostly synthetic fibers should be classified differently. How would you know based on the description "Small Womens V neck Short"?
Of course, an AI/ML system that could reliably classify a majority, and reliably classify the rest as "unknown", would be interesting. Not sure how close we are to that though.
(1) Inflated expectations from higher/middle management which trickle down the organization. AI is seen as a high-profile case which has to lead to success (and a larger budget next year for my dept.)
(2) Data quality issues. The data itself has issues, but the key issue is lack of metadata and dispersed sources. Lack of historical labels (or them being stuck in Excel or on paper) is part of this as well. Big data without any labels is mostly useless, contrary to expectations
(3) Most AI or ML projects are not about ML. In fact, they're mostly about automation or rethinking an internal or customer-facing process. In many cases, such projects could be solved much better without a predictive component at all, or by simply sourcing a 1 cent per call API. AI is somehow seen as necessary, however, without which our CX can never be improved. ("We need a chatbot" vs. "No, you just need to think about your process flow")
(4) Deployment issues and no clean ways to measure ROI leads to projects being in development indefinitely without someone daring to stop them early. This is also related to orgs starting 30 projects in parallel (2m lead times with one to two data scientists for each), which end up all doing kind of the same preprocessing and all lead to kind of the same propensity model. No one dares to invest in long-term deeply-impacting projects as "we want to go for the low hanging fruit first"
This in turn means that it may fail catastrophically when faced with adversarial examples or with examples that are drawn from a different distribution than that of the training set.
> Specifically in regards to adversarial input, humans are often the weakest link in terms of process security
I have yet to encounter a (healthy) person who looks at a photo of static and mistakes it for a cat.
More relevant however is that we humans can understand that we're faced with an optical illusion and we can make adjustments accordingly. We have formed the concept of an "optical illusion" and we just place "The dress" in that category. A machine needs to be specifically trained on adversarial examples in order to be able to predict them. Once you come up with a different class of adversarial examples it will continue to fail to detect them. There is no understanding there, just more and more refined pattern matching.
Does a machine that can match any pattern actually "understand"? I would say no. But these are already philosophical considerations :D
Broadly speaking, yes. At the same time that's not what happened in 2015. It produced so much polarizing content with people deeply entrenched in their believes. They might have recognized it as an optical illusion, but they refused to make adjustments.
> A machine needs to be specifically trained on adversarial examples in order to be able to predict them. Once you come up with a different class of adversarial examples it will continue to fail to detect them. There is no understanding there, just more and more refined pattern matching.
Moving away from image recognition examples, isn't that exactly what happens with humans predicting whether an email is a phishing attempt? I remember reading here on Hacker News this week about phishing tests at GitLab. It had a lot of comments about tests and training employees to spot adversarial emails. Some companies are more successful than the others. It is a complicated problem; otherwise we would have solved it already. But it's the same principle because phishers come up with different ways of tricking people. And some people will fail to detect them.
This doesn't mean that the kind of thinking that goes on in the human mind is the same as the pattern-matching that goes on in an ANN (for example). Think about how ppl learn to talk. It's not like we expose infants to the wikipedia corpus and then test them on it repeatedly until they learn. There are structures in the brain that have a capacity for language - not a specific language, but language as an abstract thing. These structures are not the same as a pre-trained model.
The truth is I don't know enough about cognitive science to properly express what I'm thinking, but I'm pretty sure it's not just pattern matching :D
(To your point, though, I agree that machines can make strange errors, raising trust issues. My experience is that ML is useful in cases like recommendations or search results where a person can interact with predictions rather than being a complete replacement)
https://henrikwarne.com/2020/05/19/artificial-intelligence-a...
As one of the other comments points out, technology can only change productivity when you change the process, sometimes radically. And that may mean restructuring the business.
> why can't I get a computer to translate my colleague's financial spreadsheet into the format my SAP software wants?
Because you probably expect it to be 100% or maybe 99.999% accurate, and we can't do that. Imagine "AI" translating someones financial spreadsheet into a different format and dropping a zero somewhere. Oops.. but your test set accuracy is 99.8984%. Still not good enough. Just getting 1 thing wrong breaks everything. This is fundamentally different from clicking on image search and ignoring the false positives.
This suggests to me that lot of office jobs will be lost just by modernizing systems and making them spit out JSON
Consider invoicing, you may have hundreds of distinct sources, but none of them individually would warrant automation.
It is a problem of standardization. Any standard complex enough to cover all business cases would be too expensive to implement, or you could not get all stakeholders to adopt it, and so on. Accountants would not want to make themselves redundant, obviously.
Websites have been able to produce "ready to print" html for a while now, and html (even with a lot of restriction) would be hundred times better than PDFs.
Much of the world is full of PDFs that are bad scans. Upside down, sideways, green, or maybe in French because why not.
>making them spit out JSON
This doesn’t just require fixing your business but every other biz you interact with. Modernize your own stuff isn’t enough. A bit like offices still list fax numbers
Mistakes are tolerable, but you need to be able to recover from them somehow, and recovery from an AI mistake seems to be something that people like to pretend is unnecessary or impossible.
We currently live in a world where failures are tolerable but seem adamant that moving to an AI driven world means that failures are no longer tolerable. We still haven't lost the ability to make mistakes (because we still do many things manually) but we really seem keen on losing it.
School grades are in an environment where you learned it recently, have no access to references, and have not been doing it very long.
If you had to keep doing algebra for even a month on a full time basis, you will get a lot better than 90%.
Even in many highly critical human endeavors, there are many errors.
The key to success is not absolute error-free perfection, it is no critical errors in components that are severe enough to kill the project.
Every rocket launch has some issues, but the successful ones have issues where it doesn't explode or land in the wrong orbit.
In the spreadsheet example, dropping a critical zero will cause damage akin to the rocket explosion. But dropping an "O" in a label field is utterly trivial.
Humans understand the distinction, constantly make such judgements and focus on the critical areas in their moment-to-moment work and embed it in their work processes. These constant criticality judgements are not just binary, but refined scaled, and serve to apply resources where needed.
The AI systems do not have such a judgement layer, and apply the same degree of inaccuracy to every part of their domain. So, absolute 100% accuracy is required, as errors are no less likely in the critical components.
By "not that complicated", I mean a decent CRM system to track information about the organization is approaching peak operational efficiency for most businesses. Most inefficiency I see after that is people/political problems.
By "people don't know what AI is", I mean that business owners are unable to describe their business problem as a supervised learning problem. If you can formulate your business problem as a supervised learning problem, then you can probably solve it with AI (which, yes, is really just a marketing term for supervised ML).
But most business problems are really "order taking" or "production/delivery" or "moving things through a funnel" problems and thus AI isn't the solution, CRM or CRUD apps are the solution.
I now see Google sales dragging these people out to 'AI meetings', which are about Dialog Flow. (Meeting pitch email gets around, business manager reply all: 'WTF is this?') Later they summarized the meeting, as they understood it: "You just need to drag and drop the CRM on the 'Brain' and then you can book haircut appointments and open a bank account. We were like um, we develop and support software, they were like - just drag and drop you knowledge management API on it too." Uh... what KM system?
The complicated part is that in small and midmarket business, their knowledge is tribal and processes are folklore. There are processes that are followed though they don't exist, and business rules violated as they aren't aware.
Meanwhile, no one in sales puts useful information in a CRM system if they can avoid it, the least effort principle on administrivia is important if you are ever going to hit your ever growing number.
Finally, the management must still be spooging for SPOGs, I sill see marketing for them as some sort of nirvana. Meanwhile, they have been collecting data randomly about everything, all over the place, excel documents, FTP servers, and of course databases. Here, no one in IS/IT will admit: The servers were here when they started their job and they have no idea what is on them or if they are even needed anymore. They certainly aren't mentioning that to the incoming CIO, who gets focused on improving efficiency as per his executive mandate. So he/she gets to work with cloud migrations (more hilarity ensues)
I need to stop now. It's a rat hole...
Business is about exploiting what exists. This is why the buzzword is "innovation", not "invention". Incremental improvements, not qualitative jumps. So nothing will ever be really considered "Artificial Intelligence" once it is boring enough for business.
Scheduling algorithms are incredibly useful for business. There was a time when this was considered AI, but that was the time when they didn't work well enough to be useful.
Example: my wife is an admin in a school office, and a ludicrous amount of her and her colleagues' time is spent on replicating data entry between a multiplicity of different incompatible systems. The Rolls Royce / engineer's solution to this would be to provide APIs for all these disparate systems and have some orchestration propagating the data between them, except of course that's never going to be remotely practical; instead, dumbly spoofing the typing that the workers do into the existing UIs is a far more tractable approach. My (admittedly not 1st person based) experience of these things is that they currently still require significant technical input in the form of programming and "training", but this fruit has got to be hanging a lot lower than any ML-based approach.
Also, if it's been running for months before it breaks, the humans that used to do the work are gone, or have forgotten how to do it.
Similarly, seen things like using a email as a trigger, when the source application has configurable web hooks.
Feels like there's an RPA culture of sorts to assume the things being automated only have human based interfaces.
Your first statement is a trusism, any solution can be a dangerous band aid, if applied incorrectly. RPAs solve real problems now, really the only correct measure.
Of the RPAs I have raised, I always watched them work, they just prevent mistakes. I would suggest taking a screen recording and verbally annotating it for posterity. Furthermore, nothing says your RPA has to run open-loop, one can put in checks to ensure that it hasn't gone off the rails.
I also believe that the leading edge RPA systems do take advantage of some real AI techniques. And that more AI will be deployed in RPA as time goes on.
The ethical committee required that prior to using any data, you have to make a static copy in another database. Their argument is
1. They don't want excel files flying around (which will happen regardless) and
2. To perform any analysis, you "obviously" have to have "structured data", which "obviously" means that you have to extract a csv from the base system (MongoDB) and put that into a RDBMS (redcap).
Go figure...
When I was on one I would by default let everything through unless I came across something where I thought I could help improve the experimental design. My colleagues on the other hand would go through them like little emperors and cause grief for the applicant just because they could.
I mean, what a weird combination of tech (un)savviness
This article doesn’t mention a really, really straightforward factor for why AI hasn’t invaded these domains despite billions of dollars being dumped into them.
An automated process only has to be wrong once to compel human operators to double or triple check every other result it gives. This immediately destroys the upside as now you’re 1) doing the process manually anyway and 2) fighting the automated system in order to do so.
99% isn’t good enough for truly critical applications, especially when you don’t know for sure that it’s actually 99%; there’s no way to detect which 1% might be wrong; there’s no real path to 100%; and critically: there’s no one to hold responsible for getting it wrong.
In lots of business contexts, probably most, reducing variance is much more valuable than reducing mean expenses. Variance can halt downstream production, so the loss can be some huge amount of opportunity. And variance propagates through a supply chain, so your customers will hate variance in your output, as it may mean they have to ship the variance forward, which their customers hate. Plus if you allow your supply chain to get away with inconsistency, they can start to rob you with lower average quality and it will take you time to notice.
If a company has been bothering to do some process manually and they haven't outsourced it to the cheapest humans possible, then they care more about low variance than low cost. Pitching these businesses a solution that lowers cost at the expense of unknown high variance is very unattractive. Instead, you want to tell them "I can reduce your variance even further! Here's what that would cost".
But the argument still applies then - if only one incident occurs where a catastrophic mistake was not spotted by the model failsafe, and if later investigations show that the mistake could've been easily spotted by a second human, a human will be installed as a failsafe for the failsafe.
I think the general problem is the following: if a human makes a grand mistake, it can usually be attributed to a temporal lack of care, or just random bad luck, or happened because the person was having a bad day, or... it is also generally understood that making a grand mistake will be such a shock for the person responsible that this person will most likely never make the same mistake again. On the other hand, if a machine makes a grand mistake, the intuition of the general public, trained by centuries of experience with techology, is that this machine will make the same mistake again and again and again, when prompted with the same input. If the model is not designed to learn from mistakes in production, this will of course actually be true.
Surely this will depend on the cost of failure, and the cost of the human failsafe.
Spellcheck in an email client helps prevent the minor embarrassment of typos and spelling errors in emails, and few emails are so consequential that it's worth having them carefully manually vetted.
That is frequently true. Take the job of a lifeguard for instance. A single mistake can be catastrophic, and yet we know that people have trouble staying completely focused for hours on end.
AIs have no trouble staying focused and they can be trained to spot drowing swimmers pretty well even on a crowded beach.
Having a human lifeguard plus an AI that alerts the lifeguard when it spots something suspicious could lead to better outcomes than employing two lifeguards.
I disagree. I think you'll see the same thing as with drivers falling asleep while Tesla autopilot is running. The lifeguard will let the computer do all the monitoring since a low false negative rate combined with a low incidence rate means that most bodyguards will experience the computer being 100% reliable for weeks or months at a time. In fact it's not unreasonable that if the lifeguard sees someone drowning but the computer doesn't register it as such the lifeguard may question their own judgement based on experience.
Well, that is kind of the problem with AI. How do you fix a "bug" (if you can even call it that) in a model you cannot fully understand? Do you re-train it on the catastrophic mistake and somehow give it more weight? How can you be sure that this won't lead to any problems where previously there were none? How do you explain to a customer that your model now doesn't make mistake A anymore, but now mistakes B and C frequently occur? The only safe bet is to write some auxiliary code, which first uses the AI as a black box, and afterwards explicitly checks the result for this particular mistake. If this happens again, and again, and again, you need a human to maintain and extend this auxiliary code and also adjust it to changes in the underlying model, at which point I am quite certain just using a person of average intelligence instead of AI will be cheaper, more reliable and more flexible.
It's not like Google blocking your email or YouTube account, we're taking about your friggin bank account here.
I don't know how they're still in business and growing with such a process in place.
Undoing bad automation in finance often means reversing a very cheap edit on some database and pissing off a customer who’s so powerless that it won’t affect your business anyway.
Are they?
I doubt its customers are generally aware of the AI issue, or if they are, they must assume it is not out of the ordinary.
they might have a high valuation, but thats about all the financial success they have.
And even if the inputs are great, it's hard to understand what we can do with a 99% confidence prediction. The next great unlock that society is waiting for in the AI realm applies to industries where failure is not an easily acceptable outcome.
Take self-driving cars for example. Even if we can objectively prove that current AI models can drive on current roads in current conditions with a lower fatality rate than humans (this is debatable anyway, but let's assume) - what do we do when it knows it's not confident? If we assume the driver hasn't been paying attention during the ride so far, and a scenario comes up that the AI is uncertain about...the human now has likely mere seconds (at most) to capture their surroundings, analyze the risk, and take corrective action. If we assume the driver has been paying attention during the ride, then what was the point of the AI? Moreover: what if it thought it was confident, but it was still wrong. Who do we blame? How do we mitigate future instances of it? What were once societal problems we could blame on a fallible humans are now obscure technology problems we can't introspect. That's a scary place for a lot of people to be.
Basically, we've reached the level of AI where it can be used as a backup to humans and protect us from royally fucking something up. But the next big unlock will come when it can be the primary actor. It's hard to imagine how we'll get there without the AI actually understanding what it's processing in some real way.
Edit: rephrased the 3rd paragraph.
For example, Amazon Seller Central, Google, Youtube, can "outsource" their customer service to AI, because they are pretty much the only players, so customers have to suck it and deal with the frustration of a terrible experience by not getting help and not talking with a human.
With any other business if they get automated replies that don't solve a customer issue and they are unable to talk with a human, 99% of them say "fuck it" and go somewhere else.
This is one small realm of the relationship of AI and businesses. Then you have employees, suppliers, supply chains, finances, internal processes, so many multi-variable, fragile and nuanced systems that I doubt if they're not developed in-house, they'll probably do more damages than solve problems.
I can only speculate, but while it's true humans can fail when following a process, it's also true they can sometimes spot a potential failure even then there is no process to prevent it. They can come up with new processes, or ways to improve existing ones. They can also account for their actions, and managers can account for the activities of their team. All of this builds trust that the process can be improved in ways that can be well understood.
People are also scalable. You can implement controls like four-eyes on changes and critical metrics, so you're less exposed to one person's idiosyncrasies. It's hard to do that with AI.
Think of a DL powered, visually guided robot vs. a human on a production line.
One task might be to do QC inspection at the end of the line. Suppose something gets on the camera lens. In general, the DL system will keep chugging along and the accuracy will degrade. The human will notice this and clean his glasses.
If he sees something ambiguous as it passes on the line, he might give it a bit more attention or adjust the angle he's looking at it from. If he sees a series of the same anomalies, he will notice a pattern. Perhaps one of the machines up the line from him has started to introduce a new type of defect.
Suppose in assembly a worker has a sore muscle. He might adapt his motions to compensate, slow down, go to the doctor, take some pain pills, or take a day off. Unless programmed or trained to detect this, a robot will keep driving its failing motor harder until it breaks.
Like with self-driving cars?
Pharmacy - adding 1% more of a chemical compound to a pill could be lethal, but not otherwise detected for 3-12 months (pills sit on shelves for a long time). Figuring out why that happened will be very difficult.
Manufacturing - molding plastic parts requires very specific mixes of chemicals. Too much of one and it becomes brittle under certain temperatures, too little and it warps down under the warranty period. So if you manufacture the mold of a baby car seat, and you add or subtract 1% of a chemical, that car seat could break / shatter when it's involved in a collision. Terribly large lawsuits would occur because some C-level someone adopted a process to save headcount and reduce oversight.
Business processes are a lot like software - the first 80% takes 20% of the time and the remaining 20% takes 80% of the time.
There are plenty of places where human validation shouldn’t be necessary, but is.
"... and critically: there's no one to hold responsible for getting it wrong."
Could this be part of "AI"'s appeal? A dream of absolving businesses and individuals from accountability.[2]
1. "What's more, artificial research teams lack an awareness of the specific business processes and tasks that could be automated in the first place. Researchers would need to develop an intuition of the business processes involved. We haven't seen this happen in too many areas."
2. Including the ones who designed the "AI" system.
Because whoever does achieve the next unlock - should it happen - will receive an unimaginably large windfall. This is the classic intent of venture capital. In fact, I'd suggest that AI is actually one industry where VC is doing what it does best: taking extremely risky bets with a large potential upside.
> Could this be part of "AI"'s appeal? A dream of absolving businesses and individuals from accountability.
Presently, this seems to be one of its large detractors. If I have an employee do something stupid, I can say that an employee did something stupid. People might wonder why they were allowed to do that stupid thing, and what we're going to do to prevent it from happening again, but the explanation of the source is satisfactory. We're fallible, and we understand the fallibility of others (generally speaking).
AI is not that at all. If my automation does something stupid, I still have all the blame, and yet I have nowhere else to pass it off to. "We don't understand why our AI did this really stupid thing" is, frankly, not a satisfying response (nor should it be). Businesses employing AI certainly are not absolved of any form of accountability, and are arguably exposed to more of it (since they're not able to pass the blame on to another fallible human, and have to take direct accountability of a system they built but don't fully understand).
AI also exposes the possibility of systemic error where humans would be stochastic.
A human might only identify the right number of rentable units from a spreadsheet (to pick an example from this article) 97% of the time when an AI might do it 99% of the time, but even the same human will have a different 3% error on each day. The consequences of failure are more limited and more dilute.
On the other hand, the AI may work perfectly right up until a holding company redesigns their data tables for the 100th time, whereupon it misreads every financial report with much more concentrated ill effect.
When humans are wrong, the business’ “ego” can be saved by blaming the employee who made the call, sometimes firing them. But the process goes on with the same error rate.
But when software makes the wrong call, it feels like the business itself has done the wrong thing. With no way to externalize the blame for the decision, the blame gets placed on the decision to use ml in the first place.
It has nothing to do with accuracy or the repetitiveness of the mistake. It has to do with managers preferring to make sure that they can somehow avoid being personally blamed for their subordinate's mistakes, because most companies are designed to absolve everyone at the top and blame everyone at the bottom.
I've worked in an industry (prop trading) where the costs of mistakes were real, and it is the most heavily automated industry there is. I've also worked in another part of the finance industry (hedge funds) where mistakes are paid for in other people's money, and it's super obvious that managers there are way more interested in passing off blame than fixing problems relative to how interested they are in doing these things in prop trading, and there is correspondingly less automation.
It's not about people in the population at large believing that machines are more prone to repeating mistakes than humans. Look at how many people are excited about the concept of self driving cars. It's about managers trying to convince themselves that this is true, so they can go on being unethical.
You can't fire a crappy AI and replace it with one with better judgement. If it does something weird or bad or inexplicable, you can't ask why it did that and tell it to how to be better in the future. Blame is a tool that organizations and people; you can't blame AI because you mostly can't teach AI.
I think this is the core of the problem. 99% isn't good enough. Even 99.9% isn't good enough when we are talking acceptable accepted error margins. Even if humans make more mistakes than the AI, telling our customer it was a human error is much easier for our customers to accept than telling them it was a program error without our threshold tolerance.
We see this with self driving cars. People's reactions to the machines is that the machines have to be far better than humans before humans will be okay with the risks involved. This also holds for financial aspects. Imagine your grocery store telling you that there is a X% chance of being double charged for an item and that is within acceptable error tolerance. Even if X is lower than the rate that human grocers accidentally double charge will people be okay with that as the planned error rate or will they demand perfection?
Most of these issues aren't about how many 9's of reliability there are, but whether or not some person is accountable. AI is not itself accountable, only the person in charge of it.
Many companies already give gift cards or whatever when a human mistake happens as a form of customer service, and there's not enough compensation around AI error rates to make them palatable.
The veracity of this statement greatly depends on whose mistake you are saying it was.
List of things by how hard they are to say (easiest to hardest):
1) Someone else messed up. We'll deal with him.
2) Our code is bad. We'll fix it.
3) I messed up. I'm sorry.
Wouldn't this itself be error prone? The operator is, after all, human.
Let's also not forget that most business processes are bespoke to each corporation. Finding a single process that can be successfully targeted across multiple companies is hard. Some people like AWS and Google are trying with Textract and some of these other AI-as-a-service products, but they're not having a lot of success. They still fail all the time.
It can definitely do that, but you might not like the cost/benefit analysis, depending on how many such documents you want to process. The costs are coming down steadily though as the tech improves. If you need to do millions of such documents, yeah a model will probably be worth it. But if you need to do a few hundred you probably should just do them manually.
The thing is, reality has surprisingly high resolution. When you give out a task like this to a person, they will likely come back to you for clarification about how you want to deal with some of the examples. Your initial requirements will be underspecified, or incorrect, in some details. When you are dealing with a person, these minor adjustments are pretty inconsequential, and so you don't really notice it happening. The worker might also have enough context to guess what you want and not ask, and just tell you the summary when they deliver the work.
If you're training a model, you need to work through all these annoying details about what you want, just as you would when you're creating any other sort of program. This adds some overhead, and places a lower bound on how many examples you'll need to have annotated -- you'll always need enough examples of annotation to actually specify your requirements, including various corner-cases. You need enough contact with the data to realise which of your initial expectations about the task were wrong.
So there will always be a lower scaling limit, where the automation isn't worthwhile for some small volume of work. The threshold is getting lower but there will always be a trade-off.
Why isn’t this generally solved, though?
If you tell me, "Go to ebay and get me a list of the prices of washing machines", that sounds simple, but then I'm faced with some washer/dryer combo or some hand-crank contraption. These are things you didn't think of. I can either ask you, or take a guess and hand you something that needs to be cleaned up later.
If I'm instead training a model, I need to encounter these tricky examples in order to ask you for a policy on them. If I'm collecting an unbiased sample of the training data, this could take a very long time. If there's some sampling strategy maybe it's faster, but there's still a minimum number of examples we need to think about, no matter what.
A secondary issue with business data is that vast majority of the features are categorical, for example: vendor id, client id, shipper id, etc. These usually get hot-one encoded, and you end up with hundreds of features where there's no meaningful distance metric. Random Forest and XGB are about the only that produce somewhat rational models, but in reality, they are good because they approximate reverse engineering of business process.
And lastly, the hype far outweighs the possibilities, at least until the business are ready to re-engineer the processes, if it's not too late.
You couldn't do that as well with different stores simply because malls have different demographics (meaning any conclusions could be noise) and the costs of shuffling where things are located is high so you can't just test 50 different setups and see what works.
Xerox is a more apt example of the 'missed opportunity' narrative.
The big old-line brick & mortar chains just didn't think it worthwhile to build this sort of granularity into their systems, and are paying the price for it. I suspect that many executives who grew up in the 50s-70s think in terms of "Is this change good or bad?" vs. "Why is this change good or bad?" (Note that brick & mortar retailers who have embraced extensive data operations - notably Walmart, Target, and Safeway - are doing great. It's the Sears & JC Penneys of this world that are failing.)
I wouldn't be so certain of others ignorance. Retail stores have long done studies and applied consultants to problems on layout, music, pricing, etc.
Does it actually? Specifically, what has AI done for Uber?
What makes this hard, is that for all of these companies, machine learning models are essentially used in place of heuristics of varying degrees of complexity. The models are being used to incrementally improve heuristics that in some cases are tuned quite well. Couple that with the issue that a machine learning model is only one piece of an actual product improvement for these websites/companies (ie: now you have a prediction, what are you gonna do with it to effect the product?), and all of the sudden you have actual incremental improvements completely misaligned with moonshot expectations.
We've had plenty of rankers and recommenders for decades, the current wave of deep learning variants are improvements, but they're incremental from a high level. If your business wasn't successful/is failing using simple heuristics (sears, jcp, etc), a machine learning model isn't going to magically correct that.
The people who have access to people with software and hardware have found a lot of uses for the tech - I assume AI basically is Google Image Search.
I used to work for a really well-known medical dictation/transcription, documentation, and coding (in the medical billing sense) company.
They're using ML models all over for speech to text, document analysis, etc.
It enables some very real efficiency gains but it's not positioned the same as something like IBM's Watson and it's somewhat ridiculous AI claims.
Where I've seen it used well is as a piece in a larger system of automation. In the healthcare case, it's doing a first pass at transcribing an audio dictation so that a transcriptionist can then start with a 90%+ accurate document.
This is tough, their role shifts some (more editor/correctionist than true transcriber) and not everyone makes that transition well, but the end result is 2x+ efficiency gains.
A spreadsheet has very little or no spacial component for a neural network to learn, and the location of an important number in a pdf probably has little to do with the number’s significance. Without a spacial component to do pattern recognition a lot of the recent advancements in machine learning like transformers or convolution get thrown out the window.
There are some machine learning problems that can be solved with more or better data, but I don’t think PDF to JSON is one of them.
It's really just a database developed through trial and error (aka "training") that we "hope" contains enough differentiating data points to produce a reasonable, weighted "best guess".
And of course, we should always compare the recommendations with the usual baselines. e.g. i sometimes hate youtube recommendations, but what if the baseline was videos that are trending or are watched by users in my country? I would hate them more.
Not as sexy as a general recommendation engine, but useful nonetheless.
Because it doesn't make money? A big enough revelation?
Even though what we were doing was primitive and could have benefited from a lot more ML, I suspect that even then the best that you could have gotten would have been some sort of anomaly detection system that can catch a good share of the kind of fraud that you have seen in the past, but will never be very good at detecting an intrinsic change in fraudster behavior.
On top of this, especially when dealing with humans, you often are expected to be able to explain why a certain decision was made. Setting payments aside, think of predictive policing or sentencing decisions. In those cases ML is essentially guaranteed to build in all sorts of biases regarding somebody's race, gender, place of living etc.
Ironically enough, after data robot did a layoff they also completed a large acquisition and hired more executives.