I actually was surprised how well they can simulate at conversation now. It is a fake because there is little underlying reasoning of course. That is a monumental problem and difficult to determine where to begin.
Do you start to give your AI a motivation or goal? Perception? These are vastly more complex problems than some statistical tricks on data that is widely available.
Still, it is fascinating that we came this far with a dead machine that talks.
Yeah we can handle the part where it knows how to express itself in a specific language, it can take in some facts, compare them to its internal database and spit out something sensible as a statistically probable good reply. But there's no sense of self involved there.
I remember reading an interesting article a while ago about at least in the human case the basic principle of emerging consciousness happens when the prediction system in our brain designed for figuring out what other entities around us do is used on itself, trying to explain what the subconscious is doing. As such the consciousness we experience is a bit of a bug in that system that turned out to be beneficial to some extent. All just a theory of course given how much we actually know about the brain so far, but it's always made the most sense to me.
I'm not sure how that would translate into the current ML environment though.
Do we want that if we could have it? I want machines as slaves for me: go wash the dirty dishes and then do the laundry. I don't want it to get depressed about doing those routine jobs.
Well we can have our cake and eat it too. Keep simple and limited machines for work, and intelligent ones to talk to and treat as equals to help us in other ways. It's not like every roomba needs to run a neural inference engine to do its job, nor do we hand pollinate ever flower ourselves if we can get the bees do it it in the organic world.
I've come full circle on that using voice assistants. Y'know - when you're a kid "terminal interfaces are bad, guis are great". When you get older and learn "Terminal interfaces are good for compsability and piping, they're great." Then you start developing software for users and learn "terminal interfaces have poor discoverability, GUIs present them their options in clear, informative ways". Then the promise of natural-language processing appears and it's "terminal interfaces are great, you don't have to be perfectly precise any more".
Now, with my google assistant? Text interfaces are awful, because I have no idea what combination of magic words will confuse it. What will be interpreted as a parameter to which command. Even simple commands get broken if I accidentally add phrasing where it interprets it as a different command... usually just giving up with "I don't know how to do that" and then Google gets a nice recording of elaborate cusswords.
You probably can't do this with google (rhasspy on the other hand...), but what is really needed is every-time someone tries something that doesn't work they go program the response that would have been correct.
I think speech recognition is good enough for this, but what is left is making a complex tree. See the 20 questions game where because so many people have put in new answers it now can guess almost everything after just a few questions. This is not easy and it takes a lot of effort to figure out where things didn't work and fix it so it will for the next person. At least some of those things you fix will never be asked again, but you need to do it.
Part of the above is speech recognition needs to expose a confidence and if it thinks it knows ask useful questions about what you mean. (they already do in some cases)
After years of unsuccessfully trying to get people to use modern GUI based software, I had a minor yolo moment and replaced an entire factory worth of employee facing software with a text-mode curses interface. No mouse support, company banner using ASCII art, interface in multiple languages that you toggle with a control key etc. All instances are concurrent user sessions on a linux VM in the cloud, it runs faster than anything we've paid for and no one will entertain the idea of going back. Not saying this is a typical situation and we're only talking a couple hundred daily users, but still, food for thought.
Can you provide more details on the use cases/workflows, the users etc? I could be suffering from confirmation bias, but I've assumed this would work in many places.
These interfaces are good because they are asynchronous, all the good things about old-timey terminal stuff but without the session overhead badness. If I message back and forth with a bot in a way that accomplishes a goal over a longer period of time but doesn't depend on me maintaining my attention, phone line, internet connection, or browser state for that entire period of time then in many cases it is a superior experience.
This is why I love chat-with-a-human interfaces on websites for support (although they mostly require maintaining browser state) because staying on a phone line absolutely sucks and the person helping you also doesn't have a real-time constraint so they often have the ability to get you better answers to hard questions.
Every "natural language" system, whether it's a text chatbot or Siri-style voice bot, is essentially a command-line interface without very good documentation.
In general, it was considered an advancement over the classic CLI (at least for environments where there's low expected user proficiency) to provide a menu of valid choices and put them front and centre.
Most customer service scenarios are exactly that-- low user proficiency (in company- or business-specific terminology, or sometimes even the very transaction flow itself) so they may not be able to quite enunciate exactly what they mean, but can easily pick from a clearly written menu.
My last trip in Canada used a Facebook chatbot for the airline. The neat/silly thing is many of the questions presented to me a form full of buttons and options. At that point it's almost a full application, just running within Facebook infrastructure and using the chat box as a UI.
If folks stop calling it a chatbot - it's almost just a nicely structured form, delivered in a fairly convenient way. No download, no link to external site, can be deployed easily to various interfaces - sms, chat apps, slack, etc.
They aren't actually the same article, this one (by Andrew) refers to the other (by Gary) and quotes the title.. it's a response article. Because of the HN policy it's hard to tell that from the titles though. Original article [0] 663pts, 408 comments.
Well yes. I’ve been told VCs invested in these back in 2015 (I was in a startup accelerator in the UK at the time and there were a few in my cohort) and a few years later very few of the chat bot investments have worked out.
A bazillion parameters in gpt3, but what does the training process amount to? Filling in missing characters or words in sentences taken from a huge dump of literature, news articles, reddit comments...
No wonder these things are so dumb still. The training process and the loss function used probably does not penalize poor long-range coherence between paragraphs. Also, if I'm not mistaken, these things have absolutely no internal state besides the characters you steam into them as conversion prompts.
If these things were trained more like agents having to operate in eg. Socratic dialogues maybe we'd be getting somewhere
The problem with that is how do you rate the dialogue produced as correct or not? Not exactly something that can be automated, but would probably need something like a recaptcha to gather responses and it'd take forever.
If you use openAI's gpt API, in the docs they talk about providing a prompt that primes the network to respond in a specific fashion. Like,
"this is a Q&A session between an agent knowledgeable about bash scripting.
Q: how do I check the current working dir?
A: use 'pwd'
Q: "
... And then the actual user query is concatenated to that.
Which is a fine way to customize a toy chatbot to sound like edgar allen poe, but no way to maintain state across a long conversation with a customer (the max prompt length is very much finite)
Unless someone smart finds out a fundamentally different approach I guess these transformer networks will never really solve chatbots.
For a while I was frustrated at how slow people have been to realize that GPT-3 sucks but lately I am more amused.
There a few reasons structurally why it can't do what people want it to do, two of them are: (i) it can't detect that it did the wrong thing at one level when interpreting it at a higher level, (ii) most creative tasks have an element of constraint satisfaction.
The 1st one interests me because I was struggling with the need for text analysis systems to do that circa 2005 and looking at the old blackboard systems. I went to a talk by Geoff Hinton just before he became a superstar where he said instead of having a system with up-and-down data flow during inference, build a system with 1 way data flow and train all the layers at once. As we know that strategy has been enormously effective, but text analysis is where it goes to die just as symbolic AI failed completely at visual recognition.
Like the old Eliza program, GPT-3 exploits human psychology. We are always looking to see ourselves mirrored
Awkward people are always worried that we are going to get it 90% right but get shunned for getting the last 10% wrong. GPT-3 exploits "neurotypical privilege" in which it gets it partially correct but people give it credit for the whole. People think it will get to 100% if you just add more connections and training time but because GPT-3 is structurally incorrect adding resources means you converge on an asymptote, say 92% right. It's one of the worst rabbit holes in technology development and one of the hardest ones to get people to look clearly at. (They always think stronger, faster, harder is going to get there...)
It seems to me an effective chatbot will be based around structured interactions, starting out like an interactive voice response system and maybe growing in the direction of
The most difficult thing to accept is maybe that even humans are bad at speech recognition. Put your mom in a chatroom to answer questions by clients of a bank, she'll be even more lost than the robot.
You need a ton of dimensions to be able to help someone: to be raised for years by humans to understand politeness, intertextual meaning, general tones, and then special enthusiasm for a specific domain to learn and enjoy helping on banking. Plus, getting money to spend on other even more interesting things in exchange for helping others motivates you to reach optimal results for your user, even if it means asking quickly other humans or sacrificing something personal for it.
Most humans put in the situation of these robots would just say "sorry I don't even understand the question, can you ask someone else" lol
I've seen a fantastic "chatbot" human equivalent once, at Apple of all place. Philipino guy (I'm in HK), absolutely dedicated, polite, cultured, very empathetic (phili people are usually adorable naturally but this one went above and beyond), went well beyond the minimum, and I feel weird saying that but I left the call with a smile and told colleagues around me "wow Apple, what a pleasant customer support, it's insane". I'll probably never say that of a robot however good they make them at talking so there's always going to be value in putting humans in front of clients.
Exactly. You can make a robot that transcribes audio to produce a transcript better than a human does ("superhuman") but it will garble 1 word in 20, thus massacre every other sentence, and leave customers feeling 0% understood.
Speech understanding requires sometimes stopping the other person and asking questions to clarify.
> For a while I was frustrated at how slow people have been to realize that GPT-3 sucks but lately I am more amused.
Generated text was not good before this era of GPT-X. It’s so much better and more interesting to work with now. It will probably keep getting even better and more controllable.
GPT-X is better than RNNs I grant you but people have built mad-lib and rules-based text generation systems that are absolutely great for specific applications in particular domains. (e.g. GPT-X is still a bridesmaid instead of a bride)
I think you could do better with RNNs than most people are doing because of structural problems.
Usually when people run RNNs for text generation they start out with the inner state of the system at 0 and then start flipping the coin to choose individual letters so you are starting from a very constrained region of the latent space and not sampling it very well.
I read a paper where they through out the idea that you ought to add coefficients for the latent state that you train for at the same time you train the network which means the number of coefficients goes up with the number of text samples but they never actually did it and I never found a paper where somebody tried it.
I was working on a project where we were developing models based on abstracts of case studies from pubmed as a stand in for clinical notes (certainly real medical notes are very different but you might say that medical notes should look like the abstract) I had the intuition that, as above, the author (and/or the patient) started out with a latent state (e.g. the patient had a disease before coming in) and that we'd get better results if we did something like the above.
It looked like a big and high risk project to develop that kind of model so I proposed something different around supervised training of a "magic magic marker" that could highlight certain areas and unsupervised multi-tasks such as "put the punctuation back in when it is taken out" but the client was hopeful that word2vec would be helpful.
I am still hopeful that incremental improvements, attacks on structural weaknesses, and appropriate multi-task training ("did the patient die?") would get a lot more out of RNN and CNN models.
Maybe it is just me but I never use chatbots and I don't understand why anyone would.
For everything I want to do there should be an UI that is much easier to use than explaining it, even to a human.
For help and troubleshooting chatbots are pretty much useless. If I have a problem doing something via the UI then probably the developer did a bad job and no chatbot will ever do better.
Nobody wants chatbots and companies just drum up propaganda to create that impression because they want to reduce their customer support costs while giving themselves a pat on the back and pretending that it is in the interests of the customer.
A chatbot is just a long way of saying "GO AWAY!".
Large telecoms do this so they get to shut down their customer support almost entirely while claiming that they are available for the customers.
a large telecom in Australia has applied this to an extreme degree in the last two years to the point that you almost can't contact them no matter how severe the issue is. Their message is clear "SHUT UP AND PAY".
We had a somewhat beloved chatbot at work, but we are of course a consulting company.
And it is gone now and replaced with a much more effective app, my point is just that it clearly possible to make chatbots that aren't rage inducing.
The question is just if it is worth it, something we didn't think after the novelty wore off.
I believe one way it can provide value is when it is closely backed by competent support staff so it takes more of a receptionist role instead of the role of a support engineer.
"Customer contact is always treated as a cost center."
"How can we provide the minimum acceptable level of service, at the lowest possible cost?"
Not with all products. Some companies understand, that good support keeps the people loyal to their products. But they are usually in a higher price segment.
The most surprising example I found was Dell Alienware support. Not sure if it's different from Dell consumer support, but I'd guess so.
Had to RMA an ultra widescreen monitor I use for work, and the experience was as smooth as silk and always in the hands of actual humans, who would follow up and push things forward.
Dell has always had good support, especially for business. I once bought a laptop second-hand that still had a few months of their highest support plan on it. Needed a screen first because of a few dead pixels. They had a tech come out to me at work and repair it on-site. A month before the warranty ended, the motherboard died. Same thing, replaced on-site. In both cases they offered to either send me the parts and DIY repair instructions or just send someone to do it. This was on a laptop I paid like $300 for (probably $2-3k new back then, it was a nice Latitude).
I’ve had their enterprise support on the phone with me for a dozen hours over the course of a week trying to get this SAN working. Their support remoted into it and had me manipulating physical stuff when required. They eventually got it going through a combination of hard resetting everything and updating firmware on half a dozen components + some software updates.
Dell doesn’t always have the greatest hardware, but their support really doesn’t rest until you’re satisfied. “No one ever got fired for buying Dell”.
> It's usually far easier to build and test a UI than integrate the same backend code with multiple vendor's chatbot SDKs as well.
I’m guessing that’s often not true if you’re running a Wordpress blog or some white label e-commerce platform or similar. You can probably just pase some JavaScript into an admin interface somewhere to add a third-party chatbot to your site.
Annoyingly some companies have started putting them in front of any customer support contact. It's incredibly frustrating. I'd much rather just a menu of options.
I just don't understand the problem they are trying to solve.
Because if you work in a call center/chat center 99% of your customers are asking the same 20-30 questions. You and I might be bright enough to read the Kb or search google but many people are not. A talented user support person is going to cost a company $40-60k/year and can only handle so many requests at a time.
I understand that, I'm just not sure why it has to be done with a chat bot. I've seen companies achieve the same by just having a simple option tree system before you get to the support agent (and divert you to self-service if possible).
Yes, good UI will cut down on 99% of customer interactions. We built a chatbot that companies can use by integrating with their e-commerce. The overwhelming majority of customers ask "where is my package?"
Why is that? Can't they just check the tracking number? That's a question I had until i tested a few dozen e-commerce solutions and most of them have buried their tracking number under several confusing menus. And even when you get the tracking number, carriers each have confusing ways to track packages.
> Why is that? Can't they just check the tracking number?
My experience is Amazon is an enormous pain in the butt to deal with if the delivery service drops your package at the wrong address or there's a hole in the package with nothing in it. Huge pain dealing with those people.
I get that there are scammers out there. But if I get 100 packages per year from amazon for years, some of which have been very expensive, and one outta nowhere they lose my $5 multivitamin, I'm not trying to get rich by fraud but I just want my vitamins.
Also a couple years ago I ordered an obscure stainless steel hinge for woodworking, kind of expensive for stainless steel it was about $8 (solid not plated USA made brass would have been $30+) and Amazon van delivery successfully delivered a bubblewrap envelope with a hole ripped in it because hinges are heavy and bubble wrap envelopes are weak. Its an $8 hinge I just want my hinge can't you ask the amazon employee in the van who delivered it to look on the floor of his van? I'm not angry (yet) but I just want my hinge so I can finish my woodworking project. If I was trying to rip Amazon off then I wouldn't be arguing with somechatbot or someone over a $8 hinge it would be "my box that was supposed to contain a PS5 arrived with a brick in it instead" or similar fraud-smelling situation.
This is why I won't buy consumer electronics from Amazon; they can't deliver a hinge without trying to make a federal case out of it, so if I order a TV with my luck I'm going to get a box of cracked glass with no recourse and long arguments. If I go to best buy the price is worse and the selection is worse and I don't like the experience in general but at least I can slice the box open and see if the glass is smashed before swiping my credit card.
Amazon doesn't have a process for "the delivery people messed up" probably intentionally to save money when the delivery people mess up. Even when the delivery people who messed up are Amazon employees (or contractors)
My point of this ramble, aside from F Amazon, is they set you up to take the blame. They screwed up, the chatbot can't help, the chatbot must be to blame.
I see this opinion come up on HN frequently. You are not the intended target for a chatbot. You're right; for most tech-savvy users, the ideal interaction would be through a well designed UI. However, there's a significant portion of society that is not familiar with UIs and just wants to "speak to a person" and describe the issue they're trying to deal with. Chatbots are the closest thing we have to an automated version of that.
A/B test that: a well thought through UI vs a bot. I genuinely think it would be worse in time required for resolution, choice by the users. Obviously not one size fits all but in simple cases I doubt that even the non tech savvy user would prefer trying to articulate their issue in a way a bot could help them.
Recently came across a delivery service which sent an sms announcing a delivery to connect me to a Facebook messenger bot to ask me the order number to give me the delivery information. How is this process better? Just send a link to the tracker and have all the necessary info in one place. Add a button to reach someone if you must.
That's an example of a bad use of chatbots, but I don't think that means all chatbots are a bad interface. Chat is good when you have a large potential problem space and a wide range in the tech abilities of your users. A 90 year old grandpa isn't going to happily dig through five layers of menus to solve their problem. They are able to describe their problem in a couple of sentences though.
Have you, or whoever recommends that, ever talked to an actual non tech-savy person? I don't mean it like a "me as a non tech-savy person" role-playing, but to a real flesh and bone grandma.
Yes, I'm basing this on experience. A non-savvy user sees most modern UIs as overwhelming, difficult to navigate and they don't have the same priors that the average HN reader does. When they see a chat pop-up, they see a familiar interface (SMS has been around for long enough) and it gives everyone literate the ability to describe in plain language what problem they are having.
My mom tries painfully hard to make Bixby do things for her. She will spend 20 minutes trying to get it to find her a recipe or something instead of using an app. I keep trying to tell her to stop wasting her time and just use google but I guess there is still some novelty in these conversational interfaces (clearly its not easier or faster). If she could use google home to fill out her time sheet or save a document to google drive she would. The thing she finds difficult about tech stuff is remembering the workflow of how to accomplish a task through multiple websites/apps.
Also while I’m on my soap box this is a PSA to all ui devs, stop trying to turn every menu/button text into an icon. My mom does not even remember that the button with 3 horizontal lines means menu she is not going to understand that stamp icon means “add a signature.”
I remember reading an interview with Bill Gates about how "two way mirror" user testing where devs watch REAL users interact with their application is both incredibly important and humbling at the same time.
The above quote is from the days where people bought shrink wrapped software so it was incredibly hard to observe people using your application in the wild. Therefore, companies spent money to do it in house to get better data.
Modern web applications, I would argue, make observing user behaviour in real time trivially easy but I'm sure lots of companies don't even bother. In turn, this brings us to the incredibly complicated interfaces we have today.
My sister came for a visit 3 weeks ago, actually, no, 4 weeks ago, it was the week before Christmas. We were celebrating her 31st birthday and we went out to a great local restaurant. They serve vegan and vegetarian food and we had a great time. I went Christmas shopping and found a lot of great gifts for my son and husband. Yada yada. Now Christmas rolls around and my husband opens the present - it's a Polo Ralph Lauren shirt I knew he would love. To my surprise, and you won't believe this, I got him the wrong size! I need to exchange the shirt for the right size.
Hi I'm Macy, the Macy's chatbot. Thanks for reaching out! You need to exchange an item, is that correct?
It's, of course, made up. But everyone can appreciate the anguish of having to talk to someone like this and trying to discern what it is they actually need. Macy, the Macy's chatbot? They can go through hundreds of terms that correlate with resolutions in milliseconds. if chatMessage.contains?(typoOrExactMatch('exchange')) then chatSession.initiateExchange() end
Like the grandparent explains, some people just need a way to explain what they need in their own words and I don't think chatbots completely fail at that. The best one's I've experienced have prescreened me for common issues and then sent me off to a live person as soon as it was clear my request was not easily addressed with the built in script.
_You_ don't communicate like this, but the reviews & chat logs at my company are proof that people 100% do type like this. A lot of older people are really just looking for someone to talk to it seems, so yes, they will type out 90%+ fluff to just return an item.
Very true. Many older people I've met use Siri or Google as the primary means of device interaction.
For example, my mom has used an iPad daily for many years now. She's never used a laptop or desktop. Rather, she navigates and interacts with an iPad exclusively using Siri and Google voice search--rarely, if ever, touching the keyboard. And she's a (relatively) advanced user.
Granted, to tech savvy users, these interactions are often quite funny. For example, opening the YouTube app, and then doing a voice search for "YouTube videos for how to make chocolate icing YouTube please". :-)
You wouldn't believe some of the things people say to conversational interfaces.
A long time ago – I'm talking early 2000s – I designed a voice UI for a train timetable information system. It was a phone line, you called it up, told it where you were travelling to and from, and it gave you the information you needed.
After launch, I would listen to the odd call at random to see how it was handling things. I distinctly remember one case where, when asked their destination, the user said "I'm going to a funeral".
Now, when somebody said something outside the scope of the expected responses, we would reprompt them up to twice more with a bit of extra guidance. I'm proud to say that, even though this caller started way off course, the system steered them back and they ended up going away with the information they needed.
That's actually a danger of making your conversational interfaces too human-like - people then talk to them like they're humans, and it doesn't go well. You're better to make things a little stiff and robotic, so as to manage their expectations and make them understand that they need to constrain their answers a bit to get what they need. It might feel less natural, but everybody goes home happier at the end of the day.
Ironically I'm 18th in the queue at computershare.com right now (Its a way for even minors to do direct investment in a company other than more typical employee ownership programs or broker acct, very long story about a very old account).
The website has a bug where locked accounts are told to change their password, which will change their password via an email process but then your acct is still locked out of course, but you can click here to change your password. Endless loop. Very annoying.
The idiotic chatbot helpfully suggests the way to solve a locked account is to change your password. That's exactly where the bug is, and resetting your password does not unlock the acct which is the problem.
Helpfully the support website says if you wait a day a locked account will be unlocked. That of course is not true.
I've only been working on this since Sunday.
Anyway a fucked up company that is essentially inoperable, when given a chatbot, will merely have one more thing that doesn't work. Its a lot harder to code a chatbot that can fix a locked account than it is to code a website to unlock a locked account, so if they fail the simpler task you know the harder chatbot task would be impossible for them.
I'm hopeful that within a half hour I'll be able to chat with a human rep and gain access to my account. I've been in line for a human for about ten minutes and I'm down to 15th in the queue.
I used to work in telecommunications and I'm glad I don't have to hold a phone to my ear and listen to on-hold music for hours on end like the bad old days, I have multiple monitors and I can just leave this window up for however many hours it takes while working on other monitors.
I like chatbots when I need to contact the support of a company. The chatbot can do the normal support script, sometimes it works for me, and it makes the conversation much faster if I still need a human because they can read the discussion with the chatbot first and skip the basic questions.
I only use Chatbots to ask for a human because there's usually no other way to do it. They're like the new IE for me, only used to download other browsers.
I maintain a work intranet site. Its an out of the box Django site and I have very little time to design or develop pages. While I've not _yet_ implemented a chatbot, I really want to only for the purpose of translating whatever naive search terms into domain specific tags and terms. Even better if I could say, ME: Hey Bot! What's that thing I have to do at the end of every month that has to do with payroll? BOT: Payroll Accruals? click. haha.
The charm of Eliza is that it was simply a Rogerian therapist who didn't try to be intelligent.
Eliza's talent was in getting you to express yourself, free from inhibition. That doesn't require “intelligence”, but it does require the art of listening. There's nothing dumb about that.
Sure, the overall technique of asking vague open-ended questions to elicit a response might not be dumb. But it's hard to argue that ELIZA-style chatbots are intelligent in anyway. They deliberately had no understanding at all.
Wasn’t there a story about about a Georgia Tech professor who coded a chatbot to act as a GA for his class and nobody realized it wasn’t a real person?
I saw the development of a chatbot based on the IBM Watson stuff. It was just like a phone tree map, except the system tries to guess the option selected based on the intents found in the speech/text. Of course it got no adoption, except when forced on users to drive metrics.
It was enormously expensive, it would have been cheaper to have a human on the other end.
While the mainframe is arguably still a success (after all, they still run an amazing number of core services in our society: banks, air-traffic, governmental stuff, ... [1]), it's hard to find any evidence for a successful Watson project [2].
It probably doesn't count as successful since it didn't make money (I suppose this is why it was retired) but it was great at suggesting recipes from whatever ingredients I had, and ingredient pairings I wouldn't have thought about.
I still haven't found anything that works like it.
Wikipedia seems to consider the IBM 701 as mainframe and from the customers and special numeric format it had I assume it was used to replace entire floors of human calculators that did nothing but compute the same calculation over and over again. It was also rented for a monthly fee, so the cost comparison was probably straight forward.
I wouldn't be surprised if an advanced chat bot with science fiction level AI would replace entire floors of Microsoft call centers at some point in the far future.
In almost every case, it's better to just use a regular dumb dialog tree than to try using any of the whizz-bang NLP stuff. I also have some horror stories about Watson that I'd love to tell if they weren't under disclosure agreements...
All of the advanced features like this that are built into chatbot products are only there because they look fancy when you give the demo. From a customer perspective, you only ever want to get connected to the second-level customer service agent that might possibly be able to tell you to do something you haven't already tried. And from the business side, they want to get as many interactions out of the funnel so that they don't need to be connected, and thus they can run their customer service desks even leaner.
My favorite phone tree implementation is the ones where it takes twice as long to hear all of the options because it has to tell you the keyword(s) and the equivalent number to press (since I guess they don't have enough faith in it otherwise).
I was one of the original developers on this project! It was called the Watson Assistant and was a completely different code base from the original Jeopardy code base. The dialog tree was actually fairly sophisticated in allowing for a lot of flexibility. However, I always had a concern hat developers would find it too complex to author dialogs. Apart from the dialog tree, the other components were a text classifier and an entity detection system. It has been several years since I quit IBM so I have no idea how it has evolved since then.
That makes me wonder how much of "bad chatbot" implementations are just poorly designed dialog trees? It seems like a domain-specific chatbot could be really good, if designed properly, but it's probably as much of an art as it is a science and most implementations probably don't go deep enough.
The idea of completly replacing human beings with chatbots isn't going to succeed. They have their own uses, not very advanced, for example replacing some web interface with chatting in WhatsApp/Telegram, some companies already adopted that and filtering people in case of a call center. But for something more complex that requires actual experience and real life understanding, they should connect you to a real person that can comprehend your messages.
Whether it works or not, sounds dumb or useful. Clients keep asking for it. Personal experience and opinion, they are best as a backup for human agent, when one is busy or unavailable. Costco, Amazon, Ally have good implementations on these. Chatbot discussion maybe in the air, Chat Widget is a must have form of interaction. Customers expect a site to have a "Chat Now" option on the website.
Amazon's is great in most cases. I forgot to cancel my Prime at one point (I was switching to the student price), and it renewed. I opened the chatbot expecting to have to wait for a human, but the bot refunded the charge with nothing more than an "are you sure?" question.
We were building a chatbot to use on a website until we realized how customers where using it. Most people were frustrated with something and needed help.
People who wanted to have a conversation did it for fun and had no real need for our services. We couldn't tell them how tall the Eiffel tower is.
Maybe there is a time where you want to have a conversation like the examples in the article. But I don't ever find myself wanting to talk to a human in this manner, so why a chatbot?
Have you ever watched the sci-fi show The Expanse? Have you seen how they interact with the AI? They ask a question, it provides an answer. It doesn't even use voice most of the time. It gives you the answer without trying to be sassy about it.
> Maybe there is a time where you want to have a conversation like the examples in the article.
I mean, his examples were pretty factual and to the point. I suppose it's unusual to want to know if it's dangerous to walk down stairs backwards with your eyes closed, but there's clearly a short answer. Similarly with asking who the president is.
They aren't actually the same article, this one (by Andrew) refers to the other (by Gary) and quotes the title.. it's a response article. Because of the HN policy it's hard to tell that from the titles though. Original article [0] 663pts, 408 comments.
When GPT3 was opened up so that anyone could create an account, I was excited to try it. I was quickly disappointed. Its ability to chat was quickly shown to be pretty terrible -- it could mostly make reasonable-sounding English sentences, but it was like talking to someone who was maybe a bit drunk and not really listening. I can't imagine using it as an interface for a customer to interact with product support.
The whole thing just made me a bit sad. I really was so excited. Nothing it could do was very impressive, even aside from holding a conversation. The most impressive thing I've seen is Copilot, but even that's been next to useless from a practical perspective.
Is it not unfair to expect GPT-3 to perfectly tackle this issue when it has been trained as a general purpose model? For customer support or other more specific chatbot applications there are better machine learning models.
Because a friend of mine is into Chuck Norris facts, I tried to train GPT3 to give them. Some of the more novel (as far as I can determine) facts it gave:
* A duck's quack does not echo. Chuck Norris is solely responsible for this phenomenon.
* When you open an umbrella in the rain, do not be alarmed if Chuck Norris falls out of the sky and lands on you. The rain drops are simply being pushed away by his roundhouse kick.
* In an emergency, you can use a bucket of water to put the fire out. However, if Chuck Norris is directly responsible for the emergency, use a flamethrower.
* In an airport, there is no "B" gate. There is only "C" gate. The "B" stands for the bus you will take from the plane after Chuck Norris lands on it.
* There are no weapons of mass destruction, Chuck Norris lives inside every element on the periodic table. It's why you see him in your sodium chloride.
I'll let you be the judge as to whether these are funny.
There’s a special little place in hell for whoever decided the whole world needed chat bots for every crappy website.
Why would I want to try to articulate something that could be found in a simple tree? Just give me direct access.
I don’t know where to find it: search!
The issue is not covered in the standard workflow? Get me a real person!
Did anyone implementing ever end-to-end test this for speed and user friendliness? Did they just misinterpret wanting to talk to someone? I want to talk to someone because the process doesn’t cover my case, not because I actually want to have a conversation with the broken process.
Worst part is, after answering 10 questions. Some websites offer a real agent, and they ask all the same questions or it tries to point to an article we already know everything about.
We have a broadband provider in India, that asks 5 questions about internet outage and they says call 121 to talk to someone. Thomas had never seen such BS before.
I worked with some folks that did IVR systems and they were mostly doing their best with the resources and constraints they had to make the thing useful. They were measured by dropped calls, they did not like them.
The weaknesses were mostly just ordinary business stupidity.
The marketing department demands that the first option allow the caller to express interest in buying a product.. nobody ever does that but then it takes up the primest real estate in the system #1 on the first level of the tree.
Of course anyone with a billing enquiry needs to enter the account # so that the collections department has the opportunity to intercept.. but after the arrears lookup nobody in the call centre is willing to pony-up the resources to make the system retain the account number that was typed in so every customer has the then say the damn number after having just typed it in!
At least #1 option is better than having to hear an announcement to press 1 to hear about how Bank of America supports the Military… when you call in for support (yes it is actually happening right now, try it). Whoever would want to hear that when they want assistance with their issue, unless they or their family is in the military?
To be clear, I wasn't putting the blame with IVRs or their designers. I was specifically speaking to the UX that results from businesses programming them and the menu options they set up. Like you say, those are constrained by what the business itself wants, which in turn is constrained by cost management philosophy.
Because the average user doesnt want to look at faq. They want a person to answer their question. Now. Chatbots kind of feel like that and the vast majority of questions can be answered by a FAQ.
Unfortunately there seems to be a tendency to either remove phones altogether, or to man them with the same chat bots (coupled with speech recognition and synthesis, so even worse than text-based ones).
I agree with you, but there's a lot going on here:
- Customer support is expensive. If keeping customers happy and lowering churn is important, you spend a lot of money on it.
- If you can't afford enough staff, the next best option is to put up a few barriers to slow down the incoming requests. Maybe automated means can solve a large percentage of problems.
- Not everyone knows how to search or navigate a tree. Think about the non-tech folks. You have to offer them something different. It's hard to strike a balance.
- Chat bots are being hyped and sold by new tech companies trying to build larger scale solutions. They want to build more, but they need to sell, grow cash flow, etc.
Non tech folks come up quite often: I disagree with that, chatbots often require talking to them in a precise language to find an answer or reply in a strange way if they don’t have an answer, I think that’s even worse for a non tech savvy user.
Also: what level of non tech savvy are we taking about? Test with some real users of different ages, I’m sure there’s something to be found that improves usability to a point that’s better than a bot.
Agree on the cost cutting though, if you don’t care about an individual user go ahead and waste their time. I’ve actually had enough encounters with the human counterpart of a chat bot that was equally unhelpful in resolving simple issues.
Yes, the cost of having real agents is the main reason for chat bots to exist.
The bot is like a buffer: you can deflect simple issues; capture information for the agent; show feedback about the wait time and agent availability.
Badly implemented bot AI is not the only issue. To give a good CX, companies need to invest in giving the agents the right tools. It’s extremely frustrating when the canned responses fail and there isn’t anybody to answer or when you have to repeat all the info (because the agent cannot see your chat with the bot).
One challenge for those promoting robots is that they are unpleasant to deal with and suck at their purpose (from the customer's perspective - they're deflecting traffic, so they're working for the business). Worse, everyone knows they suck, and people with a choice choose not to use them. I know I've dropped one vendor who forced me to. (I'm sorry, life's too short to talk to robots.)
Things like this make the customer very aware of how they are valued - as cattle not pets, to steal a metaphor.
I'd challenge the idea that customer support is _expensive_. Generic customer support requires no specialized expertise: you need knowledge your company's business processes, tooling related to third parties they worked with (redirecting a courier's shipment via the courier or whatever), and general customer service skills, and that's about it.
_Specialized_ customer support, e.g. support for complicated software, is expensive, because it requires technical domain knowledge equivalent to or above many engineering employees, and employees that can perform it effectively could probably move to engineering and increase their salaries twofold because compensation isn't tied to expertise, it's tied to what the market will pay, and you can more easily get away with shit support than shit engineering, but ignore that--it's not the majority of customer support.
Customer support _is_, in the simplest view, a cost center. Customers contact support when something has gone wrong that they can't resolve otherwise. If nothing goes wrong, you don't need it. If you, as support management, can reduce the time it takes to resolve things that go wrong (by any means) and reduce the number of staff needed to handle cases, you win: your department doesn't directly generate revenue, so your goal is to reduce expenses.
Tying lost revenue to customers lost after poor support experiences is difficult and noisy: there are plenty of customers who have poor experiences because they're incompetent and wouldn't succeed with any amount of support, and because of this are more likely to request support, and the high-level categorization used to describe issues often obscures what actually went wrong (a generic "Account > Creation" category isn't going to capture issues like "users can't create an account if their address includes non-ASCII characters", but execs don't see anything more than the category).
These issues (and others) combine to promote an environment where support is more a buffer to getting support with limited ability to fix issues, and a perverse incentive to _not_ fix issues since metrics are focused towards reducing time to resolution (easy to measure) versus quality of resolution (hard to measure!). Poor support is furthermore easy to ignore because you can either often focus more on new business first and foremost (hello, every startup ever!) or rest comfortably on a monopoly position where customers can't drop you even if you provide terrible support (hello, the US ISP market!).
Cost is not strictly an disincentive to providing sufficient, capable, and effective support, but it's difficult to recognize the value it provides and there are more immediate cost savings from reducing support cost than long-term revenue from customer retention by providing good support.
This compounds into even poorer service in more specialized environments because talented people will leave, because working in a cost center sucks. The flipside of support is essentially sales engineering, AKA support while trying to court a customer. The technical skills needed aren't any different, but one brings in revenue in an easily quantifiable way, so it gets more political clout and more organizational investment.
The Amazon chat bot has been pretty great for me. Instant replies and I most times I don’t have to type anything out. It’s really a decision tree with multiple choices but extensive and actually helpful. And if there is a bigger problem it instantly switches to a person who has all the context.
Sounds good - the menu based customer care is pretty good there too, and they tend to actually want to resolve issues unlike many other sites (though that might also depend on the cost benefit analysis) might be the basis many other sites are missing.
> Why would I want to try to articulate something that could be found in a simple tree? Just give me direct access.
Sometimes I feel that way about databases and dynamic websites. I suspect the reasons behind those may be similar: web UIs add branding, maybe make usage a bit more convenient to an average user, and so on. Animations are popular too, probably they are supposed to make the websites to look more fun and modern. I guess that people planning chat bots similarly view them as looking more futuristic and convenient, more advanced than boring old documentation.
I think chat bots exist in our world because people would rather "ask a question on a forum" than search and have direct access to an answer. It drives me crazy too, but it's how we get "does anyone else ever..." memes in this world. People don't want information, they want the experience of asking the question and the discussion. The answer is not the primary force of their effort.
It also seems to be the conversational model in use at parties.
You need a really good team and MlOps/DevOps pipeline to bring world class chatbot support and performance. I think there is still opportunity in this space, but you need Apple Level Design care to make it work.
I don't think they're actually intended to answer questions as much as be a cost effective attempt to instill some sense of agency and audience for the user. TL;DR they fail at this too.
What ever happened to the smarterchild bot? I remember being amazed as a kid that it was a robot on AIM that would reply just like a person. I don't remember it being dumb like modern "AI" chatbots, but it would play coy if it didn't know the answer in a reasonable way. I feel like we've regressed from there.
RIP old buddy. I hope you didn't save our chat logs from that era because man that would be cringey to look at now
Today on HN there was a post about GitHub Copilot chat between user and the AI. I thought it seemed pretty clever with its syntax completion / suggestions.
169 comments
[ 0.23 ms ] story [ 845 ms ] threadDo you start to give your AI a motivation or goal? Perception? These are vastly more complex problems than some statistical tricks on data that is widely available.
Still, it is fascinating that we came this far with a dead machine that talks.
I remember reading an interesting article a while ago about at least in the human case the basic principle of emerging consciousness happens when the prediction system in our brain designed for figuring out what other entities around us do is used on itself, trying to explain what the subconscious is doing. As such the consciousness we experience is a bit of a bug in that system that turned out to be beneficial to some extent. All just a theory of course given how much we actually know about the brain so far, but it's always made the most sense to me.
I'm not sure how that would translate into the current ML environment though.
1 - Book X 2 - Cancel Y 3 - Recieve info on Z
Everyone comments that they work really well and are super convenient.
I think these have more potential than natural language bots.
The public can 100% manage and love terminal interfaces... as long as it looks like a text message.
Now, with my google assistant? Text interfaces are awful, because I have no idea what combination of magic words will confuse it. What will be interpreted as a parameter to which command. Even simple commands get broken if I accidentally add phrasing where it interprets it as a different command... usually just giving up with "I don't know how to do that" and then Google gets a nice recording of elaborate cusswords.
I think speech recognition is good enough for this, but what is left is making a complex tree. See the 20 questions game where because so many people have put in new answers it now can guess almost everything after just a few questions. This is not easy and it takes a lot of effort to figure out where things didn't work and fix it so it will for the next person. At least some of those things you fix will never be asked again, but you need to do it.
Part of the above is speech recognition needs to expose a confidence and if it thinks it knows ask useful questions about what you mean. (they already do in some cases)
This is why I love chat-with-a-human interfaces on websites for support (although they mostly require maintaining browser state) because staying on a phone line absolutely sucks and the person helping you also doesn't have a real-time constraint so they often have the ability to get you better answers to hard questions.
Every "natural language" system, whether it's a text chatbot or Siri-style voice bot, is essentially a command-line interface without very good documentation.
In general, it was considered an advancement over the classic CLI (at least for environments where there's low expected user proficiency) to provide a menu of valid choices and put them front and centre.
Most customer service scenarios are exactly that-- low user proficiency (in company- or business-specific terminology, or sometimes even the very transaction flow itself) so they may not be able to quite enunciate exactly what they mean, but can easily pick from a clearly written menu.
[0]: https://news.ycombinator.com/item?id=29825612
No wonder these things are so dumb still. The training process and the loss function used probably does not penalize poor long-range coherence between paragraphs. Also, if I'm not mistaken, these things have absolutely no internal state besides the characters you steam into them as conversion prompts.
If these things were trained more like agents having to operate in eg. Socratic dialogues maybe we'd be getting somewhere
Training a kid to takes years, hopefully this can be sped up a bit ;)
If you use openAI's gpt API, in the docs they talk about providing a prompt that primes the network to respond in a specific fashion. Like,
"this is a Q&A session between an agent knowledgeable about bash scripting.
Q: how do I check the current working dir? A: use 'pwd'
Q: "
... And then the actual user query is concatenated to that.
Which is a fine way to customize a toy chatbot to sound like edgar allen poe, but no way to maintain state across a long conversation with a customer (the max prompt length is very much finite)
Unless someone smart finds out a fundamentally different approach I guess these transformer networks will never really solve chatbots.
Do any of these generate narrative fictions (such as characters and events they supposedly did) to interact with?
There a few reasons structurally why it can't do what people want it to do, two of them are: (i) it can't detect that it did the wrong thing at one level when interpreting it at a higher level, (ii) most creative tasks have an element of constraint satisfaction.
The 1st one interests me because I was struggling with the need for text analysis systems to do that circa 2005 and looking at the old blackboard systems. I went to a talk by Geoff Hinton just before he became a superstar where he said instead of having a system with up-and-down data flow during inference, build a system with 1 way data flow and train all the layers at once. As we know that strategy has been enormously effective, but text analysis is where it goes to die just as symbolic AI failed completely at visual recognition.
Like the old Eliza program, GPT-3 exploits human psychology. We are always looking to see ourselves mirrored
https://www.nasa.gov/multimedia/imagegallery/image_feature_6...
Awkward people are always worried that we are going to get it 90% right but get shunned for getting the last 10% wrong. GPT-3 exploits "neurotypical privilege" in which it gets it partially correct but people give it credit for the whole. People think it will get to 100% if you just add more connections and training time but because GPT-3 is structurally incorrect adding resources means you converge on an asymptote, say 92% right. It's one of the worst rabbit holes in technology development and one of the hardest ones to get people to look clearly at. (They always think stronger, faster, harder is going to get there...)
It seems to me an effective chatbot will be based around structured interactions, starting out like an interactive voice response system and maybe growing in the direction of
http://inform7.com/
You need a ton of dimensions to be able to help someone: to be raised for years by humans to understand politeness, intertextual meaning, general tones, and then special enthusiasm for a specific domain to learn and enjoy helping on banking. Plus, getting money to spend on other even more interesting things in exchange for helping others motivates you to reach optimal results for your user, even if it means asking quickly other humans or sacrificing something personal for it.
Most humans put in the situation of these robots would just say "sorry I don't even understand the question, can you ask someone else" lol
I've seen a fantastic "chatbot" human equivalent once, at Apple of all place. Philipino guy (I'm in HK), absolutely dedicated, polite, cultured, very empathetic (phili people are usually adorable naturally but this one went above and beyond), went well beyond the minimum, and I feel weird saying that but I left the call with a smile and told colleagues around me "wow Apple, what a pleasant customer support, it's insane". I'll probably never say that of a robot however good they make them at talking so there's always going to be value in putting humans in front of clients.
Speech understanding requires sometimes stopping the other person and asking questions to clarify.
Generated text was not good before this era of GPT-X. It’s so much better and more interesting to work with now. It will probably keep getting even better and more controllable.
I think you could do better with RNNs than most people are doing because of structural problems.
Usually when people run RNNs for text generation they start out with the inner state of the system at 0 and then start flipping the coin to choose individual letters so you are starting from a very constrained region of the latent space and not sampling it very well.
I read a paper where they through out the idea that you ought to add coefficients for the latent state that you train for at the same time you train the network which means the number of coefficients goes up with the number of text samples but they never actually did it and I never found a paper where somebody tried it.
I was working on a project where we were developing models based on abstracts of case studies from pubmed as a stand in for clinical notes (certainly real medical notes are very different but you might say that medical notes should look like the abstract) I had the intuition that, as above, the author (and/or the patient) started out with a latent state (e.g. the patient had a disease before coming in) and that we'd get better results if we did something like the above.
It looked like a big and high risk project to develop that kind of model so I proposed something different around supervised training of a "magic magic marker" that could highlight certain areas and unsupervised multi-tasks such as "put the punctuation back in when it is taken out" but the client was hopeful that word2vec would be helpful.
I am still hopeful that incremental improvements, attacks on structural weaknesses, and appropriate multi-task training ("did the patient die?") would get a lot more out of RNN and CNN models.
For everything I want to do there should be an UI that is much easier to use than explaining it, even to a human.
For help and troubleshooting chatbots are pretty much useless. If I have a problem doing something via the UI then probably the developer did a bad job and no chatbot will ever do better.
Is there any evidence customers prefer chatbots? The entire concept feels like it's driven by managers trying to impress their managers.
A chatbot is just a long way of saying "GO AWAY!".
Large telecoms do this so they get to shut down their customer support almost entirely while claiming that they are available for the customers.
a large telecom in Australia has applied this to an extreme degree in the last two years to the point that you almost can't contact them no matter how severe the issue is. Their message is clear "SHUT UP AND PAY".
And it is gone now and replaced with a much more effective app, my point is just that it clearly possible to make chatbots that aren't rage inducing.
The question is just if it is worth it, something we didn't think after the novelty wore off.
I believe one way it can provide value is when it is closely backed by competent support staff so it takes more of a receptionist role instead of the role of a support engineer.
It's cost cutting. Customer contact is always treated as a cost center.
Therefore, the only relevant question for its management is "How can we provide the minimum acceptable level of service, at the lowest possible cost?"
In general, the answer to "Why does X suck?" is to ask whether the company sees it as a profit or cost center. See: HR.
"How can we provide the minimum acceptable level of service, at the lowest possible cost?"
Not with all products. Some companies understand, that good support keeps the people loyal to their products. But they are usually in a higher price segment.
Had to RMA an ultra widescreen monitor I use for work, and the experience was as smooth as silk and always in the hands of actual humans, who would follow up and push things forward.
I’ve had their enterprise support on the phone with me for a dozen hours over the course of a week trying to get this SAN working. Their support remoted into it and had me manipulating physical stuff when required. They eventually got it going through a combination of hard resetting everything and updating firmware on half a dozen components + some software updates.
Dell doesn’t always have the greatest hardware, but their support really doesn’t rest until you’re satisfied. “No one ever got fired for buying Dell”.
I’m guessing that’s often not true if you’re running a Wordpress blog or some white label e-commerce platform or similar. You can probably just pase some JavaScript into an admin interface somewhere to add a third-party chatbot to your site.
I just don't understand the problem they are trying to solve.
Why is that? Can't they just check the tracking number? That's a question I had until i tested a few dozen e-commerce solutions and most of them have buried their tracking number under several confusing menus. And even when you get the tracking number, carriers each have confusing ways to track packages.
My experience is Amazon is an enormous pain in the butt to deal with if the delivery service drops your package at the wrong address or there's a hole in the package with nothing in it. Huge pain dealing with those people.
I get that there are scammers out there. But if I get 100 packages per year from amazon for years, some of which have been very expensive, and one outta nowhere they lose my $5 multivitamin, I'm not trying to get rich by fraud but I just want my vitamins.
Also a couple years ago I ordered an obscure stainless steel hinge for woodworking, kind of expensive for stainless steel it was about $8 (solid not plated USA made brass would have been $30+) and Amazon van delivery successfully delivered a bubblewrap envelope with a hole ripped in it because hinges are heavy and bubble wrap envelopes are weak. Its an $8 hinge I just want my hinge can't you ask the amazon employee in the van who delivered it to look on the floor of his van? I'm not angry (yet) but I just want my hinge so I can finish my woodworking project. If I was trying to rip Amazon off then I wouldn't be arguing with somechatbot or someone over a $8 hinge it would be "my box that was supposed to contain a PS5 arrived with a brick in it instead" or similar fraud-smelling situation.
This is why I won't buy consumer electronics from Amazon; they can't deliver a hinge without trying to make a federal case out of it, so if I order a TV with my luck I'm going to get a box of cracked glass with no recourse and long arguments. If I go to best buy the price is worse and the selection is worse and I don't like the experience in general but at least I can slice the box open and see if the glass is smashed before swiping my credit card.
Amazon doesn't have a process for "the delivery people messed up" probably intentionally to save money when the delivery people mess up. Even when the delivery people who messed up are Amazon employees (or contractors)
My point of this ramble, aside from F Amazon, is they set you up to take the blame. They screwed up, the chatbot can't help, the chatbot must be to blame.
Recently came across a delivery service which sent an sms announcing a delivery to connect me to a Facebook messenger bot to ask me the order number to give me the delivery information. How is this process better? Just send a link to the tracker and have all the necessary info in one place. Add a button to reach someone if you must.
Also while I’m on my soap box this is a PSA to all ui devs, stop trying to turn every menu/button text into an icon. My mom does not even remember that the button with 3 horizontal lines means menu she is not going to understand that stamp icon means “add a signature.”
The above quote is from the days where people bought shrink wrapped software so it was incredibly hard to observe people using your application in the wild. Therefore, companies spent money to do it in house to get better data.
Modern web applications, I would argue, make observing user behaviour in real time trivially easy but I'm sure lots of companies don't even bother. In turn, this brings us to the incredibly complicated interfaces we have today.
Hi I'm Macy, the Macy's chatbot. Thanks for reaching out! You need to exchange an item, is that correct?
Like the grandparent explains, some people just need a way to explain what they need in their own words and I don't think chatbots completely fail at that. The best one's I've experienced have prescreened me for common issues and then sent me off to a live person as soon as it was clear my request was not easily addressed with the built in script.
For example, my mom has used an iPad daily for many years now. She's never used a laptop or desktop. Rather, she navigates and interacts with an iPad exclusively using Siri and Google voice search--rarely, if ever, touching the keyboard. And she's a (relatively) advanced user.
Granted, to tech savvy users, these interactions are often quite funny. For example, opening the YouTube app, and then doing a voice search for "YouTube videos for how to make chocolate icing YouTube please". :-)
A long time ago – I'm talking early 2000s – I designed a voice UI for a train timetable information system. It was a phone line, you called it up, told it where you were travelling to and from, and it gave you the information you needed.
After launch, I would listen to the odd call at random to see how it was handling things. I distinctly remember one case where, when asked their destination, the user said "I'm going to a funeral".
Now, when somebody said something outside the scope of the expected responses, we would reprompt them up to twice more with a bit of extra guidance. I'm proud to say that, even though this caller started way off course, the system steered them back and they ended up going away with the information they needed.
That's actually a danger of making your conversational interfaces too human-like - people then talk to them like they're humans, and it doesn't go well. You're better to make things a little stiff and robotic, so as to manage their expectations and make them understand that they need to constrain their answers a bit to get what they need. It might feel less natural, but everybody goes home happier at the end of the day.
The website has a bug where locked accounts are told to change their password, which will change their password via an email process but then your acct is still locked out of course, but you can click here to change your password. Endless loop. Very annoying.
The idiotic chatbot helpfully suggests the way to solve a locked account is to change your password. That's exactly where the bug is, and resetting your password does not unlock the acct which is the problem.
Helpfully the support website says if you wait a day a locked account will be unlocked. That of course is not true.
I've only been working on this since Sunday.
Anyway a fucked up company that is essentially inoperable, when given a chatbot, will merely have one more thing that doesn't work. Its a lot harder to code a chatbot that can fix a locked account than it is to code a website to unlock a locked account, so if they fail the simpler task you know the harder chatbot task would be impossible for them.
I'm hopeful that within a half hour I'll be able to chat with a human rep and gain access to my account. I've been in line for a human for about ten minutes and I'm down to 15th in the queue.
I used to work in telecommunications and I'm glad I don't have to hold a phone to my ear and listen to on-hold music for hours on end like the bad old days, I have multiple monitors and I can just leave this window up for however many hours it takes while working on other monitors.
Eliza's talent was in getting you to express yourself, free from inhibition. That doesn't require “intelligence”, but it does require the art of listening. There's nothing dumb about that.
EDIT - Found it: Jill Watson
https://www.businessinsider.com/a-professor-built-an-ai-teac...
It was enormously expensive, it would have been cheaper to have a human on the other end.
This sounds exactly like what someone might have said about early IBM mainframes.
[1] https://www.bmc.com/blogs/state-of-mainframe/
[2] https://www.nytimes.com/2021/07/16/technology/what-happened-...
It probably doesn't count as successful since it didn't make money (I suppose this is why it was retired) but it was great at suggesting recipes from whatever ingredients I had, and ingredient pairings I wouldn't have thought about.
I still haven't found anything that works like it.
I wouldn't be surprised if an advanced chat bot with science fiction level AI would replace entire floors of Microsoft call centers at some point in the far future.
All of the advanced features like this that are built into chatbot products are only there because they look fancy when you give the demo. From a customer perspective, you only ever want to get connected to the second-level customer service agent that might possibly be able to tell you to do something you haven't already tried. And from the business side, they want to get as many interactions out of the funnel so that they don't need to be connected, and thus they can run their customer service desks even leaner.
The main issue was that it was a vanity project for a stakeholder, which they used to pivot into a new job / career as a “thought leader”
People who wanted to have a conversation did it for fun and had no real need for our services. We couldn't tell them how tall the Eiffel tower is.
Maybe there is a time where you want to have a conversation like the examples in the article. But I don't ever find myself wanting to talk to a human in this manner, so why a chatbot?
Have you ever watched the sci-fi show The Expanse? Have you seen how they interact with the AI? They ask a question, it provides an answer. It doesn't even use voice most of the time. It gives you the answer without trying to be sassy about it.
I mean, his examples were pretty factual and to the point. I suppose it's unusual to want to know if it's dangerous to walk down stairs backwards with your eyes closed, but there's clearly a short answer. Similarly with asking who the president is.
[0]: https://news.ycombinator.com/item?id=29825612
The whole thing just made me a bit sad. I really was so excited. Nothing it could do was very impressive, even aside from holding a conversation. The most impressive thing I've seen is Copilot, but even that's been next to useless from a practical perspective.
* A duck's quack does not echo. Chuck Norris is solely responsible for this phenomenon.
* When you open an umbrella in the rain, do not be alarmed if Chuck Norris falls out of the sky and lands on you. The rain drops are simply being pushed away by his roundhouse kick.
* In an emergency, you can use a bucket of water to put the fire out. However, if Chuck Norris is directly responsible for the emergency, use a flamethrower.
* In an airport, there is no "B" gate. There is only "C" gate. The "B" stands for the bus you will take from the plane after Chuck Norris lands on it.
* There are no weapons of mass destruction, Chuck Norris lives inside every element on the periodic table. It's why you see him in your sodium chloride.
I'll let you be the judge as to whether these are funny.
Why would I want to try to articulate something that could be found in a simple tree? Just give me direct access.
I don’t know where to find it: search! The issue is not covered in the standard workflow? Get me a real person!
Did anyone implementing ever end-to-end test this for speed and user friendliness? Did they just misinterpret wanting to talk to someone? I want to talk to someone because the process doesn’t cover my case, not because I actually want to have a conversation with the broken process.
To take the edge off customer complaints elsewhere?
The weaknesses were mostly just ordinary business stupidity.
The marketing department demands that the first option allow the caller to express interest in buying a product.. nobody ever does that but then it takes up the primest real estate in the system #1 on the first level of the tree.
Of course anyone with a billing enquiry needs to enter the account # so that the collections department has the opportunity to intercept.. but after the arrears lookup nobody in the call centre is willing to pony-up the resources to make the system retain the account number that was typed in so every customer has the then say the damn number after having just typed it in!
- Customer support is expensive. If keeping customers happy and lowering churn is important, you spend a lot of money on it.
- If you can't afford enough staff, the next best option is to put up a few barriers to slow down the incoming requests. Maybe automated means can solve a large percentage of problems.
- Not everyone knows how to search or navigate a tree. Think about the non-tech folks. You have to offer them something different. It's hard to strike a balance.
- Chat bots are being hyped and sold by new tech companies trying to build larger scale solutions. They want to build more, but they need to sell, grow cash flow, etc.
Etc.
Also: what level of non tech savvy are we taking about? Test with some real users of different ages, I’m sure there’s something to be found that improves usability to a point that’s better than a bot.
Agree on the cost cutting though, if you don’t care about an individual user go ahead and waste their time. I’ve actually had enough encounters with the human counterpart of a chat bot that was equally unhelpful in resolving simple issues.
The bot is like a buffer: you can deflect simple issues; capture information for the agent; show feedback about the wait time and agent availability.
Badly implemented bot AI is not the only issue. To give a good CX, companies need to invest in giving the agents the right tools. It’s extremely frustrating when the canned responses fail and there isn’t anybody to answer or when you have to repeat all the info (because the agent cannot see your chat with the bot).
Yep, and that's a key thing to get right.
One challenge for those promoting robots is that they are unpleasant to deal with and suck at their purpose (from the customer's perspective - they're deflecting traffic, so they're working for the business). Worse, everyone knows they suck, and people with a choice choose not to use them. I know I've dropped one vendor who forced me to. (I'm sorry, life's too short to talk to robots.)
Things like this make the customer very aware of how they are valued - as cattle not pets, to steal a metaphor.
_Specialized_ customer support, e.g. support for complicated software, is expensive, because it requires technical domain knowledge equivalent to or above many engineering employees, and employees that can perform it effectively could probably move to engineering and increase their salaries twofold because compensation isn't tied to expertise, it's tied to what the market will pay, and you can more easily get away with shit support than shit engineering, but ignore that--it's not the majority of customer support.
Customer support _is_, in the simplest view, a cost center. Customers contact support when something has gone wrong that they can't resolve otherwise. If nothing goes wrong, you don't need it. If you, as support management, can reduce the time it takes to resolve things that go wrong (by any means) and reduce the number of staff needed to handle cases, you win: your department doesn't directly generate revenue, so your goal is to reduce expenses.
Tying lost revenue to customers lost after poor support experiences is difficult and noisy: there are plenty of customers who have poor experiences because they're incompetent and wouldn't succeed with any amount of support, and because of this are more likely to request support, and the high-level categorization used to describe issues often obscures what actually went wrong (a generic "Account > Creation" category isn't going to capture issues like "users can't create an account if their address includes non-ASCII characters", but execs don't see anything more than the category).
These issues (and others) combine to promote an environment where support is more a buffer to getting support with limited ability to fix issues, and a perverse incentive to _not_ fix issues since metrics are focused towards reducing time to resolution (easy to measure) versus quality of resolution (hard to measure!). Poor support is furthermore easy to ignore because you can either often focus more on new business first and foremost (hello, every startup ever!) or rest comfortably on a monopoly position where customers can't drop you even if you provide terrible support (hello, the US ISP market!).
Cost is not strictly an disincentive to providing sufficient, capable, and effective support, but it's difficult to recognize the value it provides and there are more immediate cost savings from reducing support cost than long-term revenue from customer retention by providing good support.
This compounds into even poorer service in more specialized environments because talented people will leave, because working in a cost center sucks. The flipside of support is essentially sales engineering, AKA support while trying to court a customer. The technical skills needed aren't any different, but one brings in revenue in an easily quantifiable way, so it gets more political clout and more organizational investment.
Sometimes I feel that way about databases and dynamic websites. I suspect the reasons behind those may be similar: web UIs add branding, maybe make usage a bit more convenient to an average user, and so on. Animations are popular too, probably they are supposed to make the websites to look more fun and modern. I guess that people planning chat bots similarly view them as looking more futuristic and convenient, more advanced than boring old documentation.
It also seems to be the conversational model in use at parties.
RIP old buddy. I hope you didn't save our chat logs from that era because man that would be cringey to look at now