Ask HN: Is 2023 the year chat.company.com starts replacing www.company.com?
Given the step function improvement showcased by ChatGPT, it seems we’ve reached an entirely new realm of possibilities. Will all well funded companies, gurus, etc., now start assessing LLM based chat interfaces as a primary customer entry point to their organization/knowledgebase/methodsbase/etc.?
What would the supply chain look like to enable this?
Who apart from the core tech providers like OpenAI and ML experts are positioned to play key roles here?
24 comments
[ 2.6 ms ] story [ 47.5 ms ] threadDunno but the AWS cost for your knowledge base would be comical.
Everyone with enough VRAM and data.
I’m trying to get a read on how things will play out over the next year or two. I’m bracing for big change and trying to get my head around the enabling and gating factors.
If you can point me to any articles/resources that would help with that effort, I’d appreciate it.
1) chatbot are either based on a search engine (so with limited answers) or AI-based (with sometimes totally inaccurate "understanding" or "answers")
2) knowledgebases (and other kindof FAQ) are still really an open problem: wikis are a way to structure informal knowledge but need A LOT of discipline and human... FAQ are generally limited to the 10 most questions
In the end, the best way to provide knowledges right now is the old "website search engine"... but nothing more. And AI has already shown that - even if it LOOK good at first sight - it's not GOOD ENOUGH to be used (wrong answer looking "right" for example)
Moreover: a website embed a structure, so you know what you can search or how to formulate it even if you dont know what you are searching exactly... a chatbot requires you to ask the right question to get the answer you seek, and that's far more challenging for the user !
As an example: try to ask a tax chatbot about how to get family support... and compare it with looking at the website structure (that clearly show that you're not at the right place)
Right now, for what I read, ChatGPT is able to "synthetize" answers that are just false ! And even if it's right 95% of time, you just can't afford to have 5% answer being false... Just imagine what would be the consequences if ChatGPT "decides" to discount your product or say that a non-existing functionality is implemented...
ChatGPT might be some kind of answer IF it's the beginning of a pipeline that is followed by some kind of "reality check". But right now, I wouldn't trust this technology to replace some "certified and validated" information on a website representing a product or a company
One question is how far off we are from a model that is reliable enough to autonomously handle hard business transactions. But an entirely different question is how much better could many user experiences be that don’t involve an automated culmination of a hard transaction.
I think of you differentiate this way between transactions with a hard impact (ie charging a customer or shipping a product) vs. informational transactions, you can allow for a lot more to happen on a much faster timeline.
As long as the use cases are considered properly.
Consider I hit a pet emergency website that is powered by ChatGPT. I ask it what to do because my dog has just had an encounter with a specific poisonous animal. It gives me advice based on a similar, but slightly different situation and my dog dies.
Alternatively say a telecommunications vendor (or for that matter, many SaaS vendors) implement it instead of detailed product docs. It gives me wrong advice and I misconfigure a router. As a result, an emergency call is mis-routed and someone dies.
AI chat isn't even remotely ready for these scenarios yet and there are still plenty of critical infrastructure services that rely on things like email. Therefore even your basic webmail SaaS isn't a suitable target for this type of tech.
Where could you genuinely deploy it safely?
One can debate just how much of the market falls into the safer range of the spectrum, but the key question may be how far we are in time from much more reliable performance and not how far the current performance falls below some reliability cutoff.
For the sake of this discussion, I think how far we are in time is the more compelling question. And, I don’t know that we are nearly so “remote” in this sense as suggested by the phrase “isn’t even remotely ready.”
Unless there is specific reason to believe otherwise, I’d expect such improvements could appear in very short order given the recent pace of model improvements in other ways.
In 2023, the new chatbot might be allowed to do something the expert-system chatbot was already doing, but then why was the expert system doing it badly and employed? There's management inertia for anything else.
There's the assumption the chatbot won't provide outlandish results occasionally - that matters because of the asymmetry of prospect theory, that bad results can be really bad, adding to management inertia.
Others have commented on raw cost.
I acknowledge ChatGPT grade models are very expensive to operate and that there are still real performance limitations/issues that make it not yet appropriate for autonomous handling of hard transactions. Yet, we still have a new level of capability that is seriously better than anything we've had before.
I'm thinking about the acrobatics I've gone through recently trying to get certain answers from airlines, mail carriers and insurance companies just to even locate the right portion of the site or the right person to speak with. It truly seems ChatGPT would have well outperformed the www and phone channels I went through as an entry point for my journey.
So, given the new capabilities we're seeing with large language models:
1. Will WELL FUNDED companies START ASSESSING chat interfaces as a primary customer entry point - not to autonomously handle hard transactions, but to at least start the customer journey down an efficient path?
2. If so, how will this play out in terms of the evolving supply chain?
And, I would add:
3. If the answer to the first question is "no", what are the key improvements we need to see before that does happen and over what timeline should we expect to see those improvements?
It seems pretty underwhelming as far as ChatGPT applications go.
Take Amazon shopping. Right now I go to their homepage, search, browse, add to basket, check out. Occasionally I look through my order history to start a return; and very rarely I contact support if there's some problem like a refund not being processed.
What does chat.amazon.com look like under your vision? What's different?
1. For the use case where you are searching for a specific product, you could much more efficiently get from the search box to the product you want. Think about all of the text entry and clicks you would go through now to find exactly the right product that meets all of your needs including function, size, price, color, review sentiment, compatibility, etc. And compare that effort to if you had an executive assistant to whom you could quickly rattle off your needs and preferences and then maybe answer a clarifying question or two. chat.amazon.com would be like that executive assistant.
2. For all the other use cases (i.e. returns, shipping status, subscription preferences, etc.), you don't have to fiddle through the menu options to finally land at the point you wanted. You just say, "I want to change my default shipping address from ABC to XYZ", and you land a page with the new address parsed into all the correct fields, ready for you to confirm the change.
Now, I will grant you that there is some ground to travel in ML development before this particular set of functionality could be implemented, but you chose Amazon and I wanted to be directly responsive. The point is that ML technology after many years finally seems close - very close - to all this being real. It's not hard to imagine this is where things are heading, and so will 2023 be the year CEOs and CTOs of well funded companies start moving in this direction.
This sounds like a expensive solution in search of a problem.
1. The notion that chat.company.com becomes the single entry point to a company that can effectively launch your journey more efficiently than www or any given phone number or email address.
2. The idea that ML is only now at (or is fast approaching) the point where this could be done successfully.
It's not hard to imagine ChatGPT giving users dangerous advice like microwaving their phone and constructing an entirely convincing (yet utterly incorrect) explanation supporting it.
Chat interfaces might be useful for some purposes, where the user largely already knows what they’re asking for, but they would be terrible for any kind of structured information.
Just because we’ve invented a shiny new tool doesn’t mean we need to use it for everything.
The point is that having a layer of intelligence between consumers and all of the information out there could be super important and helpful. That layer doesn't exist in the form of human experts because that would be prohibitively expensive. And, the only reason that layer doesn't broadly exist now in the form of AI is because AI has not yet developed to the point where it can serve that function effectively.
What we're seeing now with ChatGPT is that we are much closer to having that layer in the form of AI. It arguably seems within striking distance for the first time.