What exactly is the bottleneck here? What is it about ChatGPT that makes it better than Google's stuff? Is it more compute power? Better algorithms used in the models? More talent? Better work culture? Some proprietary technical innovations? Quality of data? Quantity of data? Some combination of all of these?
I think for Google it may be profits. If you're already a near monopoly that's captured most of the market using a (relatively) cheap means of data processing, an extremely expensive competitor is terrible news. You're in a position where the winning move where you keep your marketshare means far lower profits, at least for some amount of time.
The same would have hypothetically been true for plenty of sites like Stack Overflow.. Which is an example that sites themselves penalize anonymous internal search over fresh Google search.
Controlling the search pages is of low value compared to staying fixed at the top of Google searches (thanks to obfuscated interaction rankings) by having a lot of people who search dozens of times a day always selecting only your page in Google's responses and of course never applying captcha to a new visitor from Google.
Google search gets worse and worse every year. It's actually becoming painful to search for things now. It gives me nothing but spammy low quality results unless I restrict resuts to reddit and HN and other old school forums where people actually talk about stuff and exchange information.
Google is definitely becoming the source of last resort for a lot of queries. I also turn to it just to search other websites.
The biggest search tragedy is Twitter. There’s an incredible amount of information on that platform but most of it is hard to find because of the awful search.
The problem with Twitter is everything has much greater semantic similarity due to brevity. How the hell do you SEO a tweet? LLM might actually come in handy for that kind of backend search usecase.
IMO the real issue with twitter is how it only loads tweets on a page to your monitor size. Impossible to CTRL+F if you know what you're looking for even on the right author's page.
Google has something to loose now: their reputation. OpenAI can be maverick, any mistakes are "move fast and break things".
Same reason why Mercedes or Audi did not bring automated driving on the road, even though they were technologically more advanced than Tesla. History proved them right.
Just from reading ycombinator on this topic I would say ChatGPT went all in on productizing - so like they hired developers to solve tasks for it to get training data etc. Maybe Google treated it as a research project and didn't pour billions in to productizing it sooner because like you said - they had the same algorithms/hardware/developer/researcher caliber and didn't view it as a threat.
And then when ChatGPT launched they got caught with their pants down and rushed to save face.
For years now Google has been shoving the word "AI" into its presentations. Just go through the past GoogleIO and see how many times they mention "AI". All along they've painted a picture of having some grand plan, that they've got something cooking internally that they're perfecting. Maybe some investors even believed it, after all if they're publishing cutting edge papers surely they must have a product backing this as well. And now Microsoft has called their bluff.
And the irony is that if they had simply worked on perfecting standard keyword based information retrieval, the thirst for a LLM integrated search engine would not be as high.
Reminds me of their experience with the iPhone. They were sitting on android until the iPhone launched to mass appeal and left Google scrambling to catch up.
Change in the status quo. Even if there AI is better than ChatGPT, it might monetize as well search queries. When you are a monopoly, any change means possible losses.
Execs: We need to implement ChatGPT functionality into Google search to stay relevant and protect our moat!
Engineers: Okay, we have LaMDA which is a more powerful language model, we'll implement it.
Execs: Great, what are the risks?
Engineers: Well each search is going to cost 5x more compute power, and may reduce the amount of clicks to ads and search results since we're answering all their questions directly.
Execs: Hold on... we need to have a bunch of meetings about why not to do this.
Google went from searching when you click the search button, to searching as you type, which increased compute spend per search, and they kind of just did it. I don't think it's cost of compute per search that's holding things back.
The bottleneck is clearly Google's hesitancy. ChatGPT is better because I can use it right now. I don't care how good the research demos are if I can't interact with them myself. Google has a track record of discontinuing products and having weird user/developer unfriendly policies.
One of the most recent was Chrome's plan to force manifest v3. But many of the features promised to devs from v3 wouldn't have been ready by the time v3 was supposed to release. I would approach Google's AI stuff with similar hesitancy as I would have approached Stadia within the last few months before it shut down. Just like with Stadia, Google very clearly does not care to prioritize this kind of AI tech. Google failed to proofread its own ad about Bard. I can't understand how rushed they would have to be for something like that to happen.
In my opinion, this is why new ChatGPT users are going to stay with ChatGPT, even as the aggressive filtering makes the output slightly worse as time goes on.
First, there is the issue of paid ads. An example is if I look up "How to learn C++", the top link is a paid ad for CodeAcademy, which claims to be free, but then has three tiers with only the basic one being free. I started looking for information on C++ and ended up in a sales room being given a payment plan. Contrast this to me typing "How to learn C++" into ChatGPT, which informs me about "variables, data types, control structures, functions, and arrays", which are a lot of nice keywords that I could jump further into.
Second, Google does not give you access to the information itself. As such, the style the information is stored in on a website is inconsistent, with each website displaying information in drastically different ways. For an example, I Googled "what is a pointer in c++".
The top link starts with "You learned from the previous chapter, that we can get the memory address of a variable by using the & operator", leads to a single example, and is about a single screen long. This would be useless to someone who has no idea what a pointer is and just heard the name of the concept.
Meanwhile the second top link starts with "In earlier chapters, variables have been explained as locations in the computer's memory which can be accessed by their identifier" and is a much more in-depth tutorial spanning over a dozen screens worth of information. This is better as it gives a lot more information, but is still assuming that I got there from following their tutorial, but instead Google just jumped me into the middle of it.
However, ChatGPT homogenizes this, so the data is already organized in an easily accessible format that does not assume you have much previous knowledge. When I tried ChatGPT with this question, it started with "A pointer is a variable in C++ that stores the memory address of another variable." That is far better than either of the websites that I tried with Google that assumed you were halfway through their personal tutorials.
Yeah, I'm going to write a one page textbook on how to learn C++. On that one page it just says "variables, data types, control structures, functions, and arrays". It's gonna be brilliant.
Google sounds like a Kodak or a Xerox, so enamored of their cash cow they lose the incentive to innovate. I'm not saying they will end up like those companies, though, as they're in an industry where catching up won't be as difficult. But they definitely got caught napping.
Except they do innovate. They created this amazing form of AI. Their problem is they don't innovate on ways to monetize their technology, instead they give it away. Satya just laid off a bunch of the Edge team. What's the point in paying for a browser team when Google will fund it for you?
It's funny that there's literally no indication that Google is losing any material market share but somehow Bing with a chat bot (that just launched this week) will, for whatever reason, dominate the market (it's almost as if Microsoft has a publicist or something). A part of me hopes people will take this seriously enough to force Google stock to sell at a discount. The expectations of consumer behavior as it relates to search and chatbots are wildly optimistic given that using a thing isn't as relevant as making money from a person using a thing. So far, chatbots don't lead to more sales or more ad revenue and assumptions of "potential" behavior aren't as speculative as they are unreasonably hopeful.
A crisis of confidence is just as dangerous as a crisis of results. If users believe Google is losing users they may try other services. Internally it could cause them to make misteps in order to maintain its position that lose marker position, or causes a bleeding of staff. Also stock prices are set based on feelings as much as anything else, and it does appear there is at least some risk that G won't be making as much money in the future.
I believe they have many publicists writing stories. I have google alerts on a somewhat niche technology and there are like 5 articles every week from small news sources writing about that particular technology. A quick google search by a decision maker will see all these articles and think, wow this is popular - so it must be good. I'm surprised more companies are not doing this. With the help of "AI" you could generate a bunch of "news" articles and they will get picked up by Google SERP. Even if noone reads the articles it will still make said technology seem popular.
If a search competitor comes about and gives better results, won't it lose market share?
I think a lot of people realize google search results aren't great anymore. There's pages of dumb things at the top because the system is optimized for profit. Hence, almost anyone on HN adding "reddit" as a keyword to search results. Or news editorials explaining why searching for a recipe for eggplant parm will have an enormous website with a story about grandma and food photography shots before the 5 step recipe at the very very bottom.
What more indication do you want? A peer reviewed research paper?
Anyone who used ChatGPT frequently enough finds it difficult to get back to using Google. One specific query is all that takes to get an answer from ChatGPT which is on point. The alternative is to rummage through the top few links from Google and trying to construct the information you need after trying different variations of search phrases.
When the iPhone first came out there were arm chair pundits who predicted that no one would buy or use an expensive phone. But the people who actually used it knew that it was the wave of the future even back in the day. Same thing with ChatGPT.
I think it depends a lot on what your regular searches are. My recent searches were: a local restaurant, find which streaming provider has a show I heard about and it's ratings, look up website for a business, understand some concept, find a product and compare prices, search flights, search for an airport related service.
Except for one query, I don't think LLM could have helped me with the rest. The rest are monetizable queries, btw.
To give Google some credit, I think people would have been equally upset if they did release an AI that ended up being used maliciously or without proper filters. Google has a habit of shipping things while they are still in beta. Gmail kept the "beta" label after launch for five years. Not releasing unfinished AI products was a calculated move and I think it was an overall good decision, but also an expensive one. I am just worried about the longevity of it, since Google clearly doesn't care to prioritize it. The Bard ad mistake implies a deeper issue about resources being allocated incorrectly.
OpenAI clearly spent a lot time red teaming the chatbot to make sure the outputs were sanitized. Chatbot as a service is an easy monetization strategy from big companies. I wonder if ChatGPT servers will share resources with Bing search servers, I can't imagine they will, but if they do, the paid subscription will be a lot more valuable. I think users will react extremely negatively if Bing AI results embed affiliate links on every response possible. On the other hand, I can see myself clicking links of products ChatGPT recommends if I am intetionally asking about some sort of product, like if I ask it types/differences of climbing carabiners.
Research is very collaborative, so I understand why those at Google stay there. But it seems like MS and OpenAI are prioritizing having a clear pathway for novel AI work to reach users, and it won't be limited only to implementations that will help serve ads better.
So, it took the first picture of a (particular) exoplanet, which is not to be confused with the first picture (ever) of an exoplanet. There is a misunderstanding here.
I feel like there's some bigger mistake I missed regarding that controversy. People are aware of chatbots making things up. Microsoft had that poor AI Tay where users' adversarial prompts effectively courrupted its working dataset and it became racist. I know AI doesn't have feelings, and yet I still feel some sadness for how much it was attacked. Maybe because I liked the idea and wanted it to be successful.
Regardless of the validity, I think it's still fair to call it a mistake because of the ambiguity. Most people would misunderstand the answer to mean something it doesn't. But there was such a strong reaction in media about it, when I would have assumed most people wouldn't care. Google never promised an AI chatbot never got anything wrong, and no one I knew expected differently. It makes me think about how many other news stories, for things out of my field, are like the Bard situation. I'll just never know because I won't gain expertise in it.
After ChatGPT's training cutoff of 2021. With the information the AI has in its dataset, it is incorrect. It's unaware of things that happened last month, and "put out wrong answers and wait to see if they come true" is not a business case OpenAI should rely on.
> So, it took the first picture of a (particular) exoplanet
c.f.
> Astronomers discovered the planet in 2017 using the SPHERE instrument on the European Southern Observatory’s Very Large Telescope in Chile and took images of it using short infrared wavelengths of light. Webb’s view, at longer infrared wavelengths, reveals new details that ground-based telescopes would not be able to detect because of the intrinsic infrared glow of Earth’s atmosphere.
The JWST news is that JWST took the first photo it has ever taken of an exoplanet.
I think this highlights a big limitation of LLMs. It doesn't "understand" the text. It's just piecing words together based on the probability. In this case, the key word was "its", which it ignored - but makes a huge difference to the meaning of the sentence.
> See, in 2017, Google researchers published the article “Attention is all you need,” introducing the concept of the transformer and vastly improving the capabilities of machine learning models.
…
You may well ask, why did Google give this wonderful thing away freely? While big private research outfits have been criticized in the past for withholding their work, the trend over the last few years has been toward publishing.
I think that whether or not they had published this article, the transformer architecture would have been established within a year or two anyway. If you read the paper, they cite plenty of references to work already using self attention. As attention itself has no trainable parameters, it’s a short jump to add an mlp to create the “memory bank” it needs.
Likely they knew this, and thought it better to publish and receive the credit for their work than to let someone else claim it.
Google has become the new case study for The Innovators Dilemma. Their main product is effectively unchanged for 15+ years and anything not related to it usually dies.
If you think Search and Ads haven't changed, that's only because Google has done an amazing job at preserving a simple usable interface. It is different in many huge ways, and 100x as complex as it was. Google switched from manually tweaked parameters to ML for search ranking. Advertising has layers upon layers of optimization and auction.
53 comments
[ 2.8 ms ] story [ 146 ms ] threadI've seen many more people appending reddit to Google searches.
google should be more worried about reddit getting their search together
Controlling the search pages is of low value compared to staying fixed at the top of Google searches (thanks to obfuscated interaction rankings) by having a lot of people who search dozens of times a day always selecting only your page in Google's responses and of course never applying captcha to a new visitor from Google.
The biggest search tragedy is Twitter. There’s an incredible amount of information on that platform but most of it is hard to find because of the awful search.
IMO the real issue with twitter is how it only loads tweets on a page to your monitor size. Impossible to CTRL+F if you know what you're looking for even on the right author's page.
Same reason why Mercedes or Audi did not bring automated driving on the road, even though they were technologically more advanced than Tesla. History proved them right.
And then when ChatGPT launched they got caught with their pants down and rushed to save face.
And the irony is that if they had simply worked on perfecting standard keyword based information retrieval, the thirst for a LLM integrated search engine would not be as high.
Engineers: Okay, we have LaMDA which is a more powerful language model, we'll implement it.
Execs: Great, what are the risks?
Engineers: Well each search is going to cost 5x more compute power, and may reduce the amount of clicks to ads and search results since we're answering all their questions directly.
Execs: Hold on... we need to have a bunch of meetings about why not to do this.
That's the literal bottom line. How is Google going to insert ads into an AI system. There's probably people being paid a fortune to try to find out.
One of the most recent was Chrome's plan to force manifest v3. But many of the features promised to devs from v3 wouldn't have been ready by the time v3 was supposed to release. I would approach Google's AI stuff with similar hesitancy as I would have approached Stadia within the last few months before it shut down. Just like with Stadia, Google very clearly does not care to prioritize this kind of AI tech. Google failed to proofread its own ad about Bard. I can't understand how rushed they would have to be for something like that to happen.
In my opinion, this is why new ChatGPT users are going to stay with ChatGPT, even as the aggressive filtering makes the output slightly worse as time goes on.
First, there is the issue of paid ads. An example is if I look up "How to learn C++", the top link is a paid ad for CodeAcademy, which claims to be free, but then has three tiers with only the basic one being free. I started looking for information on C++ and ended up in a sales room being given a payment plan. Contrast this to me typing "How to learn C++" into ChatGPT, which informs me about "variables, data types, control structures, functions, and arrays", which are a lot of nice keywords that I could jump further into.
Second, Google does not give you access to the information itself. As such, the style the information is stored in on a website is inconsistent, with each website displaying information in drastically different ways. For an example, I Googled "what is a pointer in c++".
The top link starts with "You learned from the previous chapter, that we can get the memory address of a variable by using the & operator", leads to a single example, and is about a single screen long. This would be useless to someone who has no idea what a pointer is and just heard the name of the concept.
Meanwhile the second top link starts with "In earlier chapters, variables have been explained as locations in the computer's memory which can be accessed by their identifier" and is a much more in-depth tutorial spanning over a dozen screens worth of information. This is better as it gives a lot more information, but is still assuming that I got there from following their tutorial, but instead Google just jumped me into the middle of it.
However, ChatGPT homogenizes this, so the data is already organized in an easily accessible format that does not assume you have much previous knowledge. When I tried ChatGPT with this question, it started with "A pointer is a variable in C++ that stores the memory address of another variable." That is far better than either of the websites that I tried with Google that assumed you were halfway through their personal tutorials.
I think a lot of people realize google search results aren't great anymore. There's pages of dumb things at the top because the system is optimized for profit. Hence, almost anyone on HN adding "reddit" as a keyword to search results. Or news editorials explaining why searching for a recipe for eggplant parm will have an enormous website with a story about grandma and food photography shots before the 5 step recipe at the very very bottom.
And maybe there's some wishful thinking too.
Anyone who used ChatGPT frequently enough finds it difficult to get back to using Google. One specific query is all that takes to get an answer from ChatGPT which is on point. The alternative is to rummage through the top few links from Google and trying to construct the information you need after trying different variations of search phrases.
When the iPhone first came out there were arm chair pundits who predicted that no one would buy or use an expensive phone. But the people who actually used it knew that it was the wave of the future even back in the day. Same thing with ChatGPT.
Except for one query, I don't think LLM could have helped me with the rest. The rest are monetizable queries, btw.
OpenAI clearly spent a lot time red teaming the chatbot to make sure the outputs were sanitized. Chatbot as a service is an easy monetization strategy from big companies. I wonder if ChatGPT servers will share resources with Bing search servers, I can't imagine they will, but if they do, the paid subscription will be a lot more valuable. I think users will react extremely negatively if Bing AI results embed affiliate links on every response possible. On the other hand, I can see myself clicking links of products ChatGPT recommends if I am intetionally asking about some sort of product, like if I ask it types/differences of climbing carabiners.
Research is very collaborative, so I understand why those at Google stay there. But it seems like MS and OpenAI are prioritizing having a clear pathway for novel AI work to reach users, and it won't be limited only to implementations that will help serve ads better.
So, it took the first picture of a (particular) exoplanet, which is not to be confused with the first picture (ever) of an exoplanet. There is a misunderstanding here.
Regardless of the validity, I think it's still fair to call it a mistake because of the ambiguity. Most people would misunderstand the answer to mean something it doesn't. But there was such a strong reaction in media about it, when I would have assumed most people wouldn't care. Google never promised an AI chatbot never got anything wrong, and no one I knew expected differently. It makes me think about how many other news stories, for things out of my field, are like the Bard situation. I'll just never know because I won't gain expertise in it.
No interpretation of Bard's output here is accurate. It is wrong.
c.f.
> Astronomers discovered the planet in 2017 using the SPHERE instrument on the European Southern Observatory’s Very Large Telescope in Chile and took images of it using short infrared wavelengths of light. Webb’s view, at longer infrared wavelengths, reveals new details that ground-based telescopes would not be able to detect because of the intrinsic infrared glow of Earth’s atmosphere.
The JWST news is that JWST took the first photo it has ever taken of an exoplanet.
JWST took the first picture of LHS 475 B, confirming it exists.
https://www.esa.int/Science_Exploration/Space_Science/Webb/W...
Sure, JWST has also taken the first picture of a different exoplanet-- 2 months later. But JWST did not:
* Take the first picture of an exoplanet
* Take the first picture of the exoplanet in the cited source (instead, it took its first picture of any exoplanet).
I think that whether or not they had published this article, the transformer architecture would have been established within a year or two anyway. If you read the paper, they cite plenty of references to work already using self attention. As attention itself has no trainable parameters, it’s a short jump to add an mlp to create the “memory bank” it needs.
Likely they knew this, and thought it better to publish and receive the credit for their work than to let someone else claim it.