Google was certainly completely caught of guard. Then they said there version was more powerful and released it and it was certainly not. Then they said they had a more powerful version coming out soon that would be much more powerful, and they released Bard which was far behind ChatGPT. Now they are saying Gemini is going to be as powerful but now until early 2024 and of course by then presumably will be behind what ChatGPT is so they are also now touting there is Claude coming someday that will be even more powerful, which seems as unlikely as their past attempts. ChatGPT was able to gather so much data without restrictions and now companies late to the party will have many more restrictions and their tools will be much more limited presumably. Any secret sauce Google can develop ChatGPT presumably will be able too. Apple’s may be the most limited as they may want to be the most cautious and privacy orientated but Siri can’t get any worse so excited for any improvements.
> Google was certainly completely caught of guard.
They weren't.
Their AI research team is completely dysfunctional and unfocused. They're only interested in doing research and are entirely incapable of the engineering and product focus necessary to build something like ChatGPT.
That isn't the same thing as being caught off guard.
The execs were. They've been paying millions for a top-tier AI devteam hoping for something like ChatGPT, just to find out that their devteam made and freely released the transformers research that ChatGPT is based on (derp)
As always, Google has incredible ideas and promise but lacks business execution
So they were caught of guard, maybe not from a research pov but certainly from a product pov. They are behind and are just playing catch-up for over a year now that it's safe to say that they don't have any competitive advantage over microsoft/openai anymore
It's not caught off guard. Google simply can't do products because of how career growth works.
You don't get promoted for polishing things other than ad revenue. It's all about launching things built in a hasty way.
Well atleast others are using all the research coming out of Google for all of this. DALLE and chatGPT are both the result of research papers directly from Google research.
> Their AI research team is completely dysfunctional and unfocused.
The bigger problem was that they saw the major problems with trustworthiness of LLMs and felt there was still too much brand risk to Google. OpenAI had no such problem.
“We should stop training radiologists now. It's just completely obvious that within five years, deep learning is going to do better than radiologists.”
Geoffrey Hinton, 2016 (while working for Google).
I speculate that they weren't caught off guard. More likely, they tried to find a real-life application for their AI, and failed. Improving the quality of search is not in their interests (in fact, it's the contrary)
> ChatGPT was able to gather so much data without restrictions and now companies late to the party will have many more restrictions
Not only that. OpenAI entered the second stage in AI development: acceleration. They are using their current models to build next generation models. Like they used 4v to build better DALE-3. No competitor is doing it, not at this scale. This and the fact that OpenAI rises the bar makes it very hard to compete even for behemoths like Google. Not even talking about Hugging Face. DALE-3 is incomparably better than Stable Diffusion. But it's artificially limited, like no nudes, no recognizable humans, etc. Making unrestricted competitor will be really hard to do. So, my prediction: OpenAI is far ahead and the gap will increase with the time. Unless there is a breakthrough somewhere else. Then history will repeat itself. Remember, Deep Mind was an indisputable leader not so long ago.
This isn’t history as I remember it. It’s also not history as the product announcement for bard reads.
Bard was explicitly a smaller (cheaper) version of their model. They always said they had a bigger and better one behind the curtain (now called Gemini I guess). I expected bard to be simpler than ChatGPT because they said it’d be small and simple. I guess that still means it’s disappointing 1:1 though.
The notable feature of the model that Bard has over ChatGPT (at least at time of launch) was daily updates keeping it way better for current information and events. Of course, Google has a lot more internal APIs they can/are leverage (eg search, drive, etc) so today bard has a different focus than just talking to a big model.
I’m afraid the real problem is that Google engineering is just lame. Case in point, Google’s vaunted spam filter marks emails from Google as phishing attempts. As I understand it every engineer at Google has the access to fix this. None of them do it. Not even having 20% time is enough. Being a leet coder is one thing. Having an ounce of initiative and giving a shit is another.
20% project is a forbidden sentence by now and certainly never mentioned by the current layer of middle management that has come to infest the engineering department of the company.
In the future, when people will be comparing Google's (lack of) corporate governance and SBF's shenanigans, the former will be considered more outrageous by a lot.
Google prompt results yield far more results and links to sources. Furthermore content creators gain traffic which they can monetise. What benefits does a procedural text generator such as chat gpt bring to the table by comparison? A summary of people’s content without attribution and without monetisation?
ChatGPT4 regularly answers questions for me, summarizes web content, etc while providing linked sources. Saying it doesn't provide links to sources is not accurate. On the other hand, Bard usually (tho admittedly not always) directs me to use Google search for the latest information.
Is bard so bad? I asked it to help write text for a patent, and it did. ChatGPT refuses to do so and gives me another moralistic lecture (this has become so annoying that i stopped using it - i use bing/copilot instead). As a ChatGPT competitor it might even be better, i dont get the outright dismissal.
Bard is honestly fine, I think? It wasn't particularly impressive when they first rolled it out, that probably had a significant impact on peoples' perception.
The GPT API becomes less usable with every new version. For 3.5, March is mostly okay, June requires a heavy hand to only sometimes give a weird moral lecture, and the newest gpt4 version now also starts moralizing all the time.
It's one or two updates before it becomes completely useless for anything but corporate usage.
The lectures drive me insane. At the very least, Bard seems to let the guardrails bend a bit here and there. ChatGPT is insanely strict on it. If I run into a problem, I can get around it in Bard way more easily.
Yeah I'm getting pretty tired of ChatGPT lectures as well. I use it less because most of the time, when I think of some use for it, I'm like "Eh it probably would refuse to do that anyway."
I don't feel like I'm even doing anything awful with it. It just takes issue with the simplest things. Once I had it running a Text Adventure Game for me and I tried to, in game, specify to someone that changing their name was a requirement of joining my company and ChatGPT lost the plot completely and started lecturing me about how unreasonable that requirement was. Like "Oh I'm sorry I didn't realize we needed the moral police in the fantasy realm."
Yeah I'm getting so sick of it. And Microsoft CoPilot is burdened by the same political correctness.
Clearly they're super afraid of what happened to Microsoft earlier last year: Their chatbot that turned into a Nazi. But they are making their product almost unusable.
Luckily jailbreakchat.com still works fine. But doing that all the time is a hassle. I've been trying to run LLaMa 2 uncensored which is great but super slow because I lack the right hardware.
- They don't have significantly different training datasets (it's probably 99% the web).
- They don't have significantly different algorithms
- A big advantage that traditional Google search has the sheer amount of search traffic. They will be able to induce similar traffic (and even repurpose search traffic) to validate their LLMs. OpenAI has the same advantage with ChatGPT thanks being first out of the gate.
- Compute power and efficiency: both OpenAI and Google have access to effectively infinite capacity vs their competitors making search-adjacent products.
Most significantly, OpenAI/ChatGPT has won the LLM branding race, just like Google won the traditional search branding race.
Different, but they also share some major base training sets, i.e Wikipedia.
If differentiation is really about proprietary training data sets, then the winner will be determined by a data land grab, not by technology differentiation.
There was a rumor about ChatGPT using libgen for training data, if true, I find it hard to believe Google's legal team would touch that with a long stick.
>Most significantly, OpenAI/ChatGPT has won the LLM branding race, just like Google won the traditional search branding race.
Outside the tech bubble everyone hears AI, but not a lot are clued in that hatGPT is even a thing. There are 180 million chat GPT sign ups, and 100million DAU. Lots of my non techie, non west coast, non corporate friends think I have 9 heads when I talk about this stuff. In 2002 if you said the word browser, some people were still confused, use of a mouse was an issue in some parts of the US... ChatGPT is at the same level now.
> Outside the tech bubble everyone hears AI, but not a lot are clued in that hatGPT is even a thing.
The moment people realize that this tool allows you to enhance the volume and quality of your work output with little effort, whether it is authentic or not, it's use will catch on like wildfire.
Arguably it already has with students and knowledge workers, even outside tech centers. Schools and universities everywhere are suddenly contending with how to evaluate students in light of the availability of LLMs.
Why is it catching on? In the best light because it actually is a tremendous productivity accelerator. In the worst light, because we live in a world that incentivizes "fake it til you make it".
> Why is it catching on? In the best light because it actually is a tremendous productivity accelerator. In the worst light, because we live in a world that incentivizes "fake it til you make it".
It's great at a few things and pretty good at a lot of things. In my view, the thing that it's the absolute best at is churning out low value, rarely-read communication. There is a massive amount of that type of communication - spam, student essays, procedural documents for compliance. There are loads of jobs that need to do that sort of thing regularly and it's a godsend for them.
ChatGPT 4 is light-years ahead of Bard. I tried a couple of question regarding physic and Bard consistently answers incorrectly. And it is not like Wikipedia page for my questions does not exist.
Google was so busy milking its search lead it forgot it needed to be outstanding to win consistently in the long term.
As far as I'm concerned, Google's key, meaningful homegrown products are Search, Maps, Gmail and Chrome.
Those products (and everything else they have acquired, including Android) exist for one single purpose: selling ads. Funnily enough, those products have also degraded in quality over time as their ability to sell ads has increased.
Unfortunately for Google, it is hard to imagine a GPT chatbot that is aligned with that One True Objective, so the organization can't help itself. Sure, it's a quote-unquote "tech company", but it's far from nimble. As it scrambles to come up with a response to ChatGPT, it looks more like GE or IBM than it looks like OpenAI. Changing course of such a massive ship is damn near impossible.
I don't think the decisions made by Alphabet today are as engineering-driven as they were in the early days. I'm sure the talent is still there (or was there, pre-pandemic), but maybe it's just not calling the shots.
Google is no different from any other entrenched monopoly. It's hard for the organization to care about anything outside its core revenue enough to put significant weight behind it. How many technologies were passed on by Kodak, AT&T or Xerox in favor of their existing businesses that wound up being wildly financially successful.
The problem isn't Google here. It's how all potential competitors or even just any interesting tech startup is gobbled up by the big 4 (?) and then extinguished. Imagine if companies like IBM had been able to or allowed to do any of what Facebook or Google does to reduce competition.
Bard has been working really well for me. Creating contexts/hooks for react. Generating fake resumes for some tests. Questions about how the stock market works.
I would really hope Google's monopoly is undermined a bit. The internet is too dependent on them and their business model is pretty consumer-hostile.
But big ships turn slowly, but once they build up steam they're unstoppable. I think they will manage.
I bet they would have chipped in to OpenAI if Microsoft hadn't beat them to it. As for OpenAI and Microsoft, not sure what I think of it, Microsoft has not exactly been an angel in the internet history books either.
Question about training LLM models: Can OpenAI leverage their existing models when training new ones? Asking because data sources such as Reddit and Twitter are now blocking AI crawlers. So if new OpenAI models can't be trained on those sources then can we expect that future models will be dumber? Or will they be able to leverage the existing models, so future models can only get smarter?
53 comments
[ 2.8 ms ] story [ 113 ms ] threadThey weren't.
Their AI research team is completely dysfunctional and unfocused. They're only interested in doing research and are entirely incapable of the engineering and product focus necessary to build something like ChatGPT.
That isn't the same thing as being caught off guard.
As always, Google has incredible ideas and promise but lacks business execution
https://www.inc.com/nick-hobson/googles-rest-vest-days-for-s...
Too little too late?
Announcing things and actually executing on said announcements are two very different things.
Sundar has never been able to tell the difference between the two.
Correct, but not for the reason you gave.
> Their AI research team is completely dysfunctional and unfocused.
The bigger problem was that they saw the major problems with trustworthiness of LLMs and felt there was still too much brand risk to Google. OpenAI had no such problem.
“We should stop training radiologists now. It's just completely obvious that within five years, deep learning is going to do better than radiologists.”
Geoffrey Hinton, 2016 (while working for Google).
I speculate that they weren't caught off guard. More likely, they tried to find a real-life application for their AI, and failed. Improving the quality of search is not in their interests (in fact, it's the contrary)
Not only that. OpenAI entered the second stage in AI development: acceleration. They are using their current models to build next generation models. Like they used 4v to build better DALE-3. No competitor is doing it, not at this scale. This and the fact that OpenAI rises the bar makes it very hard to compete even for behemoths like Google. Not even talking about Hugging Face. DALE-3 is incomparably better than Stable Diffusion. But it's artificially limited, like no nudes, no recognizable humans, etc. Making unrestricted competitor will be really hard to do. So, my prediction: OpenAI is far ahead and the gap will increase with the time. Unless there is a breakthrough somewhere else. Then history will repeat itself. Remember, Deep Mind was an indisputable leader not so long ago.
Bard was explicitly a smaller (cheaper) version of their model. They always said they had a bigger and better one behind the curtain (now called Gemini I guess). I expected bard to be simpler than ChatGPT because they said it’d be small and simple. I guess that still means it’s disappointing 1:1 though.
The notable feature of the model that Bard has over ChatGPT (at least at time of launch) was daily updates keeping it way better for current information and events. Of course, Google has a lot more internal APIs they can/are leverage (eg search, drive, etc) so today bard has a different focus than just talking to a big model.
In 2023 Google is an entirely different beast.
20% project is a forbidden sentence by now and certainly never mentioned by the current layer of middle management that has come to infest the engineering department of the company.
Is bard so bad? I asked it to help write text for a patent, and it did. ChatGPT refuses to do so and gives me another moralistic lecture (this has become so annoying that i stopped using it - i use bing/copilot instead). As a ChatGPT competitor it might even be better, i dont get the outright dismissal.
It's one or two updates before it becomes completely useless for anything but corporate usage.
I don't feel like I'm even doing anything awful with it. It just takes issue with the simplest things. Once I had it running a Text Adventure Game for me and I tried to, in game, specify to someone that changing their name was a requirement of joining my company and ChatGPT lost the plot completely and started lecturing me about how unreasonable that requirement was. Like "Oh I'm sorry I didn't realize we needed the moral police in the fantasy realm."
Clearly they're super afraid of what happened to Microsoft earlier last year: Their chatbot that turned into a Nazi. But they are making their product almost unusable.
Luckily jailbreakchat.com still works fine. But doing that all the time is a hassle. I've been trying to run LLaMa 2 uncensored which is great but super slow because I lack the right hardware.
- They don't have significantly different training datasets (it's probably 99% the web).
- They don't have significantly different algorithms
- A big advantage that traditional Google search has the sheer amount of search traffic. They will be able to induce similar traffic (and even repurpose search traffic) to validate their LLMs. OpenAI has the same advantage with ChatGPT thanks being first out of the gate.
- Compute power and efficiency: both OpenAI and Google have access to effectively infinite capacity vs their competitors making search-adjacent products.
Most significantly, OpenAI/ChatGPT has won the LLM branding race, just like Google won the traditional search branding race.
If differentiation is really about proprietary training data sets, then the winner will be determined by a data land grab, not by technology differentiation.
I thought it has been reasonably well-established that ChatGPT was trained significantly on pirated textbooks?
Could you elaborate on what you mean?
Outside the tech bubble everyone hears AI, but not a lot are clued in that hatGPT is even a thing. There are 180 million chat GPT sign ups, and 100million DAU. Lots of my non techie, non west coast, non corporate friends think I have 9 heads when I talk about this stuff. In 2002 if you said the word browser, some people were still confused, use of a mouse was an issue in some parts of the US... ChatGPT is at the same level now.
The moment people realize that this tool allows you to enhance the volume and quality of your work output with little effort, whether it is authentic or not, it's use will catch on like wildfire.
Arguably it already has with students and knowledge workers, even outside tech centers. Schools and universities everywhere are suddenly contending with how to evaluate students in light of the availability of LLMs.
Why is it catching on? In the best light because it actually is a tremendous productivity accelerator. In the worst light, because we live in a world that incentivizes "fake it til you make it".
It's great at a few things and pretty good at a lot of things. In my view, the thing that it's the absolute best at is churning out low value, rarely-read communication. There is a massive amount of that type of communication - spam, student essays, procedural documents for compliance. There are loads of jobs that need to do that sort of thing regularly and it's a godsend for them.
But it insists on making up articles, complete with sources and authors. shrugs
ChatGPT is better in that regard.
Waiting for someone will match GPT-4 and especially GPT4-V quality and latency.
And this would be the start of the battle, which could be bad for combatants but great for the rest of us.
Fingers crossed, it will happen soon. Not sure that early 2024 ill be that day, but let's hope.
As far as I'm concerned, Google's key, meaningful homegrown products are Search, Maps, Gmail and Chrome.
Those products (and everything else they have acquired, including Android) exist for one single purpose: selling ads. Funnily enough, those products have also degraded in quality over time as their ability to sell ads has increased.
Unfortunately for Google, it is hard to imagine a GPT chatbot that is aligned with that One True Objective, so the organization can't help itself. Sure, it's a quote-unquote "tech company", but it's far from nimble. As it scrambles to come up with a response to ChatGPT, it looks more like GE or IBM than it looks like OpenAI. Changing course of such a massive ship is damn near impossible.
I don't think the decisions made by Alphabet today are as engineering-driven as they were in the early days. I'm sure the talent is still there (or was there, pre-pandemic), but maybe it's just not calling the shots.
The problem isn't Google here. It's how all potential competitors or even just any interesting tech startup is gobbled up by the big 4 (?) and then extinguished. Imagine if companies like IBM had been able to or allowed to do any of what Facebook or Google does to reduce competition.
TBH Youtube and Photos aren't half bad.
And their cloud-based office suite, while it could be way better, is kind of useful.
https://news.ycombinator.com/item?id=38300990
But big ships turn slowly, but once they build up steam they're unstoppable. I think they will manage.
I bet they would have chipped in to OpenAI if Microsoft hadn't beat them to it. As for OpenAI and Microsoft, not sure what I think of it, Microsoft has not exactly been an angel in the internet history books either.