Looks like new competition in the dumbing down department.
Only 5 pages of search results vs regurgitated internet text.
A race to the bot. 'um...
Updated Jan. 21, 2023, 1:53 a.m. ET
Last month, Larry Page and Sergey Brin, Google’s founders, held several meetings with company executives. The topic: a rival’s new chatbot, a clever A.I. product that looked as if it could be the first notable threat in decades to Google’s $149 billion search business.
Mr. Page and Mr. Brin, who had not spent much time at Google since they left their daily roles with the company in 2019, reviewed Google’s artificial intelligence product strategy, according to two people with knowledge of the meetings who were not allowed to discuss them. They approved plans and pitched ideas to put more chatbot features into Google’s search engine. And they offered advice to company leaders, who have put A.I. front and center in their plans.
The re-engagement of Google’s founders, at the invitation of the company’s current chief executive, Sundar Pichai, emphasized the urgency felt among many Google executives about artificial intelligence and that chatbot....(which shall remain name less.)
your parent comment probably thought the article was paywalled, and was looking for an archive.ph link but was unaware that you could just append the original url
It’s interesting that Google—the inventor of the “T” in GPT—is on red alert due to ChatGPT. And Google has a bunch of research in AI (see Dean’s recent blogpost [1]), so what gives with their lack of AI product/execution/strategy? Too scared to upend their existing cash cow maybe, aka innovator’s dilemma?
Also, it doesn’t inspire confidence in Pichai that Sergei and Brin were consulted about something that should be on the forefront of Google’s strategy. Maybe that’s too harsh but I just find all this surprising.
I think it does the opposite of not inspiring confidence. A good leader knows when they’re facing a tough problem that they don’t have all the answers for and they need help solving it. Reaching out for that help rather than trying to muddle through on your own could be the difference between success and a disaster. Not sure Larry and Sergei are the ones to give it but it doesn’t hurt to have them involved.
>"A good leader knows when they’re facing a tough problem that they don’t have all the answers for and they need help solving it. Reaching out for that help rather than trying to muddle through on your own could be the difference between success and a disaster."
There is no reason why he would have ever needed to "muddle through" on his own. The article states Google acquired DeepMind an AI research lab in 2014 as well as the fact that Google has its own Advanced Technology Review Council. This is in addition to the insane hiring spree that the company went on in recent years where presumably they did or should have been hiring even more AI competencies. Given these things, having to call in the founders who stepped away from the company 3 years ago kind of make this CEO look incompetent.
I guess the founder's mindset is different from the CEO's mindset in that strategic positioning, especially when market is in flux, is second-nature to founders.
A CEO getting in position after a company is well-established is often merely the "toppest" manager.
Google proved they could pump out new business after new business. They just kept shutting them down because they weren't as profitable as search ads. None of the things they tried ever had a chance to refine or pivot like any business in the real world. They expect immediate world domination.
Big companies try to incubate startups but just can't leave shit alone. They have to micromanage everything under the terms of the core business. Startups get lucky from trying hair-brained ideas and being scrappy--not being micromanaged.
I'm working in a startup right now, building a new product that complements the core business. The new product is highly suspect, and on paper seems like it could be a real mess (handling real peoples' money). I'm interested to see how it adapts or pivots after meeting the customer, and I'm not willing to write it off because I cannot predict what it will develop into even if the original idea fails. It could all be a waste of time and money. It could also reveal another market. The original idea doesn't have to work out for the venture to be successful.
> In recent years, large neural networks trained for language understanding and generation have achieved impressive results across a wide range of tasks. GPT-3 first showed that large language models (LLMs) can be used for few-shot learning and can achieve impressive results without large-scale task-specific data collection or model parameter updating. More recent LLMs, such as GLaM, LaMDA, Gopher, and Megatron-Turing NLG, achieved state-of-the-art few-shot results on many tasks by scaling model size, using sparsely activated modules, and training on larger datasets from more diverse sources. Yet much work remains in understanding the capabilities that emerge with few-shot learning as we push the limits of model scale.
> […] More recent LLMs, such as GLaM, LaMDA, Gopher, and Megatron-Turing NLG, achieved [SOTA] few-shot results on many tasks by scaling model size, using sparsely activated modules, and training on larger datasets from more diverse sources. Yet much work remains [for few-shot LLMs] /2
DeepMind Dreamerv3 is more advanced than all the LLMs.
I don’t think the issue of Google is a lack of advanced AI technology but the transfer and realization of products. So many Google’s products are half-backed, only half-heartedly developed and then be buried at the end. For a giant like Google, they must be stuck in a rut and to get free need all the help they can get. If they continue the business as usual, I afraid they will experience a never-seen before landslide.
I believe calling back the founders at this critical time is the right move.
> Furthermore, the inefficiency of tabula rasa RL research can exclude many researchers from tackling computationally-demanding problems. For example, the quintessential benchmark of training a deep RL agent on 50+ Atari 2600 games in ALE for 200M frames (the standard protocol) requires 1,000+ GPU days. As deep RL moves towards more complex and challenging problems, the computational barrier to entry in RL research will likely become even higher.
> To address the inefficiencies of tabula rasa RL, we present “Reincarnating Reinforcement Learning: Reusing Prior Computation To Accelerate Progress” at NeurIPS 2022. Here, we propose an alternative approach to RL research, where prior computational work, such as learned models, policies, logged data, etc., is reused or transferred between design iterations of an RL agent or from one agent to another. While some sub-areas of RL leverage prior computation, most RL agents are still largely trained from scratch. Until now, there has been no broader effort to leverage prior computational work for the training workflow in RL research. We have also released our code and trained agents to enable researchers to build on this work.
Feed-Forward with Prompt Engineering is like RL; which prompt elements should remain given objective or subjective error?
16 comments
[ 2.8 ms ] story [ 30.8 ms ] threadLooks like new competition in the dumbing down department.
Only 5 pages of search results vs regurgitated internet text.
A race to the bot. 'um...
Updated Jan. 21, 2023, 1:53 a.m. ET Last month, Larry Page and Sergey Brin, Google’s founders, held several meetings with company executives. The topic: a rival’s new chatbot, a clever A.I. product that looked as if it could be the first notable threat in decades to Google’s $149 billion search business.
Mr. Page and Mr. Brin, who had not spent much time at Google since they left their daily roles with the company in 2019, reviewed Google’s artificial intelligence product strategy, according to two people with knowledge of the meetings who were not allowed to discuss them. They approved plans and pitched ideas to put more chatbot features into Google’s search engine. And they offered advice to company leaders, who have put A.I. front and center in their plans.
The re-engagement of Google’s founders, at the invitation of the company’s current chief executive, Sundar Pichai, emphasized the urgency felt among many Google executives about artificial intelligence and that chatbot....(which shall remain name less.)
Also, it doesn’t inspire confidence in Pichai that Sergei and Brin were consulted about something that should be on the forefront of Google’s strategy. Maybe that’s too harsh but I just find all this surprising.
[1] https://ai.googleblog.com/2023/01/google-research-2022-beyon...
There is no reason why he would have ever needed to "muddle through" on his own. The article states Google acquired DeepMind an AI research lab in 2014 as well as the fact that Google has its own Advanced Technology Review Council. This is in addition to the insane hiring spree that the company went on in recent years where presumably they did or should have been hiring even more AI competencies. Given these things, having to call in the founders who stepped away from the company 3 years ago kind of make this CEO look incompetent.
A CEO getting in position after a company is well-established is often merely the "toppest" manager.
Big companies try to incubate startups but just can't leave shit alone. They have to micromanage everything under the terms of the core business. Startups get lucky from trying hair-brained ideas and being scrappy--not being micromanaged.
I'm working in a startup right now, building a new product that complements the core business. The new product is highly suspect, and on paper seems like it could be a real mess (handling real peoples' money). I'm interested to see how it adapts or pivots after meeting the customer, and I'm not willing to write it off because I cannot predict what it will develop into even if the original idea fails. It could all be a waste of time and money. It could also reveal another market. The original idea doesn't have to work out for the venture to be successful.
Google already has very large LLMs online for search and other applications.
A similar take on similar spin: https://twitter.com/westurner/status/1614002846394892288
From April 2022, "Pathways Language Model (PaLM): Scaling to 540 Billion Parameters for Breakthrough Performance" https://ai.googleblog.com/2022/04/pathways-language-model-pa... :
> In recent years, large neural networks trained for language understanding and generation have achieved impressive results across a wide range of tasks. GPT-3 first showed that large language models (LLMs) can be used for few-shot learning and can achieve impressive results without large-scale task-specific data collection or model parameter updating. More recent LLMs, such as GLaM, LaMDA, Gopher, and Megatron-Turing NLG, achieved state-of-the-art few-shot results on many tasks by scaling model size, using sparsely activated modules, and training on larger datasets from more diverse sources. Yet much work remains in understanding the capabilities that emerge with few-shot learning as we push the limits of model scale.
> […] More recent LLMs, such as GLaM, LaMDA, Gopher, and Megatron-Turing NLG, achieved [SOTA] few-shot results on many tasks by scaling model size, using sparsely activated modules, and training on larger datasets from more diverse sources. Yet much work remains [for few-shot LLMs] /2
DeepMind Dreamerv3 is more advanced than all the LLMs.
"Beyond Tabula Rasa: Reincarnating Reinforcement Learning" https://ai.googleblog.com/2022/11/beyond-tabula-rasa-reincar... :
> Furthermore, the inefficiency of tabula rasa RL research can exclude many researchers from tackling computationally-demanding problems. For example, the quintessential benchmark of training a deep RL agent on 50+ Atari 2600 games in ALE for 200M frames (the standard protocol) requires 1,000+ GPU days. As deep RL moves towards more complex and challenging problems, the computational barrier to entry in RL research will likely become even higher.
> To address the inefficiencies of tabula rasa RL, we present “Reincarnating Reinforcement Learning: Reusing Prior Computation To Accelerate Progress” at NeurIPS 2022. Here, we propose an alternative approach to RL research, where prior computational work, such as learned models, policies, logged data, etc., is reused or transferred between design iterations of an RL agent or from one agent to another. While some sub-areas of RL leverage prior computation, most RL agents are still largely trained from scratch. Until now, there has been no broader effort to leverage prior computational work for the training workflow in RL research. We have also released our code and trained agents to enable researchers to build on this work.
Feed-Forward with Prompt Engineering is like RL; which prompt elements should remain given objective or subjective error?