At some point it's going to be difficult to shut everyone else out. Intel tried this a long time ago, and while they maintained dominance, they were not able to shut out competitors completely, and experienced a lot of legal battles over it.
Foreign nations aren't going to be happy about one mostly US company holding all the cards too.
Another big concern will be regulatory. It seems unlikely a couple billion people whose livelihood is significantly impacted will just chill as it happens?
I think it’s unlikely, but no less likely than the compute issues mentioned.
So the research and development will just move over to a neighboring country?
This isn't like the manhattan project. There are lots of people who know how to make this stuff, and they don't need rare volatile elements - just consumer hardware.
It depends if were talking 'Butlerian Jihad' levels of disruption here.
And no, the big models require exabytes of processing power and time, so at least at the most extreme scales if nations started punching missiles in processor factories and data centers you'd slow top end AI projects way down.
If it seems dire enough lethal military force might be used.
If Russia and China think the US is about to have AI capable of absolute total global domination they may launch a preemptive strike. Maybe hypersonic cruise missiles at datacenters, maybe a full EMP or nuclear launch.
Isn't this a "concern" whenever a new technology comes out? ie: the internet? Yet, due to how slowly government moves and how hard new technology is to understand for governments, it is barely a concern.
The US is very capitalistic. You may get regulation in France, but not the US.
The Corporate desire for profit will overshadow the livelihood of billions. It's been this way in the US since forever. Look what happened to the corporation that caused the Opioid epidemic. Nothing, they profited.
If that was true, the US would be producing orders of magnitude more nuclear energy than it is today. In reality many sectors of the US economy, such as housing, are utterly crippled by regulation.
It's rather disingenuous to say "there are always exceptions". First off, if you're just going to talk generally, there are countries with less onerous business regulations than the United States. France isn't one of them, but Norway and Denmark are. More importantly, we are talking about regulating a specific technology, namely AI. The US has already demonstrated the ability to regulate a specific technology--namely nuclear energy--to a point where it is almost completely marginalized. So the notion that the US isn't capable of regulating an entire technology into near-oblivion is demonstrably false.
>It's rather disingenuous to say "there are always exceptions".
Then what do you want me to say? It's the truth. There are always exceptions. Always. You want me to say you're right when you're actually wrong?
> but Norway and Denmark are
You kidding? Scandinavia is more or less a collection of countries closest to socialism. These countries have by far more regulations in GENERAL. It's obvious but we can find evidence if you want. Take this for instance: In Sweden, there is a law enforcing a five-week vacation policy.
>The US has already demonstrated the ability to regulate a specific technology--namely nuclear energy--to a point where it is almost completely marginalized. So the notion that the US isn't capable of regulating an entire technology into near-oblivion is demonstrably false.
Except there are multitudes of failures as well. The failures outnumber the successes by a huge margin. Take for instance:
Enron: In the early 2000s, the energy company Enron engaged in a series of fraudulent accounting practices to make it appear as if the company was more profitable than it actually was. Despite warnings from whistleblowers and others, the government failed to intervene and the company ultimately collapsed, resulting in significant financial losses for many investors and employees.
BP Oil Spill: In 2010, an explosion on an oil rig operated by BP in the Gulf of Mexico caused the largest oil spill in U.S. history. The disaster was largely attributed to lax regulation and oversight by government agencies, including the Minerals Management Service.
Volkswagen: In 2015, it was revealed that Volkswagen had installed software in its diesel cars that cheated emissions tests, leading to higher levels of pollution than were reported. The company ultimately agreed to pay billions of dollars in fines and compensation, but the government was criticized for failing to catch the deception sooner.
Equifax: In 2017, the credit reporting agency Equifax suffered a massive data breach that exposed the personal information of millions of people. Critics argued that the government had not done enough to regulate the company and protect consumer data.
Tobacco Industry: For decades, the tobacco industry engaged in deceptive marketing practices that downplayed the health risks of smoking. Despite mounting evidence of the harmful effects of tobacco, the government was slow to take action to regulate the industry, and it was not until the late 1990s that significant reforms were implemented.
Wall Street: The 2008 financial crisis was largely caused by the reckless behavior of major Wall Street banks and financial institutions. Many critics argue that the government failed to adequately regulate these institutions, allowing them to engage in risky practices that ultimately led to the collapse of the housing market and the wider economy.
Boeing: In 2019, two deadly crashes involving the Boeing 737 Max raised questions about the safety of the aircraft and the company's regulatory oversight. Critics argued that the FAA (Federal Aviation Administration) had been too close to Boeing, allowing the company to cut corners and prioritize profits over safety.
Big Pharma: The pharmaceutical industry has come under scrutiny for a variety of reasons, including skyrocketing drug prices, aggressive marketing tactics, and the opioid epidemic. Critics argue that the government has not done enough to regulate the industry, which has resulted in significant harm to patients and communities.
Meatpacking Industry: The meatpacking industry has been criticized for unsafe working conditions, low wages, and lax regulatory oversight. The COVID-19 pandemic brought these issues to the forefront, as workers in meatpacking plants became some of the hardest hit by the virus.
Tech Industry: Tech giants like Facebook, Google, and Amazon have faced criticism for a variety of reasons, including antitrust violations, privacy violations, and the spread of...
> Then what do you want me to say? It's the truth. There are always exceptions. Always.
It’s disingenuous to make an overly general statement and then immediately dismiss any counterexample to that overgeneralization by saying “there are always exceptions”. I could just as easily say the United States is chronically overregulated and your list of examples are the exceptions.
Where did you get that by the way? I can’t find any exact phrase matches online, and the writing style and overall glib superficiality reminds me of ChatGPT output. I’m not going to waste my time rebutting machine-generated garbage point by point, especially when half of it is meaningless weasel language about what “critics argue”.
Ultimately it doesn’t matter because if there are exceptions either way, then it’s still possible AI will be regulated.
> You kidding? Scandinavia is more or less a collection of countries closest to socialism.
A common misconception. In reality, they are market economies that combine a dynamic free market economy with a generous welfare state. Wikipedia even specifically lists as a characteristic of the “Nordic model”, “Little product market regulation. Nordic countries rank very high in product market freedom according to OECD rankings”(https://en.m.wikipedia.org/wiki/Nordic_model). It also mentions this quote:
In a speech at Harvard's Kennedy School of Government, Lars Løkke Rasmussen, the centre-right Danish prime minister from the conservative-liberal Venstre party, addressed the American misconception that the Nordic model is a form of socialism, which is conflated with any form of planned economy, stating: "I know that some people in the US associate the Nordic model with some sort of socialism. Therefore, I would like to make one thing clear. Denmark is far from a socialist planned economy. Denmark is a market economy."
The Heritage Foundation publishes an “Economic Freedom Index” that ranks countries by “economic freedom”, according to their own conservative, pro-free-market point of view, and they rank Denmark and Sweden as more economically free than the United States, both in the general index and in the more specific index of “business freedom”, which seems to be the measure relevant to regulatory burden in particular (https://www.heritage.org/index/)
> Given the sheer amount of counter examples of lack of regulation. It's pretty much safe to say that it's highly unlikely AI will be regulated in any meaningful way.
So then how do you explain the regulatory crippling of nuclear power?
>I can’t find any exact phrase matches online, and the writing style and overall glib superficiality reminds me of ChatGPT output.
I got it from my notes for an unrelated project. You can't find it because I wrote the notes. But this shouldn't matter. As long as the points are correct, you getting schooled by an AI is irrelevant to the conversation at hand.
>It’s disingenuous to make an overly general statement and then immediately dismiss any counterexample to that overgeneralization by saying “there are always exceptions”.
No it's not. General truths exist. You must dismiss exceptions to get at the general truth. Otherwise we'll be mired in details constantly.
>Denmark is a market economy.
I never said it wasn't. I said "closest" to a socialist economy.
>A common misconception. In reality, they are market economies that combine a dynamic free market economy with a generous welfare state.
No misconception made here. You are putting words in my mouth. I said it was closest to socialism. And it holds true. A generous welfare state is closer to socialism.
>The Heritage Foundation publishes an “Economic Freedom Index” that ranks countries by “economic freedom”, according to their own conservative, pro-free-market point of view, and they rank Denmark and Sweden as more economically free than the United States
That's bullshit. The heritage foundation is a conservative think tank with a biased agenda. How about getting a study from an unbiased source. "Economic Freedom Index" << don't fall for that.
The US isn't even on that list because we've lobbied the hell out of those laws to be lax af.
Not to mention labor regulations, mandatory five week vacations? Paternity leave? Unheard of in the US.
Just use some common sense before trusting some "Freedom" Index from a politicized foundation.
>So then how do you explain the regulatory crippling of nuclear power?
It's an exception. I mean I literally gave you tons of examples how the US fails to regulate things. The ratio of failures to successes is what matters here. And the failures outnumber the successes by a huge amount. I only copied a fraction of my notes. Would you like more?
> I got it from my notes for an unrelated project. You can't find it because I wrote the notes. But this shouldn't matter. As long as the points are correct, you getting schooled by an AI is irrelevant to the conversation at hand.
The points don't specifically pertain to the US and have enough garbage weasel words that they don't even rise to the level of "correct". I'll make another comment that goes through them though.
> General truths exist. You must dismiss exceptions to get at the general truth. Otherwise we'll be mired in details constantly.
You can't just handwave away complexity. It's entirely possible for the US to have too little regulation in some fields and too much regulation in other fields.
Your claim is that it's impossible for the US to erect regulatory barriers for AI. Whether or not the US, in general, has more or less regulations than other countries isn't sufficient to make that case.
> The heritage foundation is a conservative think tank with a biased agenda.
I never claimed otherwise. But why would a conservative think tank, which favors less business regulations, not even put the US on the top ten list of most "economically free" countries if the US is so underregulated? Why would they prefer the regulatory environment in Denmark and Sweden over the regulatory environment in the United States?
> Check out which countries rank the highest for food regulations
So when I bring up examples they're "exceptions", but while you bring up examples, they're examples of the rule. When I cite sources that do an overall survey of a country's regulatory environment, that's "biased", but when you cite a source that makes their money by helping companies comply with food regulations, that's just fine.
> Not to mention labor regulations, mandatory five week vacations? Paternity leave? Unheard of in the US.
Yeah, there are some places where the US has more regulations and some places where the US has less regulations.
If we're talking about regulating AI, I think nuclear energy regulations are a much more analogous case than vacation and paternity leave.
> It's an exception. I mean I literally gave you tons of examples how the US fails to regulate things. The ratio of failures to successes is what matters here. And the failures outnumber the successes by a huge amount. I only copied a fraction of my notes. Would you like more?
I can run ChatGPT myself, thanks. Why don't you try thinking for yourself and considering the possibility that your presuppositions are wrong?
>You can't just handwave away complexity. It's entirely possible for the US to have too little regulation in some fields and too much regulation in other fields.
I didn't handwave anything. My answer is sufficiently complex with multitudes of counter examples to your point.
The whole thing with the massive list of examples is to illustrate a general point on the lack of business regulation overall in the US.
I literally stated it's the ratio of failures to successes that matters here. If I can produce 30 examples of the US failing to regulate and you produce one, that speaks to an overall generality that eclipses your example.
The arena of scientific validation is hard to establish here. Neither of us can paint a picture of the entire domain of every single failure and success of regulatory laws in existence for the US. So given the nature of this debate just list as many general examples as possible.
You have nuclear power as one, that's it.
>I never claimed otherwise. But why would a conservative think tank, which favors less business regulations, not even put the US on the top ten list of most "economically free" countries if the US is so underregulated? Why would they prefer the regulatory environment in Denmark and Sweden over the regulatory environment in the United States?
Don't know. The motivations of such groups are complex and multifaceted. Following some breadcrumb trail to get at the root of it is too much effort. I only know that this group is biased and not a neutral party. There's no point in vetting a known compromised source. Pick a valid one.
>So when I bring up examples they're "exceptions", but while you bring up examples, they're examples of the rule. When I cite sources that do an overall survey of a country's regulatory environment, that's "biased", but when you cite a source that makes their money by helping companies comply with food regulations, that's just fine.
Yeah you cited one bogus example from the heritage foundation. All my examples are real. Unlike yours.
>I can run ChatGPT myself, thanks. Why don't you try thinking for yourself and considering the possibility that your presuppositions are wrong?
Highly disagree. Your answers are inferior to anything chatGPT can come up with so obviously you likely can't run it yourself.
> The whole thing with the massive list of examples is to illustrate a general point on the lack of business regulation overall in the US. I literally stated it's the ratio of failures to successes that matters here. If I can produce 30 examples of the US failing to regulate and you produce one, that speaks to an overall generality that eclipses your example.
Your list produces nothing of the kind, as I tediously went out of my way to demonstrate.
> I only know that this group is biased and not a neutral party. There's no point in vetting a known compromised source. Pick a valid one.
You haven't provided any valid sources yourself.
> Highly disagree. Your answers are inferior to anything chatGPT can come up with so obviously you likely can't run it yourself.
Well, that's just your opinion, and it's an opinion that reflects more on your poor judgment than on me.
I never said with certainty whether the US would or wouldn't regulate AI. Which has absolutely nothing to do with the open letter you posted in any case.
Frankly, the abrasive and belligerent way you've conducted yourself this entire conversation is the only thing that reflects poorly on anyone here. There's simply no call for making this personal.
You both broke the site guidelines badly in this thread. You unfortunately have a pattern of doing this and we had to warn you about almost exactly the same thing before: https://news.ycombinator.com/item?id=33187614.
I don't want to ban you, so would you please review https://news.ycombinator.com/newsguidelines.html and stick to the rules properly from now on? We don't want this sort of tit-for-tat spat in which people abuse each other.
Okay, fine, I'll go through your "notes" (which still sound a lot like ChatGPT garbage):
> Enron: In the early 2000s, the energy company Enron engaged in a series of fraudulent accounting practices to make it appear as if the company was more profitable than it actually was.
And in 2002, the US passed the Sarbanes-Oxley Act, which dramatically increased the regulatory burdens of running a publicly traded company.
> Volkswagen: In 2015, it was revealed that Volkswagen had installed software in its diesel cars that cheated emissions tests, leading to higher levels of pollution than were reported.
This doesn't make the point you think it makes. Even though Volkswagen is a German company that were selling these cars all around the world, and even though 8.5 million of the 11 million Volkswagens that were eventually recalled were in the European Union, it was the US EPA, not European regulators, who caught Volkswagen.
> Tobacco Industry
Almost all European countries have higher smoking rates than the United States. France has twice as many smokers as the US.
> The 2008 financial crisis was largely caused by the reckless behavior of major Wall Street banks and financial institutions. Many critics argue that the government failed to adequately regulate these institutions, allowing them to engage in risky practices that ultimately led to the collapse of the housing market and the wider economy.
"Many critics argue"--there are those weasel words again. Similarly with Enron, this led to the Dodd-Frank Act, which increased financial regulations again.
An interesting counterpoint to this is the Libor scandal, which came to light a few years afterwards. (https://en.wikipedia.org/wiki/Libor_scandal) Similarly to Volkswagen, the Libor scandal was happening in Europe (specifically the UK) under European regulatory jurisdiction, and yet it was American regulators who caught the perpetrators.
> Boeing 737 Max
This is a case of US regulators being behind other countries, but it is an exception. Also note that none of the actual 737 Max crashes happened in the US, but rather in countries with weaker regulatory regimes when it comes to airline operations.
> Big Pharma
Not US specific. Lots of these firms, like Bayer and GSK, are European, and drugs usually get approved by European regulators more quickly than they get approved by the FDA.
> Meatpacking Industry: The meatpacking industry has been criticized for unsafe working conditions, low wages, and lax regulatory oversight.
Meaningless weasel words.
> Tech Industry: Tech giants like Facebook, Google, and Amazon have faced criticism for a variety of reasons, including antitrust violations, privacy violations, and the spread of misinformation. Critics argue that the government has not done enough to regulate these companies, which have become some of the most powerful corporations in the world.
Weasel words. If you want to count this as US specific because most of these companies are American, then you don't get to blame the US for the actions of Volkswagen and British Petroleum.
> Fast Food Industry: The fast food industry has been criticized for...
More weasel words. Also not US specific.
> Industrial Agriculture
Not US specific.
> Gun Industry
Unrelated political controversy.
> Pharmaceutical Industry
You already listed Big Pharma.
> Big Oil
Not specific to the US. Canada and Norway also produce oil, and relative to their GDP, they are more dependent on oil production than the US is. In yet another example of US regulations being more stringent than other countries, it was the US, not Canada, that shut down the Keystone XL oil pipeline between Canada and the United States.
> Private Prisons
Also not US specific. Australia, New Zealand, and Canada have more of their prisoners in private prisons than the US does.
That's not entirely true. Back in the 1970s when we tightened up regulation the companies operating nuclear power plants in the US were selling power onto the grid on a cost plus basis. That is, they'd be paid for their expenses plus a reasonable percentage on top of that. And there were regulators looking at their expenses to make sure they were reasonable.
But when they, together with environmental activists, were able to get laws passed that drastically increased the cost of running a nuclear plant the regulators couldn't say no. So their costs increased, but then their profits increased as well through the magic of cost-plus contracts.
That sounds like a classic example of regulatory capture, with incumbent nuclear power plant operators managing to serve their own interests and raise a regulatory moat against anyone coming after them to build more nuclear power plants.
A more important factor than that, I think, is that the national security establishment in the US is the part of government most concerned with AI right now and they mostly see it as a matter of competition with China.
I think scaling limits and profitability are the only things that can stop the march of the AI. The utility is already there and even the current GPT4 utility is revolutionary.
It's different this time. Because this time AI is hugely more popular in the public and corporate sphere. The previous AI winters were more academic winters with few people pushing the envelope.
I don't think compute is the issue. It's an issue with LLMs. Current LLMs are just a stepping stone for true AGI. I think there's enough momentum right now that we can avoid a winter and find something better through sheer innovation.
After the massive hype around generative AI, seems likely there will be an AI winter when the promised transformation in many business areas just doesn't happen as advertised.
* More busy work will be automated taking away some of the fun and leaving the harder tasks thus making jobs shittier, pushing workers to find other kinds of work.
* More solutions on AI looking for problems will be implemented increasing the speed of success/failure, and from the lottery winner effect we can expect "the granny that created an AI solution and earned millions" that other devs will follow and fail leaving debts.
A disappointment in the hype cycle isn't equivalent to AI winter. The hype cycle always, eventually, hits a trough, even with wildly successful products.
It's very likely we'll be disappointed with AI in many oversold contexts (I share the sentiment about self-driving cars), but it can't be denied that ChatGPT is a product that's being used _right now_ to massive success and still has quite a ways to go.
It doesn't go into large value decisions and projects in a meaningful way (pretty much like all AI) - that is the critical thing AI needs to get to. Where was AI when guaranteeing SVB's deposits was decided, for example?
> It doesn't go into large value decisions and projects in a meaningful way (pretty much like all AI) - that is the critical thing AI needs to get to
Large decisions such as SVB deposits are literally the last thing that will be given to AI, it's the most important part of the business. That doesn't mean AI is useless or is not being incorporated into valuable parts of businesses.
I'm at a large tech company, LLMs are already entering some of our key product workflows. It massively lowers the floor for a number of product features e.g. recommendation systems.
In a more general sense, many features that were previously too difficult to expose (e.g. simple coding commands) to business users are now on the table as long as we give them limited access to an LLM. The current hype train means that we don't even need to do much user education.
I am not claiming AI is useless. Rather that pushing AI too hard will create disappointment as it isn't just going into every part of businesses with the same speed (or at all for still some time on some things).
Also can I add that I hate the comparison to self-driving cars. The issue with self-driving cars is the penalty for a mis-step can be huge, even if doing something mundane, e.g., driving home from the neighborhood grocery store.
If crashing a self-driving car required me to spend 5 minutes rejiggering it, I'd probably use the feature a lot. With generative models there's usually very little cost to it making a mistake -- and even better, I can determine when the cost is likely to be high and modify my behavior.
Self-driving cars do seem much better than they used to be. But I have no interest in using them until they get better still. I don't have this same bar with LLMs or DALL-Es. And I think this will contribute to more continuous improvement in the technology.
I remember the hype cycle around this thing called the "internet" back in the day. People said it was going to take over the world, even though back then it was slow and kinda sucked.
What metrics do you want to go by? At least by internet use and user growth the dotcom bubble still saw massive amounts of new user growth and online time by users.
Was there a massive reduction in completely untenable .com's? But I'm not exactly sure if that's the definition of a winter, plenty of other internet based businesses did fine and kept growing in that time.
In many cases we see that making effective use of a new technology requires a lot of other changes. Electrification brought large productivity gains to factories, but only after they were re-designed to be more spread out as to compacted around a central motive shaft. IT has brought a lot of productivity gains but business processes have to be changed to make use of it. Likewise, making use of LLMs will probably require some bottom up rethinking of how businesses work which won't be fast or automatic.
Can anyone give some color on to what extent advancements in AI are limited by the availability of compute, versus the availability of data?
I was under the impression that the size and quality of the training dataset had a much bigger impact on performance versus the sophistication of the model, but I could be mistaken.
Both matter, and returns fall off as you go further in one but not the other. The Chinchilla paper[0] established a simple scaling law for Large Language Models: model size and training tokens should grow at the same pace.
Compute is then proportional to the product of model size & data quantity.
That said, quality of data also matters a lot - OpenAI has had human labelers produce the data for their Reinforcement Learning from Human Feedback (RLHF), which has probably had a disproportionate impact on the success of ChatGPT compared to previous models, but that data is probably O(1%) of what they trained on.
At this point I'm guessing OpenAI are limited by both data & compute. Rumor has it they're training the "next big thing" now and it won't finish until December. If they had more compute they could presumably finish sooner, and if they had more data they would presumably let it train longer.
Also at this point, very few of the biggest players are going to tell us anything about which matters most and those fine tuning numbers can represent a huge strategic advantage. Forcing your competitors to spend billions in hardware and time can put you far ahead of them quickly, at least at our current rate.
AFAIK it's still an active area of research, and evidence from Meta AI [0] suggests that size and quality of data can let smaller (not necessarily less sophisticated) models do amazing things.
But a lot of the advancements we're seeing right now are the result of more sophisticated models [1], and one person is doing some interesting work [2] around achieving transformer-level performance with other architectures.
So it's not completely settled if more data is the answer. But it has a significant impact.
It's like a calf being born. It gets up and starts walking. Pretty amazing. Mesmerises everyone. The model contains everything it needs to know, to walk. But it's not going to dance nor is it capable of working out how to dance.
Babies do something completely different. They can't walk when born. Their model is to blunder about and work things out, building the model up thro a can I do this - can I do that - why not etc. Its only through this doing learning happens.
We have calf ai right now..you ask the calf what do you want to learn next or what are you curious about and you get to see how dumb it is.
So. The article starts with "I give it an estimate of 5 per cent chance..." and then explains: what if...
Is this case really worth exploring? Or was the article written by a bored AI?
I find it striking that there are still so many people downplaying the latest developments of AI. We all feel that we are at the verge of a next revolution on par or even greater than the emergence of the www, while some people just can't to seem to let it sink in.
That doesn't help your case :). Your intuition is not enough to say it's coming right at us. So where is the data? If there is no data, then it's just an intuition.
Even with or without data, every prediction is intuition. Data, if you need them, are the current events. But remember the quote 'Lies, damn lies and statistics'. It is the interpretation of the data that is what counts and that interpretation is partly subjective. Otherwise we would not be having this rather fierce but interesting conversation in the first place.
Well, it can be more or less backed by facts. That's the difficult part: making the difference between what seems like a reasonable prediction and what is merely a belief.
Climate change is coming at us. That's still a prediction (it hasn't happened yet), but not believing in it right now is irrational and dangerous.
I don't think we can say that for AI. That AI will change the world is a belief. Doesn't mean that it won't.
Crypto wasn't good enough as a currency, so what was coming in terms of that use case didn't pan out en masse. But it did work to an extent for some people.
Crypto's value fluctuation made more sense as a speculative asset class, and it clearly saw most of its traction there.
It's potential as a currency did lead to banks, financial institutions and governments taking a hard look at it. And CBDC's are an area of active investigation. So the hyper-optimists may have been off on some of their predictions, but something came out of it.
AGI-agencent tools are more grounded. They don't threaten to tear down existing entrenched institutions. They do one thing simple: commoditize and scale some level of intelligence. And they're pretty good at it, hence the hype.
Unless humanity develops a deep distaste for many things AI-made, the fascination will wear off and we'll be left with the cold hard truth of raw productivity increase and digital production on an unprecedented scale.
> AGI-agencent tools are more grounded. They don't threaten to tear down existing entrenched institutions.
I disagree with this part. (I disagree with calling this tech "AGI-adjacent", too, but that's not the part I'm calling out here).
I'm not sure what you mean by "more grounded", but if you mean "more based in reality", I don't think this is clear. But, if what the evangelizers say is correct, it absolutely threatens to tear down entrenched institutions. It even threatens to tear down society itself by destroying what little trust remains.
By "grounded", I mean the promises of crypto vs the promises of AI.
IIRC, crypto was a little too idealistic in the promise of providing a digital currency without needing a central, trusted authority to back it. From my perspective, AI as we've seen it in the last few months simply provides a tool that can automate intelligent tasks easily.
I think the distinction is fuelled more by the companies and leading figures pushing the latter. OpenAI's stewardship, concerns for safety and generally downplaying how amazing these tools are while not ignoring the real effects they could have make AI sound more "grounded" than peak crypto was. Or it could just be the folks I pay more attention to.
> I disagree with calling this tech "AGI-adjacent", too, but that's not the part I'm calling out here
I think this rests mostly on the definition of "General" here. Here, I'm talking about LLMs as general task performers, as opposed to models created for more specific tasks. LLMs have proven to be more general purpose, which I'd argue makes them closer to the AGI ideal than, say, a sentiment analysis model.
Yes it has, I see bitcoin ATMs all over the place and know that tons of people exchange cryptocurrencies daily for fiat and for other cryptocurrencies. A lot of people use it to gamble and buy drugs online. Some people are using it as a monetization scheme for their artwork (NFTs). The actual impact of those things is pretty small, but none of this was possible before their invention, so they literally changed the world, for better or worse (mostly worse).
basically this. the hype is real. the results are impressive.
but the examples of success so far seem cherrypicked leading to serious skepticism. there will be a revolution, and its already in progress dethroning entire business models but the future is still very uncertain.
what is humans role in this adventure into the singularity is even moreso clouded with more questions than answers
Because it is not clear if it is coming right at us or not, precisely.
Have you ever looked at predictions from big consulting companies like McKinsey and the likes? "This new field with unlock a market of 30 billions in the next 5 years", that kind of crap?
In the fields where I worked, I used to take those predictions seriously (after all, they know better than I do, right?). What I realized is that they just don't have a damn clue. I can't blame them for failing: predicting the future is an unsolved problem. But making that kind of money by telling that kind of crap should honestly not be legal.
All that to say, some things are clearly coming right at us (ahem, climate change), some aren't (AI, for instance).
This time it isn't a bunch of empty suits from McKinsey saying it. It's the technologists that built the Internet and smartphones.
By all means keep your own estimates of what the world will look like in 10 years and act accordingly, but I'm trusting my own technologist's judgement on this one.
I am not estimating anything. I'm saying that there is no way to know where it will go. You are the one doing what the empty suits did: you are predicting something based on your beliefs only.
Not that you can't do it: everyone can believe what they want. But it would be wise to realize that it's a belief, even if you are a technologist.
Could it be that the events at work have been coloring your perception by said consulting companies? Such companies are getting paid for making bold predictions, I would think. Of course Openai and MS as well get paid to say similar things. But this time the eating (usage) of the AI pudding already has the proof partly established. At least that is for me the sign to see it coming.
> But this time the eating (usage) of the AI pudding already has the proof partly established.
I don't know, I don't see it this way. Bitcoin at the very beginning was an impressive piece of technology (still is), but it clearly did more harm than good.
Autonomous cars had some very impressive demos, and don't seem to have evolved much in a few years (it's impressive, but not good enough).
AI is making very impressive demos now, but it's not changing the world (what seems to work really well right now is to generate convincing disinformation at scale, which is not good).
To me it's all the same: impressive demo, but what matters is not the first 80%, it's the last 20. Everyone seems to be building infrastructure around what AI will become. Just like people built entire companies betting on the fact that cryptocurrencies would become global (they have not), or that blockchain would be useful outside from cryptocurrencies (it is not). Or like people built entire companies around "the new world with autonomous vehicles".
>Autonomous cars had some very impressive demos, and don't seem to have evolved much in a few years
The problem is we got autonomous cars backwards. We needed multimodal GPT first. GPT-4 can look at a picture can tell you what's happening very clearly. Now, chain this together really fast and you have something that could be used for driving. I'm making no bets on when the hardware/models will be fast/cheap/power efficient enough to support this.
>but it's not changing the world
Eh, no, it is changed the world in a vast number of ways, but not spread very equally. On the scientific/AI side a whole bunch of "20 years away" and "We may never accomplish" have been changed to "now". On the language side things GPT has put any translation service on notice. Summerizers too.
I've never been a crypto fan. I believe there are purposes for it, but at the same time I have a bank that does all the same things.
What I do (did?) not have is an AI that actually solve real life problems that I run into now, and that's something that is both highly useful, and something that many people are willing to pay for.
lol, you may be right, but it’s both hilarious and terrifying to imagine that the future of self driving cars is effectively one person in the passenger seat verbally describing the road ahead to a blindfolded driver.
Self driving cars are not fully autonomous yet. It took longer than expected/promised, but we are only a few years away from that. Mind that technology in cars on average is 10 years behind on technology in advanced consumer products like cell phones, because of safety regulations and number of components involved. Some componies try to shortcut that lag with mixed results. But it is just lag. Delivery bots are a fact today already.
They have been "only a few years away" for over a decade. Delivery bots are not a fact today, they are not reliable on any scale of any practical sort without having humans in the loop.
My old eyes can't see perfectly anymore, so I would probably miss it - whatever it is - anyway, but if you're so sure, would you bet everything you own on it?
From TFA, a (somewhat infamous quote from Frank Rosenblatt):
> “[It] revealed an embryo of an electronic computer that it expects will be able to walk, talk, see, write, reproduce itself and be conscious of its existence. Later perceptrons will be able to recognize people and call out their names and instantly translate speech in one language to speech and writing in another language, it was predicted”
The section on "Past Winters" in TFA is a good part of the reason why you might want to not believe what you think you see coming right at us ....
What some are arguing is that the future is now. A 1-in-20 chance of another AI winter is not worth serious consideration given the enormous progress in last 5. The challenges do not involve hardware at this point.
You cannot say that with any certainty. While GPT4 shows great promise it’s still not clear at all that the current architecture can keep scaling up. Presumably they’re already using all the data they have access to. So what’s left to improve performance? Changes in architecture, some of which could require significantly different hardware.
The massive amount of money going into AI could invariably lead to the development of new and even more successful architectures. Although speculating on what and when would be like perfectly predicting the development of the Transformer architecture while living in 2010 or something.
Yes, I know. But there's no way to know if these people are right, right now. That will only be possible to know in the future, looking back, so it's still "predicting the future".
> The hard, cold truth is that nobody knows the future.
Probably, yes, especially after what happened to Cassandra.
Still a fun counterfactual to develop. Lessee, you do know the future, but do not run your mouth off. Matter of fact, you pooh-pooh any such notion ...
The time between rather large steps in advancement is small. So small that we think, oh yet another AI advancement, I'm getting used to this.
At first we were perplexed by a computer beating Karpov at chess.
These days however we're not perplexed anymore that a lot of creative jobs have been replaced by AI. (I think those people will find another passion no doubt, that is another topic).
Next thing is less creative jobs. But i'm already thinking about climate change and hunger mitigation. You could call me an idealist, only time can tell, of course.
GPT4 is clearly magic, no matter what mechanism backs it the fact that we managed to get rocks to think and reason about things in a convincing matter was absolutely unimaginable a few years ago even.
The momentum of investment even in known techniques makes it pretty clear that there will be even more useful things than GPT-4 available very shortly. Never mind in 10 years, with global training & inference capacity increased by multiples.
It's obvious at this point that we are figuring out how to create machines that think. While I agree that the future is unclear, in the sense that it's hard to make exact predictions, the trend is very clear. Significant efficiency improvements in many types of intellectual labor is almost certain.
I don't think it's priced in yet; the somewhat independent-minded among tech people are currently the only ones who have front-row seats to what's going on. But in a few years it will be obvious.
Honestly, I think the persistent exponential improvement in AI makes it obvious we're on track for superintelligent machines in a few decades as well (if that!), but the preceding paragraphs should at least be entering the realm of Overton.
When I first ran NCSA Mosaic, I dismissed it. It was another 5 years until it started to sink in — this was before google. Other than the hype, what are the latest developments of AI?
* generate digital images from natural language descriptions. ie. DALL-E
If code can be reliably generated and digital businesses deployed automatically, you can Monte-Carlo sample enterpreneurship. Automated ventures.
Edit: That may the lesson of Stable Diffusion: ideas are very cheap, if you have a heuristic for what a good idea is. Generate a business, fix it by optimizing for measurable metrics, if it doesn't work, try again. Legal constraints can be interpreted by an LLM, fines are penalties for training.
I think in terms of human ingenuity, the digital space is finished in the long run. It will all be on the level of infrastructure funneling data to corporations.
The space to innovate with the low-hanging fruit still lies in engineering physical things, where implementation and testing is still a barrier to iterating quickly.
Yes! You’ve mentioned people downplaying, but this is someone who is 95% confident there won’t be an AI winter in the next seven years. That’s really quite confident. And the definition he gave makes it compatible with AI being the biggest thing since sliced bread. But if it gets funded at 3x bread and scales down to 1x bread, that will count.
What cost was this exploration? A blog post? An HN discussion. I think it’s so valuable for people to consider the scenarios they think are unlikely. I find this analysis much more interesting than all the dismissive “my job is safe, AI can’t [friction-point-of-existing-system]” posts. It’s certainly worth the investment of a few hours of work.
Thanks. And I do share your surprise at the number of people downplaying it. It seems like some people are forecasting that things that happened last month won’t happen. That’s not a great way to get predictions right.
I'm actually fine with this argument. "I'll be fine, I can move up another abstraction level." is pretty solid.
You've made me realize the difference between the arguments I find weak vs strong. The weak argument is I can do something that ChatGPT can't. "I have people skills!". The strong argument is that as ChatGPT grows, I will grow with it.
The AI floodwaters are rising. People are climbing up the stairs when they should be building boats. If you seek shelter and are wrong about how far this goes, your attic will become a coffin.
Been noticing the same. There's this very unique flavor of comment, happening even on HN, where people downplay the level of impact of AI/MI/ML (whatever you're more comfortable with) not whether or not it will have any impact at all.
Mind you, there are always a variety of possibilities that people see, that is true, but this unique flavor of comment is unique because I haven't seen it expressed before. Not during crypto (as others are bringing it up).
It feels like there's a much more serious... insecurity? Maybe that's too aggressive of a word, but it's one that describes how this downplaying partially is rooted in this idea of a replacement, and lack of job security, and an uncertainty about what you should be spending your time on. All of this is so unique, nobody was feeling this way during crypto, not a single person. If you hated crypto, you were hating it because it sounds redundant, the "fans" were annoying, the environmental impact, etc - not a single one of these reasons came from this assumption that crypto will be the new norm.
So I think that's what a lot of the other comments are missing right now. The downplaying feels totally different now. As if even the people who are downplaying realize what's happening, but don't want it to be true.
I did like this blog post though, outside of what we're talking about.
A fair number of HN'ers are the "pull yourselves up by the bootstraps", "We don't need to stinking unions", "Social safety nets are for the weak" types. The potential for AGI causes a cognitive dissonance with them. They are intelligent enough to realize that AGI would nearly completely destroy their ability to make money, but at the same they don't want to release their ideals that "rugged individualism" is why they are better than everyone else, and they would need a more fair system to survive.
Neither of these statements are true and time will show the ignorance of it.
I'm a crypto and AI optimist, but a tempered optimist. I don't think either tech is imminently going to disrupt everything, but I think both technologies have their use cases. They are far more modest than their most ardent supporters would believe. However, over the course of decades, the tech can transform society, I still think that.
However, as with the internet itself, which never did live up to its loftiest, most optimistic ambitions, it's not to downplay the impact at all. Just thay eventually you do find a point where you grapple with the tradeoffs inherent in the technology and those are often sticky and intractable.
I was recently in a work meeting with some higher ups where we discussed the near-term future of AI. The idea that this is the interface revolution on-par with GUIs and the WWW was mentioned, and attributed to Bill Gates (I guess he must have talked about this recently). I've been dwelling on everyone's expectations for a few days and I see the hype cycle.
A good reason to consider "what if" is that it's still going to take a lot of effort to get from where we are to where expectations are. There's been many false starts recently: self-driving cars, 3D printing, VR/AR, crypto. 10 years on and they're all real technologies anyone can use and making progress every day, but the wild expectations hit reality. When we get there with AI (and we will, because expectations are a moving goalpost) it's good to have considered what we want to do about it. We don't want to waste years of resources on insurmountable roadblocks, we want ideas about why what worked before stopped and what easier paths forward might exist.
That is, I don't see many people downplaying AI but I do see people with lots of unanswerable questions about how long this current boom will last and where we'll wind up when it's over.
I agree and I find some of the statements a bit odd (not thought out?).
I think Moore’s Law could keep going for decades.[2] But even if it doesn’t...
If 1e35 FLOP is enough to train a transformative AI (henceforth, TAI) system,
which seems plausible, I think we could get TAI by 2040...
First, I don't see Moore's Law going sub-atomic without a complete change in methodology, which would delay the results. Wafer/die stacking are cool, but stop-gap measures. In particular he acknowledges that power consumption hasn't scaled since 2005 (in fact chiplets help by only sqrt2!). The challenge is that it doesn't help that much to increase the number of transistors, if their latency/power/cost don't fall as well. We're approaching that even ignoring any geopolitical issues.
Second, power consumption is not improving except by going to 8&16 bit... there's not a lot of room there. Currently, we get <1000GFlOPs/W and even if we get 100x up to 100TFLOPs/W, you still need 10^17kWhr. OK algorithms get us another 10,000x... and it costs $1T to train 1 transformative AI. How many proof of concepts trials will be do at only $100B a pop?
It all just seems a bit flip and hopeful like Feb 2000. The 10^35 number seems like its just pulled out to be a number, when it could orders of magnitude up/dn.
The major fabs all have roadmaps for approaching 1 nm, and there are other advances that could allow you to keep going either if transistor size scaling stops (e.g., vertical scaling). (That said, I definitely don't think it's a given that HW price-performance keeps doubling at the same rate 10+ years.)
> I find it striking that there are still so many people downplaying the latest developments of AI.
5-7 years ago the progress with self driving cars seemed enormous, and the end of driving was just around the corner. And then all progress seemed to stall or recede, and it turned out that what seemed like huge progress was mostly hot air.
Maybe that’s not what happening now, but I don’t think a cautious approach is unwarranted.
Progress is sigmoidal approaching a limit. If the minimum acceptable performance is into the logarithmic portion of the curve, you'll see progress seem to stall out in that manner. Keep in mind that SoTA self driving AI has a better track record than people outside of rare corner cases, but the liability issue is crippling, so it won't see widespread adoption divorced from autopilot until it's "perfect." Unlike full self driving, I don't think applications of AI have a functional floor that is well into diminishing returns.
Yep, but a lot of the current hype for GPT is that we're in the near-vertical segment of the sigmoid, and so people are like "woah, if that's where we are now, where will we be in 5-10 years?!". And the answer to that might be, pretty much where we are now. And maybe not, maybe it'll be better, but you can't assume linear improvements on a sigmoid curve.
Considering that AI is just "taking off" now, I'd say we're still pretty early in the growth curve (the alternate hypothesis, that people have been oblivious for a while, seems unlikely). Regardless, I think even if we were well into the linear portion of the curve, we could still do a lot with layers of tooling on top of marginal improvements on current models and technology.
We're in the hype cycle now at the moment, but the capabilities have been taking off for a few years. GPT-3 came out in 2020, and in retrospect, could have been trained several years before that, given that the 175 billion parameters of GPT-3 turned out to be way too many for its level of performance (as discovered by Chinchilla). I agree that there are a lot of refinements that can be made even with the current LLMs, but it's unclear how far LLMs will continue to improve.
When you go from doing 100 things to 10,000 things, it seems like a lot, until you realize you need to do 10,000,000,000 things to make the process work.
In theory if we could make multi-modal GPT-4 go very fast, we could have it operate a car right now. It's too bad we don't have access to feed it images, because my guess is if you put a picture of an inside of the car and showed a kid chasing a ball into the road in front of it, it's first guess would be "hit the brakes".
Indeed, the progress is exponential, but for things like full autonomous driving, we need near perfection, a lot better than the average driver. So does this mean 1k-fold better or 1M-fold better, we are not sure yet.
I agree with you.. but we're also seeing a lot of money get thrown at things which will ultimately be
- Redundant due to winner takes all dynamics.
- Useless
- Way to expensive for what it does.
- Produced off of sky-high seed rounds at 150 MM.
This same dynamic happened with the www leading to an eventual crash - the mobile and social versions would have been more noticeable had it not happened in private markets and simultaneously with the valuation spike of the late teens. It would be almost more surprising if there wasn't a pull back eventually.
This also doesn't say anything about the people who work in AI. It's not a given that the legions of research scientists populating corporate labs will be the right people for new startups. Similarly it may turn out that prompt engineering is the dominant method for talking to LLMs rather than code.
> Eden writes, “[Which] areas of the economy can deal with 99% correct solutions? My answer is: ones that don’t create/capture most of the value.”
And
>Take for example the sorting of randomly generated single-digit integer lists.
These seem like very confused statements to me.
For example, lets take banking. It's actually two (well far more) different parts. You have calculating things like interest rates and issues like 'sorting integers' like above. This is very well solved in simple software at extremely low energy costs. If you're having your AI model spend $20 trying to figure out if 45827 is prime, you're doing it wrong. The other half of banking is figuring out where to invest your money for returns. If you're having your AI read all the information you can feed it for consumer sentiment and passing that to other models, you're probably much closer to doing it right.
And guess what, ask SVB about 99% correct correct solutions that do/don't capture value. Solutions that have correct answers are quickly commoditized and have little value in themselves.
Really the most important statement is the last one, mostly the article is telling is the reasons why AI could fail, not that those reasons are very likely.
>I still think an AI winter looks really unlikely. At this point I would put only 5% on an AI winter happening by 2030, where AI winter is operationalised as a drawdown in annual global AI investment of ≥50%. This is unfortunate if you think, as I do, that we as a species are completely unprepared for TAI.
I'm not an expert, but I see the main threat to continued improvement as running out of high-quality data. LLM's are a cool thing you can produce only because there is a bunch of freely available high-quality text representing (essentially) the sum of human knowledge. What if GPT-4 (or 5, or 6) is the product of that, and then further improvements are difficult? This seems like the most likely way for improvement to slow or halt; the article cites synthetic data as a fix, but I'm suspicious that that could really work.
Even just finetuning these models, they will pick up on the weirdest things a human wouldn't see, and artificial datasets will contain all kinds of invisible artifacts that further "inbreeding" is going to massively amplify.
I am not speaking speculatively either. I have seen it happen finetuning ESRGAN on previous upscales that I even personally vetted as "good," and these generative models are way more sensitive than the old GANs.
More than that, I believe we will hit a ceiling when the impossibility of these models to incorporate causality becomes evident.
Right now, LLMs are trained by predicting the next word given a context (the prompt). This approach, IMHO, gives a resemblance of cause/effect because the training data is made by humans and obviously we are able to express ourselves and reason in those terms. So we have a poor proxy for that which, also IMHO, partially explains why LLMs performance degrades when asked to solve novel problems (there was an entry few days ago about this).
After seeing LangChain and the reasoning paper I think it's fairly obvious that we're just starting to scratch the surface of AI architectures, dictated largely by scalability of GPU resources. The LLMs we're playing with are at best the proof of concept of what will eventually be the message passing for the next generation of models.
That we are running out of data makes continued improvement more likely, in the medium/long term, because it means we have found close to the maximum amount of static data that models need to be trained on. This means that one of the factors for compute requirements (static training data size) has reached its upper bound while still within our computational capacity today, and further improvements must come from elsewhere.
So yes, easy data scaling is coming to an end and may or may not lead to a short term winter. But this also means that, all being equal, training a model will be cheaper compared to a situation where we still had orders of magnitude more data to go.
I've just spent a 4-5 hours session with ChatGPT trying to fix a problem with `aiohttp`.
The specific problem was what while I'm in a WebSocket handler, I cannot await on asynchronous generators which act like `asyncio.sleep()` and send messages to the client while I have `heartbeat` on that `WebSocketResponse` enabled (it sends pings to the client and waits for pongs to see if the connection is "alive"). The issue is that the incoming pong is not getting processed so the client gets disconnected.
I struggled a lot with ChatGPT's help, but it was mostly insightful, like rubberducking with a duck that does actually try to help you.
Occasionally I went to Google to search for a very specific way of solving the issue; for me Google is the gateway to Stack Overflow, I never use Stack Overflow's search.
But I didn't find anything of value there. And earlier today I noticed that I had unread messages in Stack Overflow, and when I checked how many consecutive days I have been on SO: 1. I used to be 100+ consecutive days on SO and lately I'm struggling with using it as the to-go place for my programming questions.
And this is where your comment comes in: I was thinking to myself what is going to happen if more people move away from Stack Overflow, that place is a goldmine of programming information, who will feed it?
One thing I really hope is that OpenAI adds some "modes", like what were poised to expect from Copilot X, where we get different layouts for the webpage, so that we can choose a "programmer mode" which actually lets us give proper feedback in the sense that "this code worked", "that one didn't".
There's already a feedback, but it's not task-oriented. A programmer can submit to it as well as a cook, the result won't go into a "database of usable code" which ideally would be publicly accessible or even get formed into a Question/Answer pair which is prepared to be submitted to Stack Overflow.
It may become a norm to write AI QAs instead of docs for projects. Maybe similar to sitemaps for web scrapers. But even this will not be necessary at some point.
Stack Overflow will degrade to single search box. And we'll all love it.
I see no lack of high-quality data, although it may not be the kind you prefer. How about the telephone/chat/internet logs (not "metadata") of every single person in the USA? We've got it! And how about using that to answer social science questions and, more immediately, political questions such as:
- What really happens in corporate/government/private decision-making?
- Did a particular politician really trade his vote for money? How was that done?
- Are elected judges better/more honest than appointed judges?
- Are there hidden organizations within particular governmental entities? e.g., a right-leaning group within the FBI who secretly persecutes persons/groups of other political persuasions? Are there independent entities within the CIA that are capable of financing and operating on their own under the umbrella of the government but also capable of evading the congressional oversight specified in the Constitution?
- Is my wife's cousin really screwing Hunter Biden again?
It's like worrying about running out of stables for horses when cars already started to take over.
It's going to shift from requiring quality data to producing quality data.
There is not going to be need for it just like there is no need for more Kasparovs anymore.
You can see it with artists slowly happening.
Lawyers will get the hit (or great tool to use depending how you want to see it).
Medical analysis will find the same faith. If you think that medicine will require human analysis just imagine for a second what closed loop AI could do - if it had access to data not only from all hospitals but patients as well and being able to munch through it continuosly. This together with constant access to simulations and physical trials, finding patterns and correlations on its own.
It's exciting and depressing to think that humans will evantually be left to being humans the way they wish with everything sorted out - just like chickens don't mind being free range chickens.
There is no other direction it can go and it'll just go faster.
Our generation will see some mind blowing things.
Next generation will arrive at the world unlike anything else.
> It's going to shift from requiring quality data to producing quality data.
synthetic data gives you synthetic results.
To train something requires decent quality input, otherwise it's going to mimic the crap quality stuff. There is little chance in getting around that.
Have you ever stopped to wonder _why_ large for profit companies give away free models with weights? its because once trained, they are commodity. the hard part is dataset management.
Yes, chatgpt is largely self supervised, but the training into a usable model required a fucktonne of human hours
AI winter will arrive if we don’t get the models to depend on ‘just’ more training data to get better. There is no more training data. We need models the same or better than gpt-4 but trained on roughly what an 18 year old >100 iq human would digest to get to that point. Which is vastly less than what gpt 4 gets fed.
If advancing means ever larger and more expensive systems and ever more data, we will enter a cold winter soon.
There is nearly an unlimited amount of training data. As of so far models have been eating up text. We still have sound, image, video, temperature, gravimetrics, and other sources that we can feed multimodal models. And that's not even including training models from learning from the world itself.
Sound, image and video we’ll ravish through (already happening); is temp and gravimetrics valuable? (I don’t know; it’s a question) Games might be a source too. But yes, the world itself is a good source. So maybe that won’t freeze it over; computing power/energy?
Energy is "probably" not a big deal. If you're looking at long term oversupply from green energy sources, it's probably not hard to train with bursty, but very cheap power like this.
Compute is currently the biggest limitation, and will remain so far a long time, as long as scaling continues.
Calling it now: humanity will seal its fate the day we hook up LIGO to ChatGPT and it gets corrupted by gravitational waves from the beginning of time when the Great Old Ones freely roamed the universe.
You are confusing the entirety of all text corpus to _all data_.
The next step is multi-modal data. Think videos, sounds, touch, etc.
An 18 year old in fact "trained" on much more input data than GPT4, orders of magnitudes more, considering the complete informational content of our 5 senses and implicit structure of our brains "learned" through evolution.
Not to mention recent research in LLM scaling (chinchilla) reveals how our current models are undertrained and over-parameterized.
What if it can annotate itself ? With some feedback loop it can learn the correct annotation by experiments like a child. Earlier there was a comment of an agent learning to run a python program by itself and in the process of writing code to file and install python interpreter.
I think the assumption that companies are willing to spend 10 billion dollars on AI training is unrealistic. Even the biggest companies would find such an investment to be a financial burden.
You're "probably" right, but it's not something I'm going to place bets on with the level of uncertainty. When you have some companies still sitting on war chests of tens of billions of dollars, and almost nothing to spend it on, not investing in AI is a risk in itself. If your competitor succeeds they may rapidly take parts of your market while you now attempt to reproduce their work.
Also if these 10 billion dollar models are 'AGI' level, then your model pays for itself if you can find enough interesting work to throw at it.
If they really believe that it will bring appropriate returns, a $10B investment is manageable. It is a significant burden, but even much larger investments have been made - e.g. Starlink satellite fleet on the private side, or the Large Hadron Collider from government funds.
Write a counterpoint to the article posted. Your goal is to refute all claims by giving correct facts with references. Cite your sources. Make it 3 paragraphes. As a poem. In Klingon.
We're only just seeing expectations for the tech inflate now. VCs will probably pump money into LLM-related companies for at least a couple years, and it'll be a couple years after that before things really start to decline.
It's late spring right now, a strange time to start forecasting winter.
I think the AI winter will come, but not for why the author asserts (quality, reliability, etc.).
I think the current crop of AI is good enough. It will happen because people will actually grow resentful of things that AI can do.
I anticipate a small, yet growing segment of populations worldwide to start minimizing internet usage. This, will result in fewer opportunities for AI to be used and thus the lack of investment and subsequent winter.
Being old as dirt, my observation is that potential tech revolutions take ten years after the initial exuberance to be realized broadly, or three to five years to fizzle. Of those that fizzle, some were bad ideas and some were good ideas replaced by better ideas. FWIW
Well, during the initial "tech" (broadly interpreted to mean computational processes that somehow involve the internet) revolutions, there were not 100M, nor even 1M users to be gained.
And what is a "user" when it comes to ChatGPT and it's ilk? How does that compare with the definition of a user of, say, the web? Or of SMS ?
With most other technologies before this there was the issue with rollout of physical infrastructure. And that is still true somewhat. All the big AI players want more AI processors and would take possession of as many as they could get.
Buy the other side of physical infrastructure is already here. The internet, cellphones, and computers already exist in mass and can be utilized by AI now. We didn't need to build new highways for this. No new towers. No wires put in the ground. Those fields already exist and are ready for planting.
The "winter" analogy (I remember the AAAI when marvin made that comment) was to the so-called "nuclear winter" that was widely discussed at the time: a devestating pullback. It did indeed come to pass. I don't see that any time soon.
I think the rather breathless posts (which I also remember from the 80s and apparently used to be common in the 60s when computers just appeared) will die down as the limits of the LLMs become more widely understood, and they become ubiquitous where they make sense.
idk if there will be that much of a winter, but i would welcome it
in the late 90s and early 2000s, neural network had a significant stigma for being dead ends and were unpromising grads were sent - people didn't want to go there because it was a self-fulfilled prophecy that if you went to research ANNs then you were a loser, and you were seen as such, and in academia that is all you need to be one
but, in real life, they worked
sure, not for everything because of hardware limitations among other things, but these things worked and they were a useful arrow in your quiver as everybody else just did whatever was fashionable at the time (simulated annealing, SVMs, conditional random fields, you name it)
hype or no hype, if you know what you are doing and the stuff you do works, you will be okay
The dissonance what people see in it is really gross.. maybe I'm too stupid, but session from simple to more complicated problems have all shown me: It is definitely not reasoning at all, it doesn't understand anything.. it is a different version of stackoverflow that sometimes gets you quicker to target, but for me even more often not.
IMHO the prospect of an AI winter is 0%. As someone who has done research in ML I think ML technology is moving forward much faster than we anticipated. ChatGPT shouldn’t work based on what we knew. It’s incredible that it works. Makes you think what other things that shouldn’t work we should scale up and see if they would work. And then there are things that we think they should work. Each new paper or result opens the door for many more ideas. And there are massive opportunities in applying what we already have to all industries.
You can absolutely build high precision ML models. Using a transformer LM to sum numbers is dumb because the model makes little assumptions about the data by design, you can modify the architecture to optimize for this type of number manipulation or you can change the problem to generating code for summing values. In fact Google is using RL to optimize matmul implementations. That’s the right way of doing it.
> As someone who has done research in ML I think ML technology is moving forward much faster than we anticipated.
The past AI winters have been preceded by periods of AI moving forward faster than anticipated. Its when the limits of easy advancement with the current approaches are reached without a new approach that allows continued rapid progress you get an AI winter.
I guess because we still don’t understand why it works. We still have no basis on which to predict that it works as well as it does, except for the manifest fact that it does.
I give it a 95% chance that an AI winter is coming. Winter in a sense that there won't be any new ways to move forward towards AGI. The current crop of AIs will be very useful but it won't lead to the scary AGI people predict.
Reasons:
1) We are currently mining just about all the internet data that's available. We are heading towards a limit and the AIs aren't getting much better.
2) There's a limit to the processing power that can be used to assemble the LLM's and the more that's used the more it will cost.
3) People will guard their data more and will be less willing to share it.
4) The basic theory that got us to the current AI crop was defined decades ago and no new workable theories have been put forth that will move us closer to an AGI.
It won't be a huge deal since we probably have decades of work to sort out what we have now. We need to figure out its impact on society. Things like how to best use it and how to limit its harm.
1) Is everything really being mined already? My sense is that another round of GPT-like training on YouTube, EdX and Coursera data + some other large video archives (BBC and the like) could still make quite a bit of difference. Text and images independently is one thing, having them together in context might be something else.
2) The available power seems to be growing pretty rapidly and dropping in prices. I think there are still quite some gains to be had from architectural optimizations (both in hardware and in models).
4) They were defined decades ago but I think they did actually seem to move us closer to AGI only recently.
You might be right, and there are definitely interesting times ahead! But I kind of doubt that we will have decades to sort out what we have and figure out its impact on society (which is a bit scary).
Some of these are good arguments and can turn out to be true. But they are equally likely to be false.
1/ We humans are still generating data. To live is to generate data and while it is dystopian to think about how these data can be harvested, it is still possible to get more information to feed the next gen AIs. And remember that right now, we have only used text and images. Videos, audio, sensory inputs (touch, smell, taste, etc), and even human's brainwaves are still available as more training data. We are nowhere close to running out of stuff to teach AIs yet.
2/ Fine tuning and optimizing training has shown tremendous effects in reducing the size of these LLMs. We already have LLMs running on laptops and mobile phones! With reasonable performance and in only half a year from the big release. There is lots of room to grow here.
3/ My nieces laughed when I told them TikTok is too invasive. Most people outside of HN does not care about data privacy as much as you might think.
4/Sometimes it only takes 1 big breakthrough to open the floodgate. Transistors was that point for computers and we are still developing new techs based on that 100 years old invention. We don't know how much potential there is in these decades old AI invention especially when many of them was only put into proper practice the last decade. We didn't learn the big mistake until very recently after all.
Just some ways things can go differently. It is the future, we can't really predict it. Maybe an AI can...
>4) The basic theory that got us to the current AI crop was defined decades ago and no new workable theories have been put forth that will move us closer to an AGI.
I guess it really depends on what you mean by "basic theory" but my view is that the framework that got us to our current crop of models (vision now too, not just LLMs) is much more recent, namely transformers circa 2017. If you're talking about artificial neural networks, in general, maybe. ANNs are really just another framework for a probabilistic model that is numerically optimized (albeit inspired by biological processes) so I don't know where to draw the line for what defines the basic theory...I hope you don't mean backprop either as the chain rule is pretty old too.
We are currently mining just about all the internet data that's available. We are heading towards a limit and the AIs aren't getting much better.
The entire realm of video is under-explored. Think about the amount of content that lives in video. Image + text is already being solved, so video isn't the biggest leap. Embodied learning is underexplored. Constant surveillance is underexplored.
There's a limit to the processing power that can be used to assemble the LLM's and the more that's used the more it will cost
If the scaling law papers have shown us anything, it is that that the models don't need to get much bigger. More data is enough for now.
People will guard their data more and will be less willing to share it.
Fair. Though, companies might be able to prisoner's dilemma FOMO their way into everyone's data.
was defined decades ago
The core ideas around self-attention came about around 2015-2017. The ideas are as new as new ideas get. It's like saying that the ideas for the invention of calculus existed for decades before Newton because we could compute the area & volume of things. Yes, progress is incremental. There are new ideas out there, and we'll inevitably find something new in 20 years that some sad-phd is working on today, all while regretting not working on LLMs themselves.
also most scientific data lives in silos somewhere unknown. i went to a really interesting talk last year where a scientist in uzbekistan saved a closet full of notebooks from old professors at his university. They had meticulously catalogued hydrology data, photos of glaciers, etc. and suddenly there was daily data going back to basically the formation of the soviet union for a region that had never had much data before. the department secretaries were going to burn all this as trash. stuff like this exists everywhere including on the internet.
EXACTLY! there's so much research and citizen science being done, and even the field of CS is no good at identifying, preserving, and disseminating the good ideas that our lossy proxies miss. (proxies like "university prestige" or "went to grad school" or "was accepted at top-tier conference")
I think OP was talking more about upending the universal dogma of "statistical optimization to minimize feedforward loss functions via gradient descent." We now have plenty more tricks of the trade (ReLUs were a big one, self-attention was another, making things convolutional was a third, and the smaller ones like dropout/batchnorm/LR schedules). However, in broad strokes, this paradigm remains relatively unchanged since LeCunn+Bengio+Hinton in the 80s. The fundamental idea informally stretches back to Rosenblatt's perceptrons in the 50s, which itself was built on McCulloch and Pitts' modeling of neurons as multiply-accumulate units from 1943. We've spent lots of time questioning these assumptions (e.g. branching off into Boltzmann machines or spiking neurons, which are really fascinating), but to my knowledge, there haven't been promising alternatives yet.
We have "something that appears to work" but is this itself a local minimum? Biology found its own efficient processes that look a bit different after all.
In my opinion, "fundamental breakthroughs" would answer questions like:
- is there an inherent reason that humans learn through a process akin to statistical optimization, or is it a coincidence that feedforward-only gradient descent seems to work?
- Are there other models for "neurons" that go beyond GEMM, like spikey activations? We know from biology that they're nonlinear; how might they be modeled better?
- Are there ways of training effective agents beyond just the reinforcement Q-learning techniques that we've settled on?
- What does it even mean for a model to be able to "reason;" does our current view of recursive models with hidden state really map to that philosophy one-to-one or is there more?
But he has a very good point. Transformers are such a quantum leap over anns, it's almost a whole new paradigm, Karpathy calls them a brand new way to do computation
Right, but transformers are artificial neural networks though, so it's not clear to me what you mean.
The leap from RNNs to the Transformer architecture in 2017ish is similar to vision networks' leap from fully-connected layers to convolutional layers in 1987ish. In both cases, the key is to change the architecture to exploit some implicit structure in the data that wasn't quite used effectively before.
For images, each pixel is strongly related to its close neighbors, so convolution is a natural way to simplify models while capturing that locality.
For language, the opposite is true - almost any part of the text implicitly reference anything that's been said previously, so you need representations that are somewhat position-independent. Approaches that can model that structure naturally work better than an RNN munching tokens one by one.
Each one was certainly a leap forward that unlocked its respective field, but at the end of the day, it's "just" an architecture tweak, you know?
1. I'd wager AI models will begin to learn via interacting with the world rather than just reading lots of text. That will reduce the need for a huge corpus of training data.
2. More efficient methods of training and running LLMs are emerging at an exponential rate
3. There's already enough data to train an AGI.
4. Transformer architecture isn't that old
> 3) People will guard their data more and will be less willing to share it.
I recently came across this myself when writing a reply on another forum. A feeling of reluctance to attempt to contribute something maybe-useful in a public, 'minable' space.
Almost feels like this has tarnished the 'magic' of Internet, of sharing information and knowledge. I'm not saying this is the appropriate reaction, but it is what it is.
> I recently came across this myself when writing a reply on another forum. A feeling of reluctance to attempt to contribute something maybe-useful in a public, 'minable' space
I used to be like this, but the reality is that ideas are a dime a dozen. Any idea that you or I (or anyone else) have had, have likely also been thought by thousands (or maybe add a few more 0's) of people.
What makes things happen is people not ideas. Talk to any VC or people involved in startups - it's all about the team, not the ideas they are working on.
5) AI-generated content will become more and more common over time. This will inevitably end up in the training sets of future AI. By recycling its own training material, AI will get less meaningful results over time.
>By recycling its own training material, AI will get less meaningful results over time.
I read this a lot, and it sounds intuitive on the surface. But I don't understand how it's justifiable. For example, all that exists in the Universe is a result of the application of very simple rules. It would make sense that information is not what is important towards intelligence and complexity, but computation. It ought to be possible to create a superintelligence with a few bytes of training data, given enough compute.
Computers are electrical automatons that process binary signals and little else. Without a human at the end of the chain to interpret these signals (whatever representation they’ve been given), there is no meaning whatsoever.
So an AI that simply recycles its own input ad infinitum might produce something but it won’t be meaningful to us humans. Hence, it’s unlikely to be useful apart from the novelty of it.
Current approaches to ML are largely in the camp of throw enough "found data" at it and hope for the best. The exceptions are mostly games, like AlphaGo, where the ML can play against itself.
For generative AI, the hallucinations will poison the well. And they're not random, so same/similar hallucinations will pop up all over and reinforce each other.
> The current crop of AIs will be very useful but it won't lead to the scary AGI people predict.
I argue that this won't matter. I think Generative AIs will be much scarier than any sci-fi AGI. And this is because while sci-fi AGI apocaliptic scenarios involve the AGI seeking to exterminate humanity usually as a form of revenge (see The Matrix, The Orville and many others) or as part of an optimization, scenarios involving Generative AIs simply involve humanity destroying itself, and these scenarios are already very plausible.
Prepare for the complete destruction of objectivity, the onslaught of spam, grifts and fake news at an unprecedented scale and a flood of security vulnerabilities unlike any other. This will be the annihilation of interpersonal trust and of knowledge.
I mean, imagine for a moment that Russia or China were to spend a few tens of billions of dollars on a cluster to run an AI.
They have the human capability to ensure that it is state of the art in all current ways, and could theoretically use some processes that aren't disclosed to the public. I don't think anyone would say that China at least doesn't have the brainpower to do something like that.
Then, these government leaders could ask the AI to come up with simulations on how to conquer the world.
The AI would probably say something like, "America has to fall, since it's military might is the strongest resistance to your ability to dominate the globe".
How would you do that?
Ensure weak or compromised politicians get elected, which will make the American people lose confidence in their government.
Spend resources to ensure that the financial stability of the country is negatively affected. Do this by disrupting the flow of goods into the country, viciously competing with any product based company of any merit, and purchasing housing in the major financial centers of the country.
This serves three purposes:
1: Businesses will become tight-fisted with their spending wherever they can as they won't always know if they will be able to purchase the goods they need when they need them. The easiest way to reduce spend in the short term is to underpay and overwork your employees by firing the more expensive employees and shifting their workloads to others, so that will be the first thing that happens.
2: The smarter and capable people will attempt to escape the rat race by relying on their inventiveness. They will suffer in drudgery for years while they perfect a product that someone needs and is willing to pay for, and then, since manufacturing in America is ridiculously expensive thanks to China's price competition, they will outsource the manufacturing of their product to China. This will happen frequently, and they will be taking all of the risk for themselves.
Once they have a successfully tested product, China will clone the product and flood the market with inferior quality but massively cheaper clones, ensuring that the original business never achieves the success it could have and then has to cut costs, putting the employees of that company into the same conditions that the founder started the company to escape.
3: Housing will become very expensive, so the citizens will not be able to afford the same quality of housing that their parents did. This will further increase their unhappiness and destabilize the power of America.
Once all of that is done, then you let the kettle simmer for a while. Spend the money, time, and energy to keep the pressure up. Use the time to buy or subvert more politicians.
When this is done, the country will be weakened by its internal stresses and so charged that all it will need is a single strong event to vent all of that pressure on. A match to a powder keg will blow the whole thing up and America will collapse from civil war.
While that civil war is going on and the whole world is watching, launch your attacks. Take over your neighboring countries. Threaten nuclear annihilation on anyone that stands against you as within a few years you quadruple the size and power of your country.
By the time America pulls it's head out of the fog and realizes what is going on, it will be too late.
>We are currently mining just about all the internet data that's available.
If AI starts to become economically important, there will be an incentive to create more data. If I'm OpenAI, why not pay a company to put microphones around their office, transcribing everything everyone says for training data. Buy troves of corporate (or personal) emails. If there are one hundred million office workers in the world, and you can convince/pay 10% of companies to let you spy on them, and each says or writes 5,000 words per day (these are all low estimates, in my opinion), you'd be getting ten billion more words of data per day. You'd double the size of the Chincilla training data in about a month.
If you incorporate video for a truly multimodal model, now you're talking about not only the entirety of Youtube, but potentially also CCTV footage. If you could get the data from every surveillance camera in the world, you'd generate one YouTube worth of video every ten minutes (based on 150 million hours of video on YouTube).
Imagine Open.AI purchasing troves of corporate data from asset liquidations following bankruptcies, and the AI inadvertently becoming incredibly skilled at bankrupting corporations.
1) The AI's aren't getting much better since when? Last week?
You appear to be assuming that the only way to make current LLM-based AI's better is by feeding them more and more data, but that's simply not true. These current (really first generation, not withstanding GPT's versioning) AI's are very data/parameter inefficient, and do not differentiate between data used to develop reasoning and data that is just pure knowledge. The short-term path forward is more judicious training set design to use less data, and require less model parameters, so that the model itself just learns to reason, and doesn't need to do double-duty as a database repository of facts as well. Retrieval-based plugins and similar approaches can be used to give access to external knowledge.
2) There will be many different sizes and capabilities of AI. They don't all need to be superhuman.
We are entering the AI age and just as the computer age didn't result in everyone needing a mainframe or datacenter in their garage (although there are uses for them), the AI age will not result in everyone running the world's most powerful super-human AI on their smartphone. There will be a range of AI's for different uses cases.
Current models are also very inefficient. We are essentially day 1 into the NN/LLM AI era. Think of ChatGPT as the ENIAC of the age. DeepMind's Chinchilla scaling laws have already proven how inefficient these systems are, with a suitably trained 70B model equaling the performance of a 175B one. There is tons more work to be done on efficiency and smarter model design.
3). Perhaps, but per point 1) we're not running out of data anyway, and there will anyways always be people/companies happy to sell data for a price, even if it is not for free.
Future AI's will learn the same way we do by experimentation and curiosity/surprise, and will not need to be spoon fed a training set in the same way as today's systems.
4) No idea what you're talking about here. NN optimization is hardly spent as a technique - that'd be like saying "we've reached the limit of what we can to with software development - we need a new idea".
LLMs as a path to AI, or building block of AI, are of course a very new idea - a few years old at best. A few decades ago no-one was even working on neural nets (current DeepLearning revolution basically started with AlexNet in 2012) .. it was all GOFAI symbolic systems - failed approaches like SOAR and CYC.
If you compare the Transformer architecture to the human cortex, and our brain's overall cognitive architecture, some of the missing pieces are extremely obvious. We're not at any sort of impasse as to how to move things forward.
I agree with this perspective. But I also see the current crop of AI as a "brute force" method towards AGI, which could still be combined with a symbolic system to make it significantly more efficient at basic reasoning. I sadly don't know why certain neural-symbolic aproaches haven't been explored further despite their incredibly promising results in the past.
270 comments
[ 5.5 ms ] story [ 299 ms ] threadIf they effectively shut out other hardware companies, that is going to slow scaling and price/perf reduction.
Foreign nations aren't going to be happy about one mostly US company holding all the cards too.
I think it’s unlikely, but no less likely than the compute issues mentioned.
This isn't like the manhattan project. There are lots of people who know how to make this stuff, and they don't need rare volatile elements - just consumer hardware.
And no, the big models require exabytes of processing power and time, so at least at the most extreme scales if nations started punching missiles in processor factories and data centers you'd slow top end AI projects way down.
Yeah, but we weren't. We were talking about chatgpt and llms.
Of course if the singularity arrives, all bets are off, all dominoes will topple, and the new world order will be hard to guess, if there even is one.
If Russia and China think the US is about to have AI capable of absolute total global domination they may launch a preemptive strike. Maybe hypersonic cruise missiles at datacenters, maybe a full EMP or nuclear launch.
(Swap countries around as desired.)
The Corporate desire for profit will overshadow the livelihood of billions. It's been this way in the US since forever. Look what happened to the corporation that caused the Opioid epidemic. Nothing, they profited.
Then what do you want me to say? It's the truth. There are always exceptions. Always. You want me to say you're right when you're actually wrong?
> but Norway and Denmark are
You kidding? Scandinavia is more or less a collection of countries closest to socialism. These countries have by far more regulations in GENERAL. It's obvious but we can find evidence if you want. Take this for instance: In Sweden, there is a law enforcing a five-week vacation policy.
>The US has already demonstrated the ability to regulate a specific technology--namely nuclear energy--to a point where it is almost completely marginalized. So the notion that the US isn't capable of regulating an entire technology into near-oblivion is demonstrably false.
Except there are multitudes of failures as well. The failures outnumber the successes by a huge margin. Take for instance:
It’s disingenuous to make an overly general statement and then immediately dismiss any counterexample to that overgeneralization by saying “there are always exceptions”. I could just as easily say the United States is chronically overregulated and your list of examples are the exceptions.
Where did you get that by the way? I can’t find any exact phrase matches online, and the writing style and overall glib superficiality reminds me of ChatGPT output. I’m not going to waste my time rebutting machine-generated garbage point by point, especially when half of it is meaningless weasel language about what “critics argue”.
Ultimately it doesn’t matter because if there are exceptions either way, then it’s still possible AI will be regulated.
> You kidding? Scandinavia is more or less a collection of countries closest to socialism.
A common misconception. In reality, they are market economies that combine a dynamic free market economy with a generous welfare state. Wikipedia even specifically lists as a characteristic of the “Nordic model”, “Little product market regulation. Nordic countries rank very high in product market freedom according to OECD rankings”(https://en.m.wikipedia.org/wiki/Nordic_model). It also mentions this quote:
The Heritage Foundation publishes an “Economic Freedom Index” that ranks countries by “economic freedom”, according to their own conservative, pro-free-market point of view, and they rank Denmark and Sweden as more economically free than the United States, both in the general index and in the more specific index of “business freedom”, which seems to be the measure relevant to regulatory burden in particular (https://www.heritage.org/index/)> Given the sheer amount of counter examples of lack of regulation. It's pretty much safe to say that it's highly unlikely AI will be regulated in any meaningful way.
So then how do you explain the regulatory crippling of nuclear power?
I got it from my notes for an unrelated project. You can't find it because I wrote the notes. But this shouldn't matter. As long as the points are correct, you getting schooled by an AI is irrelevant to the conversation at hand.
>It’s disingenuous to make an overly general statement and then immediately dismiss any counterexample to that overgeneralization by saying “there are always exceptions”.
No it's not. General truths exist. You must dismiss exceptions to get at the general truth. Otherwise we'll be mired in details constantly.
>Denmark is a market economy.
I never said it wasn't. I said "closest" to a socialist economy.
>A common misconception. In reality, they are market economies that combine a dynamic free market economy with a generous welfare state.
No misconception made here. You are putting words in my mouth. I said it was closest to socialism. And it holds true. A generous welfare state is closer to socialism.
>The Heritage Foundation publishes an “Economic Freedom Index” that ranks countries by “economic freedom”, according to their own conservative, pro-free-market point of view, and they rank Denmark and Sweden as more economically free than the United States
That's bullshit. The heritage foundation is a conservative think tank with a biased agenda. How about getting a study from an unbiased source. "Economic Freedom Index" << don't fall for that.
Check out which countries rank the highest for food regulations: https://www.fooddocs.com/post/food-safety-standards
The US isn't even on that list because we've lobbied the hell out of those laws to be lax af.
Not to mention labor regulations, mandatory five week vacations? Paternity leave? Unheard of in the US.
Just use some common sense before trusting some "Freedom" Index from a politicized foundation.
>So then how do you explain the regulatory crippling of nuclear power?
It's an exception. I mean I literally gave you tons of examples how the US fails to regulate things. The ratio of failures to successes is what matters here. And the failures outnumber the successes by a huge amount. I only copied a fraction of my notes. Would you like more?
The points don't specifically pertain to the US and have enough garbage weasel words that they don't even rise to the level of "correct". I'll make another comment that goes through them though.
> General truths exist. You must dismiss exceptions to get at the general truth. Otherwise we'll be mired in details constantly.
You can't just handwave away complexity. It's entirely possible for the US to have too little regulation in some fields and too much regulation in other fields.
Your claim is that it's impossible for the US to erect regulatory barriers for AI. Whether or not the US, in general, has more or less regulations than other countries isn't sufficient to make that case.
> The heritage foundation is a conservative think tank with a biased agenda.
I never claimed otherwise. But why would a conservative think tank, which favors less business regulations, not even put the US on the top ten list of most "economically free" countries if the US is so underregulated? Why would they prefer the regulatory environment in Denmark and Sweden over the regulatory environment in the United States?
> Check out which countries rank the highest for food regulations
So when I bring up examples they're "exceptions", but while you bring up examples, they're examples of the rule. When I cite sources that do an overall survey of a country's regulatory environment, that's "biased", but when you cite a source that makes their money by helping companies comply with food regulations, that's just fine.
> Not to mention labor regulations, mandatory five week vacations? Paternity leave? Unheard of in the US.
Yeah, there are some places where the US has more regulations and some places where the US has less regulations.
If we're talking about regulating AI, I think nuclear energy regulations are a much more analogous case than vacation and paternity leave.
> It's an exception. I mean I literally gave you tons of examples how the US fails to regulate things. The ratio of failures to successes is what matters here. And the failures outnumber the successes by a huge amount. I only copied a fraction of my notes. Would you like more?
I can run ChatGPT myself, thanks. Why don't you try thinking for yourself and considering the possibility that your presuppositions are wrong?
I didn't handwave anything. My answer is sufficiently complex with multitudes of counter examples to your point.
The whole thing with the massive list of examples is to illustrate a general point on the lack of business regulation overall in the US.
I literally stated it's the ratio of failures to successes that matters here. If I can produce 30 examples of the US failing to regulate and you produce one, that speaks to an overall generality that eclipses your example.
The arena of scientific validation is hard to establish here. Neither of us can paint a picture of the entire domain of every single failure and success of regulatory laws in existence for the US. So given the nature of this debate just list as many general examples as possible.
You have nuclear power as one, that's it.
>I never claimed otherwise. But why would a conservative think tank, which favors less business regulations, not even put the US on the top ten list of most "economically free" countries if the US is so underregulated? Why would they prefer the regulatory environment in Denmark and Sweden over the regulatory environment in the United States?
Don't know. The motivations of such groups are complex and multifaceted. Following some breadcrumb trail to get at the root of it is too much effort. I only know that this group is biased and not a neutral party. There's no point in vetting a known compromised source. Pick a valid one.
>So when I bring up examples they're "exceptions", but while you bring up examples, they're examples of the rule. When I cite sources that do an overall survey of a country's regulatory environment, that's "biased", but when you cite a source that makes their money by helping companies comply with food regulations, that's just fine.
Yeah you cited one bogus example from the heritage foundation. All my examples are real. Unlike yours.
>I can run ChatGPT myself, thanks. Why don't you try thinking for yourself and considering the possibility that your presuppositions are wrong?
Highly disagree. Your answers are inferior to anything chatGPT can come up with so obviously you likely can't run it yourself.
Your list produces nothing of the kind, as I tediously went out of my way to demonstrate.
> I only know that this group is biased and not a neutral party. There's no point in vetting a known compromised source. Pick a valid one.
You haven't provided any valid sources yourself.
> Highly disagree. Your answers are inferior to anything chatGPT can come up with so obviously you likely can't run it yourself.
Well, that's just your opinion, and it's an opinion that reflects more on your poor judgment than on me.
This: https://futureoflife.org/open-letter/pause-giant-ai-experime...
will never happen. No pause will occur. I'm right and you will be wrong.
If it does happen then I concede that you're right. If it doesn't then it reflects poorly on you.
Frankly, the abrasive and belligerent way you've conducted yourself this entire conversation is the only thing that reflects poorly on anyone here. There's simply no call for making this personal.
https://news.ycombinator.com/newsguidelines.html
I don't want to ban you, so would you please review https://news.ycombinator.com/newsguidelines.html and stick to the rules properly from now on? We don't want this sort of tit-for-tat spat in which people abuse each other.
> Enron: In the early 2000s, the energy company Enron engaged in a series of fraudulent accounting practices to make it appear as if the company was more profitable than it actually was.
And in 2002, the US passed the Sarbanes-Oxley Act, which dramatically increased the regulatory burdens of running a publicly traded company.
> Volkswagen: In 2015, it was revealed that Volkswagen had installed software in its diesel cars that cheated emissions tests, leading to higher levels of pollution than were reported.
This doesn't make the point you think it makes. Even though Volkswagen is a German company that were selling these cars all around the world, and even though 8.5 million of the 11 million Volkswagens that were eventually recalled were in the European Union, it was the US EPA, not European regulators, who caught Volkswagen.
> Tobacco Industry
Almost all European countries have higher smoking rates than the United States. France has twice as many smokers as the US.
> The 2008 financial crisis was largely caused by the reckless behavior of major Wall Street banks and financial institutions. Many critics argue that the government failed to adequately regulate these institutions, allowing them to engage in risky practices that ultimately led to the collapse of the housing market and the wider economy.
"Many critics argue"--there are those weasel words again. Similarly with Enron, this led to the Dodd-Frank Act, which increased financial regulations again.
An interesting counterpoint to this is the Libor scandal, which came to light a few years afterwards. (https://en.wikipedia.org/wiki/Libor_scandal) Similarly to Volkswagen, the Libor scandal was happening in Europe (specifically the UK) under European regulatory jurisdiction, and yet it was American regulators who caught the perpetrators.
> Boeing 737 Max
This is a case of US regulators being behind other countries, but it is an exception. Also note that none of the actual 737 Max crashes happened in the US, but rather in countries with weaker regulatory regimes when it comes to airline operations.
> Big Pharma
Not US specific. Lots of these firms, like Bayer and GSK, are European, and drugs usually get approved by European regulators more quickly than they get approved by the FDA.
> Meatpacking Industry: The meatpacking industry has been criticized for unsafe working conditions, low wages, and lax regulatory oversight.
Meaningless weasel words.
> Tech Industry: Tech giants like Facebook, Google, and Amazon have faced criticism for a variety of reasons, including antitrust violations, privacy violations, and the spread of misinformation. Critics argue that the government has not done enough to regulate these companies, which have become some of the most powerful corporations in the world.
Weasel words. If you want to count this as US specific because most of these companies are American, then you don't get to blame the US for the actions of Volkswagen and British Petroleum.
> Fast Food Industry: The fast food industry has been criticized for...
More weasel words. Also not US specific.
> Industrial Agriculture
Not US specific.
> Gun Industry
Unrelated political controversy.
> Pharmaceutical Industry
You already listed Big Pharma.
> Big Oil
Not specific to the US. Canada and Norway also produce oil, and relative to their GDP, they are more dependent on oil production than the US is. In yet another example of US regulations being more stringent than other countries, it was the US, not Canada, that shut down the Keystone XL oil pipeline between Canada and the United States.
> Private Prisons
Also not US specific. Australia, New Zealand, and Canada have more of their prisoners in private prisons than the US does.
> Airlines, Plastic Indus...
But when they, together with environmental activists, were able to get laws passed that drastically increased the cost of running a nuclear plant the regulators couldn't say no. So their costs increased, but then their profits increased as well through the magic of cost-plus contracts.
I don't think compute is the issue. It's an issue with LLMs. Current LLMs are just a stepping stone for true AGI. I think there's enough momentum right now that we can avoid a winter and find something better through sheer innovation.
Now the vast majority of the worlds population has a cellphone and internet service, and use services that AI can improve/affect.
But there will be a small transformation:
* More busy work will be automated taking away some of the fun and leaving the harder tasks thus making jobs shittier, pushing workers to find other kinds of work.
* More solutions on AI looking for problems will be implemented increasing the speed of success/failure, and from the lottery winner effect we can expect "the granny that created an AI solution and earned millions" that other devs will follow and fail leaving debts.
Since when is busy work fun and hard tasks shitty?
It's very likely we'll be disappointed with AI in many oversold contexts (I share the sentiment about self-driving cars), but it can't be denied that ChatGPT is a product that's being used _right now_ to massive success and still has quite a ways to go.
Large decisions such as SVB deposits are literally the last thing that will be given to AI, it's the most important part of the business. That doesn't mean AI is useless or is not being incorporated into valuable parts of businesses.
I'm at a large tech company, LLMs are already entering some of our key product workflows. It massively lowers the floor for a number of product features e.g. recommendation systems.
In a more general sense, many features that were previously too difficult to expose (e.g. simple coding commands) to business users are now on the table as long as we give them limited access to an LLM. The current hype train means that we don't even need to do much user education.
If crashing a self-driving car required me to spend 5 minutes rejiggering it, I'd probably use the feature a lot. With generative models there's usually very little cost to it making a mistake -- and even better, I can determine when the cost is likely to be high and modify my behavior.
Self-driving cars do seem much better than they used to be. But I have no interest in using them until they get better still. I don't have this same bar with LLMs or DALL-Es. And I think this will contribute to more continuous improvement in the technology.
And then it did.
After the dotcom bubble exploded. Can't we call that a winter?
Was there a massive reduction in completely untenable .com's? But I'm not exactly sure if that's the definition of a winter, plenty of other internet based businesses did fine and kept growing in that time.
I was under the impression that the size and quality of the training dataset had a much bigger impact on performance versus the sophistication of the model, but I could be mistaken.
Compute is then proportional to the product of model size & data quantity.
That said, quality of data also matters a lot - OpenAI has had human labelers produce the data for their Reinforcement Learning from Human Feedback (RLHF), which has probably had a disproportionate impact on the success of ChatGPT compared to previous models, but that data is probably O(1%) of what they trained on.
At this point I'm guessing OpenAI are limited by both data & compute. Rumor has it they're training the "next big thing" now and it won't finish until December. If they had more compute they could presumably finish sooner, and if they had more data they would presumably let it train longer.
[0] https://arxiv.org/abs/2203.15556
But a lot of the advancements we're seeing right now are the result of more sophisticated models [1], and one person is doing some interesting work [2] around achieving transformer-level performance with other architectures.
So it's not completely settled if more data is the answer. But it has a significant impact.
[0] https://ai.facebook.com/blog/large-language-model-llama-meta...
[1] https://en.wikipedia.org/wiki/Transformer_(machine_learning_...
[2] https://github.com/BlinkDL/RWKV-LM
Babies do something completely different. They can't walk when born. Their model is to blunder about and work things out, building the model up thro a can I do this - can I do that - why not etc. Its only through this doing learning happens.
We have calf ai right now..you ask the calf what do you want to learn next or what are you curious about and you get to see how dumb it is.
Is this case really worth exploring? Or was the article written by a bored AI?
I find it striking that there are still so many people downplaying the latest developments of AI. We all feel that we are at the verge of a next revolution on par or even greater than the emergence of the www, while some people just can't to seem to let it sink in.
Just because people may have opinions different from yours doesn't mean they're denying reality. They just have a different opinion.
The hard, cold truth is that nobody knows the future. Everybody is just guessing.
Climate change is coming at us. That's still a prediction (it hasn't happened yet), but not believing in it right now is irrational and dangerous.
I don't think we can say that for AI. That AI will change the world is a belief. Doesn't mean that it won't.
Crypto's value fluctuation made more sense as a speculative asset class, and it clearly saw most of its traction there.
It's potential as a currency did lead to banks, financial institutions and governments taking a hard look at it. And CBDC's are an area of active investigation. So the hyper-optimists may have been off on some of their predictions, but something came out of it.
AGI-agencent tools are more grounded. They don't threaten to tear down existing entrenched institutions. They do one thing simple: commoditize and scale some level of intelligence. And they're pretty good at it, hence the hype.
Unless humanity develops a deep distaste for many things AI-made, the fascination will wear off and we'll be left with the cold hard truth of raw productivity increase and digital production on an unprecedented scale.
I disagree with this part. (I disagree with calling this tech "AGI-adjacent", too, but that's not the part I'm calling out here).
I'm not sure what you mean by "more grounded", but if you mean "more based in reality", I don't think this is clear. But, if what the evangelizers say is correct, it absolutely threatens to tear down entrenched institutions. It even threatens to tear down society itself by destroying what little trust remains.
IIRC, crypto was a little too idealistic in the promise of providing a digital currency without needing a central, trusted authority to back it. From my perspective, AI as we've seen it in the last few months simply provides a tool that can automate intelligent tasks easily.
I think the distinction is fuelled more by the companies and leading figures pushing the latter. OpenAI's stewardship, concerns for safety and generally downplaying how amazing these tools are while not ignoring the real effects they could have make AI sound more "grounded" than peak crypto was. Or it could just be the folks I pay more attention to.
> I disagree with calling this tech "AGI-adjacent", too, but that's not the part I'm calling out here
I think this rests mostly on the definition of "General" here. Here, I'm talking about LLMs as general task performers, as opposed to models created for more specific tasks. LLMs have proven to be more general purpose, which I'd argue makes them closer to the AGI ideal than, say, a sentiment analysis model.
Ah, I see. By that definition, it seems to me that cryptocurrency was actually more grounded than gpt.
Cryptocurrency (or even blockchain) has not (yet) changed the world. That was what everyone was predicting.
Then it hasn’t changed the world. That phrase doesn’t mean “a small number of things are different” it means that everything has pivoted.
That’s the definition of NOT changing the world.
otherwise, even an ant changes the world but nobody would say that.
but the examples of success so far seem cherrypicked leading to serious skepticism. there will be a revolution, and its already in progress dethroning entire business models but the future is still very uncertain.
what is humans role in this adventure into the singularity is even moreso clouded with more questions than answers
Bonus points for things that should cause us to make major adjustments (otherwise it doesn’t seem worth it to expend too much energy).
Have you ever looked at predictions from big consulting companies like McKinsey and the likes? "This new field with unlock a market of 30 billions in the next 5 years", that kind of crap?
In the fields where I worked, I used to take those predictions seriously (after all, they know better than I do, right?). What I realized is that they just don't have a damn clue. I can't blame them for failing: predicting the future is an unsolved problem. But making that kind of money by telling that kind of crap should honestly not be legal.
All that to say, some things are clearly coming right at us (ahem, climate change), some aren't (AI, for instance).
By all means keep your own estimates of what the world will look like in 10 years and act accordingly, but I'm trusting my own technologist's judgement on this one.
I’m excluding the desk jockey pundits from the argument, of course.
I am not estimating anything. I'm saying that there is no way to know where it will go. You are the one doing what the empty suits did: you are predicting something based on your beliefs only.
Not that you can't do it: everyone can believe what they want. But it would be wise to realize that it's a belief, even if you are a technologist.
I don't know, I don't see it this way. Bitcoin at the very beginning was an impressive piece of technology (still is), but it clearly did more harm than good.
Autonomous cars had some very impressive demos, and don't seem to have evolved much in a few years (it's impressive, but not good enough).
AI is making very impressive demos now, but it's not changing the world (what seems to work really well right now is to generate convincing disinformation at scale, which is not good).
To me it's all the same: impressive demo, but what matters is not the first 80%, it's the last 20. Everyone seems to be building infrastructure around what AI will become. Just like people built entire companies betting on the fact that cryptocurrencies would become global (they have not), or that blockchain would be useful outside from cryptocurrencies (it is not). Or like people built entire companies around "the new world with autonomous vehicles".
The problem is we got autonomous cars backwards. We needed multimodal GPT first. GPT-4 can look at a picture can tell you what's happening very clearly. Now, chain this together really fast and you have something that could be used for driving. I'm making no bets on when the hardware/models will be fast/cheap/power efficient enough to support this.
>but it's not changing the world
Eh, no, it is changed the world in a vast number of ways, but not spread very equally. On the scientific/AI side a whole bunch of "20 years away" and "We may never accomplish" have been changed to "now". On the language side things GPT has put any translation service on notice. Summerizers too.
I've never been a crypto fan. I believe there are purposes for it, but at the same time I have a bank that does all the same things.
What I do (did?) not have is an AI that actually solve real life problems that I run into now, and that's something that is both highly useful, and something that many people are willing to pay for.
Again, that's a belief :). Could totally be that they never become a thing, and nobody would be surprised.
https://fortune.com/2015/12/21/elon-musk-interview/
It took longer than the likes of Tesla said, but it certainly hasn't been taking longer than a whole lot of people expected.
My prediction is that we're much more than a few years away from it becoming a reality in the way people imagine.
examples?
We haven't even seen a war coming right at us.
My old eyes can't see perfectly anymore, so I would probably miss it - whatever it is - anyway, but if you're so sure, would you bet everything you own on it?
> “[It] revealed an embryo of an electronic computer that it expects will be able to walk, talk, see, write, reproduce itself and be conscious of its existence. Later perceptrons will be able to recognize people and call out their names and instantly translate speech in one language to speech and writing in another language, it was predicted”
The section on "Past Winters" in TFA is a good part of the reason why you might want to not believe what you think you see coming right at us ....
Money does not guarantee results, even if the amount of money is massive.
Yes, I know. But there's no way to know if these people are right, right now. That will only be possible to know in the future, looking back, so it's still "predicting the future".
Probably, yes, especially after what happened to Cassandra.
Still a fun counterfactual to develop. Lessee, you do know the future, but do not run your mouth off. Matter of fact, you pooh-pooh any such notion ...
Is this something you can see so clearly that you can describe it in specific terms?
At first we were perplexed by a computer beating Karpov at chess.
These days however we're not perplexed anymore that a lot of creative jobs have been replaced by AI. (I think those people will find another passion no doubt, that is another topic).
Next thing is less creative jobs. But i'm already thinking about climate change and hunger mitigation. You could call me an idealist, only time can tell, of course.
Let people play with GPT 4 for a bit.
but the future is very unclear.
It's obvious at this point that we are figuring out how to create machines that think. While I agree that the future is unclear, in the sense that it's hard to make exact predictions, the trend is very clear. Significant efficiency improvements in many types of intellectual labor is almost certain.
I don't think it's priced in yet; the somewhat independent-minded among tech people are currently the only ones who have front-row seats to what's going on. But in a few years it will be obvious.
Honestly, I think the persistent exponential improvement in AI makes it obvious we're on track for superintelligent machines in a few decades as well (if that!), but the preceding paragraphs should at least be entering the realm of Overton.
* generate digital images from natural language descriptions. ie. DALL-E
* suggest code and entire functions. ie. Copilot
* generate textual content: Generative Pre-trained Transformer. ie. ChatGPT
The future may be in other important endeavors:
* train on ... data and learn how to ... to automate ...
I think that the creativity is in your idea and not in what is being automatically generated.
Edit: That may the lesson of Stable Diffusion: ideas are very cheap, if you have a heuristic for what a good idea is. Generate a business, fix it by optimizing for measurable metrics, if it doesn't work, try again. Legal constraints can be interpreted by an LLM, fines are penalties for training.
I think in terms of human ingenuity, the digital space is finished in the long run. It will all be on the level of infrastructure funneling data to corporations.
The space to innovate with the low-hanging fruit still lies in engineering physical things, where implementation and testing is still a barrier to iterating quickly.
Yes! You’ve mentioned people downplaying, but this is someone who is 95% confident there won’t be an AI winter in the next seven years. That’s really quite confident. And the definition he gave makes it compatible with AI being the biggest thing since sliced bread. But if it gets funded at 3x bread and scales down to 1x bread, that will count.
What cost was this exploration? A blog post? An HN discussion. I think it’s so valuable for people to consider the scenarios they think are unlikely. I find this analysis much more interesting than all the dismissive “my job is safe, AI can’t [friction-point-of-existing-system]” posts. It’s certainly worth the investment of a few hours of work.
If that's the case, I think those, like most people, are taking the advancements seriously. I would say to those people: don't worry too much.
The impact will be big, but it doesn't have to mean people will not have a job anymore, the tools they use will be at another level.
You've made me realize the difference between the arguments I find weak vs strong. The weak argument is I can do something that ChatGPT can't. "I have people skills!". The strong argument is that as ChatGPT grows, I will grow with it.
The AI floodwaters are rising. People are climbing up the stairs when they should be building boats. If you seek shelter and are wrong about how far this goes, your attic will become a coffin.
Mind you, there are always a variety of possibilities that people see, that is true, but this unique flavor of comment is unique because I haven't seen it expressed before. Not during crypto (as others are bringing it up).
It feels like there's a much more serious... insecurity? Maybe that's too aggressive of a word, but it's one that describes how this downplaying partially is rooted in this idea of a replacement, and lack of job security, and an uncertainty about what you should be spending your time on. All of this is so unique, nobody was feeling this way during crypto, not a single person. If you hated crypto, you were hating it because it sounds redundant, the "fans" were annoying, the environmental impact, etc - not a single one of these reasons came from this assumption that crypto will be the new norm.
So I think that's what a lot of the other comments are missing right now. The downplaying feels totally different now. As if even the people who are downplaying realize what's happening, but don't want it to be true.
I did like this blog post though, outside of what we're talking about.
A fair number of HN'ers are the "pull yourselves up by the bootstraps", "We don't need to stinking unions", "Social safety nets are for the weak" types. The potential for AGI causes a cognitive dissonance with them. They are intelligent enough to realize that AGI would nearly completely destroy their ability to make money, but at the same they don't want to release their ideals that "rugged individualism" is why they are better than everyone else, and they would need a more fair system to survive.
Maybe ChatGPT would have suggested that had I ran the comment through it prior instead of sounding like a bumbling fool lol
I'm a crypto and AI optimist, but a tempered optimist. I don't think either tech is imminently going to disrupt everything, but I think both technologies have their use cases. They are far more modest than their most ardent supporters would believe. However, over the course of decades, the tech can transform society, I still think that.
However, as with the internet itself, which never did live up to its loftiest, most optimistic ambitions, it's not to downplay the impact at all. Just thay eventually you do find a point where you grapple with the tradeoffs inherent in the technology and those are often sticky and intractable.
A good reason to consider "what if" is that it's still going to take a lot of effort to get from where we are to where expectations are. There's been many false starts recently: self-driving cars, 3D printing, VR/AR, crypto. 10 years on and they're all real technologies anyone can use and making progress every day, but the wild expectations hit reality. When we get there with AI (and we will, because expectations are a moving goalpost) it's good to have considered what we want to do about it. We don't want to waste years of resources on insurmountable roadblocks, we want ideas about why what worked before stopped and what easier paths forward might exist.
That is, I don't see many people downplaying AI but I do see people with lots of unanswerable questions about how long this current boom will last and where we'll wind up when it's over.
https://youtu.be/bHb_eG46v2c
https://www.gatesnotes.com/The-Age-of-AI-Has-Begun
Second, power consumption is not improving except by going to 8&16 bit... there's not a lot of room there. Currently, we get <1000GFlOPs/W and even if we get 100x up to 100TFLOPs/W, you still need 10^17kWhr. OK algorithms get us another 10,000x... and it costs $1T to train 1 transformative AI. How many proof of concepts trials will be do at only $100B a pop?
It all just seems a bit flip and hopeful like Feb 2000. The 10^35 number seems like its just pulled out to be a number, when it could orders of magnitude up/dn.
https://www.researchgate.net/publication/354573934_Compute_a...
The 1e35 FLOP number is meant as a conservative upper bound and comes from here: https://www.lesswrong.com/s/5Eg2urmQjA4ZNcezy/p/rzqACeBGycZt...
The major fabs all have roadmaps for approaching 1 nm, and there are other advances that could allow you to keep going either if transistor size scaling stops (e.g., vertical scaling). (That said, I definitely don't think it's a given that HW price-performance keeps doubling at the same rate 10+ years.)
5-7 years ago the progress with self driving cars seemed enormous, and the end of driving was just around the corner. And then all progress seemed to stall or recede, and it turned out that what seemed like huge progress was mostly hot air.
Maybe that’s not what happening now, but I don’t think a cautious approach is unwarranted.
Yep, but a lot of the current hype for GPT is that we're in the near-vertical segment of the sigmoid, and so people are like "woah, if that's where we are now, where will we be in 5-10 years?!". And the answer to that might be, pretty much where we are now. And maybe not, maybe it'll be better, but you can't assume linear improvements on a sigmoid curve.
In theory if we could make multi-modal GPT-4 go very fast, we could have it operate a car right now. It's too bad we don't have access to feed it images, because my guess is if you put a picture of an inside of the car and showed a kid chasing a ball into the road in front of it, it's first guess would be "hit the brakes".
- Redundant due to winner takes all dynamics. - Useless - Way to expensive for what it does. - Produced off of sky-high seed rounds at 150 MM.
This same dynamic happened with the www leading to an eventual crash - the mobile and social versions would have been more noticeable had it not happened in private markets and simultaneously with the valuation spike of the late teens. It would be almost more surprising if there wasn't a pull back eventually.
This also doesn't say anything about the people who work in AI. It's not a given that the legions of research scientists populating corporate labs will be the right people for new startups. Similarly it may turn out that prompt engineering is the dominant method for talking to LLMs rather than code.
And
>Take for example the sorting of randomly generated single-digit integer lists.
These seem like very confused statements to me.
For example, lets take banking. It's actually two (well far more) different parts. You have calculating things like interest rates and issues like 'sorting integers' like above. This is very well solved in simple software at extremely low energy costs. If you're having your AI model spend $20 trying to figure out if 45827 is prime, you're doing it wrong. The other half of banking is figuring out where to invest your money for returns. If you're having your AI read all the information you can feed it for consumer sentiment and passing that to other models, you're probably much closer to doing it right.
And guess what, ask SVB about 99% correct correct solutions that do/don't capture value. Solutions that have correct answers are quickly commoditized and have little value in themselves.
Really the most important statement is the last one, mostly the article is telling is the reasons why AI could fail, not that those reasons are very likely.
>I still think an AI winter looks really unlikely. At this point I would put only 5% on an AI winter happening by 2030, where AI winter is operationalised as a drawdown in annual global AI investment of ≥50%. This is unfortunate if you think, as I do, that we as a species are completely unprepared for TAI.
I am not speaking speculatively either. I have seen it happen finetuning ESRGAN on previous upscales that I even personally vetted as "good," and these generative models are way more sensitive than the old GANs.
More than that, I believe we will hit a ceiling when the impossibility of these models to incorporate causality becomes evident.
Right now, LLMs are trained by predicting the next word given a context (the prompt). This approach, IMHO, gives a resemblance of cause/effect because the training data is made by humans and obviously we are able to express ourselves and reason in those terms. So we have a poor proxy for that which, also IMHO, partially explains why LLMs performance degrades when asked to solve novel problems (there was an entry few days ago about this).
1) langchain paper? 2) reasoning paper?
Thanks!
So yes, easy data scaling is coming to an end and may or may not lead to a short term winter. But this also means that, all being equal, training a model will be cheaper compared to a situation where we still had orders of magnitude more data to go.
The specific problem was what while I'm in a WebSocket handler, I cannot await on asynchronous generators which act like `asyncio.sleep()` and send messages to the client while I have `heartbeat` on that `WebSocketResponse` enabled (it sends pings to the client and waits for pongs to see if the connection is "alive"). The issue is that the incoming pong is not getting processed so the client gets disconnected.
I struggled a lot with ChatGPT's help, but it was mostly insightful, like rubberducking with a duck that does actually try to help you.
Occasionally I went to Google to search for a very specific way of solving the issue; for me Google is the gateway to Stack Overflow, I never use Stack Overflow's search.
But I didn't find anything of value there. And earlier today I noticed that I had unread messages in Stack Overflow, and when I checked how many consecutive days I have been on SO: 1. I used to be 100+ consecutive days on SO and lately I'm struggling with using it as the to-go place for my programming questions.
And this is where your comment comes in: I was thinking to myself what is going to happen if more people move away from Stack Overflow, that place is a goldmine of programming information, who will feed it?
One thing I really hope is that OpenAI adds some "modes", like what were poised to expect from Copilot X, where we get different layouts for the webpage, so that we can choose a "programmer mode" which actually lets us give proper feedback in the sense that "this code worked", "that one didn't".
There's already a feedback, but it's not task-oriented. A programmer can submit to it as well as a cook, the result won't go into a "database of usable code" which ideally would be publicly accessible or even get formed into a Question/Answer pair which is prepared to be submitted to Stack Overflow.
Stack Overflow will degrade to single search box. And we'll all love it.
- What really happens in corporate/government/private decision-making?
- Did a particular politician really trade his vote for money? How was that done?
- Are elected judges better/more honest than appointed judges?
- Are there hidden organizations within particular governmental entities? e.g., a right-leaning group within the FBI who secretly persecutes persons/groups of other political persuasions? Are there independent entities within the CIA that are capable of financing and operating on their own under the umbrella of the government but also capable of evading the congressional oversight specified in the Constitution?
- Is my wife's cousin really screwing Hunter Biden again?
It's going to shift from requiring quality data to producing quality data.
There is not going to be need for it just like there is no need for more Kasparovs anymore.
You can see it with artists slowly happening.
Lawyers will get the hit (or great tool to use depending how you want to see it).
Medical analysis will find the same faith. If you think that medicine will require human analysis just imagine for a second what closed loop AI could do - if it had access to data not only from all hospitals but patients as well and being able to munch through it continuosly. This together with constant access to simulations and physical trials, finding patterns and correlations on its own.
It's exciting and depressing to think that humans will evantually be left to being humans the way they wish with everything sorted out - just like chickens don't mind being free range chickens.
There is no other direction it can go and it'll just go faster.
Our generation will see some mind blowing things.
Next generation will arrive at the world unlike anything else.
synthetic data gives you synthetic results.
To train something requires decent quality input, otherwise it's going to mimic the crap quality stuff. There is little chance in getting around that.
Have you ever stopped to wonder _why_ large for profit companies give away free models with weights? its because once trained, they are commodity. the hard part is dataset management.
Yes, chatgpt is largely self supervised, but the training into a usable model required a fucktonne of human hours
If advancing means ever larger and more expensive systems and ever more data, we will enter a cold winter soon.
Energy is "probably" not a big deal. If you're looking at long term oversupply from green energy sources, it's probably not hard to train with bursty, but very cheap power like this.
Compute is currently the biggest limitation, and will remain so far a long time, as long as scaling continues.
Calling it now: humanity will seal its fate the day we hook up LIGO to ChatGPT and it gets corrupted by gravitational waves from the beginning of time when the Great Old Ones freely roamed the universe.
The next step is multi-modal data. Think videos, sounds, touch, etc.
An 18 year old in fact "trained" on much more input data than GPT4, orders of magnitudes more, considering the complete informational content of our 5 senses and implicit structure of our brains "learned" through evolution.
Not to mention recent research in LLM scaling (chinchilla) reveals how our current models are undertrained and over-parameterized.
We have not plateaued yet.
Remember that the data must be annotated. This puts limits on what can be usefully ingested.
The entertainment industry disagrees with this.
These systems are transformative for any creative works and in first world countries, this is no small part of the economy.
Also if these 10 billion dollar models are 'AGI' level, then your model pays for itself if you can find enough interesting work to throw at it.
When this happened in the 60s-70s, the psychedelic revolution was crushed by the government. And we entered an AI winter.
I’m not implying causation. Just pointing out a curious correlation between the two things.
I wonder what will happen now.
See https://digital-strategy.ec.europa.eu/en/policies/regulatory... for the rationale and general approach.
It's late spring right now, a strange time to start forecasting winter.
pray for a winter but prepare for a societal upheaval
I think the current crop of AI is good enough. It will happen because people will actually grow resentful of things that AI can do.
I anticipate a small, yet growing segment of populations worldwide to start minimizing internet usage. This, will result in fewer opportunities for AI to be used and thus the lack of investment and subsequent winter.
And what is a "user" when it comes to ChatGPT and it's ilk? How does that compare with the definition of a user of, say, the web? Or of SMS ?
Buy the other side of physical infrastructure is already here. The internet, cellphones, and computers already exist in mass and can be utilized by AI now. We didn't need to build new highways for this. No new towers. No wires put in the ground. Those fields already exist and are ready for planting.
given that they need a _huge_ amount of GPU power to server that kind of traffic, I suspect that 100million seems out by an order of magnitude.
Humans are unreliable AF and we employ them just fine. Better reliability would certainly be nice but I don’t think it is strictly speaking necessary
I think the rather breathless posts (which I also remember from the 80s and apparently used to be common in the 60s when computers just appeared) will die down as the limits of the LLMs become more widely understood, and they become ubiquitous where they make sense.
In which case progress becomes unlimited and there’s never an AI winter again.
in the late 90s and early 2000s, neural network had a significant stigma for being dead ends and were unpromising grads were sent - people didn't want to go there because it was a self-fulfilled prophecy that if you went to research ANNs then you were a loser, and you were seen as such, and in academia that is all you need to be one
but, in real life, they worked
sure, not for everything because of hardware limitations among other things, but these things worked and they were a useful arrow in your quiver as everybody else just did whatever was fashionable at the time (simulated annealing, SVMs, conditional random fields, you name it)
hype or no hype, if you know what you are doing and the stuff you do works, you will be okay
ChatGPT as is, is already transformative. It CAN do human level reasoning really well.
The only winter I can see, is the AI gets so good, there is little incentive to improve upon it.
shrug
>It is definitely not reasoning at all
read the paper. GPT4 is the real deal.
[1]https://arxiv.org/pdf/2303.12712.pdf
You can absolutely build high precision ML models. Using a transformer LM to sum numbers is dumb because the model makes little assumptions about the data by design, you can modify the architecture to optimize for this type of number manipulation or you can change the problem to generating code for summing values. In fact Google is using RL to optimize matmul implementations. That’s the right way of doing it.
The past AI winters have been preceded by periods of AI moving forward faster than anticipated. Its when the limits of easy advancement with the current approaches are reached without a new approach that allows continued rapid progress you get an AI winter.
Why?
Reasons:
1) We are currently mining just about all the internet data that's available. We are heading towards a limit and the AIs aren't getting much better.
2) There's a limit to the processing power that can be used to assemble the LLM's and the more that's used the more it will cost.
3) People will guard their data more and will be less willing to share it.
4) The basic theory that got us to the current AI crop was defined decades ago and no new workable theories have been put forth that will move us closer to an AGI.
It won't be a huge deal since we probably have decades of work to sort out what we have now. We need to figure out its impact on society. Things like how to best use it and how to limit its harm.
Like they say,"interesting times are ahead."
2) The available power seems to be growing pretty rapidly and dropping in prices. I think there are still quite some gains to be had from architectural optimizations (both in hardware and in models).
4) They were defined decades ago but I think they did actually seem to move us closer to AGI only recently.
You might be right, and there are definitely interesting times ahead! But I kind of doubt that we will have decades to sort out what we have and figure out its impact on society (which is a bit scary).
1/ We humans are still generating data. To live is to generate data and while it is dystopian to think about how these data can be harvested, it is still possible to get more information to feed the next gen AIs. And remember that right now, we have only used text and images. Videos, audio, sensory inputs (touch, smell, taste, etc), and even human's brainwaves are still available as more training data. We are nowhere close to running out of stuff to teach AIs yet.
2/ Fine tuning and optimizing training has shown tremendous effects in reducing the size of these LLMs. We already have LLMs running on laptops and mobile phones! With reasonable performance and in only half a year from the big release. There is lots of room to grow here.
3/ My nieces laughed when I told them TikTok is too invasive. Most people outside of HN does not care about data privacy as much as you might think.
4/Sometimes it only takes 1 big breakthrough to open the floodgate. Transistors was that point for computers and we are still developing new techs based on that 100 years old invention. We don't know how much potential there is in these decades old AI invention especially when many of them was only put into proper practice the last decade. We didn't learn the big mistake until very recently after all.
Just some ways things can go differently. It is the future, we can't really predict it. Maybe an AI can...
I guess it really depends on what you mean by "basic theory" but my view is that the framework that got us to our current crop of models (vision now too, not just LLMs) is much more recent, namely transformers circa 2017. If you're talking about artificial neural networks, in general, maybe. ANNs are really just another framework for a probabilistic model that is numerically optimized (albeit inspired by biological processes) so I don't know where to draw the line for what defines the basic theory...I hope you don't mean backprop either as the chain rule is pretty old too.
We have "something that appears to work" but is this itself a local minimum? Biology found its own efficient processes that look a bit different after all.
In my opinion, "fundamental breakthroughs" would answer questions like:
- is there an inherent reason that humans learn through a process akin to statistical optimization, or is it a coincidence that feedforward-only gradient descent seems to work?
- Are there other models for "neurons" that go beyond GEMM, like spikey activations? We know from biology that they're nonlinear; how might they be modeled better?
- Are there ways of training effective agents beyond just the reinforcement Q-learning techniques that we've settled on?
- What does it even mean for a model to be able to "reason;" does our current view of recursive models with hidden state really map to that philosophy one-to-one or is there more?
The leap from RNNs to the Transformer architecture in 2017ish is similar to vision networks' leap from fully-connected layers to convolutional layers in 1987ish. In both cases, the key is to change the architecture to exploit some implicit structure in the data that wasn't quite used effectively before.
For images, each pixel is strongly related to its close neighbors, so convolution is a natural way to simplify models while capturing that locality.
For language, the opposite is true - almost any part of the text implicitly reference anything that's been said previously, so you need representations that are somewhat position-independent. Approaches that can model that structure naturally work better than an RNN munching tokens one by one.
Each one was certainly a leap forward that unlocked its respective field, but at the end of the day, it's "just" an architecture tweak, you know?
Just note that this is not at all what most people mean when they say "AI Winter".
It is usually defined as less funding and general interest in investing in AI. Not "we don't know how to move forward towards AGI".
I recently came across this myself when writing a reply on another forum. A feeling of reluctance to attempt to contribute something maybe-useful in a public, 'minable' space.
Almost feels like this has tarnished the 'magic' of Internet, of sharing information and knowledge. I'm not saying this is the appropriate reaction, but it is what it is.
I used to be like this, but the reality is that ideas are a dime a dozen. Any idea that you or I (or anyone else) have had, have likely also been thought by thousands (or maybe add a few more 0's) of people.
What makes things happen is people not ideas. Talk to any VC or people involved in startups - it's all about the team, not the ideas they are working on.
I read this a lot, and it sounds intuitive on the surface. But I don't understand how it's justifiable. For example, all that exists in the Universe is a result of the application of very simple rules. It would make sense that information is not what is important towards intelligence and complexity, but computation. It ought to be possible to create a superintelligence with a few bytes of training data, given enough compute.
So an AI that simply recycles its own input ad infinitum might produce something but it won’t be meaningful to us humans. Hence, it’s unlikely to be useful apart from the novelty of it.
For generative AI, the hallucinations will poison the well. And they're not random, so same/similar hallucinations will pop up all over and reinforce each other.
I argue that this won't matter. I think Generative AIs will be much scarier than any sci-fi AGI. And this is because while sci-fi AGI apocaliptic scenarios involve the AGI seeking to exterminate humanity usually as a form of revenge (see The Matrix, The Orville and many others) or as part of an optimization, scenarios involving Generative AIs simply involve humanity destroying itself, and these scenarios are already very plausible.
Prepare for the complete destruction of objectivity, the onslaught of spam, grifts and fake news at an unprecedented scale and a flood of security vulnerabilities unlike any other. This will be the annihilation of interpersonal trust and of knowledge.
They have the human capability to ensure that it is state of the art in all current ways, and could theoretically use some processes that aren't disclosed to the public. I don't think anyone would say that China at least doesn't have the brainpower to do something like that.
Then, these government leaders could ask the AI to come up with simulations on how to conquer the world.
The AI would probably say something like, "America has to fall, since it's military might is the strongest resistance to your ability to dominate the globe".
How would you do that?
Ensure weak or compromised politicians get elected, which will make the American people lose confidence in their government.
Spend resources to ensure that the financial stability of the country is negatively affected. Do this by disrupting the flow of goods into the country, viciously competing with any product based company of any merit, and purchasing housing in the major financial centers of the country.
This serves three purposes:
1: Businesses will become tight-fisted with their spending wherever they can as they won't always know if they will be able to purchase the goods they need when they need them. The easiest way to reduce spend in the short term is to underpay and overwork your employees by firing the more expensive employees and shifting their workloads to others, so that will be the first thing that happens.
2: The smarter and capable people will attempt to escape the rat race by relying on their inventiveness. They will suffer in drudgery for years while they perfect a product that someone needs and is willing to pay for, and then, since manufacturing in America is ridiculously expensive thanks to China's price competition, they will outsource the manufacturing of their product to China. This will happen frequently, and they will be taking all of the risk for themselves.
Once they have a successfully tested product, China will clone the product and flood the market with inferior quality but massively cheaper clones, ensuring that the original business never achieves the success it could have and then has to cut costs, putting the employees of that company into the same conditions that the founder started the company to escape.
3: Housing will become very expensive, so the citizens will not be able to afford the same quality of housing that their parents did. This will further increase their unhappiness and destabilize the power of America.
Once all of that is done, then you let the kettle simmer for a while. Spend the money, time, and energy to keep the pressure up. Use the time to buy or subvert more politicians.
When this is done, the country will be weakened by its internal stresses and so charged that all it will need is a single strong event to vent all of that pressure on. A match to a powder keg will blow the whole thing up and America will collapse from civil war.
While that civil war is going on and the whole world is watching, launch your attacks. Take over your neighboring countries. Threaten nuclear annihilation on anyone that stands against you as within a few years you quadruple the size and power of your country.
By the time America pulls it's head out of the fog and realizes what is going on, it will be too late.
If AI starts to become economically important, there will be an incentive to create more data. If I'm OpenAI, why not pay a company to put microphones around their office, transcribing everything everyone says for training data. Buy troves of corporate (or personal) emails. If there are one hundred million office workers in the world, and you can convince/pay 10% of companies to let you spy on them, and each says or writes 5,000 words per day (these are all low estimates, in my opinion), you'd be getting ten billion more words of data per day. You'd double the size of the Chincilla training data in about a month.
If you incorporate video for a truly multimodal model, now you're talking about not only the entirety of Youtube, but potentially also CCTV footage. If you could get the data from every surveillance camera in the world, you'd generate one YouTube worth of video every ten minutes (based on 150 million hours of video on YouTube).
You appear to be assuming that the only way to make current LLM-based AI's better is by feeding them more and more data, but that's simply not true. These current (really first generation, not withstanding GPT's versioning) AI's are very data/parameter inefficient, and do not differentiate between data used to develop reasoning and data that is just pure knowledge. The short-term path forward is more judicious training set design to use less data, and require less model parameters, so that the model itself just learns to reason, and doesn't need to do double-duty as a database repository of facts as well. Retrieval-based plugins and similar approaches can be used to give access to external knowledge.
2) There will be many different sizes and capabilities of AI. They don't all need to be superhuman.
We are entering the AI age and just as the computer age didn't result in everyone needing a mainframe or datacenter in their garage (although there are uses for them), the AI age will not result in everyone running the world's most powerful super-human AI on their smartphone. There will be a range of AI's for different uses cases.
Current models are also very inefficient. We are essentially day 1 into the NN/LLM AI era. Think of ChatGPT as the ENIAC of the age. DeepMind's Chinchilla scaling laws have already proven how inefficient these systems are, with a suitably trained 70B model equaling the performance of a 175B one. There is tons more work to be done on efficiency and smarter model design.
3). Perhaps, but per point 1) we're not running out of data anyway, and there will anyways always be people/companies happy to sell data for a price, even if it is not for free.
Future AI's will learn the same way we do by experimentation and curiosity/surprise, and will not need to be spoon fed a training set in the same way as today's systems.
4) No idea what you're talking about here. NN optimization is hardly spent as a technique - that'd be like saying "we've reached the limit of what we can to with software development - we need a new idea".
LLMs as a path to AI, or building block of AI, are of course a very new idea - a few years old at best. A few decades ago no-one was even working on neural nets (current DeepLearning revolution basically started with AlexNet in 2012) .. it was all GOFAI symbolic systems - failed approaches like SOAR and CYC.
If you compare the Transformer architecture to the human cortex, and our brain's overall cognitive architecture, some of the missing pieces are extremely obvious. We're not at any sort of impasse as to how to move things forward.
For sure we will get a lot stuff automated with it in the near future but this is far away from anything real intelligent.
It just doesn’t really understand and or feel things. It’s dead cause it just outputs data based on it’s model.
Intelligence contains a will and chaos.