This is like when VCs were funding all kinds of ride share, bike share, food delivery, cannabis delivery, and burning money so everyone gets subsidized stuff while the market figures out wtf is going on.
I'm already keeping an eye on what NVidia gets into next... because that will inevitably be the "Next big thing". This is the third(ish) round of this pattern that I can recall, I'm probably wrong about the exact count, but NVidia is really good at figuring out how to be powering the "Next big thing". So alternatively... I should probably invest in the utilities powering whatever Datacenters are using the powerhungry monsters at the center of it all.
One thing I'm not clear on is how much of this is cause and how much effect: that is, does NVidia cheerleading for something make it more popular with the tech press and then everyone else too? There are definitely large parts of the tech press that serve more as stenographers than as skeptical reporters, and so I'm not sure how much is NVidia picking the right next big thing and how much is NVidia announcing the next big thing to the rest of us?
Yep, you will probably lose. The VCs aren't out there to advance the technology. They are there to lay down bets on who's going to be the winner. "Winner" has little to do with quality, and rides much more on being the one that just happens to resonate with people.
The ones without money will usually lose because they get less opportunity to get in front of eyeballs. Occasionally they manage it anyway, because despite the myth that the VCs love to tell, they aren't really great at finding and promulgating the best tech.
At least OAI and MSFT offer "copyright shields" for enterprise customers.
Say what you want about the morals associated to said offerings, but it sounds very interesting for companies that may want to use a plagiarizing machine for copying GPL code while covering their asses in some way (at least in terms of cost).
> when VCs were funding all kinds of ride share, bike share, food delivery, cannabis delivery, and burning money so everyone gets subsidized stuff while the market figures out wtf is going on
I’m reminded of slime molds solving mazes [1]. In essence, VC allows entrepreneurs to explore the solution space aggressively. Once solutions are found, resources are trimmed.
VC is good for high-risk, capital-intensive, scalable bets. The high risk and scalability cancel out, thereby leaving the core of finance: lending to enable economics of scale.
Plenty of entrepreneurship is low to moderate risk, bootstrappable and/or unscalable. That describes where VC is a raw deal. It also does not describe AI.
That's exactly the short term thinking they're hoping they can use to distract.
Tech companies purchased television away from legacy media companies and added (1) unskippable ads, (2) surveillance, (3) censorship and revocation of media you don't physically own, and now they're testing (4) ads while shows are paused.
Where I live the ridesharing/delivering startups didn't bring goodies, they just made everything worse.
They destroyed the Taxi industry, I used to be able to just walk out to the taxi rank and get in the first taxi, but not anymore. Now I have to organize it on an app or with a phone call to a robot, then wait for the car to arrive, and finally I have to find the car among all the others that other people called.
Food delivery used to be done by the restaurants own delivery staff, it was fast, reliable and often free if ordering for 2+ people. Now it always costs extra, and there are even more fees if I want the food while it's still hot. Zero care is taken with the delivery, food/drinks are not kept upright and can be a total mess on arrival. Sometimes it's escaped the container and is just in the plastic bag. I have ended up preferring to go pickup food myself over getting it delivered, even when I have a migraine, it's just gone to shit.
Add the abuse of gig workers, expansion of the toxic tipping culture, increase in job count but reduction in pay, concentration of wealth in fewer hands.
These rideshare and delivery companies are disgusting and terrible.
For where I live (Asia), I disagree with both of these examples.
Getting a taxi was awful before ride-sharing apps. You'd have to walk to a taxi stop, or wait on the side of the road and hope you could hail one. Once the ride-sharing apps came in, suddenly getting a ride became a lot simpler. Our taxi companies are still alive, though they have their own apps now -- something that wouldn't have happened without competition -- and they also work together with the ride-hailing companies as a provider. You could still hail taxis or get them from stops too, though that isn't recommended given that they might try to run the meter by taking a longer route.
For food delivery, before the apps, most places didn't deliver food. Nowadays, more places deliver. Even if a place already had their own delivery drivers, they didn't get rid of them. We get a choice, to use the app or to use the restaurant's own delivery. Usually the app is better for smaller meals since it has a lower minimum order amount, but the restaurant provides faster delivery for bigger orders.
I assume you are talking about airports. Guess what, they still exist in many places. And on the other hand, for US, other than a few big cities, the "normal" taxi experience is that you call a number and maybe a taxi shows up in half an hour. With Uber, that becomes 10 minutes or less, with live map updates. Give me that and I'll be happy to forget about Uber.
If drivers are actually driving people around and earning money, potentially getting more income in the end, with a trade-off of inconvenience for riders to wait a few minutes for the next ride to arrive, do you consider that positive or negative? Sounds like a much more efficient model to me.
taxis were the greatest example of regulatory capture. the post-event/airport Uber pickup situation is stupid and has obvious fixes, but, again, that's the taxicab regulatory capture where Uber has to thread a needle in order to not be a taxi, for them to legally operate. if we could clean slate, and make a working system, that would be great but we can't, because of the taxicab regulatory commission.
I can now summon a cab from the comfort of the phone I'm holding, and know that they'll accept my credit card. I know the price before I get in and I know the route they should take. I'm not going to get taken for an unnecessary scenic tourist surcharge detour.
people don't like feeling they got cheated, and pre-uber, taxis did that all the time.
Companies in the business of building models are forced to innovate on two things at once.
1. Training the next generation of models
2. Providing worldwide scalable infrastructure to serve those models (ideally at a profit)
It's hard enough to accomplish #1, without worrying about competing against the hyperscalers on #2. I think we'll see large licensing deals (similar to Anthropic + AWS, OpenAI + Azure) as one of the primary income sources for the model providers.
With the second (and higher margin) being user facing subscriptions. Right now 70% of OpenAI's revenue comes from chatgpt + enterprise gpt. I imagine Anthropic is similar, given the amount of investment in their generative UI. At the end of the day, model providers might just be consumer companies.
the hype will never die. all the smartest people in industry and government believe that there is a very high probability that this technology is near the edge of starting the AGI landslide. you dont need AGI to start the AGI landslide, you just need AI tools that are smart enough to automate the process of discovering and building the first AGI models. every conceivable heuristic indicates that we are near the edge. and because of this, AI has now become a matter of national security. the research and investment wont stop, because it cant, because it is now an arms race. this wont just fizzle out. it will be probed and investigated to absolute exhaustion before anyone feels safe enough to stop participating in the race. if you have been keeping up you will know that high level federal bureaucrats are now directly involved in openAI.
I am not among those smartest, so take my opinion with a mountain of salt. But I'm just not convinced that this is going in the direction of AGI.
The recent advances are truly jaw dropping. It absolutely merits being investigated to the hilt. There is a very good chance that it will end up being a net profit.
But intuitively they don't feel to me like they're getting more human. If anything I feel like the recent round of "get it to reason aloud" is the opposite of what makes "general intelligence" a thing. The vast majority of human behavior isn't reasoned, aloud or otherwise.
It'll be super cool if I'm wrong and we're just one algorithm or extra data set or scale factor of CPUs away from It, whatever It turns out to be. My intuition here isn't worth much. But I wouldn't be surprised if it was right despite that.
Did you criticize the turing test as being meaningless before it was easily passed by LLMs? if not i don't see how you can avoid updating on "this is getting more human" or at least "this is getting closer to intelligence" to avoid the human-bias
I never gave much thought to the Turing test one way or the other. It never struck me as especially informative.
I've always been more interested in the non-verbal aspects of human intelligence. I believe that "true AGI", whatever that is, is likely to be able to mimic sentient but non-verbal species. I'd like to see an AGI do what a dog or cat does.
LLMs are quite astonishing at mimicking something humans specifically do, the most "rational" parts of our brain. But they seem to jump past the basic, non-rational parts of the brain. And I don't think we'll see it as "true AGI" until it does that -- whatever that is.
I'm reminded of the early AI researchers who taught AI to play chess because it's what smart people do, but it couldn't do any of the things dumb people do. I think the biggest question right now is whether our present techniques are a misleading local maximum, or if we're on the right slope and just need to keep climbing.
I don't know. I know that any particular thing I say could be met with an LLM that did that thing. But at the moment an LLM doesn't seem to be capable of coding itself up that way. Yet.
I guess that was kind of my question when I thought about what you said. I could think of any things that a cat or dog do that _current_ LLMs can't do but nothing that seems fundamentally out of reach
What does it mean for something to "act on its own"? Every action taken is in some part due to what's happening in the environment right? I would argue that nothing "acts on it's own"
Possibly, but as of now it's a completely unsolved problem and to my knowledge nobody has shown even a tiny model being able to perform it.
Based on the top page today I may even be able to make the argument we can't even simulate the abilities of a fruit fly.
The absolute frontier models can perform only a fraction of a fraction of what a typical work day looks like for a human. I calculated the chance that a frontier model today has about a 1^-29 chance of performing a single day of connected tasks based on GAIA benchmarks.
> I never gave much thought to the Turing test one way or the other
then how are you qualified to even discuss this? shouldn't you have to have actually had a thought about something before you pretend to know something about it? i mean its pretty clear from your comment that you havent thought about AI for more than a year or two
itll be super cool huh? no, it wont be super cool. AGI will cause the total and complete destabilization of the world and with almost mathematical certainty it can be said that it will lead to a global war. a war fueled by AGI is the last war humans will ever witness. AGI is something that everyone should be very afraid of. unfortunately, the only experience that most people have with AI is watching star trek, playing halo, and consuming other media that depict sentient machines in a benign and very positive light. dont be fooled. the machines will not be your friends. its an outcome that must be avoided at all costs.
This article, and all the articles like it, are missing most of the puzzle.
Models don’t just compete on capability. Over the last year we’ve seen models and vendors differentiate along a number of lines in addition to capability:
- Safety
- UX
- Multi-modality
- Reliability
- Embeddability
And much more. Customers care about capability, but that’s like saying car owners care about horsepower — it’s a part of the choice but not the only piece.
To me, the vast majority of "consumers" as in B2C only care about price, specifically free. Pro and enterprise customers may be more focused on the capabilities you listed, but the B2C crowd is vastly in the free tier only space when it comes to GenAI.
One somewhat obsessive customer here: I pay for and use Claude, ChatGPT, Gemini, Perplexity, and one or two others.
The UX differences among the models are indeed becoming clearer and more important. Claude’s Artifacts and Projects are really handy as is ChatGPT’s Advanced Voice mode. Perplexity is great when I need a summary of recent events. Google isn’t charging for it yet, but NotebookLM is very useful in its own way as well.
When I test the underlying models directly, it’s hard for me to be sure which is better for my purposes. But those add-on features make a clear differentiation between the providers, and I can easily see consumers choosing one or another based on them.
I haven’t been following recent developments in the companies’ APIs, but I imagine that they are trying to differentiate themselves there as well.
The competition for big LLM AI companies is not other big LLM AI companies, but rather small LLM AI companies with good enough models. This is a classic innovator dilemma.
For example, I can imagine a team of cardiologists creating a fine tune LLM model.
The cardiologist checks the ECG, compare with the LLM results and checks the difference. If it can reduce error rate by like 10%, that's already really good.
My current stance on LLM is that it's good for stuff which is painful to generate, but easy to check (for you). It's easier/faster to read an email than to write it. If you're a domain expert, you can check the output, and so on.
The danger is in using it for stuff you cannot easily check, or trusting it implicitly because it is usually working.
> trusting it implicitly because it is usually working
I think this danger is understated. Humans are really prone to developing expectations based on past observations and then not thinking very critically about or paying attention to those things once those expectations are established. This is why "self driving" cars that work most of the time but demand that the driver remain attentive and prepared to take over are such a bad idea.
> The cardiologist checks the ECG, compare with the LLM results and checks the difference.
Perhaps you're confusing the acronym LLM (Large Language Model) with ML (Machine Learning)?
Analyzing electrocardiogram waveform data using a text-predictor LLM doesn't make sense: No matter how much someone invests in tweaking it to give semi-plausible results part of the time, it's fundamentally the wrong tool/algorithm for the job.
#2 is dead wrong, and shows that the author is not aware of the current exciting research happening in parameter efficient fine-tuning or representation/activation engineering space.
The idea that you need huge amounts of compute to innovate in a world of model merging and activation engineering shows a failure of imagination, not a failure to have the necessary resources.
PyReft, Golden Gate Claude (Steering/Control Vectors), Orthogonalization/Abliteration, and the hundreds of thousands of Lora and other adapters available on websites like civit.ai is proof that the author doesn't know what they're talking about re: point #2.
And I'm not even talking about the massive software/hardware improvements we are seeing for training/inference performance. I don't even need that, I just need evidence that we can massively improve off the shelf models with almost no compute resources, which I have.
The original SQL was essentially Prolog restricted to relational algebra and tuple relational calculus. SQL as is happened when a lot of cruft was added to the mathematical core.
On an amusing note, I've read something similar: Everything that works stops being called philosophy. Science and math being the two familiar examples.
I won't say that things like stoicism or humanism never worked. But they never got to the level of strict logical or experimental verifiability. Physics may be hard science, but the very notion of hard science, hypotheses, demand to replicate, demand to be able to falsify, etc, are all philosophy.
There is a reason for that. People who inquired into the actual functioning of the world used to be called philosophers. That's why so many foundations of mathematics actually come from philosophers. The split happened around the 17th century. Newton still called his monumental work "Natural Philosophy", not "Physics".
Just in case anyone's curious, this is from Bertrand Russell's "the history of philosophy".
> As soon as definite knowledge concerning any subject becomes possible, this subject ceases to be called philosophy, and becomes a separate science.
I'm not actually sure I agree with it, especially in light of less provable schools of science like string theory or some branches of economics, but it's a great idea.
PhD itself is an abbreviation for "Doctor of Philosophy." The title is more about the original Greek "lover of wisdom" than about the modern academic discipline of philosophy. https://en.wikipedia.org/wiki/Doctor_of_Philosophy
Doctor is similar - in the US, when someone says "Doctor" they usually mean "Medical Doctor" but "Doctor" just comes from the Greek "teacher" / "scholar" which is more broad and the title can still be used officially and correctly for PhDs. https://en.wikipedia.org/wiki/Doctor_(title)
Just a little correction. Doctor is Latin and roughly means "someone who has learned a lot."
Science also originally referred to knowledge. What we think of as "science" used to be called the natural sciences. Sometimes people get confused because I have a B.S. in Classics because science has lost that broader meaning.
It is still called natural science, but it used to be called natural philosophy.
And it is interesting, as you say, that when it comes to Bachelor/Master/Doctor of Science/Art/Philosophy (even professor), these are all titles formed from arbitrary terms that have been enshrined by the institutions that give people these titles.
Indeed, in the Summa Theologica, Thomas Aquinas asks if theology is a science, and concludes that it is. He also gave lip service to logical rigor and falsifiability, in the latter case by encouraging the discipline of asking contrary questions and answering them. What he didn't do was appeal to empirical data, to any great extent.
I think the reasoning behind "doctor of philosophy" may be lost to history. All knowing Wikipedia suggests that it didn't happen at once. My take was that the requirements for a modern PhD were added long after the title was adopted.
I suspect there was a time when a person could be well versed in multiple of what are now separated fields, and that you had to be a philosopher to make sense of science and math. Also, as science was flexing its own wings, claiming to be a philosopher might have been a way to gain an air of respectability, just like calling a physician "doctor" when the main impact of medicine was to kill rich people.
In college my Professor told me he was disappointed that the string theory hype was dying down because it made for a great subfield of algebraic geometry.
This is also true for consciousness or sentience. No matter how surprising abilities of non-human beings (and computer agents), it is something mysterious that only humans do.
Bert is already not an LLM and the vector embedding it generates are not AI. It is also first general solution for natural language search anyone has come up with. We call them vector databases. Again I'd wager this is because they actually work.
Lisp's inventor, John McCarthy, was an AI researcher. (The US government started funding AI research in the 1950s, expecting progress to be much faster than it actually was.)
People innately believe that intelligence isn't an algorithm. When a complex problem presents itself for the first time, people think "oh, this must be so complex that no algorithm can solve it, only AI," and when an algorithmic solution is found, people realise that the problem isn't that complex.
Indeed, if AI was an algorithm, imagine what would it feel like to be like one: at every step of your thinking process you are dragged by the iron hand of the algorithm, you have no agency in decision making, for every step is pre-determined already, and you're left the role of an observer. The algorithm leaves no room for intelligence.
it mostly depends on one's definition of an algorithm.
our brain is mostly scatter-gather with fuzzy pattern matching that loops back on itself. which is a nice loop, inputs feeding in, found patterns producing outputs and then it echoes back for some learning.
but of course most of it is noise, filtered out, most of the output is also just routine, most of the learning happens early when there's a big difference between the "echo" and the following inputs.
it's a huge self-referential state-machine. of course running it feels normal, because we have an internal model of ourselves, we ran it too, and if things are going as usual, it's giving the usual output. (and when the "baseline" is out of whack then even we have the psychopathologies.)
Is that not the human experience? I have no “agency” over the next thought to pop into my head. I “feel” like I can choose where to focus attention, but that too is a predictable outcome arising from the integration of my embryology, memories, and recently reinforced behaviors. “I” am merely an observer of my own mental state.
But that is an uncomfortable idea for most people.
If this was true, you could lay back relaxed and watch where your brain takes you. But we experience life as a never ending stream of choices, usually between what's easy and what's right, and pursuing the right choice takes a constant effort. We are presented with problems and have to figure out solutions on our own, with no guarantees of success.
This "I'm just an observer" idea may be true at some higher level, if you're a monk on the threshold of nirvana, but for common folk this mindset leads nowhere.
The „making choices“ could just be an illusion where it’s already clear from the inputs what you’re going to do. You rejecting determinism could already be determined and you couldn’t even choose to believe it. That’s not falsifiable of course, but your point doesn’t really contradict the idea of determinism due to the possibility.
The other option you don't mention is "algorithms can solve it, but they do something different to what humans do". That's what happened with Go and Chess, for example.
I agree with you that people don't consider intelligence as fundamentally algorithmic. But I think the appeal of algorithmic intelligence comes from the fact that a lot of intelligent behaviours (strategic thinking, decomposing a problem into subproblems, planning) are (or at least feel) algorithmic.
Exactly what I wrote recently: "The "AI effect" is behind some of the current confusion. As John McCarthy, AI pioneer who coined the term "artificial intelligence," once said: "As soon as it works, no one calls it AI anymore." This is why we often hear that AI is "far from existing." This led to the formulation of the Tesler's Theorem: "AI is whatever hasn't been done yet.""
https://www.lycee.ai/blog/there-are-indeed-artificial-intell...
> As soon as it works, no one calls it AI anymore.
So, what are good examples of some things that we used to call AI, which we don't call AI anymore because they work? All the examples that come to my mind (recommendation engines, etc.) do not have any real societal benefits.
Some examples are navigation algorithms, machine learning, neural networks, fuzzy logic, or computer vision. I personally learned several of those in a "Artificial Intelligence" CS course ~15 years ago, but most people would never think to call Google Maps, a smart thermostat learning their habits, or their doorbell camera recognizing faces "AI".
It's only recently with generative AI that you see any examples of the opposite, people outside the field calling LLMs or image generation "AI".
Chess engines. People once believed that in order for a computer to play chess we would first have to create an intelligence comparable to humans. Turns out you can win by evaluating millions of positions per turn instead. It would be somewhat disappointing for someone who was looking for a human-like opponent, that it can simply be brute forced. But it may be that a lot of “intelligence” is like that.
It’s the cognitive equivalent of the Indiana Jones sword vs gun scene.
Yeah but honestly we all know LLMs are different then say some chess ai.
You can thank social media for dumbing down a human technological milestone in artificial intelligence. I bet if there was social media around when we landed on the moon you’d get a lot of self important people rolling their eyes at the whole thing too.
This is a pretty common perspective that was introduced to me as “shifting the goalposts” in school. I have always found it a disingenuous argument because it’s applied so narrowly.
Humans are intelligent + humans play go => playing go is intelligent
Humans are intelligent + humans do algebra => doing algebra is intelligent
Meanwhile, humans in general are pretty terrible at exact, instantaneous arithmetic. But we aren’t claiming that computers are intelligent because they’re great at it.
Building a machine that does a narrowly defined task better than a human is an achievement, but it’s not intelligence.
Although, in the case of LLMs, in context learning is the closest thing I’ve seen to breaking free from the single-purpose nature of traditional ML/AI systems. It’s been interesting to watch for the past couple years because I still don’t think they’re “intelligent”, but it’s not just because they’re one trick ponies anymore. (So maybe the goalposts really are shifting?) I can’t quite articulate yet what I think is missing from current AI to bridge the gap.
> Meanwhile, humans in general are pretty terrible at exact, instantaneous arithmetic. But we aren’t claiming that computers are intelligent because they’re great at it.
"The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." - Edsger Dijkstra
Someday if we have computers that are capable of doing 100% of the cognitive tasks humans do, better than any human can, we might still say it’s “just” doing X or Y. It might even be disappointing that there isn't a “special sauce” to intelligence. But at the end of the day, the mechanism isn’t important.
We are already playing with some incredible ingredients. Machines that can instantly recall information, (in principle) connect to any electronic device, and calculate millions of times faster than brains, and perfectly self-replicate. Just using these abilities in a simple way is already pretty darn powerful.
> breaking free from the single-purpose nature of traditional ML/AI systems
it is really breaking free? so far LLMs in action seem to have a fairly limited scope -- there are a variety of purposes to which they can be applied but it's all essentially the same underlying task
It _is_ all the same task (generating text completions) but that pretraining on that task has seemed to be suitably abstract for the model to work decently well on more narrow problems—certainly they work dramatically better at a collection of tasks like “sentiment analysis of movie ratings” and “spam classifier” than if I took purposes built models for either of those tasks and tried using them like I could an LLM.
If you trace the etymology of computer in English, it means something like "small chalk/limestone pebbles used for counting do-er." (See Latin "calx".)
In Turkish, computer is "bilgisayar" which is a combination of "bilgi" meaning "knowledge" and "sayar" which is an intentional double-entendre of "counter"/"respects".
In the same line, there are also a phrase about technology, "is everything that doesn’t work yet." by Danny Hillis, "Electric motors were once technology – they were new and did not work well. As they evolved, they seem to disappear, even though they proliferated and were embedded by the scores into our homes and offices. They work perfectly, silently, unminded, so they no longer register as “technology.” https://kk.org/thetechnium/everything-that/
What about computer characters in video games? They're usually controlled by a system that everyone calls AI, but there's almost never any machine learning involved. What about fitting a line to a curve, i.e. linear regression? Definitely ML, but most people don't call it AI.
I would say people call NPCs "AI" only for historical reasons. Typically handwritten algorithms are not called AI nowadays.
I'm not sure, but I think statistical regression already specifies a simple model (e.g. a linear one), while the approach with neural networks doesn't make such strong assumptions about the model.
I actually think linear regression isn't usually called ML in practice. Anyway, I meant something like neural networks.
Interestingly, a big barrier to voice recognition is the same as with AI assistants - they don't understand context, and so they have a difficult time navigating messy inputs that you need assumed knowledge and contextual understanding for. Which is kinda the baseline for how humans communicate things to eachother with words in the first place.
There are really very few situations where people really want voice recognition. The main one is hands free controls when driving or for as a remote control for TV or music.
Exactly, yeah. The high frequency of mistakes from voice commands is a specific point of friction that makes me still prefer tactile buttons for cars - ideally somewhere you don't have to take your hands off the wheel or your eyes off the road. If voice command got really good, that would probably change for me.
Touchscreens are great for phones, but I'm really not a fan of them in cars where I prefer the tactile feedback of knowing what button is under my finger.
A lot of it just comes down to having the right tool for the right usecase, really.
Exactly. Machine learning used to be AI and "AI-driven solutions" were peddled over a decade ago. Then that died down. Now suddenly every product has to once again be "powered by AI" (even if under the hood all you're running is a good 'ol SVM).
I get the sense that the value prop of LLMs should first be cut into two categories: coding assistant, and everything else.
LLMs as coding assistants seem to be great. Let’s say that every working programmer will need an account and will pay $10/month (or their employer will).. what’s a fair comp for valuation? GitHub? That’s about $10Bn. Atlassian? $50Bn
The “everything else” bin is hard to pin down. There are some clear automation opportunities in legal, HR/hiring, customer service, and a few other fields - things that feel like $1-$10Bn opportunities.
Sure, the costs are atrocious, but what’s the revenue story?
> I get the sense that the value prop of LLMs should first be cut into two categories: coding assistant, and everything else.
Replace coding assistant with artists and you have the vibe of AI 2 years ago.
The issue is that these models are easy to make (if expensive) so the open source community (of which many, maybe most, are programmers themselves) will likely eat up any performance moat given enough time.
This story already played out with AI art. Nothing beats SD and comfyUI if you really need high quality and control.
Having been there for the dotcom boom and bust from pre-IPO Netscape to the brutal collapse of the market, it’s hard to say dotcoms don’t work. There was clearly something there of immense value, but it took a lot experimentation with business models and maturation of technology as well as the fundamental communications infrastructure of the planet. All told it feels like we’ve really only gained a smooth groove in the last 10 years.
I see no reason why AI will be particularly different. It seems difficult to make the case AI is useless, but it’s also not particularly mature with respect to fundamental models, tool chains, business models, even infrastructure.
In both cases speculative capital flowed into the entire industry, which brought us losers like pets.com but winners like Amazon.com, Netflix.com, Google.com, etc. Which of the AI companies today are the next generation of winners and losers? Who knows. And when the music stops will there be a massive reckoning? I hope not, but it’s always possible. It probably depends on how fast we converge to “what works,” how many grifters there are, how sophisticated equity investors are (and they are much more sophisticated now than they were in 1997), etc.
It seems very difficult to build a moat around a product when the product is supposed to be a generally capable tool and the input is English text. The more truly generally intelligent these models get the more interchangeable they become. It's too easy to swap one out for another.
Humans are the ultimate generally intelligent agents available on this planet. Even though most of them (us) are replaceable for mundane tasks, quite some are unique enough so that people seek their particular services and no one else's. And this is among the pool of about eight billion such agents.
It's even more relevant to AI, because the differences between the models (not just training data) may make them pretty uniquely suited to some areas, with any competition being markedly worse (in that area), or at least markedly different. And you only have maybe dozens, not billions, to choose from.
> Even though most of them (us) are replaceable for mundane tasks, quite some are unique enough so that people seek their particular services and no one else's.
Very few people manage that - indeed I can't think of anyone. Even movie stars get replaced with other movie stars if they try to charge too much. Certainly everyone in the tech industry (including the CEOs, the VCs, the investors etc.) has a viable substitute.
The moat is/will be the virtuous cycle of feeding user usage back into the model like it is for Google. That historically has been a powerful tool and it's something thats nearly impossible to get as a newcomer to the marketplace.
ChatGPT benefits from network effects, where user feedback on the quality of its answers helps improve the model over time. This reduces its reliance on external services like ScaleAI, lowering development costs.
Larger user base = increased feedback = improved quality of answers = moat
It is ironic that this article seems to focus on the business logic, considering that is the same myopia at these AI companies.
Not that physical/financial constraints are unimportant, but they often can be mitigated in other ways.
Some background: I was previously at one of these companies that got hoovered up in the past couple years by the bigs. My job was sort of squishy, but it could be summarized as 'brand manager' insofar as it was my job to aide in shaping the actual tone, behaviors, and personality of our particular product.
I tell you this because in full disclosure, I see the world through the product/marketing lens as opposed to the engineer lens.
They did not get it.
And by they I mean founds whose names you've heard of, people with absolute LOADS of experience in building a shipping technology products. There were no technical or budgetary constraints at this early stage, we were moving fast and trying shit. But they simply could not understand why we needed to differentiate and how that'd make us more competitive.
I imagine many technology companies go through this, and I don't blame technical founders who are paranoid about this stuff; it sounds like 'management bullshit' and a lot of it is, but at some point all organizations who break even or take on investors are going to be answerable to the market, and that means leaving no stone unturned in acquiring users and new revenue streams.
All of that to say, I do think a lot of these AI companies have yet to realize that there's a lot to be done user experience-wise. The interface alone - a text prompt(!?) is crazy out-of-touch to me. The fact that average users have no idea how to set up a good prompt and how hard everyone is making it for them to learn about that.
All of these decisions are pretty clearly made by someone who is technology-oriented, not user-oriented. There's no work I'm aware of being done on tone, or personality frameworks, or linguistics, or characterization.
Is the LLM high on numeracy? Is it doing code switching/matching, and should it? How is it qualifying its answers by way of accuracy in a way that aids the user learning how to prompt for improved accuracy? What about humor or style?
It just completely flew over everyone's heads. This may have been my fault. But I do think that the constraints you see to growth and durability of these companies will come down to how they're able to build a moat using strategies that don't require $$$$ and that cannot be easily replicated by competition.
Nobody is sitting in the seats at Macworld stomping their feet for Sam Altman. A big part of that is giving customers more than specs or fiddly features.
These companies need to start building a brand fast.
> If the proprietary models stop moving forward, the open source ones will quickly close the gap.
This is the Red Queen hypothesis in evolution. You have to keep running faster just to stay in place.
On it's face, this does seem like a sound argument that all the $$ following LLMs is irrational:
1. No matter how many billions you pour into your model, you're only ever, say, six months away from a competitor building a model that's just about as good. And so you already know you're going to need to spend an increased number of billions next year.
2. Like the gambler who tries to beat the house by doubling his bet each time, at some point there must be a number where that many billions is considered irrational by everybody.
3. Therefore it seems irrational to start putting in even the fewer billions of dollars now, knowing the above two points.
> it seems irrational to start putting in even the fewer billions of dollars now, knowing the above two points
This doesn’t follow. One, there are cash flows you can extract in the interim—standing in place is potentially lucrative.
And two, we don’t know if the curve continues until infinity or asymptotes. If it asymptotes, being the first at the asymptote means owning a profitable market. If it doesn’t, you’re going to get AGI.
Side note: bought but haven’t yet read The Red Queen. Believe it was a comment of yours that lead me to it.
Why would being the first at the (horizontal) asymptote mean owning a profitable market? In six months, somebody else is at the asymptote as well. If different models are mostly interchangable quality-wise, then the decision will mostly be made on price.
For a market to be both winner-take-all as well as lucrative, you need some kind of a feedback cycle from network effets, economies of scale, or maybe even really brutal lock-in. For example, for operating systems applications provide a network effect -- more apps means an OS will do better, a higher install base for an OS means it'll attract more apps. For social networks it's users.
One could imagine this being the case for LLMs; e.g. that LLMs with a lot of users can improve faster. But if the asymptote has been reached, then by definition there's no more improvement to be had, and none of this matters.
> In six months, somebody else is at the asymptote as well
But you were there first. That gives you all of a first mover’s advantages plus the profits from that period to set the terms of the market sharing.
None of this requires winner-takes-all economics. And given OpenAI’s revenues, it’s unclear they need to be the sole winner to make an investment thesis work.
The first mover advantage isn't as big an advantage as being the one to monopolize a commodity. While there are some historical advantages to first mover's doing this, it is the common case. I also think that is very nearly a winner take all sort of thing, or at least a very API providers will still matters, if this asymptotes.
Compare this to toilet paper or fuel, if AI asymptotes and everyone catches up in 6 months it will be be just like these. There are some minor differences in quality, but if a customer can buy something good enough to keep their ass clean or their car powered then they will tend to pick the cheapest option.
Sometimes it is subterfuge like ms stealing the OS market from IBM or there might be a state sponsored AI lab like so many state run industries pushing out competition, but there are a ton of ways to beat first movers.
If there is no niche, first-mover is useless. But if there is value to be had, it’s valuable. Fighting to tread water in a moving stream with the expectation the flow will slow isn’t irrational.
Are you claiming there is no Niche to making commodities? I haven't really considered it I could see an argument being made either way. Or are you saying that it's a product these API based llms are just not going to be how we use these tools? Or are you saying something else entirely?
Ok, so let's say I was there first. But now you're offering a product that's (by definition) just as good for a lower price. Why will the users stick to my product rather than paying your cheaper prices?
As for your second point, the profits from the first six months are going to be basically nothing compared to the years of massive investments needed to make this happen. Why would those profits give any kind of sustainable leverage?
Excellent point, but now consider the availability of open source models as well. They’re not as good as the frontier models, but at some point they get close enough, right?
I run Llama 3.x locally on my 5+ year old GPU. It’s pretty decent. It’s not as smart as GPT-4 or Claude, but it’s good enough for a lot of what I do, and most importantly there’s no cap on the number of prompts I can send in a given timeframe.
For many consumers, it’s fine to be a little bit behind the cutting edge. My smartphone is 2 models behind now. It’s fast enough, the camera is good enough, and the battery is holding up just fine. I don’t feel any need to fork over $1,000 for a new one right now.
I think the same phenomenon will become more common for AI enthusiasts. It’s hard to beat free.
And what happens to the economics of building frontier models then? I suspect a lot of air is going to be released from the AI bubble in the next couple of years.
>Therefore, if you are OpenAI, Anthropic, or another AI vendor, you have two choices. Your first is to spend enormous amounts of money to stay ahead of the market. This seems very risky though
Regarding point 6, Amazon invested heavily in building data centers across the US to enhance customer service and maintain a competitive edge. it was risky.
This strategic move resulted in a significant surplus of computing power, which Amazon successfully monetized. In fact, it became the company's largest profit generator.
After all, startups and businesses is all about taking risk, ain't it?
No, this an incorrect analogy on most points. Amazon retail marketplaces are absolutely not correlated with distributed data centers. Most markets are served out of a single cluster, which is likely to be in a different country; eg historically all of Europe from dublin, the americas from virginia, and asia pacific from seattle/portland.
The move of the retail marketplace hosting from seattle to virginia happened alongside, and continued long after, the start of AWS.
It is an utter myth, not promoted by Amazon, that AWS was some scheme to use “surplus computing power” from retail hosting/operations. It was an intentional business case to get in to a different B2B market as a service provider.
They eventually have to turn a profit or pass the hot potato. Maybe they’ll be the next generation of oligarchs supported by the state when they all go bankrupt but are too essential to fail.
My guess is that profitability is becoming increasingly difficult and nobody knows how yet… or whether it will be possible.
Seems like the concentration of capital is forcing the tech industry to take wilder and more ludicrous bets each year.
Don't mixup LLM with AI. Not every AI company works on top of LLM's, many are doing vision or robotics or even old-school AI.
Our system works, is AI, is profitable, doing vision. Vision scales. There's a little bit of LLM classification. And robotics also, but this part is not really AI, just a generic industry robot.
Enjoyed the article and thought many of the points were good.
Here's a counterargument.
> In other words, the billions that AWS spent on building data centers is a lasting defense. The billions that OpenAI spent on building prior versions of GPT is not, because better versions of it are already available for free on Github.
The money that OpenAI spends on renting GPUs to build the next model is not what builds the moat. The moat comes from the money/energy/expertise that OpenAI spends on the research and software development. Their main asset is not the current best model GPT-4; it is the evolving codebase that will be able to churn out GPT-5 and GPT-6. This is easy to miss because the platform can only churn out each model when combined with billions of dollars of GPU spend, but focusing on the GPU spend misses the point.
We're no longer talking about a thousand line PyTorch file with a global variable NUM_GPUs that makes everything better. OpenAI and competitors are constantly discovering and integrating improvements across the stack.
The right comparison is not OpenAI vs. AWS, it's OpenAI vs. Google. Google's search moat is not its compute cluster where it stores its index of the web. Its moat is the software system that incorporates tens of thousands of small improvements over the last 20 years. And similar to search, if an LLM is 15% better than the competitors, it has a good shot at capturing 80%+ of the market. (I don't have any interest in messing around with a less capable model if a clearly better one exists.)
Google was in some sense "lucky" that when they were beginning to pioneer search algorithms, the hardware (compute cluster) itself was not a solved problem the way it is today with AWS. So they had a multidimensional moat from the get-go, which probably slowed early competition until they had built up years' worth of process complexity to deter new entrants.
Whereas LLM competition is currently extremely fierce for a few reasons: NLP was a ripe academic field with a history of publishing and open source, VC funding environment is very favorable, and cloud compute is a mature product offering. Which explains why there is currently a proliferation of relatively similar LLM systems:
> Every LLM vendor is eighteen months from dead.
But the ramp-up time for competitors is only short right now because the whole business model (pretrain massive transformers -> RLHF -> chatbot interface) was only discovered 18 months ago (ChatGPT launched at the end of 2022) - and at that point all of the research ideas were published. By definition, the length of a process complexity moat can't exceed how long the incumbent has been in business! In five years, it won't be possible to raise a billion dollars and create a state of the art LLM system, because OpenAI and Anthropic will have been iterating on their systems continuously. Defections of senior researchers will hurt, and can speed up competitor ramp-time slightly, but over time a higher proportion of accumulated insights is stored in the software system rather than the minds of individual researchers.
Let me emphasize: the billions of dollars of GPU spend is a distraction; we focus on it because it is tangible and quantifiable, and it can feel good to be dismissive and say "they're only winning because they have tons of money to simply scale up models." That is a very partial view. There is a tremendous amount of incremental research going on - no longer published in academic journals - that has the potential to form a process complexity moat in a large and relatively winner-take-all market.
The infrastructure hardware and software is a commodity. Any real moat comes from access to data. I think we've seen data close up quite a bit since people realized that you can train LLMs with it, so I don't know that OpenAI's data access is better than when they trained GPT 4. In fact, it's probably worse unless they've cut independent deals with massive data providers.
Inference costs many many times less than training. I think we may end up with one more round of training a larger foundational model, and then it's going to have to wait for new, cheaper, more efficient hardware. It's quite possible that all of these first mover companies go bankrupt, but the models will still exist generating value.
I think we are in the middle of a steep S-curve of technology innovation. It is far from plateauing and there are still a bunch of major innovations that are likely to shift things even further. Interesting time and these companies are riding a wild wave. It is likely some will actually win big, but most will die - similar to previous technology revolutions.
The ones that win will win not just on technology, but on talent retention, business relationships/partnerships, deep funding, marketing, etc. The whole package really. Losing is easy, miss out on one of these for a short period of time and you've easily lost.
There is no major moat, except great execution across all dimensions.
Training took hundreds of thousands of years. Everyone just gets a foundation model to fine tune for another couple of decades before it can start flipping burgers.
"There is, however, one enormous difference that I didn’t think about: You can’t build a cloud vendor overnight. Azure doesn’t have to worry about a few executives leaving and building a worldwide network of data centers in 18 months."
This isn't true at all. There are like 8 of these companies stood up in the last three or four years fueled by massive investment of sovereign funds - mostly the saudi, dubai, northern europe, etc. oil-derived funds - all spending billions of dollars doing exactly that and getting something done.
The real problem is the ROI on AI spending is.. pretty much zero. The commonly asserted use cases are the following:
Chatbots
Developer tools
RAG/search
Not a one of these is going to generate $10 of additional revenue per sollar spent, nor likely even $2. Optimizing your customer services representatives from 8 conversations at once to an average of 12 or 16 is going to save you a whopping $2 per hour per CSR. It just isn't huge money. And RAG has many, many issues with document permissions that make the current approaches bad for enterprises - where the money is - who as a group haven't spent much of anything to even make basic search work.
> And RAG has many, many issues with document permissions
Why can't these providers access all documents and when answers are prompted, self-censor if the reply has references to documents that the end users do not have access permissions? In fact, I'm pretty sure that's how existing RAGaaS providers are handling document/file permissions.
Would the texts of those documents be part of the LLM training data? If so, there's an reliable way to keep a determined user from fetching stuff back out.
> The real problem is the ROI on AI spending is.. pretty much zero.
Companies in user acquisition/growth mode tend to have low internal ROI, but remember both Facebook and Google has the same issue -- then they introduced ads and all was well with their finances. Similar things will happen here.
LLMs are a utility, not a platform, and utility markets exert a downward pressure on pricing. Moreover, it's not obvious that -- once trained models hit the wild -- any one actor has or can develop significant competitive moats that would allow them to escape that price pressure. Beyond that, the digital marginal cost of services needs to be significantly reduced to keep these companies in business, but more efficient models leads to pushing inference out to end-user compute, which hollows out their business model (I assume that Apple dropping out of the OpenAI investment round was partially due to the wildly optimistic valuations involved, partially because they're betting on being able to optimize runtime costs down to iPhone levels).
Basically, I'd argue that LLMs look less like a Web 2.0 social media opportunity and more like Hashicorp or Docker, except with operational expenses running many orders of magnitude higher with costs scaling linearly to revenue.
And on other side, how many more paying users will there be? And of users that known about AI or have tried it are happy to use whatever is free at the moment. Or just whatever is on Google or Bing with add next to it?
Social media is free, subscription services are for stuff you can't get for free easily. But will there actually be similar need for AI? Be it any generation.
> LLMs are a utility, not a platform, and utility markets exert a downward pressure on pricing.
I think competition exerts a downward pressure on pricing, not being a utility personally. But I guess I agree with the utility analogy in that there are massively initial upfront costs and then the marginal costs are low.
> more efficient models leads to pushing inference out to end-user compute, which hollows out their business model
Faster CPUs have been coming forever but we keep coming up with ways of keeping them busy. I suspect the same pattern with AI. Thus server-based AI will always be better than local. In the future, I expect to be served by many dozens of persistent agents acting on my behalf (more agents as you go further into the future) and they won't be hosted on my smartphone.
> 10 years ago a decade old computer vs a current one would have made a huge difference.
For running Word or Excel or Node.js server apps, I would agree with you. But this is where new applications come in. Modern PCs with either a GPU or an NPU can run circles around your PC when it comes to running Llama or StableDiffusion locally. Same with regards to high end graphics, old PCs can not do real-time raytracing or upscaling with their lesser capabilities. I personally do rendering via Blender or astrophotography via PixInsight and I need all the cores + memory I can get for that.
Faster PCs make for more opportunities that were not possible earlier. But if you do not change your workloads as the hardware evolves, then you do not need to upgrade.
So I go to ask an LLM to answer to a question and it starts trying to sell me products I don't want or need? I'll need to filter it's output through a locally run adblock LLM which detects/flags/strips out advertising text before delivering the output. Hmmm.
I still have this pipe-dream of an augmented reality system where you can walk through the grocery store and it will superimpose a warning label over any brand connected to a company on the individual user's shit-list.
For many decades consumers have been told that the magnanimous "Free Market" (multiple definitions) is empowering them to vote with their wallets, however some of those same groups start acting rather suspiciously when faced with the possibility consumers might ever exercise that power.
> I still have this pipe-dream of an augmented reality system where you can walk through the grocery store and it will superimpose a warning label over any brand connected to a company on the individual user's shit-list.
There are boycott apps that let you scan barcodes, e.g.:
"The real problem is the ROI on AI spending is.. pretty much zero. The commonly asserted use cases are the following:
Chatbots Developer tools RAG/search"
I agree with you that ROI on _most_ AI spending is indeed poor, but AI is more than LLM's. Alas, what used to be called AI before the onset of the LLM era is not deemed sexy today, even though it can still make very good ROI when it is the appropriate tool for solving a problem.
AI is a term that changes year to year. I don't remember where I heard it but I like that definition that "as soon as computers can do it well it stops becoming AI and just becomes standard tech". Neural Networks were "AI" for a while - but if I use a NN for risk underwriting nobody will call that AI now. It is "just ML" and not exciting. Will AI = LLM forever now? If so what is the next round of advancements called?
While it might be possible for a deep-pocketed organization to spin up a cloud provider overnight, it doesn't mean that people will use it. In general, the switching cost of migrating compute infrastructure from one service to another is much higher than the switching cost of changing the LLM used for inference.
Amazon doesn't need to worry about suddenly losing its entire customer base to Alibaba, Yandex, or Oracle.
Amazon has spent a ton on developing features that lock in users beyond what is fundamentally 'cloud', i.e. the ability to lease computers as a commodity. I have always warned employers about the downside of adopting all that extraneous stuff, with little effect.
> I think we are in the middle of a steep S-curve of technology innovation
We are? What innovation?
What do we need innovation for? What present societal problems can tech innovation possibly address? Surely none of the big ones, right? So then is it fit to call technological change - 'innovation'?
I'd agree that LLMs improve upon having to read Wikipedia for topics I'm interested in but would investing billions in Wikipedia and organizing human knowledge have produced a better outcome than relying on a magic LLM? Almost certainly, in my mind.
You see, people are pouring billions into LLMs and not Wikipedia not because it is a better product - but because they foresee a possibility of an abusive monopoly and that really excites them.
That's not innovation - that's more of the same anti-social behaviour that makes any meaningful innovation extremely difficult.
In a way, that is in alignment with their founding goal of organizing human knowledge.
An ad is just a piece of data (eg Bob is selling shovels for $10 + shipping), with the additional metadata that someone really wants you to see it (eg Bob paid Google $100 to tell every person who searched for "shovels" that he's selling shovels for $10 + shipping).
Selling ads is organizing human knowledge.
Yes, you could argue using a market system is not the best way to do this, but it is a way to do it.
I'm not sure the Wikipedia example is a strong one as that site has it's own serious problems with "abusive monopolies" in its moderator cliques and biases (as with any social platform).
At least with the current big AI players there is the potential for differentiation through competition.
Unless there is some similar initiative with the Wikipedias, the problem of single supplier dominance is a difficult one to see as the way forward.
I can solve Wikipedia's woes quite easily - Wikipedia should limit itself to math, science, engineering, medicine, physics, chemistry, geography and other disciplines that are not at all in dispute.
Politics, history, religion and other topics of conversation that are matters of opinion, taste and state sponsored propaganda need to be off limits.
Its mission ought to be to provide a PhD level education in all technical fields, not engage in shortening historical events and/or opinions/preferences/beliefs down to a few pages and disputing which pages need to be left in or out. Let fools engage in that task on their own time.
Well for medicine, there's alternative medicines like chiropractic and homeopathy. According to some people, these are snake oil and quackery, but many others swear by them, and many healthcare systems even provide these treatments despite the complete lack of scientific backing.
A lot of things that seem like simple facts are actually abstractions based on abstractions based on abstractions. The further up the chain, the more theoretical it gets. This is true of the humanities and science.
Nonetheless, Microsoft is firing up a nuclear reactor to power a new data center. My money is in the energy sector right now. Obvious boom coming with solar, nuclear and AI.
Yeah, I moved into a vanguard energy ETF. Some of this is already priced in (which makes me whince a little when buying) but the actual upside hasn't even hit yet, which will bring in a rush of activity on energy stocks. I have noticed over the decades that energy companies always do very well, especially in the long run.
I was looking for renewables investments (particularly solar) but there is just too much uncertainty and the companies are so small.
The car wasn't a horse that was better, but a car has not changed drastically since they went mainstream.
They've gotten better, more efficient, loaded with tech, but are still roughly 4 seats, 4 doors, 4 wheels, driven by petroleum.
I know that this is a massive oversimplification, but I think we have seen the "shape" of LLMs\Gen AI\AI products already and it's all incremental improvements from here on out with more specialization.
We are going to have SUVs, sports cars, and single seater cars, not flying cars. AI will be made more fit for purpose for more people to use, but isn't going to replace people outright in their jobs.
Feels like someone might have said this in 1981 about personal computers.
"We've pretty much seen their shape. The IBM PC isn't fundamentally very different from the Apple II. Probably it's just all incremental improvements from here on out."
I would agree with your counter if it weren't for the realities of power usage, hardware constraints, evident diminishing returns on training larger models, and as always the fact that AI is still looking for the problem it solves, aside from mass employment.
Computers solved a tangible problem in every area of life, AI is being forced everywhere and is arguably failing to make a big gain in areas that it should excel.
I think the big game changer in the PC space was graphics cards, but since their introduction, it has all been incremental improvement -- at first, pretty fast, then... slower. Much like CPU improvements, although those started earlier.
I can't think of a point where the next generation of PCs was astoundingly different from the prior one... just better. It used to be that they were reliably faster or more capable almost every year, now the rate of improvements is almost negligible. (Yes, graphics are getting better, but not very fast if you aren't near the high end.)
I would say smartphones and tablets are new devices, not, per se, PCs.
Smartphones did bring something huge to the table, and exploded accordingly. But here we are, less than 2 decades from their introduction and... the pace of improvement has gone back to being incremental at best.
They're, at a minimum, PC replacements in the sense that I'm currently on the Internet and posting on a message board from my bathtub instead of needing to go over to my desk to do that.
Smartphones and tablets aren't PC replacements, they're TV replacements. Even we, deep into the throes of the smartphone age, still need to use real computers to do the tasks we were using them for in 1995, e.g. programming and word processing.
The fact that computing went through yet another phase transition with mobile is pretty much undisputed. Battery tech, energy efficient chips, screens, radios, solid-state storage, etc. Of course it's not the same as a desktop/notebook, because it's optimized for being in our pockets/hands not put on a desk. (But the compute power is there, plug in peripherals and many phones easily beat a desktop even from ~10-15 years ago.)
It remains to be seen where LLMs and this crop of ML tech can go in the garden of forking paths, and whether they can reach truly interesting places.
Smartphones let us do some tasks we couldn't do at all in 1995, such as GPS turn-by-turn navigation.
Sure, they also do other tasks we were already doing in 1995: mobile telephony, mobile TV watching (we had portable handheld TVs back then--they were awful but they worked), mobile music listening (Sony Walkman), easy-to-use photography, but there's some things that we couldn't do at all before this technology became mainstream, like mobile banking.
GPS turn-by-turn wasn't commonly available in the consumer space in 1995, but the first successful consumer turn-by-turn GPS came out 1998 with the Garmin StreetPilot. So close to 1995 but not quite there.
Still though, GPS navigation was definitely a common thing pre-iPhone. I remember being gifted a cheap one a few years before the iPhone came out. We didn't need smartphones to do GPS navigation. TomTom came out in 2004, three years before the iPhone launched and four years before Android had Google Maps with Turn by Turn navigation.
You need smartphones to do GPS navigation in the modern sense, which is:
1) look for e.g. "Italian restaurant" in your area
2) look at choices nearby, screen out ones currently closed, too expensive, bad reviews, etc., and pick one
3) navigate there
In the early days of GPS, you needed an actual address to navigate to. That's not very useful if you don't know where you want to go in the first place. Smartphones changed all that: now you don't need to know a specific place you want to go, you just need to know what you want to do generally, and the nav app will help you find where exactly you want to go and take you there. That's impossible without either an internet connection or a very large local database (that's out of date).
The POI database was largely the big selling point of TomTom though.
And the Magellan GPS I had pre-iPhone had quite a POI database as well. I think it had monthly updates available online. I could search "Blockbuster" or "gas station" or "public parking" or "hotel" and it would know locations. Obviously it wasn't making dinner recommendations but it did have a lot of restaurants in it.
Also you specifically called out turn by turn. Knowing the one off holiday hours of a hole in the wall restaurant isn't necessary for turn by turn GPS.
> The IBM PC isn't fundamentally very different from the Apple II. Probably it's just all incremental improvements from here on out.
Honestly? I'm not sure I even disagree with that!
Amazon is just an incremental improvement over mail order catalogs. Netflix is just an incremental improvement over Blockbusters. UberEats is just an incremental improvement over calling a pizzeria. Google Sheets is just an incremental improvement over Lotus 1-2-3.
Most of the stuff we're doing these days could have been done with 1980s technology - if a bit less polished. Even the Cloud is just renting a time slice on a centralized mainframe.
With the IBM PC it was already reasonably clear what the computer was going to be. Most of the innovation since then is "X, but on a computer", "Y, but over the internet", or just plain market saturation. I can only think of two truly world-changing innovations: 1) smartphones, and 2) social media.
The current AI wave is definitely producing interesting results, but there is still a massive gap between what it can do and what we have been promised. Considering current models have essentially been trained on the entire internet and they have now poisoned the well and made mass gathering of more training data impossible, I doubt we're going to see another two-orders-of-magnitude improvement any time soon. If anything, for a lot of applications it's probably going to get worse as the training set becomes out of date.
And if people aren't willing to pay for the current models, they aren't going to pay for a model which hallucinates 50% less often. They're going to need that two-orders-of-magnitude improvement to actually become world-changing. Taking into account how much money those companies are losing, are they going to survive the 5-10 years or more until they reach that point?
And even smartphone is just combination of with computer, but smaller and over internet. It just got pretty good eventually. https://en.wikipedia.org/wiki/HP_200LX arguably is "smartphone" supporting modem or network connectivity, albeit not wireless...
the difference is a personal computer is a vaguely-shaped thing that has a few extremely broad properties and there is tremendous room inside that definition for things to change shape, grow, and improve. processor architecture, board design, even the methods of powering the machine can change and you still meet the basic definition of "a computer."
for better or worse, people saying AI in any capacity right now are referring to current-generation generative AI, and more specifically diffusion image generation and LLMs. that's a very specific category of narrowly-defined technologies that don't have a lot of room to grow based on the way that they function, and research seems to be bearing out that we're starting to reach peak functionality and are now just pushing for efficiency. for them to improve dramatically or suddenly and radically change would require so many innovations or discoveries that they are unrecognizable.
what you're doing is more akin to looking at a horse and going "i forsee this will last forever because maybe someday someone will invent a car, which is basically the same thing." it's not. the limitations of horses are a feature of their biology and you are going to see diminishing returns on selectively breeding as you start to max out the capabilities of the horse's overall design, and while there certainly will be innovations in transportation in the future, the horse is not going to be a part of them.
And that's more or less true? By 1989 certainly, we had word processors, spreadsheets, email, BBSes. We have better versions of everything today, but fundamentally, yes, the "shape" of the personal computer was firmly established by the late 1980s.
Anyone comfortable with MS-DOS Shell would not be totally lost on a modern desktop.
Yes, you can say all of them are Turing machines... but now those Turing machines are very fast and in your pocket and beat you at chess, etc. The UI/UX has changed humans.
The big missing thing between both the metaphor in the OP's link and yours is that I just can't fathom any of these companies being able to raise a paying subscriber base that can actually cover the outrageous costs of this tech. It feels like a pipe dream.
Putting aside that I fundamentally don't think AGI is in the tech tree of LLM, if you will, that there's no route from the latter to the former: even if there is, even if it takes, I dunno, ten years: I just don't think ChatGPT is a compelling enough product to fund about $70 billion in research costs. And sure, they aren't having to yet thanks to generous input from various commercial and private interests but like... if this is going to be a stable product at some point, analogous to something like AWS, doesn't it have to... actually make some money?
Like sure, I use ChatGPT now. I use the free version on their website and I have some fun with AI dungeon and occasionally use generative fill in Photoshop. I paid for AI dungeon (for awhile, until I realized their free models actually work better for how I like to play) but am now on the free version. I don't pay for ChatGPT's advanced models, because nothing I've seen in the trial makes it more compelling an offering than the free version. Adobe Firefly came to me free as an addon to my creative cloud subscription, but like, if Adobe increased the price, I'm not going to pay for it. I use it because they effectively gave it to me for free with my existing purchase. And I've played with Copilot a bit too, but honestly found it more annoying than useful and I'm certainly not paying for that either.
And I realize I am not everyone and obviously there are people out there paying for it (I know a few in fact!) but is there enough of those people ready to swipe cards for... fancy autocomplete? Text generation? Like... this stuff is neat. And that's about where I put it for myself: "it's neat." OpenAI supposedly has 3.9 million subscribers right now, and if those people had to foot that 7 billion annual spend to continue development, that's about $150 a month. This product has to get a LOT, LOT better before I personally am ready to drop a tenth of that, let alone that much.
And I realize this is all back-of-napkin math here but still: the expenses of these AI companies seem so completely out of step with anything approaching an actual paying user base, so hilariously outstripping even the investment they're getting from other established tech companies, that it makes me wonder how this is ever, ever going to make so much as a dime for all these investors.
In contrast, I never had a similar question about cars, or AWS. The pitch of AWS makes perfect sense: you get a server to use on the internet for whatever purpose, and you don't have to build the thing, you don't need to handle HVAC or space, you don't need a last-mile internet connection to maintain, and if you need more compute or storage or whatever, you move a slider instead of having to pop a case open and install a new hard drive. That's absolutely a win and people will pay for it. Who's paying for AI and why?
> The big missing thing between both the metaphor in the OP's link and yours is that I just can't fathom any of these companies being able to raise a paying subscriber base that can actually cover the outrageous costs of this tech. It feels like a pipe dream.
I suspect these companies will introduce ads at some point similar to Google and Facebook for similar reasons, and it will be highly profitable.
I mean that's quite an assertion given how the value of existing digital ad space is already cratering and users are more than ever in open rebellion against ad supported services. And besides which, isn't the whole selling point of AI to be an agent that accesses the internet and filters out the bullshit? So what, you're going to do that, then add your own bullshit to the output?
I agree with everything you said in your previous post. The LLM as a search engine might basically eat Google's lunch in the same way the internet ate cable tv's lunch. Cable tv was a terrible product due to the advertising, and you could escape the BS by watching stuff online. Now look where we are.
Fun fact: Cable itself was originally the ad-free "premium" version of over-the-air television! And it eventually added some ads and then more ads and now we're at present day, where you get roughly 8 minutes of ads per 22 minutes of content.
I noticed visiting a hotel recently (like fuck I pay for cable!) that their cable service had sped up the intro of Law and Order SVU, presumably to make room for more advertisements.
>I noticed visiting a hotel recently (like fuck I pay for cable!) that their cable service had sped up the intro of Law and Order SVU, presumably to make room for more advertisements.
That's pretty smart of them! It makes sense to try to increase the time for ads, and decrease the time for the TV show; maybe next they can just cut the credits, or cut out parts of each act in the show. There's no reason not to. After all, the only people still paying for cable TV subscriptions are elderly people who absolutely refuse to give up their reruns of decades-old shows and also refuse to learn how to use Hulu, so they'll happily pay $150/month so they can watch 70s shows with even worse video quality than TVs in the 70s had.
> Cable itself was originally the ad-free "premium" version of over-the-air television
Often repeated but not true. Cable was originally just all the ad-supported OTA stations delivered over a wire. It had ads on day 1. It would be over a decade before the first premium cable channels would launch, and even then most of the first channels also had ads.
> I fundamentally don't think AGI is in the tech tree of LLM
There is a lot of hype and investor money running on the idea that if you make a really good text prediction engine, it will usefully impersonate a reasoning AI.
Even worse is the hype that a extraordinarily good text prediction engine will usefully impersonate a reasoning AI that usefully impersonating a specialized AI.
The car was very much a horse that was better though. It has replaced the horse ( or other draught animal ) and that's basically it. I'm not even sure it has brought fundamentally new and different use cases.
It did. It changed the entire way society operates and how and where we live by greatly increasing the speed of travel and the ability to transport goods (without laying rail everywhere).
I see 2 paths:
- Consumers - the Google way: search and advertise to consumers
- Businesses - the AWS way: attrack businesses to use your API and lock them in
The first is fickle. Will OpenAI become the door to the Internet? You'll need people to stop using Google Search and rely on ChatGPT for that to happen. Will become a commodity. Short term you can charge a subscription but long term will most likely become a commondity with advertising.
The second is tangible. My company is plugged directly to the OpenAI API. We build on it. Still very early and not so robust. But getting better and cheaper and faster over time. Active development. No reason to switch to something else as long as OpenAI leads the pack.
There would need to be significant capabilities that openai doesn't have or wouldn't be built on a short-ish timeline to have the enterprise switch. There's tons of bureaucratic work going on behind the scenes to approve a new vendor.
Sure it does. Ask any common mortal about AI and they'll mention ChatGPT - not Claude, Gemini or whatever else. They might not even know OpenAI. But they do know ChatGPT.
Has it become a verb yet? Waiting to peole to replace "I googled how to..." with "I chatgpted how to...".
You’re moving the goalposts a little here. In your other post you implied you were using OpenAI for its technical properties. “But getting better and cheaper and faster over time.”
Whether something has more name recognition isn’t completely related here. But if that’s what you really mean, as you state, “any common mortal about AI and they'll mention ChatGPT - not Claude, Gemini or whatever else. They might not even know OpenAI. But they do know ChatGPT,” then I mostly agree, but as an outsider it doesn’t seem like this is a robust reason to build on top of.
OpenAI's sole focus is serving AI to consumers and businesses. I trust them more to remain backwards compatible over time.
Google changed their AI name multiple times. I've built on them before and they end of lifed the product I was using. Zero confidence Gemini will be there tomorrow.
20 years ago people asked that exact question. E-Commerce emerged. People knew the physical process of buying things would move online. Took some time. Sure, more things emerged but monetizing the Internet still remains about selling you something.
Assuming AI progress continues, AI could replace both Microsoft's biggest product, OS, and Google's biggest product, search and ads. And there is a huge tail end of things autonomous driving/flying, drug discovery, robotics, programming, healthcare etc.
Too vague. How would it replace Windows? How would it replace search?
The latter is more believable to me, but how would the AI-enhanced version generate the additional revenue that its costs require? And I think a lot of improvement would be required... people are going to be annoyed by things like hallucinations while trying to buy products.
In reality, as soon as a competitor shows up, Google will add whatever copycat features it needs to search. So it isn't clear to me that search is a market that can be won, short of antitrust making changes.
You saw "Her" or Iron man? That's how it could replace windows. Basically entire OS working in natural language. Imagine a great human personal assistant who operates computer for you.
And searchGPT could replace Google. Also, wasn't the point of genAI is that it is cheaper than the entire stack of search? At least I know for some recommendation GPT-4 is literally cheaper than many companies in house models and I know companies who saved money using GenAI.
Not saying any of these would likely happen, but still it is not in the fantasy realm.
> people are going to be annoyed by things like hallucinations
That's like saying people are going to be annoyed by no delivery in online shopping. Yes it happened more often earlier, but we are arguing more on the ideal case if it could get solved. That's why I said if AI progress is good in my message, which means we solve hallucination etc.
OK, then I'm pretty sure it won't happen. I don't want to have a personal assistant replace an OS. I don't even want to talk to Alexa. And I'm not alone...
I meant human personal assistant as I was clear in previous post. If you are given human personal assistant for free who could replace your screen usage, would you be open to replace your OS with it?
Also we are just talking on theoretical level if AI is able to imitate human assistant(which I personally give 10% chance of happening, but not out of realm of possibility).
My guess would be using "AI" to increase/enhance sales with your existing processes. Pay for this product, get 20% increased sales, ad revenue, yada yada.
The mistake in that article is the assumption that these companies collecting those gigantic VC funding rounds are looking to stay ahead of the pack and be there even 10 years down the road.
That's a fundamental misunderstanding of the (especially) US startup culture in the last maybe 10-20 years. Only very rarely is the goal of the founders and angel investors to build an actual sustainable business.
In most cases the goal is to build enough perceived value by wild growth financed by VC money & by fueling hype that an subsequent IPO will let the founders and initial investors recoup their investment + get some profit on top. Or, find someone to acquire the company before it reaches the end of its financial runway.
And then let the poor schmucks who bought the business hold the bag (and foot the bill). Nobody cares if the company becomes irrelevant or even goes under at that point anymore - everyone who did has has recouped their expense already. If the company stays afloat - great, that's a bonus but not required.
456 comments
[ 3.3 ms ] story [ 287 ms ] threadI love it. More goodies for us
Monetizing all of this is frankly...not my problem.
Getting lucky twice is a row is really really lucky. Getting lucky three times in a row is not more likely because they were lucky two times in a row
Very few other large tech companies have deep technical competence at CEO level leadership
LLMs are capital intensive. They’re a natural fit for financing.
The ones without money will usually lose because they get less opportunity to get in front of eyeballs. Occasionally they manage it anyway, because despite the myth that the VCs love to tell, they aren't really great at finding and promulgating the best tech.
Say what you want about the morals associated to said offerings, but it sounds very interesting for companies that may want to use a plagiarizing machine for copying GPL code while covering their asses in some way (at least in terms of cost).
I’m reminded of slime molds solving mazes [1]. In essence, VC allows entrepreneurs to explore the solution space aggressively. Once solutions are found, resources are trimmed.
[1] https://www.mbl.edu/news/how-can-slime-mold-solve-maze-physi...
Except for all the others.
Plenty of entrepreneurship is low to moderate risk, bootstrappable and/or unscalable. That describes where VC is a raw deal. It also does not describe AI.
Tech companies purchased television away from legacy media companies and added (1) unskippable ads, (2) surveillance, (3) censorship and revocation of media you don't physically own, and now they're testing (4) ads while shows are paused.
There's no excuse for getting fooled again.
They destroyed the Taxi industry, I used to be able to just walk out to the taxi rank and get in the first taxi, but not anymore. Now I have to organize it on an app or with a phone call to a robot, then wait for the car to arrive, and finally I have to find the car among all the others that other people called.
Food delivery used to be done by the restaurants own delivery staff, it was fast, reliable and often free if ordering for 2+ people. Now it always costs extra, and there are even more fees if I want the food while it's still hot. Zero care is taken with the delivery, food/drinks are not kept upright and can be a total mess on arrival. Sometimes it's escaped the container and is just in the plastic bag. I have ended up preferring to go pickup food myself over getting it delivered, even when I have a migraine, it's just gone to shit.
These rideshare and delivery companies are disgusting and terrible.
Getting a taxi was awful before ride-sharing apps. You'd have to walk to a taxi stop, or wait on the side of the road and hope you could hail one. Once the ride-sharing apps came in, suddenly getting a ride became a lot simpler. Our taxi companies are still alive, though they have their own apps now -- something that wouldn't have happened without competition -- and they also work together with the ride-hailing companies as a provider. You could still hail taxis or get them from stops too, though that isn't recommended given that they might try to run the meter by taking a longer route.
For food delivery, before the apps, most places didn't deliver food. Nowadays, more places deliver. Even if a place already had their own delivery drivers, they didn't get rid of them. We get a choice, to use the app or to use the restaurant's own delivery. Usually the app is better for smaller meals since it has a lower minimum order amount, but the restaurant provides faster delivery for bigger orders.
I assume you are talking about airports. Guess what, they still exist in many places. And on the other hand, for US, other than a few big cities, the "normal" taxi experience is that you call a number and maybe a taxi shows up in half an hour. With Uber, that becomes 10 minutes or less, with live map updates. Give me that and I'll be happy to forget about Uber.
There used to be several taxi ranks on the streets with restaurants/bars/nightclubs, now there are none.
The airport is the only place left that still has a taxi rank, and it doesn't even have taxies all the time anymore.
I can now summon a cab from the comfort of the phone I'm holding, and know that they'll accept my credit card. I know the price before I get in and I know the route they should take. I'm not going to get taken for an unnecessary scenic tourist surcharge detour.
people don't like feeling they got cheated, and pre-uber, taxis did that all the time.
In every city I have lived, that is a good thing, despite all the bad things ridesharing startups may have done.
Leaves you in a worse position than you started with.
1. Training the next generation of models
2. Providing worldwide scalable infrastructure to serve those models (ideally at a profit)
It's hard enough to accomplish #1, without worrying about competing against the hyperscalers on #2. I think we'll see large licensing deals (similar to Anthropic + AWS, OpenAI + Azure) as one of the primary income sources for the model providers.
With the second (and higher margin) being user facing subscriptions. Right now 70% of OpenAI's revenue comes from chatgpt + enterprise gpt. I imagine Anthropic is similar, given the amount of investment in their generative UI. At the end of the day, model providers might just be consumer companies.
The recent advances are truly jaw dropping. It absolutely merits being investigated to the hilt. There is a very good chance that it will end up being a net profit.
But intuitively they don't feel to me like they're getting more human. If anything I feel like the recent round of "get it to reason aloud" is the opposite of what makes "general intelligence" a thing. The vast majority of human behavior isn't reasoned, aloud or otherwise.
It'll be super cool if I'm wrong and we're just one algorithm or extra data set or scale factor of CPUs away from It, whatever It turns out to be. My intuition here isn't worth much. But I wouldn't be surprised if it was right despite that.
I've always been more interested in the non-verbal aspects of human intelligence. I believe that "true AGI", whatever that is, is likely to be able to mimic sentient but non-verbal species. I'd like to see an AGI do what a dog or cat does.
LLMs are quite astonishing at mimicking something humans specifically do, the most "rational" parts of our brain. But they seem to jump past the basic, non-rational parts of the brain. And I don't think we'll see it as "true AGI" until it does that -- whatever that is.
I'm reminded of the early AI researchers who taught AI to play chess because it's what smart people do, but it couldn't do any of the things dumb people do. I think the biggest question right now is whether our present techniques are a misleading local maximum, or if we're on the right slope and just need to keep climbing.
Based on the top page today I may even be able to make the argument we can't even simulate the abilities of a fruit fly.
The absolute frontier models can perform only a fraction of a fraction of what a typical work day looks like for a human. I calculated the chance that a frontier model today has about a 1^-29 chance of performing a single day of connected tasks based on GAIA benchmarks.
then how are you qualified to even discuss this? shouldn't you have to have actually had a thought about something before you pretend to know something about it? i mean its pretty clear from your comment that you havent thought about AI for more than a year or two
Models don’t just compete on capability. Over the last year we’ve seen models and vendors differentiate along a number of lines in addition to capability:
- Safety
- UX
- Multi-modality
- Reliability
- Embeddability
And much more. Customers care about capability, but that’s like saying car owners care about horsepower — it’s a part of the choice but not the only piece.
According to who?
https://www.theinformation.com/articles/openai-coo-says-chat...
[1] https://www.bloomberg.com/news/articles/2024-09-05/openai-hi...
The math doesn't work for them to have 10m paying customers. It's not even close.
The UX differences among the models are indeed becoming clearer and more important. Claude’s Artifacts and Projects are really handy as is ChatGPT’s Advanced Voice mode. Perplexity is great when I need a summary of recent events. Google isn’t charging for it yet, but NotebookLM is very useful in its own way as well.
When I test the underlying models directly, it’s hard for me to be sure which is better for my purposes. But those add-on features make a clear differentiation between the providers, and I can easily see consumers choosing one or another based on them.
I haven’t been following recent developments in the companies’ APIs, but I imagine that they are trying to differentiate themselves there as well.
Safely and effectively, that is. Dangerous and inappropriate is obviously a much wider set of possibilities.
The cardiologist checks the ECG, compare with the LLM results and checks the difference. If it can reduce error rate by like 10%, that's already really good.
My current stance on LLM is that it's good for stuff which is painful to generate, but easy to check (for you). It's easier/faster to read an email than to write it. If you're a domain expert, you can check the output, and so on. The danger is in using it for stuff you cannot easily check, or trusting it implicitly because it is usually working.
I think this danger is understated. Humans are really prone to developing expectations based on past observations and then not thinking very critically about or paying attention to those things once those expectations are established. This is why "self driving" cars that work most of the time but demand that the driver remain attentive and prepared to take over are such a bad idea.
Perhaps you're confusing the acronym LLM (Large Language Model) with ML (Machine Learning)?
Analyzing electrocardiogram waveform data using a text-predictor LLM doesn't make sense: No matter how much someone invests in tweaking it to give semi-plausible results part of the time, it's fundamentally the wrong tool/algorithm for the job.
The idea that you need huge amounts of compute to innovate in a world of model merging and activation engineering shows a failure of imagination, not a failure to have the necessary resources.
PyReft, Golden Gate Claude (Steering/Control Vectors), Orthogonalization/Abliteration, and the hundreds of thousands of Lora and other adapters available on websites like civit.ai is proof that the author doesn't know what they're talking about re: point #2.
And I'm not even talking about the massive software/hardware improvements we are seeing for training/inference performance. I don't even need that, I just need evidence that we can massively improve off the shelf models with almost no compute resources, which I have.
Logic programming? AI until SQL came out. Now it's not AI.
OCR, computer algebra systems, voice recognition, checkers, machine translation, go, natural language search.
All solved, all not AI any more yet all were AI before they got solved by AI researchers.
There's even a name for it: https://en.m.wikipedia.org/wiki/AI_effect?utm_source=perplex...
logic programming is not directly linked to SQL, and has its own AI term now: https://en.wikipedia.org/wiki/GOFAI.
> As soon as definite knowledge concerning any subject becomes possible, this subject ceases to be called philosophy, and becomes a separate science.
I'm not actually sure I agree with it, especially in light of less provable schools of science like string theory or some branches of economics, but it's a great idea.
Doctor is similar - in the US, when someone says "Doctor" they usually mean "Medical Doctor" but "Doctor" just comes from the Greek "teacher" / "scholar" which is more broad and the title can still be used officially and correctly for PhDs. https://en.wikipedia.org/wiki/Doctor_(title)
Science also originally referred to knowledge. What we think of as "science" used to be called the natural sciences. Sometimes people get confused because I have a B.S. in Classics because science has lost that broader meaning.
And it is interesting, as you say, that when it comes to Bachelor/Master/Doctor of Science/Art/Philosophy (even professor), these are all titles formed from arbitrary terms that have been enshrined by the institutions that give people these titles.
I think the reasoning behind "doctor of philosophy" may be lost to history. All knowing Wikipedia suggests that it didn't happen at once. My take was that the requirements for a modern PhD were added long after the title was adopted.
https://en.wikipedia.org/wiki/Doctor_of_Philosophy
I suspect there was a time when a person could be well versed in multiple of what are now separated fields, and that you had to be a philosopher to make sense of science and math. Also, as science was flexing its own wings, claiming to be a philosopher might have been a way to gain an air of respectability, just like calling a physician "doctor" when the main impact of medicine was to kill rich people.
Disclosure: PhD, but ambivalent about titles.
But it may be a while.
[1] https://taylor.town/synthetic-intelligence
Just don't use the word Champaign anywhere near a Miller Highlife in France. I think they're digging up old guillotines for that.
Edit: Your Dijkstra quote may be missing an "is":
..the question whether machines can think is as relevant
Indeed, if AI was an algorithm, imagine what would it feel like to be like one: at every step of your thinking process you are dragged by the iron hand of the algorithm, you have no agency in decision making, for every step is pre-determined already, and you're left the role of an observer. The algorithm leaves no room for intelligence.
our brain is mostly scatter-gather with fuzzy pattern matching that loops back on itself. which is a nice loop, inputs feeding in, found patterns producing outputs and then it echoes back for some learning.
but of course most of it is noise, filtered out, most of the output is also just routine, most of the learning happens early when there's a big difference between the "echo" and the following inputs.
it's a huge self-referential state-machine. of course running it feels normal, because we have an internal model of ourselves, we ran it too, and if things are going as usual, it's giving the usual output. (and when the "baseline" is out of whack then even we have the psychopathologies.)
https://www.youtube.com/watch?v=-rxXoiQyQVc
But that is an uncomfortable idea for most people.
There is not much of a difference in considering it deterministic or not, because we can't determine it.
Has there been a serious effort to, using something that might be capable of doing so, like neural nets?
This "I'm just an observer" idea may be true at some higher level, if you're a monk on the threshold of nirvana, but for common folk this mindset leads nowhere.
I don't agree with this framing. A person's opinion of what a "good" or "right" outcome is is one of the inputs to the algorithm.
When you decide to put in less effort it's affecting the algorithm because the weight of the high-effort outcome is reduced.
The trouble is, you see, that if you do know Right from Wrong you can't choose Wrong. You just can't do it and live.
It is quite a convincing book and maybe even a dangerous book in a sense.
I agree with you that people don't consider intelligence as fundamentally algorithmic. But I think the appeal of algorithmic intelligence comes from the fact that a lot of intelligent behaviours (strategic thinking, decomposing a problem into subproblems, planning) are (or at least feel) algorithmic.
So, what are good examples of some things that we used to call AI, which we don't call AI anymore because they work? All the examples that come to my mind (recommendation engines, etc.) do not have any real societal benefits.
It's only recently with generative AI that you see any examples of the opposite, people outside the field calling LLMs or image generation "AI".
It’s the cognitive equivalent of the Indiana Jones sword vs gun scene.
I don't doubt that there were marketing people wanting to attach an AI label to them, but that's just marketing BS.
You can thank social media for dumbing down a human technological milestone in artificial intelligence. I bet if there was social media around when we landed on the moon you’d get a lot of self important people rolling their eyes at the whole thing too.
Humans are intelligent + humans play go => playing go is intelligent
Humans are intelligent + humans do algebra => doing algebra is intelligent
Meanwhile, humans in general are pretty terrible at exact, instantaneous arithmetic. But we aren’t claiming that computers are intelligent because they’re great at it.
Building a machine that does a narrowly defined task better than a human is an achievement, but it’s not intelligence.
Although, in the case of LLMs, in context learning is the closest thing I’ve seen to breaking free from the single-purpose nature of traditional ML/AI systems. It’s been interesting to watch for the past couple years because I still don’t think they’re “intelligent”, but it’s not just because they’re one trick ponies anymore. (So maybe the goalposts really are shifting?) I can’t quite articulate yet what I think is missing from current AI to bridge the gap.
"The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." - Edsger Dijkstra
We are already playing with some incredible ingredients. Machines that can instantly recall information, (in principle) connect to any electronic device, and calculate millions of times faster than brains, and perfectly self-replicate. Just using these abilities in a simple way is already pretty darn powerful.
it is really breaking free? so far LLMs in action seem to have a fairly limited scope -- there are a variety of purposes to which they can be applied but it's all essentially the same underlying task
A hundred years ago computers were 'electronic brains'.
It goes dormant after loosing its edge and then re-emerges when some research project gains traction again.
But more seriously, I wonder what the etymology of 'völva' is...
AI = machine learning
Yes, that makes a convolutional net trained on recognizing digits an AI.
I'm not sure, but I think statistical regression already specifies a simple model (e.g. a linear one), while the approach with neural networks doesn't make such strong assumptions about the model.
I actually think linear regression isn't usually called ML in practice. Anyway, I meant something like neural networks.
There are really very few situations where people really want voice recognition. The main one is hands free controls when driving or for as a remote control for TV or music.
Touchscreens are great for phones, but I'm really not a fan of them in cars where I prefer the tactile feedback of knowing what button is under my finger.
A lot of it just comes down to having the right tool for the right usecase, really.
LLMs as coding assistants seem to be great. Let’s say that every working programmer will need an account and will pay $10/month (or their employer will).. what’s a fair comp for valuation? GitHub? That’s about $10Bn. Atlassian? $50Bn
The “everything else” bin is hard to pin down. There are some clear automation opportunities in legal, HR/hiring, customer service, and a few other fields - things that feel like $1-$10Bn opportunities.
Sure, the costs are atrocious, but what’s the revenue story?
Replace coding assistant with artists and you have the vibe of AI 2 years ago.
The issue is that these models are easy to make (if expensive) so the open source community (of which many, maybe most, are programmers themselves) will likely eat up any performance moat given enough time.
This story already played out with AI art. Nothing beats SD and comfyUI if you really need high quality and control.
I see no reason why AI will be particularly different. It seems difficult to make the case AI is useless, but it’s also not particularly mature with respect to fundamental models, tool chains, business models, even infrastructure.
In both cases speculative capital flowed into the entire industry, which brought us losers like pets.com but winners like Amazon.com, Netflix.com, Google.com, etc. Which of the AI companies today are the next generation of winners and losers? Who knows. And when the music stops will there be a massive reckoning? I hope not, but it’s always possible. It probably depends on how fast we converge to “what works,” how many grifters there are, how sophisticated equity investors are (and they are much more sophisticated now than they were in 1997), etc.
Very few people manage that - indeed I can't think of anyone. Even movie stars get replaced with other movie stars if they try to charge too much. Certainly everyone in the tech industry (including the CEOs, the VCs, the investors etc.) has a viable substitute.
Larger user base = increased feedback = improved quality of answers = moat
Not that physical/financial constraints are unimportant, but they often can be mitigated in other ways.
Some background: I was previously at one of these companies that got hoovered up in the past couple years by the bigs. My job was sort of squishy, but it could be summarized as 'brand manager' insofar as it was my job to aide in shaping the actual tone, behaviors, and personality of our particular product.
I tell you this because in full disclosure, I see the world through the product/marketing lens as opposed to the engineer lens.
They did not get it.
And by they I mean founds whose names you've heard of, people with absolute LOADS of experience in building a shipping technology products. There were no technical or budgetary constraints at this early stage, we were moving fast and trying shit. But they simply could not understand why we needed to differentiate and how that'd make us more competitive.
I imagine many technology companies go through this, and I don't blame technical founders who are paranoid about this stuff; it sounds like 'management bullshit' and a lot of it is, but at some point all organizations who break even or take on investors are going to be answerable to the market, and that means leaving no stone unturned in acquiring users and new revenue streams.
All of that to say, I do think a lot of these AI companies have yet to realize that there's a lot to be done user experience-wise. The interface alone - a text prompt(!?) is crazy out-of-touch to me. The fact that average users have no idea how to set up a good prompt and how hard everyone is making it for them to learn about that.
All of these decisions are pretty clearly made by someone who is technology-oriented, not user-oriented. There's no work I'm aware of being done on tone, or personality frameworks, or linguistics, or characterization.
Is the LLM high on numeracy? Is it doing code switching/matching, and should it? How is it qualifying its answers by way of accuracy in a way that aids the user learning how to prompt for improved accuracy? What about humor or style?
It just completely flew over everyone's heads. This may have been my fault. But I do think that the constraints you see to growth and durability of these companies will come down to how they're able to build a moat using strategies that don't require $$$$ and that cannot be easily replicated by competition.
Nobody is sitting in the seats at Macworld stomping their feet for Sam Altman. A big part of that is giving customers more than specs or fiddly features.
These companies need to start building a brand fast.
This is the Red Queen hypothesis in evolution. You have to keep running faster just to stay in place.
On it's face, this does seem like a sound argument that all the $$ following LLMs is irrational:
1. No matter how many billions you pour into your model, you're only ever, say, six months away from a competitor building a model that's just about as good. And so you already know you're going to need to spend an increased number of billions next year.
2. Like the gambler who tries to beat the house by doubling his bet each time, at some point there must be a number where that many billions is considered irrational by everybody.
3. Therefore it seems irrational to start putting in even the fewer billions of dollars now, knowing the above two points.
This doesn’t follow. One, there are cash flows you can extract in the interim—standing in place is potentially lucrative.
And two, we don’t know if the curve continues until infinity or asymptotes. If it asymptotes, being the first at the asymptote means owning a profitable market. If it doesn’t, you’re going to get AGI.
Side note: bought but haven’t yet read The Red Queen. Believe it was a comment of yours that lead me to it.
For a market to be both winner-take-all as well as lucrative, you need some kind of a feedback cycle from network effets, economies of scale, or maybe even really brutal lock-in. For example, for operating systems applications provide a network effect -- more apps means an OS will do better, a higher install base for an OS means it'll attract more apps. For social networks it's users.
One could imagine this being the case for LLMs; e.g. that LLMs with a lot of users can improve faster. But if the asymptote has been reached, then by definition there's no more improvement to be had, and none of this matters.
But you were there first. That gives you all of a first mover’s advantages plus the profits from that period to set the terms of the market sharing.
None of this requires winner-takes-all economics. And given OpenAI’s revenues, it’s unclear they need to be the sole winner to make an investment thesis work.
Compare this to toilet paper or fuel, if AI asymptotes and everyone catches up in 6 months it will be be just like these. There are some minor differences in quality, but if a customer can buy something good enough to keep their ass clean or their car powered then they will tend to pick the cheapest option.
Sometimes it is subterfuge like ms stealing the OS market from IBM or there might be a state sponsored AI lab like so many state run industries pushing out competition, but there are a ton of ways to beat first movers.
As for your second point, the profits from the first six months are going to be basically nothing compared to the years of massive investments needed to make this happen. Why would those profits give any kind of sustainable leverage?
I run Llama 3.x locally on my 5+ year old GPU. It’s pretty decent. It’s not as smart as GPT-4 or Claude, but it’s good enough for a lot of what I do, and most importantly there’s no cap on the number of prompts I can send in a given timeframe.
For many consumers, it’s fine to be a little bit behind the cutting edge. My smartphone is 2 models behind now. It’s fast enough, the camera is good enough, and the battery is holding up just fine. I don’t feel any need to fork over $1,000 for a new one right now.
I think the same phenomenon will become more common for AI enthusiasts. It’s hard to beat free.
And what happens to the economics of building frontier models then? I suspect a lot of air is going to be released from the AI bubble in the next couple of years.
A clever AI algorithm run on rented compute is not a moat.
Regarding point 6, Amazon invested heavily in building data centers across the US to enhance customer service and maintain a competitive edge. it was risky.
This strategic move resulted in a significant surplus of computing power, which Amazon successfully monetized. In fact, it became the company's largest profit generator.
After all, startups and businesses is all about taking risk, ain't it?
The move of the retail marketplace hosting from seattle to virginia happened alongside, and continued long after, the start of AWS.
It is an utter myth, not promoted by Amazon, that AWS was some scheme to use “surplus computing power” from retail hosting/operations. It was an intentional business case to get in to a different B2B market as a service provider.
My guess is that profitability is becoming increasingly difficult and nobody knows how yet… or whether it will be possible.
Seems like the concentration of capital is forcing the tech industry to take wilder and more ludicrous bets each year.
Our system works, is AI, is profitable, doing vision. Vision scales. There's a little bit of LLM classification. And robotics also, but this part is not really AI, just a generic industry robot.
Here's a counterargument.
> In other words, the billions that AWS spent on building data centers is a lasting defense. The billions that OpenAI spent on building prior versions of GPT is not, because better versions of it are already available for free on Github.
The money that OpenAI spends on renting GPUs to build the next model is not what builds the moat. The moat comes from the money/energy/expertise that OpenAI spends on the research and software development. Their main asset is not the current best model GPT-4; it is the evolving codebase that will be able to churn out GPT-5 and GPT-6. This is easy to miss because the platform can only churn out each model when combined with billions of dollars of GPU spend, but focusing on the GPU spend misses the point.
We're no longer talking about a thousand line PyTorch file with a global variable NUM_GPUs that makes everything better. OpenAI and competitors are constantly discovering and integrating improvements across the stack.
The right comparison is not OpenAI vs. AWS, it's OpenAI vs. Google. Google's search moat is not its compute cluster where it stores its index of the web. Its moat is the software system that incorporates tens of thousands of small improvements over the last 20 years. And similar to search, if an LLM is 15% better than the competitors, it has a good shot at capturing 80%+ of the market. (I don't have any interest in messing around with a less capable model if a clearly better one exists.)
Google was in some sense "lucky" that when they were beginning to pioneer search algorithms, the hardware (compute cluster) itself was not a solved problem the way it is today with AWS. So they had a multidimensional moat from the get-go, which probably slowed early competition until they had built up years' worth of process complexity to deter new entrants.
Whereas LLM competition is currently extremely fierce for a few reasons: NLP was a ripe academic field with a history of publishing and open source, VC funding environment is very favorable, and cloud compute is a mature product offering. Which explains why there is currently a proliferation of relatively similar LLM systems:
> Every LLM vendor is eighteen months from dead.
But the ramp-up time for competitors is only short right now because the whole business model (pretrain massive transformers -> RLHF -> chatbot interface) was only discovered 18 months ago (ChatGPT launched at the end of 2022) - and at that point all of the research ideas were published. By definition, the length of a process complexity moat can't exceed how long the incumbent has been in business! In five years, it won't be possible to raise a billion dollars and create a state of the art LLM system, because OpenAI and Anthropic will have been iterating on their systems continuously. Defections of senior researchers will hurt, and can speed up competitor ramp-time slightly, but over time a higher proportion of accumulated insights is stored in the software system rather than the minds of individual researchers.
Let me emphasize: the billions of dollars of GPU spend is a distraction; we focus on it because it is tangible and quantifiable, and it can feel good to be dismissive and say "they're only winning because they have tons of money to simply scale up models." That is a very partial view. There is a tremendous amount of incremental research going on - no longer published in academic journals - that has the potential to form a process complexity moat in a large and relatively winner-take-all market.
The ones that win will win not just on technology, but on talent retention, business relationships/partnerships, deep funding, marketing, etc. The whole package really. Losing is easy, miss out on one of these for a short period of time and you've easily lost.
There is no major moat, except great execution across all dimensions.
The fact that human intelligence exists means that the idea of human level intelligence is not a pipe dream.
The question is whether or not the basic underlying technology of the LLM can achieve that level of intelligence.
It was all explained in that recent paper, "Attention is all your meat"
"There is, however, one enormous difference that I didn’t think about: You can’t build a cloud vendor overnight. Azure doesn’t have to worry about a few executives leaving and building a worldwide network of data centers in 18 months."
This isn't true at all. There are like 8 of these companies stood up in the last three or four years fueled by massive investment of sovereign funds - mostly the saudi, dubai, northern europe, etc. oil-derived funds - all spending billions of dollars doing exactly that and getting something done.
The real problem is the ROI on AI spending is.. pretty much zero. The commonly asserted use cases are the following:
Chatbots Developer tools RAG/search
Not a one of these is going to generate $10 of additional revenue per sollar spent, nor likely even $2. Optimizing your customer services representatives from 8 conversations at once to an average of 12 or 16 is going to save you a whopping $2 per hour per CSR. It just isn't huge money. And RAG has many, many issues with document permissions that make the current approaches bad for enterprises - where the money is - who as a group haven't spent much of anything to even make basic search work.
Why can't these providers access all documents and when answers are prompted, self-censor if the reply has references to documents that the end users do not have access permissions? In fact, I'm pretty sure that's how existing RAGaaS providers are handling document/file permissions.
Companies in user acquisition/growth mode tend to have low internal ROI, but remember both Facebook and Google has the same issue -- then they introduced ads and all was well with their finances. Similar things will happen here.
Basically, I'd argue that LLMs look less like a Web 2.0 social media opportunity and more like Hashicorp or Docker, except with operational expenses running many orders of magnitude higher with costs scaling linearly to revenue.
Social media is free, subscription services are for stuff you can't get for free easily. But will there actually be similar need for AI? Be it any generation.
I think competition exerts a downward pressure on pricing, not being a utility personally. But I guess I agree with the utility analogy in that there are massively initial upfront costs and then the marginal costs are low.
> more efficient models leads to pushing inference out to end-user compute, which hollows out their business model
Faster CPUs have been coming forever but we keep coming up with ways of keeping them busy. I suspect the same pattern with AI. Thus server-based AI will always be better than local. In the future, I expect to be served by many dozens of persistent agents acting on my behalf (more agents as you go further into the future) and they won't be hosted on my smartphone.
For running Word or Excel or Node.js server apps, I would agree with you. But this is where new applications come in. Modern PCs with either a GPU or an NPU can run circles around your PC when it comes to running Llama or StableDiffusion locally. Same with regards to high end graphics, old PCs can not do real-time raytracing or upscaling with their lesser capabilities. I personally do rendering via Blender or astrophotography via PixInsight and I need all the cores + memory I can get for that.
Faster PCs make for more opportunities that were not possible earlier. But if you do not change your workloads as the hardware evolves, then you do not need to upgrade.
For many decades consumers have been told that the magnanimous "Free Market" (multiple definitions) is empowering them to vote with their wallets, however some of those same groups start acting rather suspiciously when faced with the possibility consumers might ever exercise that power.
Also once you've shitlisted all of these 11 companies:
We're probably gonna find it hard to find something to eat.Tongue planted slightly in cheek of course.
There are boycott apps that let you scan barcodes, e.g.:
https://play.google.com/store/apps/details?id=com.boycat.app
But there are a ton of others.
It is a natural extension to add this to AR/MR when the time is opportune.
I agree with you that ROI on _most_ AI spending is indeed poor, but AI is more than LLM's. Alas, what used to be called AI before the onset of the LLM era is not deemed sexy today, even though it can still make very good ROI when it is the appropriate tool for solving a problem.
Amazon doesn't need to worry about suddenly losing its entire customer base to Alibaba, Yandex, or Oracle.
We are? What innovation?
What do we need innovation for? What present societal problems can tech innovation possibly address? Surely none of the big ones, right? So then is it fit to call technological change - 'innovation'?
I'd agree that LLMs improve upon having to read Wikipedia for topics I'm interested in but would investing billions in Wikipedia and organizing human knowledge have produced a better outcome than relying on a magic LLM? Almost certainly, in my mind.
You see, people are pouring billions into LLMs and not Wikipedia not because it is a better product - but because they foresee a possibility of an abusive monopoly and that really excites them.
That's not innovation - that's more of the same anti-social behaviour that makes any meaningful innovation extremely difficult.
An ad is just a piece of data (eg Bob is selling shovels for $10 + shipping), with the additional metadata that someone really wants you to see it (eg Bob paid Google $100 to tell every person who searched for "shovels" that he's selling shovels for $10 + shipping).
Selling ads is organizing human knowledge.
Yes, you could argue using a market system is not the best way to do this, but it is a way to do it.
At least with the current big AI players there is the potential for differentiation through competition.
Unless there is some similar initiative with the Wikipedias, the problem of single supplier dominance is a difficult one to see as the way forward.
Politics, history, religion and other topics of conversation that are matters of opinion, taste and state sponsored propaganda need to be off limits.
Its mission ought to be to provide a PhD level education in all technical fields, not engage in shortening historical events and/or opinions/preferences/beliefs down to a few pages and disputing which pages need to be left in or out. Let fools engage in that task on their own time.
Medicine - COVID vaccines.
Science - global warming.
Unfortunately, humans and their politics are intertwined into more aspects of life than one might deem rational.
In gold rush, shell shovels.
In computing power rush, sell energy.
I was looking for renewables investments (particularly solar) but there is just too much uncertainty and the companies are so small.
They've gotten better, more efficient, loaded with tech, but are still roughly 4 seats, 4 doors, 4 wheels, driven by petroleum.
I know that this is a massive oversimplification, but I think we have seen the "shape" of LLMs\Gen AI\AI products already and it's all incremental improvements from here on out with more specialization.
We are going to have SUVs, sports cars, and single seater cars, not flying cars. AI will be made more fit for purpose for more people to use, but isn't going to replace people outright in their jobs.
"We've pretty much seen their shape. The IBM PC isn't fundamentally very different from the Apple II. Probably it's just all incremental improvements from here on out."
Computers solved a tangible problem in every area of life, AI is being forced everywhere and is arguably failing to make a big gain in areas that it should excel.
I think the big game changer in the PC space was graphics cards, but since their introduction, it has all been incremental improvement -- at first, pretty fast, then... slower. Much like CPU improvements, although those started earlier.
I can't think of a point where the next generation of PCs was astoundingly different from the prior one... just better. It used to be that they were reliably faster or more capable almost every year, now the rate of improvements is almost negligible. (Yes, graphics are getting better, but not very fast if you aren't near the high end.)
I would say incremental, but it does solve a different set of problems (connectedness, mobility) than the PC.
Smartphones did bring something huge to the table, and exploded accordingly. But here we are, less than 2 decades from their introduction and... the pace of improvement has gone back to being incremental at best.
It remains to be seen where LLMs and this crop of ML tech can go in the garden of forking paths, and whether they can reach truly interesting places.
Sure, they also do other tasks we were already doing in 1995: mobile telephony, mobile TV watching (we had portable handheld TVs back then--they were awful but they worked), mobile music listening (Sony Walkman), easy-to-use photography, but there's some things that we couldn't do at all before this technology became mainstream, like mobile banking.
Still though, GPS navigation was definitely a common thing pre-iPhone. I remember being gifted a cheap one a few years before the iPhone came out. We didn't need smartphones to do GPS navigation. TomTom came out in 2004, three years before the iPhone launched and four years before Android had Google Maps with Turn by Turn navigation.
In the early days of GPS, you needed an actual address to navigate to. That's not very useful if you don't know where you want to go in the first place. Smartphones changed all that: now you don't need to know a specific place you want to go, you just need to know what you want to do generally, and the nav app will help you find where exactly you want to go and take you there. That's impossible without either an internet connection or a very large local database (that's out of date).
And the Magellan GPS I had pre-iPhone had quite a POI database as well. I think it had monthly updates available online. I could search "Blockbuster" or "gas station" or "public parking" or "hotel" and it would know locations. Obviously it wasn't making dinner recommendations but it did have a lot of restaurants in it.
Also you specifically called out turn by turn. Knowing the one off holiday hours of a hole in the wall restaurant isn't necessary for turn by turn GPS.
Honestly? I'm not sure I even disagree with that!
Amazon is just an incremental improvement over mail order catalogs. Netflix is just an incremental improvement over Blockbusters. UberEats is just an incremental improvement over calling a pizzeria. Google Sheets is just an incremental improvement over Lotus 1-2-3.
Most of the stuff we're doing these days could have been done with 1980s technology - if a bit less polished. Even the Cloud is just renting a time slice on a centralized mainframe.
With the IBM PC it was already reasonably clear what the computer was going to be. Most of the innovation since then is "X, but on a computer", "Y, but over the internet", or just plain market saturation. I can only think of two truly world-changing innovations: 1) smartphones, and 2) social media.
The current AI wave is definitely producing interesting results, but there is still a massive gap between what it can do and what we have been promised. Considering current models have essentially been trained on the entire internet and they have now poisoned the well and made mass gathering of more training data impossible, I doubt we're going to see another two-orders-of-magnitude improvement any time soon. If anything, for a lot of applications it's probably going to get worse as the training set becomes out of date.
And if people aren't willing to pay for the current models, they aren't going to pay for a model which hallucinates 50% less often. They're going to need that two-orders-of-magnitude improvement to actually become world-changing. Taking into account how much money those companies are losing, are they going to survive the 5-10 years or more until they reach that point?
for better or worse, people saying AI in any capacity right now are referring to current-generation generative AI, and more specifically diffusion image generation and LLMs. that's a very specific category of narrowly-defined technologies that don't have a lot of room to grow based on the way that they function, and research seems to be bearing out that we're starting to reach peak functionality and are now just pushing for efficiency. for them to improve dramatically or suddenly and radically change would require so many innovations or discoveries that they are unrecognizable.
what you're doing is more akin to looking at a horse and going "i forsee this will last forever because maybe someday someone will invent a car, which is basically the same thing." it's not. the limitations of horses are a feature of their biology and you are going to see diminishing returns on selectively breeding as you start to max out the capabilities of the horse's overall design, and while there certainly will be innovations in transportation in the future, the horse is not going to be a part of them.
Anyone comfortable with MS-DOS Shell would not be totally lost on a modern desktop.
I think in fact the previous point stands: we've seen the shape of what's coming, it's gonna be now about incremental steps.
Putting aside that I fundamentally don't think AGI is in the tech tree of LLM, if you will, that there's no route from the latter to the former: even if there is, even if it takes, I dunno, ten years: I just don't think ChatGPT is a compelling enough product to fund about $70 billion in research costs. And sure, they aren't having to yet thanks to generous input from various commercial and private interests but like... if this is going to be a stable product at some point, analogous to something like AWS, doesn't it have to... actually make some money?
Like sure, I use ChatGPT now. I use the free version on their website and I have some fun with AI dungeon and occasionally use generative fill in Photoshop. I paid for AI dungeon (for awhile, until I realized their free models actually work better for how I like to play) but am now on the free version. I don't pay for ChatGPT's advanced models, because nothing I've seen in the trial makes it more compelling an offering than the free version. Adobe Firefly came to me free as an addon to my creative cloud subscription, but like, if Adobe increased the price, I'm not going to pay for it. I use it because they effectively gave it to me for free with my existing purchase. And I've played with Copilot a bit too, but honestly found it more annoying than useful and I'm certainly not paying for that either.
And I realize I am not everyone and obviously there are people out there paying for it (I know a few in fact!) but is there enough of those people ready to swipe cards for... fancy autocomplete? Text generation? Like... this stuff is neat. And that's about where I put it for myself: "it's neat." OpenAI supposedly has 3.9 million subscribers right now, and if those people had to foot that 7 billion annual spend to continue development, that's about $150 a month. This product has to get a LOT, LOT better before I personally am ready to drop a tenth of that, let alone that much.
And I realize this is all back-of-napkin math here but still: the expenses of these AI companies seem so completely out of step with anything approaching an actual paying user base, so hilariously outstripping even the investment they're getting from other established tech companies, that it makes me wonder how this is ever, ever going to make so much as a dime for all these investors.
In contrast, I never had a similar question about cars, or AWS. The pitch of AWS makes perfect sense: you get a server to use on the internet for whatever purpose, and you don't have to build the thing, you don't need to handle HVAC or space, you don't need a last-mile internet connection to maintain, and if you need more compute or storage or whatever, you move a slider instead of having to pop a case open and install a new hard drive. That's absolutely a win and people will pay for it. Who's paying for AI and why?
I suspect these companies will introduce ads at some point similar to Google and Facebook for similar reasons, and it will be highly profitable.
I noticed visiting a hotel recently (like fuck I pay for cable!) that their cable service had sped up the intro of Law and Order SVU, presumably to make room for more advertisements.
That's pretty smart of them! It makes sense to try to increase the time for ads, and decrease the time for the TV show; maybe next they can just cut the credits, or cut out parts of each act in the show. There's no reason not to. After all, the only people still paying for cable TV subscriptions are elderly people who absolutely refuse to give up their reruns of decades-old shows and also refuse to learn how to use Hulu, so they'll happily pay $150/month so they can watch 70s shows with even worse video quality than TVs in the 70s had.
Often repeated but not true. Cable was originally just all the ad-supported OTA stations delivered over a wire. It had ads on day 1. It would be over a decade before the first premium cable channels would launch, and even then most of the first channels also had ads.
There is a lot of hype and investor money running on the idea that if you make a really good text prediction engine, it will usefully impersonate a reasoning AI.
Even worse is the hype that a extraordinarily good text prediction engine will usefully impersonate a reasoning AI that usefully impersonating a specialized AI.
I see 2 paths: - Consumers - the Google way: search and advertise to consumers - Businesses - the AWS way: attrack businesses to use your API and lock them in
The first is fickle. Will OpenAI become the door to the Internet? You'll need people to stop using Google Search and rely on ChatGPT for that to happen. Will become a commodity. Short term you can charge a subscription but long term will most likely become a commondity with advertising.
The second is tangible. My company is plugged directly to the OpenAI API. We build on it. Still very early and not so robust. But getting better and cheaper and faster over time. Active development. No reason to switch to something else as long as OpenAI leads the pack.
Has it become a verb yet? Waiting to peole to replace "I googled how to..." with "I chatgpted how to...".
Whether something has more name recognition isn’t completely related here. But if that’s what you really mean, as you state, “any common mortal about AI and they'll mention ChatGPT - not Claude, Gemini or whatever else. They might not even know OpenAI. But they do know ChatGPT,” then I mostly agree, but as an outsider it doesn’t seem like this is a robust reason to build on top of.
Google changed their AI name multiple times. I've built on them before and they end of lifed the product I was using. Zero confidence Gemini will be there tomorrow.
There are so many ways, it makes the question seem nonsensical.
Ways to monetize AI so far:
Metered APIs (OpenAI and others)
Subscription products built on it (Copilot, ChatGPT, etc.)
Using it as a feature to give products a competitive edge (Apple Intelligence, Tesla FSD)
Selling the hardware (Nvidia)
What similar parallel can we think of for AI?
The latter is more believable to me, but how would the AI-enhanced version generate the additional revenue that its costs require? And I think a lot of improvement would be required... people are going to be annoyed by things like hallucinations while trying to buy products.
In reality, as soon as a competitor shows up, Google will add whatever copycat features it needs to search. So it isn't clear to me that search is a market that can be won, short of antitrust making changes.
And searchGPT could replace Google. Also, wasn't the point of genAI is that it is cheaper than the entire stack of search? At least I know for some recommendation GPT-4 is literally cheaper than many companies in house models and I know companies who saved money using GenAI.
Not saying any of these would likely happen, but still it is not in the fantasy realm.
> people are going to be annoyed by things like hallucinations
That's like saying people are going to be annoyed by no delivery in online shopping. Yes it happened more often earlier, but we are arguing more on the ideal case if it could get solved. That's why I said if AI progress is good in my message, which means we solve hallucination etc.
Also we are just talking on theoretical level if AI is able to imitate human assistant(which I personally give 10% chance of happening, but not out of realm of possibility).
That's a fundamental misunderstanding of the (especially) US startup culture in the last maybe 10-20 years. Only very rarely is the goal of the founders and angel investors to build an actual sustainable business.
In most cases the goal is to build enough perceived value by wild growth financed by VC money & by fueling hype that an subsequent IPO will let the founders and initial investors recoup their investment + get some profit on top. Or, find someone to acquire the company before it reaches the end of its financial runway.
And then let the poor schmucks who bought the business hold the bag (and foot the bill). Nobody cares if the company becomes irrelevant or even goes under at that point anymore - everyone who did has has recouped their expense already. If the company stays afloat - great, that's a bonus but not required.