For people who are in AI companies or have heard their pitches: What's the typical response to "What makes your AI special that can't be replicated by a dozen competitors?"
I've sold various "AI" consulting projects, I tell people that all the AI hard- tech is open source and that there's nothing that differentiates it. What is different is implementation experience and industry customization. For example everyone has datasets scraped from the internet, but there are not deep application specific datasets publicly available. Likewise experience with the workflows in an industry.
It's just software, there's little "secret sauce" in the engineering, it's the knowledge of the customer problem that's the differentiator.
I worked at a place where we thought there was value into putting it together in one neat package with a bow.
That is, a lot of people are thinking at the level of "let's build a model" but for a business you will need to build a model and then update it repeatedly with new data as the world changes and your requirement changes.
There would be a lot to say for a solution that includes tools for managing training sets, foundation models, training and evaluation, packages stuff up for inference in a repeatable way, etc.
One trouble though is that you have to make about 20 decisions or so about how you do those things and developing that kind of framework people get some of them wrong and it will drive you crazy because other people will make different wrong decisions than you will. (To take an example, look at the model selection tools in scikit-learn and huggingface. Both of these are pretty good for certain things but they don't work together and both have serious flaws... And don't get me started with all the people who are hung up on F1 when they really should be using AUC...)
So given the choice of (a) building out something half baked vs (b) fighting with various deficiencies in a packaged system, you can't blame people for picking (a) and "Just doing it". (Funny enough I always told people at that startup that we'd get bought by one of our customers, I thought it was going to be a big four accounting firm, a big telecom, or an international aerospace firm but... it turned out to be a famous shoe and clothing brand.)
(1) In the current environment things are moving so fast that the model of "get VC funding", "hire up a team", "talk to customers", "find product market fit" is just not fast enough.
Contrast that how quickly Adobe rolled out Generative Fill, a product that will keep people subscribed to Photoshop. (e.g. it changed my photography practice in that now I can quickly remove power lines, draw an extra row of bricks, etc. I don't do "AI art" but I now have a buddy that helps retouch photos while keeping it real)
If they went and screwed around with some startup they'd add six months to a project like that unless it was absolutely in the place where they needed to be.
(2) If you were like Pinecone and working on this stuff before it was cool you might be a somebody but if you just got into A.I. because it was hot, or if you pivoted from "blockchain" or if you've ever said both of those things in one sentence I am sorry but you are a nobody, you are somebody behind the curve not ahead of the curve.
(3) I've worked for startups and done business development in this area years before it was cool and I can say it is tough.
#1 is a really interesting point. Traditionally startups have a velocity advantage over the big companies because they don't have all the red tape, but in AI the big companies seem to have the advantage. The amount of data you need for training and the compute resources required means that startups are stuck with APIs that someone else provides, but a giant company like Adobe can train their own very quickly just based on the research papers that are out there and their own data.
Some of it is that a VC-based company that just got funding today is not going to have any product at all for at least six months or a year if not longer.
Somebody who needs a system built for their business right now gains very little talking to them.
If a startup is a year or two post funding it might really have something to offer, but the huge crop of A.I. startups funded in the last six months have missed the bus.
Big co's can frequently move very fast when there is a lot on the line.
From memory, they raised a huge round just before chat gpt went viral. Not sure if they would have been able to do so well if they were raising now. Very much doubt it.
We focus on "selling" the market size, customer problem-solution fit and not so much the AI part. AI is just the means to an end, a better way to solve the problem that we are solving. I saw some interesting stats the other day that the majority of investments in AI focus on infrastructure (databases etc) and foundational models.
1. research talent. There's not actually that many people in the world that can adequately fine-tune a large cutting-edge model, and far fewer that can explore less mainstream paths to produce value from models. Only way to get good researchers is to have name-brand leaders, like a top ML professor.
2. data. Can't do anything custom without good training data! How to get this varies widely across industry. Partnerships with established non-tech companies are a common path, which tend to rely on the network and background of founders.
Even with both those things it's not easy to outcompete a large, motivated company in the same space, like a FAANG. They have the researchers, they have the data and partnerships, so the way to beat them is to move quickly and hope their A- and B-teams are working on something else.
Interestingly enough I think there is lack of talent on the investment side of things too. Very few investors have the right skillsets in their teams to be able to do deep technical due diligence required for true AI solutions.
I think there's rather more to it than that. Two of these guys are on the Llama paper; the hype and momentum from that is surely responsible for a huge chunk of their valuation. If you take the big LLM-relevant papers, most of the folks with this kind of profile are already off doing some kind of startup.
The Mistral folks have impeccable timing, but are leaving FAANG somewhat late compared to their peers.
Definitely. Just to be clear, my comment wasn't to diminish the Mistral folks, they are certainly a very impressive group, but rather to contest your implication about the audience here.
No, not really. The script does the vast majority of the work. The only challenges here would be knowing how to use Google Colab and formatting/splitting your training and test data. That’s the computer science equivalent of adding wiper fluid to your car.
Ok, but that’s goalpost shifting. We’ve gone from there are not many people who can fine-tune a model (demonstrably untrue) to there are not many people who can do {???} that AI startups do. It’s unclear what is this special AI startup thing you’re referring to, but given that various fine-tuning strategies, like QLORA, emerged out of the open source community, this also seems unlikely to be true.
Yeah, that's fair. I could have been more precise about what an advantage research talent can be.
As an example, the startup-employed AI researchers I know had already PEFT'd llama2 within a day or two of the weights being out, determined that wasn't good enough for their needs, and began a deeper fine tuning effort. That's not something I can do, nor can most people, and it's a serious competitive advantage for those who can. It's a rather different interpretation of "can adequately fine-tune" than "can follow a tutorial".
When I think "AI startup", I think of the places where these people work. I don't think there's many of those people, and I think their presence is a big competitive advantage for their employers.
Understood. Apologies, I wasn’t trying to be combative. I agree that what you describe requires a special emphasis on AI stuff or at least a part of the org that has a research focus. I work on an R&D team at a legacy org and we do the latter.
So after waiting for more responses falling for that trick question, no-one here has still realized that companies with the hardware are the ones that have the 'moat' which CANNOT be replicated by a dozen competitors.
So all these AI startup companies depending on cloud AI services or even open source models have no moat.
I work on Milvus at Zilliz and we encounter people working on LLM companies or frameworks often, I don't ask this question a lot a lot, but it looks like at the moment many companies don't have a real moat, they are just building as fast as they can and using talent/execution/funding as their moat
I've also heard some companies that build the LLMs say that those LLMs are their moat, the time, money, and research that goes into them is high
This isn't unique to AI. If you are hesitant to invest in startups because their products could be duplicated by competitors/big tech then you should not be a VC at all.
Investors already know that this is a race to zero. There are some companies in tech that are already at the finish line in this race, like Meta and can afford to release their AI model for free, undercutting cloud based AI models unless they also do the same.
They are also realizing that the many of these new 'AI startups' using ChatGPT or a similar AI service as their 'product' are a prompt away from being copied or duplicated.
The moat is quickly getting evaporated by $0 free AI models. All that needs to happen is for these models to be shrunken down and be better than the previous generation whilst still being available for free.
Whoever owns a model close to that is winning or has already won the race to zero.
So many AI startups are really just paper-thin layers over publicly-available models like GPT. There's value there, but probably not enough to support $100M+ valuations.
We've barely scratched the surface of what generative AI can do from a product perspective, but there's a mad dash to build "chatbots for $x vertical" and investors should be a little skeptical.
That's my take too. Companies are spending money in the wrong area. GPT and similar should be used to re-categorize all the data, or used to enhance their existing UIs by surfacing related information.
By just replicating a ChatGPT interface but for Your Taxes (TM) it's really a huge slap in the face to computer users that already can't tolerate typing data in.
The developer experience of AWS is so bad it creates a lot of opportunity to provide value there. The same was also true for Salesforce for a long time.
Seems like Investors are cautious and not getting on the hype train blindly (cough.. crypto/blockchain cough..). I think that is a good thing. AI has real use cases but currently it is going through the hype cycle especially with every Tom Dick and Harry starting an "AI Startup" which are mostly a wrapper around ChatGPT etc. I think in next 5-7 years, AI will stabilize and most of the "get rich quick" types would have disappeared. Whatever is left then will be the AI and its future.
Could this time be different? The tools are now in the hands of the "masses", not behind closed doors or in lofty ivory towers. People can run this stuff on their laptops etc
Every time someone says “this time it’s different” (e.g. 1998 internet bubble, 2007 housing bubble, 2020 crypto bubble, etc) time proves that this time was not really that different.
During that time period, the internet and smartphones alone have completely changed society (for better and worse) in the span of only three decades, despite the former going causing a minor economic crash in its infancy.
Almost everything is different except human nature. The scammers are innovating just like everyone else.
When someone says a technology completely changed society, I think of the hypothetical singularity that Kurzweil and company predict, where it's basically impossible for us to predict what the future looks like after. But when you look back at the world before the rise of the web and then smartphones, it's just taking preexisting technologies and making them available in more mobile formats. TV, radio, satellite and computers existed before then (1968 mother of all demos had word processing, hypertext, networking, online video). And some people did more or less foresee what we've done online since.
We still burn fossil fuels to a large extent, still drive but not fly cars, still live on Earth not in space, still die of the same causes, etc.
I watch a long cargo train that looks like it's form the 80s go by and wonder how much the internet changed cargo hauling. I'm sure with the logistics the internet made things a lot more efficient, but the actual hauling is not much different. It's not like we teleport things around now. You can order online instead of out of a catalog, but brick stores remain. You can read digital books, but still plenty of printed materials, bookstores, libraries.
> the internet and smartphones alone have completely changed society
I honestly think this overstates the case pretty severely. They have certainly caused societal change, but from what I can see, society as a whole is not actually all that different from what it was before all of that.
Well, of those 3 the 1998 internet bubble actually was different and modern society actually was fundamentally changed by the technology in question, so idk if that's the best counterargument. The other two, sure yeah those amounted to essentially nothing. But there have been plenty of bubbles where the concept underlying the bubble actually did have large societal impacts even if the investors all lost money, like tulipmania with futures contracts and railway mania with trains
But the internet did change things? Even crypto is debatable. BTC still exists and still has pretty high value, just not as high as at its peak.
TBH, I'm not sure how to quantify housing bubbles either. I'd bet most of the country has much higher home prices now than in 2007. I bet they were higher than 2007 in most places and most years between then and now too.
There was also a voice recognition thing in the 90s, the whole self-driving car/computer vision thing early to mid last decade, and a _very_ short-lived period when everyone was a chatbot startup in 2016 (I think Microsoft Tay just poured so much cold water over this that it died almost immediately).
I can remember hyped up news after Watson; personally I was hyped up after Creatures (though in my defence I was a teenager and hadn't really encountered non-fictional AI before); before those there was famously the AI winter, following hype that turned out to be unreasonable.
Not the OP, but I've been through a few AI hype cycles and know of earlier ones, depending on what you count: ML more generally over the last half decade or so¹, the excitement around Watson (and Deep Blue before it), the second big bump in neural network interest in the mid/late 80s², there have been a couple of cycles regarding expert-system-like methods over the decades, etc.
--
[1] though that has produced more useful output than some of the previous hype cycles, as I think will the current one as it seemingly already is doing
[2] I was barely born for the start of the “AI winter” following the first such hype cycle
I think people are starting to realize that "AI" in the present context is just the new vehicle for people who were yelling, "NFT's and Cyrpto" just a year ago.
Even with my rose-tinted glasses on about the future of AI, it's not clear who will be the "winner" here, or even if any business making them will be a winner.
If open source models are good enough (within the category of image generators it looks like many Stable Diffusion clone models are), what's the business case for Stability AI or Midjourney Inc.?
Same for OpenAI and LLMs — even though for now they have the hardware edge and a useful RLHF training set from all the ChatGPT users giving thumbs up/down responses, that's not necessarily enough to make an investor happy.
Early signals to me are that regulatory capture will end up being the moat that gets used here. I think it's a horrible outcome for society, but likely one that will make some companies a lot of money. Early grumblings around regulation for a lot of AI models seem at risk of making open source models (and even open-weigh models) effectively illegal. Training from scratch is also going to both remain prohibitively expensive for individuals and most bootstrapped startups, plus with more of the common sources of data locking out companies from using training data it's going to be hard for new entrants to catch up.
I personally think the only way AI will end up being a benefit to society is if we end up with unencumbered free and open models that run locally and can be refined locally. Every financial incentive is pushing in the other direction though.
This should be one of the highest voted comments in all of the AI threads this year.
Meta is no doubt doing this because it’s in their best interest, but if both the quality and licensing of LLaMA 2 start a trend that’s a pretty effective counter-weight to eyeball scanner world.
And there’s other stuff. George Hotz is pretty unpopular because he does kind of put the crazy in crazy smart (which I personally find a refreshing change to the safe space for relatively neurotypical people in the land of aspy nerds), but tinygrad is a fundamentally more optimizable design than its predecessors with an explicit technical emphasis on accelerator portability and an implicit idealistic agenda around ruining the whole day of The AI Cartel. And it runs the marquee models. Serious megacorp CEOs seem to be glancing nervously in his direction, which is healthy.
If what you’ve read and listened to is a lot, please elaborate?
If it isn’t a lot, I encourage you to look more closely.
Let’s skip all the disputes about his achievements prior to this year (and you really need to take with a grain of salt anything bad you hear about someone who has been actively fucking up the afternoon of powerful people their entire life).
Tinygrad is so obvious in retrospect, but hindsight is 20/20 and I missed it.
Why the hell do we have kernels for any composite tensor operation? The machine economics are that starting a training run is who cares. Doing a training run is machine hours, days, centuries, millennia.
Spend like a drunken sailor customizing the kernel, exploit the aforementioned “who cares” blank check to just dump the friggin CUDA or PTX or OpenCL or Metal or whatever into /tmp/ and hit it with a fork/execve to nvcc or whatever. You can fuse and columnize and whatever to your hears content in the IR.
We spent how much time rigging up a trouś convolutions via IM2Col by hand with template instantiation in ATen? Well, we’re never getting that back. And while dating NVIDIA is fun, being chained in their soundproof basement not so fun. “It lives with the same 24Gb of GDDR6 on it paid for last year on its skin or it gets the p4d.24xlarge pricing again”.
Nobody has jacked anyone so usuriously via API lock-in since Gates and Win32.
Anyone who is doing a YouTube video called “Get in Losers, We’re building a Chatbot” and then live codes for 5 hours to turn 2700 lines of Python that can’t run LLaMA into 3200 lines of Python that can run it on three times the number of platforms that PyTorch?
I think many people's first introduction to George Hotz was with Twitter.
Where his arrogance, ignorance and complete lack of experience with how teams and companies function effectively was on full display.
You can spout off technical terms all day long but I've worked at two of the FAANG companies and met hundreds of incredible engineers and it wasn't their grasp of the technology that made them great. It was their ability to lead, deliver, communicate, relate etc which are skills that sound far less impressive but are infinitely more important.
This meme isn't completely wrong, but it's not completely right either. Before you listen to anything I have to say about it I'd refer you to the guy who co-designed a Lisp variant and the site we're having this conversation on, who is also probably the most successful tech investor in history (who presumably knows a thing or two about what's important if your goal is building successful technology businesses) [0]:
"Hackers like to work for people with high standards. But it's not enough just to be exacting. You have to insist on the right things. Which usually means that you have to be a hacker yourself. I've seen occasional articles about how to manage programmers. Really there should be two articles: one about what to do if you are yourself a programmer, and one about what to do if you're not. And the second could probably be condensed into two words: give up."
I've only worked for one FAANG, but it was for like 7-8 years out of a 20+ year career, and it was mostly on stuff where even small mistakes cost a lot of money so I'll counter your anecdote with one of my own: at the beginning of the century the software business was really weak on demographic diversity (a problem we still have and still needs to be addressed) but it was unparalleled in neurodiversity. In particular there's a longitudinal component to my observation. I'm not here to diagnose anyone via the Internet, but it's pretty clear that George (like a lot of us) is pretty friggin "different".
During the period of time (say 1995-2015 +/- 3 years) when most of the current empires of tech were built, staggeringly successful managers in both open-source (e.g. Linus) and industry (e.g. Sheryl) created a lot of space for aspy nerds who "spouted off" about tech stuff in blunt "this is stupid" kinds of ways, and all kinds of other stuff that's now called "being toxic" rather than "being a weird nerd". It was only during that latter half of this time (and really the end of it) that software became a sufficiently high-status occupation to draw in a bunch of people who decided that it was a good idea to outmaneuver those folks on the org chart and re-brand an entire industry of people from "probably on the spectrum at least a bit" to "toxic asshole".
I don't know if packing Linus off to charm school because he was simply too important to write off was a better or worse move than just letting him be Linus on a mailing list he founded, I kinda liked the original Linus. But even if we decided that old-school hacker aspiness couldn't get a blank check anymore (on the Internet we fucking built), it's just a strictly better idea to strike some sort of compromise with the aspy nerds than to redefine the vibe that had prevailed for decades as basically up there with racism, misogyny, and homophobia (you might notice that aspy tech assholes are on average some of the least racist, misogynistic, and homophobic people in any industry: they on average accept anyone who kicks ass at code or is trying hard and are short with anyone who is dismissive of code, irrespective of demo).
I wrote the `im2col` and dilated convolution kernels on NVIDIA and Intel that went into Caffe2 originally, and AFAIK that stuff migrated over to like ATen or whatever (though I imagine someone has rewritten it since), so I don't exactly love the "spout off technical terms all day log" characterization, but mostly I'll admonish you that calling leadership and communication skills "infinitely" more important than technical expertise is dangerously close to "arrogance and ignorance", and that overlooking that someone has been in the public eye on e.g. Twitter since they were 16 doesn't seem like a particularly serious effort to "relate".
In a way rather reminiscent of early Linus, George has organized a few thousand iconoclastic accelerated c...
History has shown time and time again that you need the guardrails that regulation/laws provides in order to channel progress in an effective direction. Otherwise the negatives of human nature destroy progress. We have seen this recently with the rampant criminality and negative behaviours permanently crippling the crypto space.
In the AI space there is no evidence that allowing models with illegal/unethical content is going to magically translate to some boost in performance, capability or efficacy than would otherwise happen.
In fact again history has shown that the opposite is likely to happen. That having models that don't exhibit negative effects e.g. racism, sexism will result in them being used in more places and exposed to wide audiences. Thus translating to a bigger net benefit for society.
Make models usable is really valuable, for Stability AI I discussed business models with Sam Lessin here: https://www.youtube.com/watch?v=mOOYJONenWU but basically the edge is data and distribution given how widely used this technology will be.
Open is its own area, proprietary general models are a race to zero vs OpenAI and Google who are non-economic actors.
Most AI right now is just features tho, very basic without the real thinking needed.
> Open is its own area, proprietary general models are a race to zero vs OpenAI and Google who are non-economic actors.
Agreed
> Make models usable is really valuable
Sure... but is there really a moat here? Seems like OSS can address "making models usable" and without lockin from a startup.
Business model is called open core hundred of billions in market cap here as most folk don’t want to build/train their own models and will pay for support and help https://en.m.wikipedia.org/wiki/Open-core_model
I know nothing about AI or stocks, so please correct me if I'm wrong here: Isn't NVIDIA a clear winner already (bar any major technological advances allowing all of us to run LLMs on our phones?) I just checked the stock on google and it went up 200% since the beginning of the year!
Intel and AMD are both trying to win the supercomputer-scale AI battle, and both of them have chips that put up impressive paper results. Nvidia's user experience is just better, though, and they treat hobbyists with slightly less disdain than the other chip companies.
I think it’s more to do with high rate environment, with most AI firms having no clear path to profitability. Where-as many traditional tech venture rounds (now of days) have a solid business model and current profitability per deal, using raised capital to accelerate growth at current loss for long term profit.
I think that's precisely why the investors aren't as interested - bitcoin had very little value by itself, so investors got dollar signs in their eyes when a startup claimed to be able to add the value it was missing.
ChatGPT already has a lot of value by itself, the value added by any startup is going to be marginal at best.
I think this is a good example of the VC mindset, but I think it is also flawed on their part.
LLMs are a lot more like a generalized processor than people are admitting right now. Granted you can talk to it, but it becomes significantly more capable when you learn how to program it -- and thats where the value will be added.
I don't know if you mean, like, LoRAs and similar (actual substantive changes), but the vast majority of "learning how to program" LLMs (accounting for the majority of startup pitches as well) is "prompt engineering" - which, as the meme goes, isn't a moat. There's a skill to it, yes, but if your singular advantage boils down to a few lines of English prose, your product isn't able to control a market - and VCs are (rightly) not interested unless you have the possibility to be a near-monopoly.
This is the error of the thinking. It would be like saying software doesn't have a moat because thats just clever talking to a processor.
But no one would say that now, thats ridiculous. There is a sufficient degree of prompt engineering that is already defensible, I'm already doing it myself IMO. You'll see very sophisticated hybrid programming/prompting systems being developed in the next year that will prove out the case.
For example 30 parallel prompts that then amalgamate into a decision and an audit, with 10 simulation level prompts running chained afterwards to clean the output. These types of atomic configurations will become sufficiently complex to not be just for 'anybody'.
Again, this is a misunderstanding of what LLMs are capable of. These aren't chained, you can run parallel prompts of 15 personas with X diversity of perspective, that reason on a singular request, string, input or variable, they provide output plus audit explanation. You then run an amalgamation or committee decision (sort of like mixture of experts) on it to output variable or string. Then run parallel simulation or reflection prompts based on X different context personas to double check their application to outside cases, unconsidered context, etc.
It's pretty effective on complex problems like Spam, Trust and Safety, etc. And the applications of these sort of reasoning atomic configurations I think are unlimited. It's not just 'talking fancy' to an AI, its building processes that systematically improve reasoning to different very hard applied problems.
> the applications of these sort of reasoning atomic configurations I think are unlimited
They are limited to applications in which the latency slo is O(seconds), knowledge of 2021-present doesn’t matter, and you’re allowed to make things up when you don’t know the answer.
There are, to be fair, many such applications. But it’s not unlimited.
Interesting. Were you inspired by the Delphi decision algorithm, or did you happen to rediscover it? I agree it could provide a moat, though not a very wide one. All it takes is for your competition to talk to non-engineers, and all sorts of clever usages become possible.
> These aren't chained, you can run parallel prompts of 15 personas with X diversity of perspective, that reason on a singular request, string, input or variable, they provide output plus audit explanation. You then run an amalgamation or committee decision (sort of like mixture of experts) on it to output variable or string. Then run parallel simulation or reflection prompts based on X different context personas to double check their application to outside cases, unconsidered context, etc.
Sure, so you ensemble some results. You're back to the classical "hyperparameter" problem though that's faced ML for a long time-- what those personas are, what those subsequent prompts are, etc. require a fair amount of manual verification and tuning. And the search space is extremely vast.
Not to mention that something like this is likely to be very unperformant.
We’ve found way that works for us. Nothing magical, but it works in a generalized sense. And doesn’t have this need for significant manual tuning. Search space is vast, but since they can write their own prompts or select strategies, you can have then tree nav a library of strategies and they do pretty well.
That’s a lot of compute power you’re expending for funny computer parrot to hallucinate some text for you. Seems not even machines get to escape “decision by committee” lol.
Decision by committee is actually a pretty good technique for improving the performance of ml models but there's probably cheaper ways of doing it than hitting OpenAI's api 10 times a request
Been working on ML for 10 years at goog. The difference is the context, subtlety and complexity of the problems you can feed this vs older models. Delphi, ensemble, whatever before lacked audit ability and explain ability —- LLMs excel at this. It allows you to apply it to much softer problems.
> I don't know if you mean, like, <extensions> and similar (actual substantive changes), but the vast majority of "learning how to program" <computers> (accounting for the majority of startup pitches as well) is "<punchcard> engineering" - which, as the meme goes, isn't a moat. There's a skill to it, yes, but if your singular advantage boils down to a few lines of <clever pointer dereferences>, your product isn't able to control a market - and VCs are (rightly) not interested unless you have the possibility to be a near-monopoly.
Ha I think you get it. The only moat is either user engagement, sales or sufficient sophistication of otherwise known techniques. There is not dark arts magic moat.
There is a lot of moat for intelligent people who leverage emerging properties of huge LLMs. I am not going to give you a hint but I already use those LLMs in unexpected ways that weren't possible before.
> LLMs are a lot more like a generalized processor than people are admitting right now. Granted you can talk to it, but it becomes significantly more capable when you learn how to program it -- and thats where the value will be added.
So... prompt engineering? They're an extremely inefficient processor though and very prone to error (despite what synthetic benchmarks may show).
This is absolutely correct. As soon as I was able to get access I built my own... GPT proxy to generate marketing copy and all that for people and while it was neat it comes down to a regular crud application that has a wrapper around an OpenAI API, the moat isn't there, the app was alright but I realized pretty quickly my "value" being that I'm basically using a template engine against a text prompt - I probably shouldn't shut down my consulting business over pursuing it.
> ChatGPT already has a lot of value by itself, the value added by any startup is going to be marginal at best.
This assumes that all AI startups are squarely competing with ChatGPT, or that ChatGPT is some kind of AGI that can do most machine learning tasks making AI startups redundant "thin wrappers" around ChatGPT.
How does ChatGPT make say Weights and Biases irrelevant, or a startup detecting bank transaction fraud, or say a product that detects when someone at your door is a stranger.
Depends on what you mean by ’wrapper’. For most AI startups it isn’t viable to train their own models. For most customer use-cases, ChatGPT interface isn’t enough. Wrappers are currently the only logical implementation of AI to production.
This is approximately true at the moment - but it's an open question how much that is worth to customers. The market will sort it out, but it's not clear that all of these "wrapper" startups have a workable business model.
True, especially regarding how easily their services can be replicated. Their margins are low, and customer acquisition does not provide them with network effects that would yield a moat.
The extreme irony is that the automated web will largely be used by AI to begin with, and the automated web is powered by decentralized computational efforts such as smart contracts and digital currency. It's like people completely forget/ignore that cryptocurrency is a mathematical problem still in its infancy.
If you think these aren't all fundamental units of the next web, you're not thinking about it from the right perspective. If you can't pick apart the real mathematical utility and origin behind crypto efforts from a generation of scammers who hijacked a very real thing, then you just lack understanding or nuance.
We are decades away from the most obvious solution but it very likely involves cryptographically-backed digital currency and smart contract systems used by automated neural networks.
that's commie thinking, that somehow if we split the work up enough we can compensate labor equitably.
AI benefits from the same economies of scale as all the other means of production, and the winners are going to be the ones that can reinvest their profit into growth and outpace competitors.
tl;dr I don't see distributed multi party computation doing a better job than a rack of H100s
I'm speaking AI run on the edge. The kind of AI that will become pervasive. Digital personas on autopilot. The days of me having to do things like book my own flight are coming to an end.
OK but come on, booking your own flight is already automated. I put my preferences into a form and choose from like 3 options and get a confirmation email. I really don't see how a digital twin trained on my preferences making that choice for me is an improvement. And my web browser already remembers my credit card number, so I can hit "book" and the computer spends my money. What advantage does crypto play here? That's why I jumped to distributed computation, I assumed that's what crypto adds to the mix, the ability to pay other actors in an automated way, but again, if it was necessary to rent compute I can already setup my credit card to autopay, so what does crypto have to do with it?
[PS despite my dismissive tone I'm curious if there's something I'm missing]
The Bitcoin whitepaper laid out a way to prevent double-spend in cryptographically-backed digital currencies using proof-of-work. Proof-of-work was just a brute-force approach but lots of research is going into other ways to solve the problem.
All cryptocurrency is, is cryptographically-backed digital currency. The current wave of cryptocurrencies are rooted in the blockchain scheme laid out in the whitepaper, with various forms of proof. The only thing new is a way to prevent double-spend; cryptographically-backed currencies were not invented with Bitcoin.
This stuff absolutely has a fundamental place in future commerce, especially as generations grow tired of the payment processor mafia acting as global moral arbiters. Smart contracts build upon this, asset classes such as NFTs give distributed ways to work with authentic data. All of the scammers who jumped on to these technologies have nothing to do with the underlying technologies themselves, nor the core group of people who are still interested in progressing this tech.
AI is just automation, which can make use of these tools in a trustless environment, without disruption from untrusted/unwelcome parties. It's not life-changing stuff, but these tools will fundamentally drive the web in ways you won't even notice if you don't look for it.
The reality is also that the "holy shit" moment around LLMs seems to have largely passed. Models are incrementally improving, sure, but the fundamental limitations are here to stay (at least for now). That means a long tail of adding guard rails, etc.
At the end of the day, the value add is also around integration and implementation and that is very difficult to generalize.
GPT-4 has been entirely unimpressive to me. I first used it 3 months after launch, when they had already degraded the quality significantly, and there was no "magic" moment. LAMDA (internal to Google) was actually the last time I had that sense of AI wonder, and since then, the GPTs are basically only incrementally better.
Edit: fixed the number - I thought it launched in January. It turns out late March was the launch, while the first hints/discussion about it were January. I got around to it in early July.
There's plenty of holy shit moments left to develop I think.
For example, it's relatively straightforward to generate dialogue or other text for a game, but structurally connecting any of that text to game mechanics is unsolved (though AI Roguelite is trying).
Sounds like implementation and integration? Also how do you invest in that? Are video game companies going to offload dialogue generation to a firm that charges a license fee on every bit of dialogue generated?
Well, the bigger ones could probably afford to take an existing open source/free model and fine tune it for their use case with more training. Then run it themselves in some cloud (they already have some experience with servers after all).
Many users' GPU's are powerful enough to run models locally, but the rub there is that you can't really run the model at the same time you're playing the game normally, unless you want, like, massive frameskips.
There is also the wrong uses of the tech. Some people think you can train a model with any amount of data you may have and that the model will be useful somehow.
There’s a SaaS company in Brazil which has a solution for recruitment that is a text book case of bad usage of machine learning. Not going to mention them here, but their “resume matching by AI” is completely bonkers because of that.
But, enterprise don’t care if it works, it just needs to have this bullet point so the HR director can say he implemented a selection tool powered by AI on her next presentation.
Right now AI produces code that competent coders can massage into the real thing or copy that competent writers can massage into the real thing. I've not seen any evidence it's ever going to turn that corner, but obviously the future is unpredictable.
anecdotally, it was better at generating code ~4 months ago. It makes one wonder if there is a market for "super-premium AIs" for software engineers/other technical users.
Curiously, I observed that many investors (or at least people proposing to pitch investors) focus on the simplest of use cases. Perhaps the investor crowd is betting that they can consolidate a few of the successful stories? Or are simply looking to fund many "app" teams to complement their large VC bets in foundation models?
This is healthy skepticism and the acknowledgement that there are lots of free tools out there. You need to be much better than what's freely available. You need to persuade buyers to buy when they don't want to. I don't think any of that is new.
This is explained by the simple idea that only a few companies are in an arms race to create a general purpose intelligence, and when they do all of the ai-powered systems will naturally consolidate or “become flavors” of this GPI AI.
So what substantive and defensible advantage is your money buying in the AI ethos when this effect is essentially inevitable?
Answer: not much.
So it’s very logical that the team, book of business and the tech platform itself are what are driving valuations.
Or another coworker telling me they use it to generate regexes… I get it, but for me writing and testing it is part of the same process so I’m good, thanks.
All of these wannabe startups just wrapping ChatGPT or other models and trying to resell them are pretty laughable.
Chatbots are going to become "features" in more tangible products, not "products" themselves that people are going to buy (or at least, they sure aren't going to buy them from "value add" resellers).
I wonder how much of AI will be winner take all and how much will be value destruction. From an investor standpoint in LLM you have a privately held business leading the market and open source software following closely.
During the PC revolution you could buy apple hp and Microsoft and know that you were capturing the hardware market. Here we see Nvidia, AMD, Apple, and Microsoft (somewhat) looking like the major beneficiaries and the market is following that. Maybe it becomes a Omni-platform market and people rush into OpenAI once public.
Amazon, okay, but who else? Nearly all of the big .com-era startups (and major non-startup beneficiaries, like Sun) are _gone_. Yahoo somehow still exists, I suppose.
I suppose you could argue Google, but it's an odd one; it was right at the tail end, and was really only taking off as everything else was collapsing
I would argue Google because the dot-com bubble did not burst till 2000, and Google was founded in 1998.
Also, Netflix was founded Sep 1997. Also, Paypal, founded Oct 1999. Also, Ebay, founded 1995, [del: though Ebay and Paypal together today are worth only 23.8 billion dollars. :del]
I think a lot of the money made by founders and angel investors during the dot-com era was made by selling a startup to an established company for 100s of millions of dollars, which is pretty good money since it often took only took 5 years to go from the founding event to such an exit, but I cannot think of an easy way to get a list of such exits.
ADDED. In writing the above, I assumed Paypal was still part of Ebay. In reality they are separate companies again with market caps of 23.8 and 84.59 billion dollars.
While I think that some AI startups and new AI products will be successful I also think that from AI revolution mostly will benefit companies that will integrate new AI technologies in their existing product.
I have the feeling that we are at the MRP stage (https://en.wikipedia.org/wiki/Material_requirements_planning) when companies started using computers but writing software to handle production processes was so new that nobody could write anything truly universal. The next will be the ERP stage where we know some abstractions that apply to many companies, companies like SAP can sell some software - but most money is in 'implementation' by consulting agencies.
You really have to have your own existing moat for AI to augment (ala Adobe, Microsoft, etc). Anything built directly on AI can be replicated rather quickly once someone figures out what combination of prompt + extra data was used.
That said, you don't have to be the mega players to have an existing small moat. If your product does something great already, you get to improve it and add value for users very quickly. That's been my experience anyway.
I've seen some hype waves in my life, but it's probably the first one that truly unleashed the sleazy "influencer" types that regurgitate the same carousels they steal from each other. Even more intense than "crypto" now. That really kind of distracts from trying to gauge the meaning of this.
I feel like these hype cycles are getting quicker and quicker.
2030, day 8 of the 17th AI boom: A starry-eyed founder shows up to a VC office with a pitch-deck for their GPT-47-based startup which automatically responds to Yelp reviews, only to be turned away; the VCs are done with that now, and will be doing robot dogs for the next week.
I think the guys behind the idea pictured something a little different. "...Appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue" was apparently what John von Neumann said.
What I have heard from YC folks was that typically the hard costs (GPU compute) and data moats of large players make the space virtually impossible for an upstart to make a meaningful difference that isn’t immediately copied wholesale by a major player.
Software velocity is increasing. Investors should be considering what that means for their investments.
I would be worried if I were tied up in a company that depends on bloated professional services. LLM-enabled senior engineers are 100X more efficient and safe than brand new junior devs. These organizations that embrace the best people using the best tech ought to make Oracle and their famous billion dollar cost overruns quake in their boots.
> LLM-enabled senior engineers are 100X more efficient and safe than brand new junior devs.
Come on man. Having seen the inside of big-tech-TM and the senior engineers there, yes they are fast and good, but they are not 100x better than the new guy. Maybe 3--5x at best.
Anyway how do you train good senior engineers? They don't just pop up out of thin air.
Even an LLM enabled senior engineer was 100x better, the situation has to be the right one where all 100x can be applied productively.
An analogy -- if I am an athlete who runs a mile 100x faster than my competition, does that mean that my country will win 100x more medals at the Olympics? Not unless I compete in every competition.
Now, if you can make a system that consistently creates or recruits (or even just prevents your competition from finding) more productive developers you can get a real advantage. And, as soon as you do it you will have people imitating your strategy and tactics and poaching the people who are most important to building the system. It is way more complex than providing an LLM.
> Anyway how do you train good senior engineers? They don't just pop up out of thin air.
Don't underestimate the industry's capacity to self-sabotage on the long-term for a short-term pay-off. This happened with de-industrialization in the West and might happen with Tech. You only need these senior developers until you realize that you haven't trained any new senior devs. But that problem is for 20-years-in-the-future CEO. No one cares now as long as you can keep next quarter revenue/profit up.
The thing you need in this space to make money is "owning a platform". It's not clear yet what shapes such platforms are going to take. Right now, the only real condender seems to be owning and API-letting a base model. If OSS models get good, that proposition disappears.
Other than that, I haven't seen anything with monopoly potential coming from startups so far. This is what VC is looking for, though: The potential to make a platform to rent out to people who produce the value, curate it and pay for it. Basically, the next "academic publishing".
I'm excited for the metric boatload of incremental value over a long period of time that modern AI is going to deliver for businesses.
Because that seems to be the path we're actually on. I suppose there's plenty for investors to love in that equation, but it's a little harder to attach a hype machine to that.
It seems a tad bit ironic that the thing that might actually work sooner rather than later (conversational LLM) to have some applications is being shunned.
The LLM solutions I am seeing in my space are lackluster and feel like solutions looking for a problem. There are key things LLMs can help with that aren't "revolutionary" or "sexy" but a lot of what I am seeing is saving time on not so tedious things or content generation where I have to carefuly comb through the content for hallucinations or incorrect stuff and risk wild goose chases.
The type of solutions I am sorely missing have to do with adding data/work. E.g.: don't tell me what code to use, find me stackoverflow entries that might be highly relevant instead! Don't tell me what the data I am looking at means and have me google that separately, use LLM contextual "undersranding" to find best source material describing what I am looking at or helping me piece together a bigger picture!
187 comments
[ 3.6 ms ] story [ 211 ms ] threadIt's just software, there's little "secret sauce" in the engineering, it's the knowledge of the customer problem that's the differentiator.
That is, a lot of people are thinking at the level of "let's build a model" but for a business you will need to build a model and then update it repeatedly with new data as the world changes and your requirement changes.
There would be a lot to say for a solution that includes tools for managing training sets, foundation models, training and evaluation, packages stuff up for inference in a repeatable way, etc.
One trouble though is that you have to make about 20 decisions or so about how you do those things and developing that kind of framework people get some of them wrong and it will drive you crazy because other people will make different wrong decisions than you will. (To take an example, look at the model selection tools in scikit-learn and huggingface. Both of these are pretty good for certain things but they don't work together and both have serious flaws... And don't get me started with all the people who are hung up on F1 when they really should be using AUC...)
So given the choice of (a) building out something half baked vs (b) fighting with various deficiencies in a packaged system, you can't blame people for picking (a) and "Just doing it". (Funny enough I always told people at that startup that we'd get bought by one of our customers, I thought it was going to be a big four accounting firm, a big telecom, or an international aerospace firm but... it turned out to be a famous shoe and clothing brand.)
Contrast that how quickly Adobe rolled out Generative Fill, a product that will keep people subscribed to Photoshop. (e.g. it changed my photography practice in that now I can quickly remove power lines, draw an extra row of bricks, etc. I don't do "AI art" but I now have a buddy that helps retouch photos while keeping it real)
If they went and screwed around with some startup they'd add six months to a project like that unless it was absolutely in the place where they needed to be.
(2) If you were like Pinecone and working on this stuff before it was cool you might be a somebody but if you just got into A.I. because it was hot, or if you pivoted from "blockchain" or if you've ever said both of those things in one sentence I am sorry but you are a nobody, you are somebody behind the curve not ahead of the curve.
(3) I've worked for startups and done business development in this area years before it was cool and I can say it is tough.
Somebody who needs a system built for their business right now gains very little talking to them.
If a startup is a year or two post funding it might really have something to offer, but the huge crop of A.I. startups funded in the last six months have missed the bus.
Big co's can frequently move very fast when there is a lot on the line.
Just like forms over SQL, there seems to be a never ending demand.
If one can scrape the data from the web, I can't imagine having much of a moat or selling point.
2. data. Can't do anything custom without good training data! How to get this varies widely across industry. Partnerships with established non-tech companies are a common path, which tend to rely on the network and background of founders.
Even with both those things it's not easy to outcompete a large, motivated company in the same space, like a FAANG. They have the researchers, they have the data and partnerships, so the way to beat them is to move quickly and hope their A- and B-teams are working on something else.
They were L7/L6 ML researchers/eng at FAANG, I'd bet there are quite a few people like that lurking here.
The Mistral folks have impeccable timing, but are leaving FAANG somewhat late compared to their peers.
If you know how to run a Python script, you can fine-tune a LLama model:
https://huggingface.co/blog/llama2#fine-tuning-with-peft
As an example, the startup-employed AI researchers I know had already PEFT'd llama2 within a day or two of the weights being out, determined that wasn't good enough for their needs, and began a deeper fine tuning effort. That's not something I can do, nor can most people, and it's a serious competitive advantage for those who can. It's a rather different interpretation of "can adequately fine-tune" than "can follow a tutorial".
When I think "AI startup", I think of the places where these people work. I don't think there's many of those people, and I think their presence is a big competitive advantage for their employers.
As you can see with all the responses here, they have failed to realize that this is a trick question.
The real answer is that none are special and can be replicated by tons of competitors.
So all these AI startup companies depending on cloud AI services or even open source models have no moat.
Only the same big tech incumbents.
All the usual things.
First mover
Features
Integrations
Platform synergies
I've also heard some companies that build the LLMs say that those LLMs are their moat, the time, money, and research that goes into them is high
They are also realizing that the many of these new 'AI startups' using ChatGPT or a similar AI service as their 'product' are a prompt away from being copied or duplicated.
The moat is quickly getting evaporated by $0 free AI models. All that needs to happen is for these models to be shrunken down and be better than the previous generation whilst still being available for free.
Whoever owns a model close to that is winning or has already won the race to zero.
We've barely scratched the surface of what generative AI can do from a product perspective, but there's a mad dash to build "chatbots for $x vertical" and investors should be a little skeptical.
By just replicating a ChatGPT interface but for Your Taxes (TM) it's really a huge slap in the face to computer users that already can't tolerate typing data in.
1980 after the foundations of neural networks, but it was too computationally intensive to be useful
2009 with Watson
https://www.hiig.de/en/a-brief-history-of-ai-ai-in-the-hype-...
That's some extreme cherry picking.
During that time period, the internet and smartphones alone have completely changed society (for better and worse) in the span of only three decades, despite the former going causing a minor economic crash in its infancy.
Almost everything is different except human nature. The scammers are innovating just like everyone else.
We still burn fossil fuels to a large extent, still drive but not fly cars, still live on Earth not in space, still die of the same causes, etc.
I watch a long cargo train that looks like it's form the 80s go by and wonder how much the internet changed cargo hauling. I'm sure with the logistics the internet made things a lot more efficient, but the actual hauling is not much different. It's not like we teleport things around now. You can order online instead of out of a catalog, but brick stores remain. You can read digital books, but still plenty of printed materials, bookstores, libraries.
I honestly think this overstates the case pretty severely. They have certainly caused societal change, but from what I can see, society as a whole is not actually all that different from what it was before all of that.
TBH, I'm not sure how to quantify housing bubbles either. I'd bet most of the country has much higher home prices now than in 2007. I bet they were higher than 2007 in most places and most years between then and now too.
It's the very nature of the hype cycle that it is very hard to distinguish from a real thing.
--
[1] though that has produced more useful output than some of the previous hype cycles, as I think will the current one as it seemingly already is doing
[2] I was barely born for the start of the “AI winter” following the first such hype cycle
If open source models are good enough (within the category of image generators it looks like many Stable Diffusion clone models are), what's the business case for Stability AI or Midjourney Inc.?
Same for OpenAI and LLMs — even though for now they have the hardware edge and a useful RLHF training set from all the ChatGPT users giving thumbs up/down responses, that's not necessarily enough to make an investor happy.
I personally think the only way AI will end up being a benefit to society is if we end up with unencumbered free and open models that run locally and can be refined locally. Every financial incentive is pushing in the other direction though.
Meta is no doubt doing this because it’s in their best interest, but if both the quality and licensing of LLaMA 2 start a trend that’s a pretty effective counter-weight to eyeball scanner world.
And there’s other stuff. George Hotz is pretty unpopular because he does kind of put the crazy in crazy smart (which I personally find a refreshing change to the safe space for relatively neurotypical people in the land of aspy nerds), but tinygrad is a fundamentally more optimizable design than its predecessors with an explicit technical emphasis on accelerator portability and an implicit idealistic agenda around ruining the whole day of The AI Cartel. And it runs the marquee models. Serious megacorp CEOs seem to be glancing nervously in his direction, which is healthy.
It’s not locked-in yet.
If it isn’t a lot, I encourage you to look more closely. Let’s skip all the disputes about his achievements prior to this year (and you really need to take with a grain of salt anything bad you hear about someone who has been actively fucking up the afternoon of powerful people their entire life).
Tinygrad is so obvious in retrospect, but hindsight is 20/20 and I missed it.
Why the hell do we have kernels for any composite tensor operation? The machine economics are that starting a training run is who cares. Doing a training run is machine hours, days, centuries, millennia.
Spend like a drunken sailor customizing the kernel, exploit the aforementioned “who cares” blank check to just dump the friggin CUDA or PTX or OpenCL or Metal or whatever into /tmp/ and hit it with a fork/execve to nvcc or whatever. You can fuse and columnize and whatever to your hears content in the IR.
We spent how much time rigging up a trouś convolutions via IM2Col by hand with template instantiation in ATen? Well, we’re never getting that back. And while dating NVIDIA is fun, being chained in their soundproof basement not so fun. “It lives with the same 24Gb of GDDR6 on it paid for last year on its skin or it gets the p4d.24xlarge pricing again”.
Nobody has jacked anyone so usuriously via API lock-in since Gates and Win32.
Anyone who is doing a YouTube video called “Get in Losers, We’re building a Chatbot” and then live codes for 5 hours to turn 2700 lines of Python that can’t run LLaMA into 3200 lines of Python that can run it on three times the number of platforms that PyTorch?
I’m struggling here.
Where his arrogance, ignorance and complete lack of experience with how teams and companies function effectively was on full display.
You can spout off technical terms all day long but I've worked at two of the FAANG companies and met hundreds of incredible engineers and it wasn't their grasp of the technology that made them great. It was their ability to lead, deliver, communicate, relate etc which are skills that sound far less impressive but are infinitely more important.
"Hackers like to work for people with high standards. But it's not enough just to be exacting. You have to insist on the right things. Which usually means that you have to be a hacker yourself. I've seen occasional articles about how to manage programmers. Really there should be two articles: one about what to do if you are yourself a programmer, and one about what to do if you're not. And the second could probably be condensed into two words: give up."
I've only worked for one FAANG, but it was for like 7-8 years out of a 20+ year career, and it was mostly on stuff where even small mistakes cost a lot of money so I'll counter your anecdote with one of my own: at the beginning of the century the software business was really weak on demographic diversity (a problem we still have and still needs to be addressed) but it was unparalleled in neurodiversity. In particular there's a longitudinal component to my observation. I'm not here to diagnose anyone via the Internet, but it's pretty clear that George (like a lot of us) is pretty friggin "different".
During the period of time (say 1995-2015 +/- 3 years) when most of the current empires of tech were built, staggeringly successful managers in both open-source (e.g. Linus) and industry (e.g. Sheryl) created a lot of space for aspy nerds who "spouted off" about tech stuff in blunt "this is stupid" kinds of ways, and all kinds of other stuff that's now called "being toxic" rather than "being a weird nerd". It was only during that latter half of this time (and really the end of it) that software became a sufficiently high-status occupation to draw in a bunch of people who decided that it was a good idea to outmaneuver those folks on the org chart and re-brand an entire industry of people from "probably on the spectrum at least a bit" to "toxic asshole".
I don't know if packing Linus off to charm school because he was simply too important to write off was a better or worse move than just letting him be Linus on a mailing list he founded, I kinda liked the original Linus. But even if we decided that old-school hacker aspiness couldn't get a blank check anymore (on the Internet we fucking built), it's just a strictly better idea to strike some sort of compromise with the aspy nerds than to redefine the vibe that had prevailed for decades as basically up there with racism, misogyny, and homophobia (you might notice that aspy tech assholes are on average some of the least racist, misogynistic, and homophobic people in any industry: they on average accept anyone who kicks ass at code or is trying hard and are short with anyone who is dismissive of code, irrespective of demo).
I wrote the `im2col` and dilated convolution kernels on NVIDIA and Intel that went into Caffe2 originally, and AFAIK that stuff migrated over to like ATen or whatever (though I imagine someone has rewritten it since), so I don't exactly love the "spout off technical terms all day log" characterization, but mostly I'll admonish you that calling leadership and communication skills "infinitely" more important than technical expertise is dangerously close to "arrogance and ignorance", and that overlooking that someone has been in the public eye on e.g. Twitter since they were 16 doesn't seem like a particularly serious effort to "relate".
In a way rather reminiscent of early Linus, George has organized a few thousand iconoclastic accelerated c...
History has shown time and time again that you need the guardrails that regulation/laws provides in order to channel progress in an effective direction. Otherwise the negatives of human nature destroy progress. We have seen this recently with the rampant criminality and negative behaviours permanently crippling the crypto space.
In the AI space there is no evidence that allowing models with illegal/unethical content is going to magically translate to some boost in performance, capability or efficacy than would otherwise happen.
In fact again history has shown that the opposite is likely to happen. That having models that don't exhibit negative effects e.g. racism, sexism will result in them being used in more places and exposed to wide audiences. Thus translating to a bigger net benefit for society.
Open is its own area, proprietary general models are a race to zero vs OpenAI and Google who are non-economic actors.
Most AI right now is just features tho, very basic without the real thinking needed.
Next year we go enterprise.
> Make models usable is really valuable Sure... but is there really a moat here? Seems like OSS can address "making models usable" and without lockin from a startup.
Business model is called open core hundred of billions in market cap here as most folk don’t want to build/train their own models and will pay for support and help https://en.m.wikipedia.org/wiki/Open-core_model
Can look at databricks and many kfheds
Edit: if AMD plays their cards right, I'd expect that they can get a lot more in on the action, too.
ChatGPT already has a lot of value by itself, the value added by any startup is going to be marginal at best.
LLMs are a lot more like a generalized processor than people are admitting right now. Granted you can talk to it, but it becomes significantly more capable when you learn how to program it -- and thats where the value will be added.
I don't know if you mean, like, LoRAs and similar (actual substantive changes), but the vast majority of "learning how to program" LLMs (accounting for the majority of startup pitches as well) is "prompt engineering" - which, as the meme goes, isn't a moat. There's a skill to it, yes, but if your singular advantage boils down to a few lines of English prose, your product isn't able to control a market - and VCs are (rightly) not interested unless you have the possibility to be a near-monopoly.
But no one would say that now, thats ridiculous. There is a sufficient degree of prompt engineering that is already defensible, I'm already doing it myself IMO. You'll see very sophisticated hybrid programming/prompting systems being developed in the next year that will prove out the case.
For example 30 parallel prompts that then amalgamate into a decision and an audit, with 10 simulation level prompts running chained afterwards to clean the output. These types of atomic configurations will become sufficiently complex to not be just for 'anybody'.
It's pretty effective on complex problems like Spam, Trust and Safety, etc. And the applications of these sort of reasoning atomic configurations I think are unlimited. It's not just 'talking fancy' to an AI, its building processes that systematically improve reasoning to different very hard applied problems.
But overall, hasn't that theme been true for like... all tech ever? You have to set up and build your own innovation path at some point.
They are limited to applications in which the latency slo is O(seconds), knowledge of 2021-present doesn’t matter, and you’re allowed to make things up when you don’t know the answer.
There are, to be fair, many such applications. But it’s not unlimited.
See https://en.wikipedia.org/wiki/Delphi_method
But in general many configurations are possible, but also you need to refine the personas a great deal to get it to work well.
Sure, so you ensemble some results. You're back to the classical "hyperparameter" problem though that's faced ML for a long time-- what those personas are, what those subsequent prompts are, etc. require a fair amount of manual verification and tuning. And the search space is extremely vast.
Not to mention that something like this is likely to be very unperformant.
So... prompt engineering? They're an extremely inefficient processor though and very prone to error (despite what synthetic benchmarks may show).
I think this is very much like the CCD sensor, that Kodak couldn't envision using because it was "so expensive, slow and low resolution."
This assumes that all AI startups are squarely competing with ChatGPT, or that ChatGPT is some kind of AGI that can do most machine learning tasks making AI startups redundant "thin wrappers" around ChatGPT.
How does ChatGPT make say Weights and Biases irrelevant, or a startup detecting bank transaction fraud, or say a product that detects when someone at your door is a stranger.
If you think these aren't all fundamental units of the next web, you're not thinking about it from the right perspective. If you can't pick apart the real mathematical utility and origin behind crypto efforts from a generation of scammers who hijacked a very real thing, then you just lack understanding or nuance.
We are decades away from the most obvious solution but it very likely involves cryptographically-backed digital currency and smart contract systems used by automated neural networks.
AI benefits from the same economies of scale as all the other means of production, and the winners are going to be the ones that can reinvest their profit into growth and outpace competitors.
tl;dr I don't see distributed multi party computation doing a better job than a rack of H100s
[PS despite my dismissive tone I'm curious if there's something I'm missing]
All cryptocurrency is, is cryptographically-backed digital currency. The current wave of cryptocurrencies are rooted in the blockchain scheme laid out in the whitepaper, with various forms of proof. The only thing new is a way to prevent double-spend; cryptographically-backed currencies were not invented with Bitcoin.
This stuff absolutely has a fundamental place in future commerce, especially as generations grow tired of the payment processor mafia acting as global moral arbiters. Smart contracts build upon this, asset classes such as NFTs give distributed ways to work with authentic data. All of the scammers who jumped on to these technologies have nothing to do with the underlying technologies themselves, nor the core group of people who are still interested in progressing this tech.
AI is just automation, which can make use of these tools in a trustless environment, without disruption from untrusted/unwelcome parties. It's not life-changing stuff, but these tools will fundamentally drive the web in ways you won't even notice if you don't look for it.
At the end of the day, the value add is also around integration and implementation and that is very difficult to generalize.
Edit: fixed the number - I thought it launched in January. It turns out late March was the launch, while the first hints/discussion about it were January. I got around to it in early July.
I’m not sure what you mean by this; GPT-4 launched 4 months ago.
For example, it's relatively straightforward to generate dialogue or other text for a game, but structurally connecting any of that text to game mechanics is unsolved (though AI Roguelite is trying).
Many users' GPU's are powerful enough to run models locally, but the rub there is that you can't really run the model at the same time you're playing the game normally, unless you want, like, massive frameskips.
Producthunt has basically become that these days, none of it is inspirational nor value adding, just constant "X but with AI"
So what substantive and defensible advantage is your money buying in the AI ethos when this effect is essentially inevitable?
Answer: not much.
So it’s very logical that the team, book of business and the tech platform itself are what are driving valuations.
Chatbots are going to become "features" in more tangible products, not "products" themselves that people are going to buy (or at least, they sure aren't going to buy them from "value add" resellers).
During the PC revolution you could buy apple hp and Microsoft and know that you were capturing the hardware market. Here we see Nvidia, AMD, Apple, and Microsoft (somewhat) looking like the major beneficiaries and the market is following that. Maybe it becomes a Omni-platform market and people rush into OpenAI once public.
Also,
> Unlike during the dot-com bubble of the 2000s, AI isn’t entirely based on speculation.
I'd say the dot-com bubble was backed by a revolutionary product: the Internet. That doesn't change that expectations were too high.
Some of the companies involved are now worth trillions.
I suppose you could argue Google, but it's an odd one; it was right at the tail end, and was really only taking off as everything else was collapsing
Also, Netflix was founded Sep 1997. Also, Paypal, founded Oct 1999. Also, Ebay, founded 1995, [del: though Ebay and Paypal together today are worth only 23.8 billion dollars. :del]
I think a lot of the money made by founders and angel investors during the dot-com era was made by selling a startup to an established company for 100s of millions of dollars, which is pretty good money since it often took only took 5 years to go from the founding event to such an exit, but I cannot think of an easy way to get a list of such exits.
ADDED. In writing the above, I assumed Paypal was still part of Ebay. In reality they are separate companies again with market caps of 23.8 and 84.59 billion dollars.
Interestingly, Netflix has its origins in a... more typical dot-com-era story: https://en.wikipedia.org/wiki/Pure_Software
They were just too early I reckon. I wonder what we're "too early" for today.
That said, you don't have to be the mega players to have an existing small moat. If your product does something great already, you get to improve it and add value for users very quickly. That's been my experience anyway.
This is assuming your thing is one call to GPT-n rather than a complex app with many LLM-core functions, and it also assumes that data is easy to get.
you can start by googling the above, for example
2030, day 8 of the 17th AI boom: A starry-eyed founder shows up to a VC office with a pitch-deck for their GPT-47-based startup which automatically responds to Yelp reviews, only to be turned away; the VCs are done with that now, and will be doing robot dogs for the next week.
Software velocity is increasing. Investors should be considering what that means for their investments.
I would be worried if I were tied up in a company that depends on bloated professional services. LLM-enabled senior engineers are 100X more efficient and safe than brand new junior devs. These organizations that embrace the best people using the best tech ought to make Oracle and their famous billion dollar cost overruns quake in their boots.
Come on man. Having seen the inside of big-tech-TM and the senior engineers there, yes they are fast and good, but they are not 100x better than the new guy. Maybe 3--5x at best.
Anyway how do you train good senior engineers? They don't just pop up out of thin air.
Even an LLM enabled senior engineer was 100x better, the situation has to be the right one where all 100x can be applied productively.
An analogy -- if I am an athlete who runs a mile 100x faster than my competition, does that mean that my country will win 100x more medals at the Olympics? Not unless I compete in every competition.
Now, if you can make a system that consistently creates or recruits (or even just prevents your competition from finding) more productive developers you can get a real advantage. And, as soon as you do it you will have people imitating your strategy and tactics and poaching the people who are most important to building the system. It is way more complex than providing an LLM.
More like infinity since juniors usually can't independently progress on non trivial tasks (and will probably regres if left to try).
I sort of get what OP is saying, if LLMs get better you can delegate stuff you usually delegate to juniors.
But that's nowhere near reality right now, LLMs are not even a net positive with all the failure modes and poor workflow.
It takes too much compute to get meaningful context. Having to summarize and chunk problems is more work than the benefits.
Copilot is great at speeding up obvious typing, but that's like 10% bump.
Don't underestimate the industry's capacity to self-sabotage on the long-term for a short-term pay-off. This happened with de-industrialization in the West and might happen with Tech. You only need these senior developers until you realize that you haven't trained any new senior devs. But that problem is for 20-years-in-the-future CEO. No one cares now as long as you can keep next quarter revenue/profit up.
Other than that, I haven't seen anything with monopoly potential coming from startups so far. This is what VC is looking for, though: The potential to make a platform to rent out to people who produce the value, curate it and pay for it. Basically, the next "academic publishing".
Because that seems to be the path we're actually on. I suppose there's plenty for investors to love in that equation, but it's a little harder to attach a hype machine to that.
The type of solutions I am sorely missing have to do with adding data/work. E.g.: don't tell me what code to use, find me stackoverflow entries that might be highly relevant instead! Don't tell me what the data I am looking at means and have me google that separately, use LLM contextual "undersranding" to find best source material describing what I am looking at or helping me piece together a bigger picture!