is it unreasonable to expect some kind of good enough baseline guided by the prior of these llm to be the standard ? Should google's priors (or their training dataset and recipe) be allowed to guide societal's prior? same problem, different era ?
I have often thought about how computers are significantly faster than they were in the early 2000s, but they are significantly harder to use. Using Linux for the first time in college was a revelation, because it gave me the tools to tell the computer "rename all of the files in this directory, keeping only the important parts of the name."
But instead of iterating on better interfaces to effectively utilize the N thousands of operations per second a computer is capable of, the powers that be behind the industry have decided to invest billions of dollars in GPUs to get a program that seems like it understands language, but is incapable of counting the number of B's in "blueberry."
If you really want to, you know have this super generic indexing thing, why don't you go organize the web with hypercard and semantic web crap and tell us how it worked out for you
> Remember Semantic Web? The web was supposed to evolve into semantically structured, linked, machine-readable data that would enable amazing opportunities. That never happened.
I think the lesson to be learned is in answering the question "Why didn't the semantic web happen?"
> If all knowledge were stored in a structured way with rich semantic linking, then very primitive natural language processing algorithms could parse question like the example at the beginning of the article, and could find the answer using orders of magnitude fewer computational resources.
In vertical markets, can LLMs generate a "semantic web of linked data" knowledge graph to be parsed with efficient NLP algorithms?
leveraging LLMs to build the special markup so that it can be applied towards other uses.. some type of semantic web format, like JSON-LD or OWL, or some database that can process SPARQL queries.. Palantir is using ontologies as guardrails to prevent LLM hallucinations
My read is that the author is saying it would have been really nice if there had been a really good protocol for storing data in a rich semantically structured way and everyone had been really really good at adhering to that standard.
I think every field has its own version of this thought, where if we could just manage to categorise and tag things properly we could achieve anything. Our lack of a valid overarching ontology is what is holding us back from greatness.
It might be short lived, who knows, but it's interesting that the recent progress came from capturing/consuming rather than systematically eliminating the nuance in language.
One can speak in their native language to a computer now and it mostly understands what is meant and can retrieve information or even throw together a scaffold of a project somewhat reliably.
It's not particularly good at writing software, however. Still feels like a human is needed to make sure it doesn't generate nonsense or apparently pretty insecure code.
So I'm not sure the author got the point across that they wished, but aren't vector databases basically a semantic storage/retrieval technology?
I feel like I see this attitude a lot amongst devs: "If everyone just built it correctly, we wouldn't need these bandaids"
To me, it feels similar to "If everyone just cooperated perfectly and helped each other out, we wouldn't need laws/money/government/religion/etc."
Yes, you're probably right, but no that won't happen the way you want to, because we are part of a complex system, and everyone has their very different incentives.
Semantic web was a standard suggested by Google, but unless every browser got on board to break web pages that didn't conform to that standard, then people aren't going to fully follow it. Instead, browsers (correctly in my view) decided to be as flexible as possible to render pages in a best-effort way, because everyone had a slightly different way to build web pages.
I feel like people get too stuck on the "correct" way to do things, but the reality of computers, as is the reality of everything, is that there are lots of different ways to do things, and we need to have systems that are comfortable with handling that.
Fresh plums right off the tree taste significantly better than the ones you can get in the produce isle, which are in turn better than canned, which are themselves still better than re-hydrated prunes.
In scaling out computation to the masses, we went from locally grown plums that took a lot of work and were only available to a small number of people that had a plum tree or knew someone that had one, to building near magical prune-cornucopia devices that everyone could carry around in their pockets, giving them an effectively unlimited supply of prunes.
LLMs re-hydrate these for us, making them significantly more palatable; if you're used to gnawing dried fruit, they seem amazing.
I always appreciate our weekly Crankypants Take on LLMs.
> AI is not a triumph of elegant design, but a brute-force workaround
You can read and understand Attention Is All You Need in one hour, and then (after just scaling out by a few billion) a computer talks to you like a human. Pretty elegant, if you ask me.
> The web was supposed to evolve into semantically structured, linked, machine-readable data that would enable amazing opportunities.
I missed that memo. The web was, and forever shall be, a giant, unstructured, beautiful mess. In fact, LLMs show just how hopeless the semantic web approach was. Yes, it's useful to attach metadata to objects, but you will still need massive layering and recursion to derive higher-order, non-trivial information.
This entire article is someone unable to let go of an old idea that Did Not Work.
Alan Kay has also written about his disappointment with what personal computing delivered. I think he'd agree with this.
Most people can't use the power of the computers they have effectively. Maintaining the data in Excel spreadsheets for most people is like manual labour.
Another angle: we're super over provisioned in compute resources because of reasonable limitations in how quickly we can collectively absorb and learn how to use computers. "AI" is simply a new paradigm in our understanding of how to program computers. This isn't a failure, it's just an evolution.
Yeah dream on. I’m an engineer and know what structured data is. And yet I miserably fail to store my private files in a way that I can find them back without relying on search tools. So how on earth are we ever going to organize all the world’s data and knowledge? Thank god we found this sub-optimal “band aid” called LLMs!
How do LLMs help Google, an ad company, generate more ad revenue. LLMs drive the traffic away from websites, give you answer directly without needing bother with ads. How does it benefit Google's ad business?
I think I broadly agree with this. I am super frustrated that everythign always wants me to search for things. As an example Finders default search while looking at a folder is the whole machine instead of a filter in the directory you are viewing in seems totally insane to me. It's almost like they don't want me to know where my files are.
I can understand that it's a result, to a degree of cloud services and peoples primary mode swapping to opening and app and opening recents or searching instead of opening a file to open an app but it does mean that you're at the mercy of what I experience as some pretty crap search algorithms that don't seem to want you to find the information you're looking for. I keep encountering searches that rank fuzzy matches over exact matches or aren't stable as you continue to complete the same word and I just don't understand how that's acceptable after being pointed out if search is what I'm supposed to be using.
56 comments
[ 2.8 ms ] story [ 60.4 ms ] thread"Standing on the shoulders of giants, it is clear that the giants failed to reach the heights we have reached."
But instead of iterating on better interfaces to effectively utilize the N thousands of operations per second a computer is capable of, the powers that be behind the industry have decided to invest billions of dollars in GPUs to get a program that seems like it understands language, but is incapable of counting the number of B's in "blueberry."
?
If you really want to, you know have this super generic indexing thing, why don't you go organize the web with hypercard and semantic web crap and tell us how it worked out for you
> Remember Semantic Web? The web was supposed to evolve into semantically structured, linked, machine-readable data that would enable amazing opportunities. That never happened.
I think the lesson to be learned is in answering the question "Why didn't the semantic web happen?"
In vertical markets, can LLMs generate a "semantic web of linked data" knowledge graph to be parsed with efficient NLP algorithms?
https://news.ycombinator.com/item?id=43914227#43926169
Is that the main thrust of it?
It might be short lived, who knows, but it's interesting that the recent progress came from capturing/consuming rather than systematically eliminating the nuance in language.
One can speak in their native language to a computer now and it mostly understands what is meant and can retrieve information or even throw together a scaffold of a project somewhat reliably.
It's not particularly good at writing software, however. Still feels like a human is needed to make sure it doesn't generate nonsense or apparently pretty insecure code.
So I'm not sure the author got the point across that they wished, but aren't vector databases basically a semantic storage/retrieval technology?
if it's a space issue, "semantic web" is far more relevant to the article than "personal computing".
To me, it feels similar to "If everyone just cooperated perfectly and helped each other out, we wouldn't need laws/money/government/religion/etc."
Yes, you're probably right, but no that won't happen the way you want to, because we are part of a complex system, and everyone has their very different incentives.
Semantic web was a standard suggested by Google, but unless every browser got on board to break web pages that didn't conform to that standard, then people aren't going to fully follow it. Instead, browsers (correctly in my view) decided to be as flexible as possible to render pages in a best-effort way, because everyone had a slightly different way to build web pages.
I feel like people get too stuck on the "correct" way to do things, but the reality of computers, as is the reality of everything, is that there are lots of different ways to do things, and we need to have systems that are comfortable with handling that.
In scaling out computation to the masses, we went from locally grown plums that took a lot of work and were only available to a small number of people that had a plum tree or knew someone that had one, to building near magical prune-cornucopia devices that everyone could carry around in their pockets, giving them an effectively unlimited supply of prunes.
LLMs re-hydrate these for us, making them significantly more palatable; if you're used to gnawing dried fruit, they seem amazing.
But there's still a lot of work to be done.
> AI is not a triumph of elegant design, but a brute-force workaround
You can read and understand Attention Is All You Need in one hour, and then (after just scaling out by a few billion) a computer talks to you like a human. Pretty elegant, if you ask me.
> The web was supposed to evolve into semantically structured, linked, machine-readable data that would enable amazing opportunities.
I missed that memo. The web was, and forever shall be, a giant, unstructured, beautiful mess. In fact, LLMs show just how hopeless the semantic web approach was. Yes, it's useful to attach metadata to objects, but you will still need massive layering and recursion to derive higher-order, non-trivial information.
This entire article is someone unable to let go of an old idea that Did Not Work.
Most people can't use the power of the computers they have effectively. Maintaining the data in Excel spreadsheets for most people is like manual labour.
Everything that is bad in UI is a direct consequence of that.
1. No tooltips, right click, middle click behavior because touch doesn't have that. No ctrl+click either.
2. Large click areas wasting screen space with padding and margins.
3. Low density UI so it can shape-shift into mobile version.
4. Why type on a phone when you can talk? Make everything a search box.
5. Everything must be flat instead of skeumorphic because it's easier to resize for other screen sizes.
6. Everything needs a swipe animation and views instead of dialogs because smartphones can't have windows.
I can understand that it's a result, to a degree of cloud services and peoples primary mode swapping to opening and app and opening recents or searching instead of opening a file to open an app but it does mean that you're at the mercy of what I experience as some pretty crap search algorithms that don't seem to want you to find the information you're looking for. I keep encountering searches that rank fuzzy matches over exact matches or aren't stable as you continue to complete the same word and I just don't understand how that's acceptable after being pointed out if search is what I'm supposed to be using.