Yes and working out how to disentangle the information storage mechanisms from say language processing is a massive area of interest. Only problem with Attention Transformers imo is that they're a bit too good :p
Imagine a slightly lossy compression algorithm which can store 10x, 100x the current best lossless and be able to maintain 99.999% fidelity when recalling that information.
Probably, very probably a pipe dream. But why do large on device models seem to be able to remember adjust everything from Wikipedia and store that in smaller format than a direct archive of the source Material. (Look at the current best from diffusion models as well)
Chain of thought seems to be an extraction algorithm for information buried deeper.
The models hold more information than they can immediately extract, but CoT can find a key to look it up or synthesise by applying some learned generalisations.
It's also important to say what it isn't. LLM detractors, for lack of a better word, expect an oracle and then when they find out it's just a lossily compressed blob of human knowledge with natural language as a query interface they say the tool is useless.
I've got my opinion on whether that's useful or not and it's quite a bit more nuanced. You don't zoom-enhance JPEGs for a reason either.
I disagree with that analogy, because LLMs have a lot of connections between text fragments, which an encyclopedia doesn't have to such a deep degree. An encyclopedia also can't interpret and output relevant knowledge from an input prompt.
The first thing I tell the juniors under my supervision: any LLM is not a fact machine, even though sometimes it pretends to be. Double check everything!
Yes, LLM is a lossy encyclopedia with a human-language answering interface. This has some benefits, mostly in terms of convenience. You don't have to browse or read through so many pages of a real encyclopedia to get a quick answer.
However, there is also a clear downside. Currently, LLM is unable to judge if your question is formulated incorrectly or if your question opens up more questions that should be answered first. It always jumps to answering something. A real human would assess the questioner first and usually ask for more details before answering. I feel this is the predominant reason why LLM answers feel so dumb at times. It never asks for clarification.
I think models will also become snapshots or time capsules (with obvious and non-obvious biases) that archaeologists of tomorrow (like in 500 years) will use to understand us and the change in society (e.g. how the models themselves change over timefrom encyclopaedic standpoint).
I find this a very useful analogy. Although it does not factor all the LLMs capabilties in.
I can also see them as very clever search engines, since this is one way I use them a lot: ask hard questions about a huge and legacy codebase.
These analogies do not really work for generating new code. A new metaphor I am starting to use is "translator engine": it is translating from human language to programming language. It in a way explains a lot of the stupidity I am seeing.
A lossy encyclopaedia should be missing information and be obvious about it, not making it up without your knowledge and changing the answer every time.
When you have a lossy piece of media, such as a compressed sound or image file, you can always see the resemblance to the original and note the degradation as it happens. You never have a clear JPEG of a lamp, compress it, and get a clear image of the Milky Way, then reopen the image and get a clear image of a pile of dirt.
Furthermore, an encyclopaedia is something you can reference and learn from without a goal, it allows you to peruse information you have no concept of. Not so with LLMs, which you have to query to get an answer.
You are absolutely right, and exactly the same thing came into my head while reading this. Some of the replies to you here are very irritating and seem not to grasp the point you're making, so I thought I'd chime in for moral support.
There are a lot of parallels between AI and compression.
In fact the best compression algorithms and LLMs have in common that they work by predicting the next word. Compression algorithms take an extra step called entropy coding to encode the difference between the prediction and the actual data efficiently, and the better the prediction, the better the compression ratio.
What makes a LLM "lossy" is that you don't have the "encode the difference" step.
And yes, it means you can turn a LLM into a (lossless) compression algorithm, and I think a really good one in term of compression ratio on huge data sets. You can also turn a compression algorithm like gzip into a language model! A very terrible one, but the output is better than a random stream of bytes.
I like this lossy compression / decompression analogy for coding too: when you prompt for a feature, you are basically asking to decompress the meaning of your ask into your existing code. Any semantic gap in your prompt will be filled with plausible glue, ie. the LLM makes decisions for you. A good prompt minimizes the glue needed and reduces the potential for really crappy outcome, but it's always a possibility!
I totally agree with the author. Sadly, I feel like that's not what the majority of LLM users tend to view LLMs. And it's definitely not what AI companies marketing.
> The key thing is to develop an intuition for questions it can usefully answer vs questions that are at a level of detail where the lossiness matters
the problem is that in order to develop an intuition for questions that LLMs can answer, the user will at least need to know something about the topic beforehand. I believe that this lack of initial understanding of the user input is what can lead to taking LLM output as factual. If one side of the exchange knows nothing about the subject, the other side can use jargon and even present random facts or lossy facts which can almost guarantee to impress the other side.
> The way to solve this particular problem is to make a correct example available to it.
My question is how much effort would it take to make a correct example available for the LLM before it can output quality and useful data? If the effort I put in is more than what I would get in return, then I feel like it's best to write and reason it myself.
Another metaphor: LLMs are sketches, not technical drawings. A sketch is not supposed to be the final product; it is exploratory, not definitive.
If you used sketches to build a house, it has a nonzero chance of falling down. Likewise, if you made technical drawings as a way to brainstorm house designs, the process would be overly rigid and extremely inefficient.
It is but when you take that stochastic token prediction machine and combine it with post, you can extract a graph that resembles intelligence that is then stored in a very disorganized fashion. This works because it is how we process and express /communicate information.
I think we will start seeing stateful AI models within the next couple of years and that will be a major milestone that could shake up the space. LLM is merely a stepping stone.
103 comments
[ 3.9 ms ] story [ 62.7 ms ] threadImagine a slightly lossy compression algorithm which can store 10x, 100x the current best lossless and be able to maintain 99.999% fidelity when recalling that information. Probably, very probably a pipe dream. But why do large on device models seem to be able to remember adjust everything from Wikipedia and store that in smaller format than a direct archive of the source Material. (Look at the current best from diffusion models as well)
llm is a pretty good librarian who has read a ton of books (and doesn't have perfect memory)
even more useful when allowed to think-aloud
even more useful when allowed to write stuff down and check in library db
even more useful when allowed to go browse and pick up some books
even more useful when given a budget for travel and access to other archives
even more useful when …
brrrrt
The models hold more information than they can immediately extract, but CoT can find a key to look it up or synthesise by applying some learned generalisations.
I've got my opinion on whether that's useful or not and it's quite a bit more nuanced. You don't zoom-enhance JPEGs for a reason either.
I can also see them as very clever search engines, since this is one way I use them a lot: ask hard questions about a huge and legacy codebase.
These analogies do not really work for generating new code. A new metaphor I am starting to use is "translator engine": it is translating from human language to programming language. It in a way explains a lot of the stupidity I am seeing.
When you have a lossy piece of media, such as a compressed sound or image file, you can always see the resemblance to the original and note the degradation as it happens. You never have a clear JPEG of a lamp, compress it, and get a clear image of the Milky Way, then reopen the image and get a clear image of a pile of dirt.
Furthermore, an encyclopaedia is something you can reference and learn from without a goal, it allows you to peruse information you have no concept of. Not so with LLMs, which you have to query to get an answer.
In fact the best compression algorithms and LLMs have in common that they work by predicting the next word. Compression algorithms take an extra step called entropy coding to encode the difference between the prediction and the actual data efficiently, and the better the prediction, the better the compression ratio.
What makes a LLM "lossy" is that you don't have the "encode the difference" step.
And yes, it means you can turn a LLM into a (lossless) compression algorithm, and I think a really good one in term of compression ratio on huge data sets. You can also turn a compression algorithm like gzip into a language model! A very terrible one, but the output is better than a random stream of bytes.
> The key thing is to develop an intuition for questions it can usefully answer vs questions that are at a level of detail where the lossiness matters
the problem is that in order to develop an intuition for questions that LLMs can answer, the user will at least need to know something about the topic beforehand. I believe that this lack of initial understanding of the user input is what can lead to taking LLM output as factual. If one side of the exchange knows nothing about the subject, the other side can use jargon and even present random facts or lossy facts which can almost guarantee to impress the other side.
> The way to solve this particular problem is to make a correct example available to it.
My question is how much effort would it take to make a correct example available for the LLM before it can output quality and useful data? If the effort I put in is more than what I would get in return, then I feel like it's best to write and reason it myself.
I can't think of any other tools like this. An LLM can multiply your efforts, but only if you were capable of doing it yourself. Wild.
indeed, Ted's piece (ChatGPT Is a Blurry JPEG of the Web) is here:
https://archive.is/iHSdS
If you used sketches to build a house, it has a nonzero chance of falling down. Likewise, if you made technical drawings as a way to brainstorm house designs, the process would be overly rigid and extremely inefficient.
https://www.youtube.com/watch?v=boiW5qhrGH4
https://www.youtube.com/watch?v=-V-vOXLyKGw
I think we will start seeing stateful AI models within the next couple of years and that will be a major milestone that could shake up the space. LLM is merely a stepping stone.