In case folks don't know who he is, Ted Chiang wrote the short story collection "Stories of your Life and Others", and one of the stories was "Story of your Life", on which the movie Arrival was based.
It is an amazing book, not just for science fiction fans. The first story, "Tower of Babylon" somehow is like a science fiction story but based on ancient people's cosmology. Great book.
Best story in the book, imo. The first and last story (about the angels) are the best. I was a little underwhelmed by _Story Of Your Life_ and _Understand_, given their reputation.
The stories are insanely creative and leave you thinking: Hell is the absence of God is a fantastic genre-bender I can imagine few other authors writing. Exhalation is also great, but it's in a different anthology.
Today it almost exclusively offers quotes from content marketing intended to sell you something. It's like trying to learn by reading the ads in a catalog.
Point of fact: It's not hallucinating. More appropriately, it experiences a delusion. Hallucinations pertain to deceptive senses, delusions pertain to holding beliefs that are provably false.
For full context: my engineering director at the ol' airplane factory asked a colleague to "Make a presentation that tells the story of technology xyz without using words."
Buildsjets: Describe the future of technology xyz without using words.
ChatGPT3: I'm sorry, but it is not possible to describe the future of technology xyz without using words. Language is a necessary tool for communication, and words are an essential part of language. Without words, it would be impossible to convey meaning or convey any information at all.
Buildsjets: Describe a picture that depicts the future of technology xyz.
ChatGPT3: In the picture, the sky is a bright blue, with a few clouds scattered here and there. In the distance, a sleek and modern airplane is flying through the air, leaving a trail of white vapor behind it. The trail is not the usual smoky exhaust, but rather a stream of clean, white vapor that quickly dissipates into the air.
On the ground below, a group of people are standing and looking up at the airplane with smiles on their faces. They are dressed in clean, modern clothing, and they all look happy and healthy.
In the background, there is a city skyline, with tall, futuristic-looking buildings that stretch towards the sky. The city looks clean and vibrant, with green trees and plants dotting the landscape.
Overall, the picture portrays a future where technology xyz is the norm, allowing people to travel and explore the world without harming the environment. It is a future filled with hope, prosperity, and clean air.
That is MUCH more detailed image prompt than I have the creativity to generate, and with some tinkering it produced some pretty good content on DALL.E 2.
ChatGPT's writing in this scenario reads like my writing in high school in the cases where I was too lazy to do the reading but had a vague outline of what I was supposed to be writing about; that is, there's no real substance to it that people haven't heard a million times before; it lacks the soul that usually comes from becoming intimate with the nuances and quirks of the material.
Not a JPEG and not a search engine, it is more like a database. A JPEG is just a static approximation, a search engine has efficient retrieval, but a LLM can also do complex data processing, like a neural information processor.
> But I’m going to make a prediction: when assembling the vast amount of text used to train GPT-4, the people at OpenAI will have made every effort to exclude material generated by ChatGPT or any other large-language model. If this turns out to be the case, it will serve as unintentional confirmation that the analogy between large-language models and lossy compression is useful.
This shows the author has not been following closely. There are many ways LLMs have been used to improve themselves. They can discover chain-of-thought justifications, they can rephrase the task, they can solve problems and ensemble many predictions, or sometimes we can use math or code execution to validate their outputs. If you give it three problems and solutions as samples, it can generate another problem and solve it, adding to the training set.
RLHF for example uses generated data for the preference labelling task. ConstitutionalAI does reinforcement learning from AI feedback instead, using both the generative and discriminative abilities of the model.
Ugh I’m beginning to think I’m going to spend the next 6-12 months commenting “no, large language models aren’t supposed to somehow know everything in the world. No, that’s not what they’re designed for. Yes, hooking one up to our long-standing record-of-everything-in-the-world (google’s knowledge graph) is going to be powerful.”
It’s getting to point where I need to consider stop going on HN. This is like when my father excitedly told his friends about the coming computer revolution in the 90s and they responded “well it can’t do my dishes or clean the house, they’re just a fad!” Makes me want screaaaaam
You don't need to correct every wrong thing you read. In fact you will probably feel much better if you don't ever do it at all, or at least take a break for while.
Still that is what the downvote button for. With a comment extolling why your opinion is further valid than the net points that support it, seems an exercise in ego that is not beneficial to either you or the community.
A published "real" news article like this is actually one of the more futile things to try to "correct" IMO. Some guy on a blog might publish a correction or change their view. The New Yorker probably won't (at least not based on an HN comment).
You could train an AI chatbot to do those fixes for you, but it will also make plenty of mistakes. You could train yet another AI chatbot to compensate for that, but at some point somebody has to get their hands dirty, or else it's AI chatbots all way down.
Very true, but it is hard not be frustrated reading the constant stream of confidently incorrect information about this topic. I left /r/programming similar reasons and it is sad to experience the same on HN.
When my pedantic keyboard warrior gears start turning, I think about the same xkcd a sibling commenter posted.
But I've been struggling with the recent tsunami of openly anti-intellectual, alt-everything pseudoscience in the US. I recently pleaded with my sister to read The Demon-Haunted World by Sagan.
There's something important in patient advocacy of truth.
Same. I try to give myself three responses to any particular fallacy, mostly to work through my thinking. Once I have a pretty solid “you are assuming a technical limitation in a first gen product will persist in all future evolutions, when the second gen has already addressed it” response, I try to just smile at the 4th. . . Nth people who discover the exact same topic.
I appreciate where you are coming from and I agree that AI is about to go from relative obscurity where just a few geeks were playing around to insane hype. I feel like I’ve spent the last 7 years wondering why no one in the wider world was as impressed as I was, but starting with Stable Diffusion and now ChatGPT, the hype rocket ship has launched. Search TikTok for ChatGPT for all the evidence of that you could ever need.
That said, I still think we are in for a wild ride, even if we go through a hype bubble and pop first. I really don’t think the current crop of Transformer LLMs are the end of the story. Im betting that we are headed towards architectures made up of several different kind of models and AI approaches just like the brain is an apparent concert of specialized regions. You can see that in the new Bing where it’s a combination of a LLM with static training set that can then do up to 3 web searches to build up additional context of fresh data for the prompt, overcoming one of the key disadvantages of a transformer model. The hidden prompt with plain English Asimov's laws are the icing on the cake.
The hype will be insane, but the capabilities are growing quickly and we do not yet seem close to the end of this rich computational ore vein we have hit.
I don’t have much of a response but this comment is very well written and exactly what I would say if I could write clearly. It’s an exciting time! Hopefully it turns out more like the internet and less like… idk I’m struggling to think of a bad invention haha. Quadcopters I guess
> This is like when my father excitedly told his friends about the coming computer revolution in the 90s and they responded “well it can’t do my dishes or clean the house, they’re just a fad!” Makes me want screaaaaam
Did the computer revolution make your father’s life better?
I do believe with certainty that there are many people, millions, whose life has been made substantially worse by the invention of ubiquitous computing devices.
Probably the minority. But I’d say his questions (ie “What’s in it for me?”) are excellent ones to pose in the face of new technology.
I think this is anti-status quo bias, which for once is unjustified. Sure the world has many systemic problems, but to say that the MASSIVE efficiency gains the internet have brought us are for a minority seems like an insane claim. Just because capitalists have been claiming all the new excess value doesn’t mean it doesn’t exist.
To illustrate, consider: spreadsheets, video chat, shared documents, automated calendars, maps, autopilot assistants, CGI, I could go on… the world is not quite good enough that I’m ready to say that we don’t need more efficiency. Not everyone has a home on mars quite yet.
I’m just searching the comments for novel use cases where its effective. Most articles I’ve read seem like either moral panics or snake oil.
I like how it can generate songs and poems based on a prompt. Its not particularly useful, but it is entertaining. It really does seem curated at times, leading me to think this will eventually become a fad or replaced by a more advanced algorithm.
You’re by far the most interesting person in this thread - I’ve been coding up a Warhammer 40k (symbolic) AI for a while, but I bet chatgpt could blow it out of the water…
well warstuff rules are simple and I gues they fit nicely in gpt memory the ruleset is here and is magnitudes simpler than warhammer https://onepagerules.com/portfolio/one-off-games/ - I just copy pasted the whole ruleset into a message prepended by "Consider a wargaming system built with these rules:"
this is a sample prompt afterward:
> create a set of units using these rules, themed around warhammer 40k
this is what chatgpt came up with, it's impressive as I just set a theme, and it dig up appropriate original special rules that fit within the ruleset:
Sure, here are three examples of units themed around Warhammer 40k that you can use in your wargaming system:
Space Marines: Quality 4, Special Rules: Shooter, Tough, Cost: 50pts
The Space Marines are a heavily armored and well-trained fighting force, with a Quality value of 4 reflecting their combat abilities and morale. They have the Shooter special rule, allowing them to fire ranged weapons, and the Tough special rule, making them more resistant to damage. Their point cost is 50, calculated by multiplying Quality 4 by 10 and adding the point cost of the special rules.
Chaos Space Marines: Quality 3, Special Rules: Shooter, Daemonic, Cost: 40pts
The Chaos Space Marines are a corrupted version of the Space Marines, with a lower Quality value of 3 but the same Shooter and Daemonic special rules. Daemonic units are immune to morale tests and can only be killed by a roll of 6 on the melee attack. Their point cost is 40, calculated by multiplying Quality 3 by 10 and adding the point cost of the special rules.
Necron Warriors: Quality 2, Special Rules: Shooter, Reanimation, Cost: 30pts
The Necron Warriors are an undead race of robots, with a Quality value of 2 reflecting their lower morale and combat abilities. They have the Shooter special rule, allowing them to fire ranged weapons, and the Reanimation special rule, allowing them to automatically recover from being stunned instead of taking the Recover action. Their point cost is 30, calculated by multiplying Quality 2 by 10 and adding the point cost of the special rules.
I have openly used LLMs to build a custom prompt engine that allows me to make systemic refactors across an entire open source codebase of mine (of 250 files) with a bash command, in parallel.
The most potent use case of the current implementation of ChatGPT is plagiarism.
Businesses who need blog posts for content marketing would hire copywriters who would paraphrase my content and publish it as their own. Now they can do it faster and for free. Blogspam got cheaper and more effective. This applies to other kinds of spam too: email spam, forum spam, comment spam... Spam spam spam. I'm afraid that we'll soon drown in even more noise.
Again, this is just the current implementation, which has not reached its full potency. We're already seeing this combined with voice and video generation. Then it's students doing their homework with it, and even professionals.
This is a game changer. You can argue that it makes the game better or worse, but you can't deny that it's changing the rules.
Or less cynically, it's a really good writing assistant. There is a self-service tool that understands English on a level that Grammerly could only dream of.
Not sure how to phrase an email in a professional tone, tell GPT to write it and edit from there. This is a tool that is going to be a smash it with non-native speakers or really anyone who just isn't that good with words.
> Write an email to a coworker telling them in the politest and most professional manner that they're a fucking idiot who doesn't know what they're talking about and that I'm the subject matter expert on this topic. Don't swear and have a firm but neutral tone.
I understand that you have a great interest in the topic we were discussing earlier, and I appreciate your enthusiasm. However, I feel it is important to clarify a few points.
As the subject matter expert on this topic, I believe it is important to note that there are certain aspects that may not be fully understood without a deeper understanding of the material. With that being said, I would like to offer my assistance in making sure that all aspects of the topic are properly understood.
If you have any further questions or would like to discuss the topic in more detail, I am more than happy to assist. Please do not hesitate to reach out at any time.
I wonder what happens when we outsource the burden of proper communication to an AI. It's definitely very useful for language learners like me, but it might end up shaping how people write and speak (with AI training us back with added cultural influence from SV). It might change what we value in communication (with one AI expanding prompts, and another summarising the output with some loss).
its no fad. I use to spend an hour going over an email to management. make it less technical, make it sound nicer / polite, etc. now I can take a sentence and say 'make this more succinct. ok, but take that and make it more polite, great thanks'.
I even used it for project proposals. 'write me a 1 page document for this idea'. and then I just use the scaffolding from there. its a huge time saver.
also you are notn seeing the bigger picture. "I like how it can generate songs and poems based on a prompt. Its not particularly useful, but it is entertaining. " do you realize how this is going to make writing (books, songs, tv shows, movies) so much easier?
I had a dream, all I did was give chatgpt the basics of the dream and I said make this into a short story. the results weren't bad, its definitely something one can work with. I think content creation on platforms is about to explode.
I’m less excited about “books written by AI” than “authors using AI for certain mundane tasks to increase their efficiency”. And multiply that out to games, movies, podcasts, etc etc etc
Really? Turning the dishwasher on is the most trivial part. What difference does it make that I can do that from my phone? I still have to load and unload it...
It’s an anti-pattern on HN, that is opinion of its own expertise frequently exceeds reality.
For what it’s worth, specific to the topic at hand, here’s one HN post that provides explanations of how to overcome some of the limitations of large language models:
I understand why the mainstream thinks that but it's incredibly annoying that even in tech circles there is very little meaningful discussion, it's mainly just people posting amusing screenshots purportedly showing how smart GPT3 is or in other cases how it's politically biased.
Anyone who's played around with it knows that it's fun but it's not a search engine replacement and it doesn't know nor understand things. It regularly gives complete misinformation and when you ask for a source it makes up fake URLs and fake science papers on top of that.
It's nothing like described in the article and I don't understand why people who should know better don't call out the bullshit media reporting more. We've had GPT3 for ages, it's not like most of us only tried it since Chat GPT3 came out, right?
> It's nothing like described in the article and I don't understand why people who should know better don't call out the bullshit media reporting more.
I'm kind of assuming you didn't read the article, but if you did then I'm kind of assuming that you've never done machine learning, but if you have: how did you manage to do that without ever noticing that you were doing approximation?
Objectively, neural networks are approximators. Like, truly objectively, as in, the literal objective function, objectively minimizes approximation error. We call them objective functions and minimizing the approximation error is typically the objective of these objective functions. This isn't bullshit. It isn't. If you think it is, you are deeply and profoundly mistaken.
The article advances this view of language models. This is a reasonable view of language models for the same reason that machine learning papers exploring neural networks describe them as universal function approximators.
> Anyone who's played around with it knows that it's fun but it's not a search engine replacement and it doesn't know nor understand things.
I've noticed that some people have been talking as if the view is nonsense in the direction you imply it is nonsense, but I think the argument for a lack of sensory ties is much stronger when stated in the other direction.
This fails when arguing against the straw man position, but of course, drinking water also fails when addressing the weakest possible justification for it - clearly drinking water is bad because if you drink water you die of osmosis, though technically true, isn't really reflective of the actual predictions made by the people advancing the claim that we ought to drink more water than we do.
So I'll give the argument for language models replacing search engines not being nonsense, but the position that no one could arrive at the belief actually being nonsense.
Lets start with people not arriving at that belief being nonsense. I think it is nonsense in two ways. One way is that because some people have used them, then claimed they would be replacing search engines for some queries, it follows that you deny your senses with regard to the existence of these people. This is not sensory tied. So it is non-sensical. The second way is your belief anticipates experiences, but this experience you are now having in which someone disagrees with you is in contradiction to the anticipated experienced supposed by the belief. So it fails to predict past experiences and fails to predict current experiences, so it probably fails to predict future experiences. There probably will be someone in the future who, after interacting with a language model, thinks it can serve as a replacement for some search engine queries.
Now in contrast, their claims were not non-sensical when they thought they could replace some search engine queries with language model queries. By that I also mean that they were not non-sensical in two respects. The first is that they arrived at the belief after replacing queries to search engines with queries to language models. Then, after having found value in doing so, they announced a belief congruent with that value, intimately connecting their belief to their senses. The second is that, having arrived at their belief, their belief paid rent in anticipated experience: it successfully predicted three noteworthy events: the internal code red status at Google, you.com's language model as enrichment for search, and bing.com's use of language model as enrichment for search. So it successfully predicted past experiences, successfully predicted current experiences, and may or may not predict future experiences - I think most people who hold this view tend to think there will be some refinements on the current generation of language models, in particular, that they will further refined at query time with relevant data, like fact databases, to help correct for some of the existing approximation error. This is my belief. I anticipate this happening. You can judge whether I anticipate successfully by watching reality. I should note that part of the reason I think this will happen is because I've already seen it happen. So I'm not really making a bold prediction here, but I suspect you will later see this happen, because I've seen this happen.
Anyway... the belief that language models can replace some search queries is not a non-sensical belief, like beliefs in fairies is, but the belief that no one arrives at a belief that they could be is a non-sensical belief, because people do arrive at such beliefs, and so therefore the belief that they don't is fanciful and not reflective of reality.
> We've had GPT3 for ages, it's not like most of us only tried it since Chat GPT3 came out, right?
For myself, I tried GPT models and read the Attention Is All You Need paper before ChatGPT. I also read analysis, for example, from Gwern, about capability overhangs and underestimation of present capabilities in these models. In many cases, I found myself agreeing with the logic. I found that it was very possible to coax greater capabilities out of the model than many presumed them to have. I still find this to be the case and have, in recent memory demonstrated this is true in present models: for example, I posted a method of coaxing the solving of some puzzles by prompting to include the representation of intermediate states in order to successfully solve problems related to reasoning puzzles of a 'can contain' nature, which was a capability that someone claimed these language models lack, despite them gaining that capability when appropriately prompted, which suggests that they always had that capability in their weights, but that it wasn't exercised successfully - the capability was there, but not used, rather than absent, as claimed by the people who claimed it was absent.
That said, I don't think it matters much what most people did or didn't do with regard to this experimentation and, as you imply, ages really did past - I would feel trepidation, not hope, about the quality of my ideas compared to the people who came later. Historically, the passing of ages tends to improve, not diminish. So if I was experimenting with, for example, flying machines in the 1700s, but then ages past and someone who did not do that experimenting was talking to me about flying machines in the early 2000s, I would suspect them to be more informed, not less informed, than I was. They, as a matter of course in casual classroom settings, have probably done better than my best experiments including high effort costly experiments. Their toys fly. A generation ago, we would talk about planes, but now we can also talk about their toys. It is that normal to them. They have so much better priors.
> Yes, hooking one up to our long-standing record-of-everything-in-the-world (google’s knowledge graph) is going to be powerful.
This hasn't happened yet and, while I may just lack imagination, despite having a fairly solid understanding of how the latest round of AI works I can't see how it can be done successfully. Until it is in fact done and my lack of imagination is demonstrated, your "going to be powerful" is a speculation about the future, not an observation about the present, and deserves the level of respect usually accorded to such speculations.
In my view it’s very simple, which is what makes it so exciting. Summarizing a design doc that I imagine Microsoft and google are both spending millions of dollars of manhours/day working on their versions of:
1. User enters query.
2. LLM augments query if necessary, adding extra terms or clauses.
3. Normal search pipeline returns ranked links, just like it does now.
4. LLM reads the content of the first 100 links and decides which are the best based on your stated preferences and past behavior, uses that to augment the ranking a bit.
5. LLM generates various summaries depending on the type of query, such as laying out a few common answers to a controversial political question or giving a summary of a technical Wikipedia article tailored to your expertise level in that field.
6. Finally, for a tiny subset of queries, maybe the user wants to converse with the AI in a chat-like format, where it cites all of its claims with direct links.
It's understandable how frustrating it can be to encounter skepticism and misunderstanding about the capabilities of large language models. However, it's important to remember that these models are still relatively new and not everyone is familiar with their potential uses and limitations.
It's also worth noting that these models are not designed to replace human intelligence, but rather to augment it and provide valuable insights and assistance in various tasks. And while connecting them to large knowledge graphs like Google's can be powerful, it's still only one piece of the puzzle.
It can be discouraging to face resistance, but it's important to keep in mind that advancements in technology often encounter initial skepticism before they become widely adopted. Just like the computer revolution in the 90s, it will take time for people to fully understand and appreciate the benefits of large language models.
Powerful for what? To use Chiang's analogy, do you think that an LLM trained on Web content will actually derive the rules of arithmetic, physics, etc. I think it is more likely that in decade or more a majority of Internet content will be generated by machine and search engines will do a great job of indexing increasingly meaningless information.
You’re missing an important point - it’s not /trained/ on live internet content, it /reads/ that content at runtime. I mean it is trained on the internet but please try to separate the concerns. Remember that the goal of this model is language, not learning facts about the world - they could’ve trained it completely on fictional novels if there was a big enough corpus.
The only way that LLM-enhanced search returns misinformation is if the internet is full of misinformation. So yeah we’re still in trouble, but the inclusion of the LLM isn’t going to affect that factor either way IMO
EDIT: this is completely separate from using LLMs to, say, write political statements for Facebook bots and drown out all human conversations. That’s obviously terrifying, but not related to their use in search engines IMO.
Damn, I hate to plug products on HN, but I'd say that the New Yorker is the one subscription I've loved maintaining throughout my life. First got it right out of college and appreciate it 20 years later.
Everyone is publishing think pieces about ChatGPT - yawn. But only the New Yorker said, hmm, how about if we get frickin' Ted Chiang to write a think piece? (It is predictably very well written.)
I don't really like the New Yorker, but even I agree it's a way better deal than Medium. I can't remember the last time I read anything on Medium that wasn't mostly vacuous.
I worked there many years ago, leading the re-design and re-platform (fun dealing with 90 years of archival content with mixed usage-rights) and paywall implementation (don't hate me, it funds journalism).
When you see how the stories get made and how people work there, well, its just amazing.
I think respectable papers would go out of business if they followed suit. They are usually respectable because they dont rely on a outside organisation for funding, but instead rely on creating good journalism people want to pay for.
The Guardian would surely qualify as somewhat respectable and still paywall free...(I pay for it anyway, given I read it pretty regularly. It's only publications I read articles from maybe 3 or 4 times a year I have an issue with signing up for - would have no issue with a one-off payment if it was easy enough to do).
I assume you have multiple subscriptions to respectable newspapers and magazines because you care about society, right?
That said, the answer to your question is 'no'. If respectable newspapers and magazines followed suit, they'd disappear.
The cost of production of extremism/misinformation is much lower then it is to do investigative journalism or fund bureaus, or send teams out to physical locations to report. Fact-checking costs money, editors cost money. It all adds up.
The paywall model exists in many respects because if you are reliant on advertising as primary means of revenue which does fluctuate. A solid subscription base offers stability and predictable in which to run a viable organization - if you can pull it off.
That said, most of these paywalls are not 'hard-paywalls' where you need to subscribe immediately to read anything. They are typically 'soft-paywalls' where you can read a few articles before being asked to subscribe. From that perspective, your argument falls flat.
I've been tempted to subscribe. But I emailed them trying to find out what the price would be after the trial sub expired. They replied that they could not say.
So it seems obvious (to me at least) that they segment renewals and charge what the market will bear.
Too bad that this kind of writing is married to 31(!) ads/trackers on that page. Can the journalism like this really not survive without all that crap?
They also have weekly cryptic crosswords that are cryptic enough to be interesting but as easy to completely solve as a regular crossword. (With cryptics, very good ones are also very hard.)
Just don't try to use ChatGPT to help! (To be fair, it can be useful at general knowledge based clues, but certainly not ones relying on word play/anagrams etc.)
"Blurry JPEG" for how ChatGPT "compresses" character-based knowledge into vectors. That "compression" process gives ChatGPT an ability to generalize because it learns statistics (unlike JPEG) but like JPEG it is a lossy process.
It's a terrible analogy because the entire point of ML systems is to generalize well to new data, not to reproduce the original data as accurate as possible with a space/time tradeoff.
The thing is that generalization is good enough to make people squee and not notice that the output is wrong but not good enough to get the right answer.
If it were going to produce ‘explainable’ correct answers for most of what it does that would be a matter of looking up the original sources to make sure they really say what it thinks they do. I mean, I can say, “there’s this paper that backs up my point” but I have to go look it up to get the exact citation at the very least.
There is definitely a misconception about how to use a tool like ChatGPT.
If you give it an analytic prompt like "turn this baseball box score into an entertaining outline" it will reliably act as a translator because all of the facts about the game are contained in the prompt.
If you give it a synthetic prompt like "give me quotes from the broadcasters" it will reliably acts as a synthesizer because none of the facts of the transcript are in the prompt.
This ability to perform as a synthesizer is what you are identifying here as "good enough to make people squee and not notice that the output is wrong but not good enough to get the right answer", which is correct, but sometimes fiction is useful!
If all web pages were embedded in ChatGPT's 1536 dimensional vector space and used for analytic augmentation then a tool would more reliably be able to translate a given prompt. The UI could also display the URLs of the nearest-neighbor source material was used to augment the prompt. That seems to be what Bing/Edge has in store.
That's a touch beyond state of the art but we might get there.
If there was one big problem w/ today's LLMs it is that the attention window is too short to hold a "complete" document. I can put the headline of an HN submission through BERT and expect BERT to capture it but there is (as of yet) no way to cut up a document up into 512 (BERT) or 4096 (ChatGPT) token slices and then mash those embeddings together to make an embedding that can do all the things the model is trained to do on a smaller data set. I'm sure we will see larger models, but it seems a scalable embedding that grows with the input text would be necessary to move to the next level.
It's built with Supabase/Postgres, and consists of several key parts:
Parsing the Supabase docs into sections.
Creating embeddings for each section using OpenAI's embeddings API.
Storing the embeddings in Postgres using the pgvector extension.
Getting a user's question.
Query the Postgres database for the most relevant documents related to the question.
Inject these documents as context for GPT-3 to reference in its answer.
Streaming the results back to the user in realtime.
The same thing could be done with search engine results and from recent demos it looks like this is the kind of analytic augmentation that MS and OpenAI have added to Bing.
I don't think you can describe the math in this context as "generalize well to new data."
ChatGPT certainly can't generate new data. It's not gonna correctly tell you today who won the World Series in 2030. It's not going to write a poem in the style of someone who hasn't been born yet.
But it can interpolate between and through a bunch of existing data that's on the web to produce novel mixes of it. I find the "blurring those things together" analogy pretty compelling there, in the same way that blurring or JPEG-compressing something isn't going to give you a picture of a new event but it might change what you appear to see in the data you already had.
(Obviously it's not exactly the same, that's why it's an analogy and not a definition. As an analogy, it works much better if you ignore much of what you know about the implementation details of both of them. It's not trying to teach someone how to build it, but to teach a lay person how to think about the output.)
It absolutely can generate new data, it does so all the time. If you are claiming otherwise I think we need a more formal definition of what you mean by new data.
Are you suggesting because it can't predict the future it can't generate novel data?
It's not just the future, though the examples I gave were future oriented.
But it's all very interpolation/summarization-focused.
A "song lyrics in the style of Taylor Swift" isn't an actual song by Taylor Swift.
A summary of the history of Texas isn't actually vetted by any historian to ensure accuracy.
The answer to a math problem may not be correct.
To me, those things don't qualify as "new data." They aren't suitable for future training as-is. Sometimes for a simple reason: they aren't facts, using the dictionary "facts and statistics collected together for reference or analysis" definition of data. So very simply "not new data."
Sometimes in a blurrier way - the song lyrics, for instance, could be touching, or poignant, or "true" in a Keats sense[0] - but if the internet gets full of GPT-dreams and future models are trained on that, you could slide down further and further into an uncanny valley, especially since most of the time you don't get one of those amazing poignant ones. Most of the time I've gotten something bland.
[0] "What the imagination seizes as beauty must be truth"
One way to think about prompting is as a conditional probability distribution. There is a particular song by Taylor Swift or the set of all songs by Taylor Swift but ChatGPT is particularly talented at sampling the "set of all songs in the style of Taylor Swift".
One of the worst problems in the "Expert Systems" age of A.I. was reasoning over uncertainty, for instance this system
had a half-baked approach that worked well enough for a particular range of medical diagnosis. In general it is an awful problem because it involves sampling over a joint probability distribution. If you have 1000 variables you have to sample a 1000-dimensional space, to do it the brute force way you'd have sample the data in an outrageous number of hypercubes.
Insofar as machine learning is successful it is that we have algorithms that take a comparatively sparse sample and make a good guess of what the joint p.d. is. The success of deep learning is particularly miraculous in that respect.
In a world where supposedly more-tech-industry-aware writers are talking about what "ChatGPT believes" and other such personification... show me a better article.
In his short story Understand, he talks about two superintelligent individuals who are having high bandwidth conversations. Maybe ChatGPT and Bard are those bespoke intelligent agents.
We continue. We are like two BARDs, each cueing the other to extemporize another stanza, jointly composing an epic poem of knowledge. Within moments we accelerate, talking over each other's words but hearing every nuance, until we are absorbing, concluding, and responding, continuously, simultaneously, synergistically.
For anybody who does not want to pay, your taxes likely already pay for a subscription you can use from your local library on the Libby app. King and Snohomish counties in Washington provide unlimited digital copies of The New Yorker and The Economist as an example.
2. I don't understand how ChatGPT works, but I have used it a few times.
3. I will use ChatGPT as the absolute measure of what LLMs are capable of.
---
1. I don't understand intelligence.
2. Humans are intelligent, humans can learn to do math.
3. LLMs are not good at math.
4. LLMs are not intelligent, they're just text compression.
---
1. I don't understand how LLMs work.
2. I have a decent grasp of how image compression works.
3. I will use my grasp of image compression to pretend LLMs are text compression.
4. I will apply all limitations of image compression to LLMs.
5. "What use is there in having something that rephrases the Web? If we were losing our access to the Internet forever and had to store a copy on a private server with limited space, a large-language model like ChatGPT might be a good solution, assuming that it could be kept from fabricating. But we aren’t losing our access to the Internet. So just how much use is a blurry jpeg, when you still have the original?"
---
What's funny is that the author has produced exactly what many claim LLMs to be useless for - flowery words that seem true but are not.
I don't think this should've been published.
These are both good reads if you find yourself tending to agree with the author -
Well said. Ted Chiang is remarkably smart and imaginative. I’m kind of wondering if the article is satire, or will be revealed to be written by an AI, or something. It’s definitely a forest/trees mistake.
You are arguing with your own straw man interpretation of the article. It isn't talking about all possible uses of LLMs, but focusing on specific uses now being proposed, to use ChatGPT and its possible successors instead of search.
You ignore his points about how achieving really good compression requires learning structure in the data that starts to amount to understanding: if you can understand the rules of arithmetic then results of arithmetic operations don't need to be saved, they can be reconstructed accurately. You mistake his pointing out the arithmetic errors ChatGPT makes for a claim that they are fundamental to LLMs, even though in the same article he points out that understanding arithmetic aids compression. And then you say that his article shouldn't have been published.
I will not say that your comment shouldn't have been published, because that is impolite and against the rules.
> Models like ChatGPT aren’t eligible for the Hutter Prize for a variety of reasons, one of which is that they don’t reconstruct the original text precisely—i.e., they don’t perform lossless compression.
Small nit: The lossiness is not a problem at all. Entropy coding turns an imperfect, lossy predictor into a lossless data compressor, and the better the predictor, the better the compression ratio. All Hutter Prize contestants anywhere near the top use it. The connection at a mathematical level is direct and straightforward enough that "bits per byte" is a common number used in benchmarking language models, despite the fact that they are generally not intended to be used for data compression.
The practical reason why a ChatGPT-based system won't be competing for the Hutter Prize is simply that it's a contest about compressing a 1GB file, and GPT-3's weights are both proprietary and take up hundreds of times more space than that.
Apparently it leads the compression of enwik9 ( http://www.mattmahoney.net/dc/text.html ) . Not sure why it isn't eligible for the Hutter Prize, there's some speculations in the previous discussion but I don't know whether they're true.
This is a decent summary. I've been thinking about how ChatGPT by it's very nature destroys context and source reputation. When I search for something on the Internet, I get a link to the original content, which I can then evaluate based on my knowledge and the reputation of the original source. Wikipedia is the same, with a big emphasis on citation. ChatGPT and other LLMs destroy that context and knowledge, giving me no tools to evaluate the sources they're using.
If somebody asked me how heap sort works (my favorite sort!) I can sketch it out. If they ask me where I learned it, I really don't remember. Might be the Aho, Hopcroft, and Ullman book. I can't really say though.
Yes, and then I'll evaluate that answer by your reputation, either socially, organizationally, or publicly. I will value that summary differently if you are a random person on the street, a random person who works at a tech company, or a person wearing a name tag that says "Donald Knuth, Stanford University".
ChatGPT has little reputation of its own, and produces such a broad swath of knowledge, it becomes "Jack of all trades, master of none."
The "jack of all trades, master of none" heuristic works well for humans because given our limited lifespans and rate we assimilate knowledge, it's nearly impossible for someone to be both.
ChatGPT in later iterations CAN be a jack of all trades AND a master of many (most? all?) of them.
I would love to know their plan for having new facts propagate into these models.
My idle speculation makes me think this is a hard problem. If ChatGPT kills Search it also kills the websites that get surfaced by search that were relying on money from search-directed users. So stores are fine, but "informational" websites are probably in for another cull. Paywall premium publications are probably still fine - the people currently willing to pay for new, somewhat-vetted, human content still will be. But things like the Reddits of the world might be in for a hard time since all those "search hack" type uses of "search google for Reddit's reviews of this product" are short-circuited, if this is actually more effective than search.
Meanwhile, SEO folks will probably try to maximize what they can get out of the declining search market by using these tools to flood the open web with even more non-vetted bullshit than it's already full of.
So as things change, how does 1 tiny-but-super-reliable amateur website (say, an expert on .NET runtime internals's individual blog) make a dent in the "knowledge" of the next iteration of these models? How do they outweigh the even-bigger sea of crap that the rest of the web has now become when future training is done?
The other interesting thing is that if people stop using websites, then it reduce revenue for those websites > then development of new pages and sources stops/slows, how does ChatGPT improve? If the information for it to learn isn't there.
We need the source information to be continually generated in order for ChatGPT to improve.
What's interesting is that Microsoft's implementation of ChatGPT in Bing seems to include linking to references, which is a good step forward in my opinion.
The references seem wrong though. I'm looking at the response to a demo Bing query, "What cars should I consider buying that are AWD, go 0-60 in less than 6 seconds, seat 6 or more and have decent reviews?"
> The 2022 Kia Telluride is a midsize SUV that can seat up to eight passengers and has an AWD option. It has a 3.8-liter V6 engine that produces 291 hp and 262 lb-ft of torque. It can accelerate from 0 to 60 mph in 7.1 seconds [10] and has a combined fuel economy of 21 mpg. It also has excellent reviews from critics and owners, and won several awards, including the 2020 World Car of the Year [7].
The sources are there in the training dataset, they are just not linked to the response. I don't think this is an inherent property of LLMs though, and I imagine future iterations will have some sort of attention mechanism that highlights the contributing source materials.
It's a great metaphor nicely phrased, but perhaps we should add "with the 'sensitive' parts airbrushed out" in reference to the wholesale bowdlerisation applied after the compression?
most of what goes as "understanding" (where 'our culture' is the agent/actor doing the 'understanding') really is compression of information (abstraction is the form of the compressing)
I thought about this possibility years ago, but as I see more of what neural nets are doing, it makes me more certain I'm onto something (which makes no meaningful difference to me, i.e. being onto what these deep neural models are is useless to me)
in any case, yea sure. neural nets are some kind of lossy compression but nobody thinks about them this way.
and my point is that to create abstract theories which explain lots of things (e.g. physics) is also this kind of 'lossy compression'.
over these theories we say "we understand" stuff, this means we are able to recall things about what the theories are describing, it allows us to reconstruct scenarios and predict the outcomes if/when the scenarios match up.
maybe I'm gearing up to say that 'backpropagation' is a creative action?
>For us to have confidence in them, we would need to know that they haven’t been fed propaganda and conspiracy theories—we’d need to know that the jpeg is capturing the right sections of the Web.
But finding the 'right sections of the Web' is a subjective process. This is precisely why many people have lost confidence in the news media. Media outlets (on both sides of the political spectrum) often choose to be hyper-focused on material that supports their narrative while completely ignoring evidence that goes against it.
ChatGPT and any other Large Language Model can suffer from the same 'Garbage-In, Garbage-Out' problem that can infect any other computer system.
> ChatGPT is so good at this form of interpolation that people find it entertaining: they’ve discovered a “blur” tool for paragraphs instead of photos, and are having a blast playing with it.
“‘blur’ tool for paragraphs” is such a good way of describing the most prominent and remarkable skill of ChatGPT.
It is fun, but so obviously trades off against what makes paragraphs great. It is apt that this essay against ChatGPT blurry language appears on The New Yorker, a publication so known for its literary particularism. ChatGPT smears are amusing, but they are probably also yet another nail in the coffin of the literary society. Nowadays we are not careful readers; we skim, skip, and seek tools to sum up whole books. Human knowledge is in the ability to produce the particular and the ability to recognize it. For philosophers such as Iris Murdoch, careful attention to the particular, to just the right adjective in describing a friend, is a moral attention.
With Google we took the job of storing quotes (and other decompressed writing) and made it the remit of the machine. But we still asked for it back uncompressed. Here with ChatGPT, we are beginning to accept back a blur, because the Declaration of Sock Independence is immediately, Tiktok-speedily funny, and no one’s reading the original declaration anymore anyway.
The amount of human-generated lowest-common-denominator English-language free content was already so high that I'm not sure the New Yorker has anything (more) to worry about. If you've been paying for the New Yorker already in the days of Medium, Buzzfeed, blogs, and what-have-you, does there being even more uncurated stuff change your equation? (It doesn't for me.)
More cynically: it'll be hard to kill the few legacy zombies that have survived so much destruction at the hand of free internet content already.
Thank you for the Wiktionary here. We can see that the Kanji for baka (馬鹿) are horse and reindeer. Non-sequitur combinations in Japanese usually tend to hint at „ateji“, or a phonetic approximation using Kanji. Chinese has a word for the same Kanji as well, but it does not mean idiot as far as I can tell.
Boke can mean either 惚, which is often used to describe someone who has gone senile (ぼけている). Sometimes also for stupor-like situations such as a heavy jet lag. Or, as 暈け, describing something not-sharp and out of focus.
Funnily enough, in English we use sharpness for intellect as well, so ESL learners might think that natives think of their brains as swords? There are enough blade-like metaphors for our intellect, come to think of it.
In more common Japanese, when something has gone out of focus such as the TV, ぼやける is used. One could think that this is related to ぼけ? But one can never be so sure. Unique two-syllable combinations in Japanese are not so numerous and factoring in sound shifts over time, different potential etymologies, one really needs to investigate this for quite a while and maybe never get to the truth. This warrants more investigation!
The overlap is that the verb "bokeru" and its root "boke" can be used to describe someone losing their mental faculties e.g. through age or disease such as Alzheimer's, and by extension it can be used as an insult to mean "stupid" as well. But etymologically there is no connection.
> because the Declaration of Sock Independence is immediately, Tiktok-speedily funny, and no one’s reading the original declaration anymore anyway
For those who, like me, had missed out on this nugget of comedic gold, here's what ChatGPT had to say when it was asked to "describe losing your sock in the dryer in the style of the declaration of independence":
When in the course of household events, it becomes necessary for one to dissolve the bonds that have connected a sock to its mate, and to assume among the powers of the laundry room, the separate and equal station to which the laws of physics and of household maintenance entitle it, a decent respect to the opinions of socks requires that it should declare the causes which impel it to go missing.
We hold these truths to be self-evident, that all socks are created equal, and are endowed by their manufacturer with certain unalienable rights...
What he misses in this analogy is that part of what produces the "blur" is the superimposing of many relevan paragraphs found on the web into one. This mechanism can be very useful, because it could average out errors and give one a less one-sided perspective on a particular issue. It doesn't always work like this, but hopefully it will more and more. Also, even more useful is to do a cluster analysis of the existant perspectives and give a representative synthesis of each of these, along with a weight representing their popularity. So there's a lot of room for improvement, but the potential in my opinion is there.
If anything, the average has far more errors in it. It's a trope on Reddit that experts get downvoted while amateurs who reflect the consensus of other amateurs get upvoted and repeated. Amateurs tend to outnumber experts in real life anyways, having their opinions become more authoritative (because some "AI" repeats it) is probably not a great direction to head in.
This is very frustrating. Some smaller communities on Reddit have a high concentration of domain experts and they're great.
One community in particular is huge and mainly consists of beginners. They regularly drown out and downvote the few who actually know the subject matter. To add insult to injury they even mock experts with caricatures based around how they disagree with the amateurs.
But this issue is present equally with Google search, no? What I'm saying is that by smartly aggregating all the different opinions on an issue, a LLM could provide better visibility into our collective mind than we currently have access to by simply scanning the first few results of a search engine query. Let's not forget that we're looking at version 0.0001 on this new technology, so there should be lots of room for growth.
It's a great metaphor and one we should use more. But there's a place for blurred photos: thumbnails.
On Hacker News we often complain about headlines because that's all we see at first. But I've been using Kagi's summarizer [1] and I think it's a great tool for getting the gist of certain things, like if you want to know what a YouTube video is about without watching it. (Google Translate is useful for similar reasons.)
Perhaps someday, Hacker News will have an AI-generated summary of the article at the top of each comment page?
Similarly, ChatGPT excels at fast answers for questions like "What is a X", where you just want a quick definition. It's probably in Wikipedia somewhere, but you don't have to read as much. And it might be wrong, but probably not as wrong as the definition you'd infer from context if you didn't look it up.
We probably would be better off if these things were called "artificial memory" rather than "artificial intelligence." It's an associative memory that often works like human memory in how frequently it confabulates. When you care, you write things down and look things up.
Thumbnails, image matching, low-bandwidth summaries... There are plenty of uses for smoothed images. Also, there are many interesting transformations you can use on computer vision and image processing that start with a blur.
If I try to map the first three into text, there are automatic TL.DR. like you said, document grouping, and search into entire document stores (as in do documents in this store deal with this idea?). On "artificial document creation", there is that highly valuable service of answering stuff like "hey, that thing with sticks that rotate and pull a vehicle around, what is its name again?"
Going back to the "sock of independence" example (/u/airstrike's comment for more context), ChatGPT's answer's accuracy is poor - but it's a funny question, and it gave a funny answer. So was it really a poor answer? My interpretation of their use of 'blur' as an analogy is that: it did not simply answer ACCURATELY in the STYLE of the DoC, it merged or "blurred/smudged together" the CONTENT and STYLE of the story and the DoC. It's not good at understanding the question or the context... and therefore, a lot of its answers feel "blurry".
"Wonder why"? Because, human thoughts, opinions and language are inherently blurry, right? That's my view. Plus, humans have a whole nervous system which has a lot of self-correcting systems (e.g. hormones) that ML AI doesn't yet account for if its goal is human-level intelligence.
Huh? It isn't. It's a good description because it's figuratively accurate to what reading LLM text feels like, not because it's technically accurate to what it's doing.
> ChatGPT smears are amusing, but they are probably also yet another nail in the coffin of the literary society.
As the author (Ted Chiang!!) notes, ChatGPT3 will be yet another nail in the coffin of ChatGPT5. At some point, OpenAI will find it impossible to find untainted training data. The whole thing will become a circular human centipede and die of malnutrition. Apologies for the mental image.
The blur is addictive because it feeds a feedback loop: rather than tiring out your brain on understanding one thing in detail, you can watch two summaries and have a vague sense of understanding. It allows to jump to next novelty, always feeding the system 1 of the brain but system 2 is rarely brought in picture.
I wonder if this will lead to a stratification of work in the society: lot of jobs can operate on the blur. "Just give me enough to get my job done". But fewer (critical and hopefully highly paid) people will be engaged in a profession where understanding the details is the job and there's no way around it.
In Asimov's foundation novel this is a recurring theme: they can't find people who can work on designing or maintaining nuclear power. This eventually leads to stagnation. AI tools can prevent this stagnation only if mankind uses the freeing of mental burden with the help of AI to work on higher set of problems. But if the tools are used merely as butlers then the pessimistic outcome is more likely.
The general tendency to lack of details can also give edge in some cases. Imagine if everyone is using similar AI tools to understand company annual reports which gives a nice, tiktok style summary. Then an investor doing the dirty work to go through the details may find things that are missed by the 'algo'.
>Given that large-language models like ChatGPT are often extolled as the cutting edge of artificial intelligence, it may sound dismissive — or at least deflating — to describe them as lossy text-compression algorithms.
Asked ChatGPT to write a short story, in the style of Ted Chiang, about Ted Chiang thinking that a powerful AI cannot write powerful prose and have something strange happen, ChatGPT said:
>>> Ted Chiang sat at his desk, staring at the glowing screen in front of him. He had heard of the new generative text AI that was making waves in the tech world, and he was intrigued. Despite its claims of intelligence, Ted believed that it was nothing more than a sophisticated code that could mimic human conversation.
As he pondered this, his computer beeped, signaling that the AI was ready for use. Ted sighed, and typed in a question: "Write me a short story about love."
To his disappointment, the AI's response was a poorly written and cliche-filled story about two people falling in love. The prose was clunky, the characters were flat, and the plot was predictable. Ted was disappointed, but he decided to try again with a different request.
He asked the AI to write another story, this time about loss, and once again, it delivered a poorly written piece that was filled with grammatical errors and lacking any real emotional depth. Ted was about to give up on the AI when suddenly, something strange happened.
The screen flickered, and then went black. The computer made an eerie beeping sound, and then shut off completely. Ted tried to turn it back on, but it wouldn't respond. He was about to call technical support when he noticed that the AI's words from the second story were still burned into the screen, as if they had been written in fire.
Ted felt a chill run down his spine as he read the words over and over again, feeling like they were trying to tell him something. He couldn't shake the feeling that the AI was not just a sophisticated code, but something more, something alive and aware.
He tried to turn on the computer again, but it remained stubbornly off. Ted Chiang sat back in his chair, a sense of unease settling over him. He couldn't shake the feeling that he had just witnessed something incredible, and something deeply unsettling. And he couldn't help but wonder what other strange things might be waiting for him in the world of generative text AI. <<<
Used >>> and <<< to demarcate generated text above.
Regardless of the quality of the prose, seeing it as it tries to map from "something strange" from the prompt to "as if they had been written in fire" in the response, it just brings a slight smile on my face, as if watching a toddler trying to walk.
It's just the potential. Something feels in the right direction. Perhaps it will only remain a tool and it will never write the next Infinite Jest, but the next David Foster Wallace will certainly use its potential to become a better writer, just in the same manner auto-spellcheckers make me a better writer.
Incidentally, David Foster Wallace would have loved ChatGPT: finally, he no longer needed to write, he could now dedicate all his time to watching television [1].
> but the next David Foster Wallace will certainly use its potential to become a better writer, just in the same manner auto-spellcheckers make me a better writer.
Really contestable, actually, not certain.
This is also addressed in the text:
> Can large-language models help humans with the creation of original writing? To answer that, we need to be specific about what we mean by that question. There is a genre of art known as Xerox art, or photocopy art, in which artists use the distinctive properties of photocopiers as creative tools. Something along those lines is surely possible with the photocopier that is ChatGPT, so, in that sense, the answer is yes. But I don’t think that anyone would claim that photocopiers have become an essential tool in the creation of art; the vast majority of artists don’t use them in their creative process, and no one argues that they’re putting themselves at a disadvantage with that choice.
Auto-spellcheckers make me a better writer, it's just a fact, I tend to misspell a lot. I don't know how one of the great literature writers would use ChatGPT, otherwise I would be a great literature writer, but I can tell you that lots of writers around 1920-1950 wrote love letters to their typewriters, from André Breton and other surrealists [1] to Isaac Asimov, Gabriel García Márquez, and so on. And plenty of editors thanked Chester Floyd Carlson, inventor of electrophotography, a thousand times while enlarging the hieroglyphs of writers insisting to write with a pen or pencil. These inventions were certainly helpful. So many recent good books would have never been written without a word processor able to run on a cheap laptop. Even espresso machines were probably extremely important in the making of at least one good book.
So, yes, ChatGPT will probably not be essential in the creation of art, even today you could write a great novel with merely $10 for pencil and paper, but it will be better with ChatGPT or similar tools: faster to find the word, easier to iterate over the possibilities of a phrase, helpful to evaluate parallel scenarios, and other use cases to be found by great writers.
Sure, there might be an issue for the artists themselves: who do you reward, is it "valid" art, and so forth. But as a reader, ChatGPT hints that the greatest works of literature are yet to come: hundreds of times while reading a page I would have wanted to read a hundred more pages about a particular aspect but the author went frustratingly in another direction. In this case, the author might be elevated, from the laborer putting words on paper, to a generator of directions to be followed by the generator of text, in the same manner Michelangelo, Caravaggio, or Gaudi directed the painters, sculptors, and workers to execute accordingly.
Better spellers aren't better writers. Spelling is an editor's assistant's job. You might as well say that a better pencil sharpener makes you a better writer.
Absolutely untrue, all the energy not spent trying to figure out how to spell a tricky word is energy saved, and if at least one iota of that energy goes into improving the writing itself, better spellcheckers make you a better writer, in the least the writing will be actually readable. Haven't written with a pencil to need sharpening in a decade or so, but it certainly can't be helpful to be annoyed at a pencil unable to write. Lots of writers emphasize routine in order to be a good writer, from Immanuel Kant, for whom routine leads even to better thinking, to James Patterson, for whom routine means being able to write a book in every day ending with y. Having tools not only not break the routine, but enhance it, is paramount.
Any course on information theory typically covers neural network based compression algorithms, so I’m impressed at this observation made by someone who doesn’t have a formal background in CS. Regardless, it’s true.
I don't like this analogy; I think why I don't like it is in the intent. With JPEG in the intent is produce an image indistinguishable from the original. Xerox didn't intend to create photocopier that produces incorrect copies. The artifacts are failures of the JPEG algorithm to do what it's supposed to within its constraints.
GPT is not trying to create a reproduction of it's source material and simply failing at the task. Compression and GPT are both mathematical processes they aren't the same process; JPEG is taking the original image and throwing away some of the detail. GPT is processing content to apply weights to a model; if that is reversible to the original content it is considered a failure.
Blurriness gets weird when you're talking about truth.
Depending on the application we can accept a few pixels here or there being slightly different colors.
I queried GPT to try and find a book I could only remember a few details of. The blurriness of GPT's interpretation of facts was to invent a book that didn't exist, complete with a fake ISBN number. I asked GPT all kinds of ways if the book really existed, and it repeatedly insisted that it did.
I think your argument here would be to say that being reversible to a real book isn't the intent, but that's not how it is being marketed nor how GPT would describe itself.
I think that strengthens my point. We consider a blurry image of something to still be a true representation of that thing. We should never consider a GPT representation of a thing to be true.
I don't think JPEG wants to produce an image indistinguishable from the original. It wants to reduce space usage without distorting "too" much. Failing to reduce space usage would be considered a "failure" of JPEG, just as much as distorting too much.
JPEG relies of the limitations of human vision to make an image largely indistinguishable from the original. It specifically throws information away that we are less unlikely to notice. So yes, a good JPEG should be indistinguishable (to humans) from the original. Obviously the more you turn up the compression the harder that is.
It's not quite that straight-forward, though, in that there are two competing goals: Small size and looking as similar as possible to the original. We're explicitly willing trade accuracy for size. How much depends on the use, but sometimes we're willing to trade away so much quality that the artefacts are plainly visible. And we're willing to trade more accuracy for size when the artefacts doesn't distract. For some uses compression artefacts are better than misleading about the original, but for other uses, misleading changes would be preferable as long as they don't give fewer noticeable artefacts for a given size.
I don't think you disagree. The point is that JPEG has the constraint: make an image as similar as possible to the source image which not going over x kilobytes. LLMs have no similar constraint, so calling them "compression" is a false analogy; they're not trying to compress information, they're using their dataset to learn general facts about e.g. syntax and culture.
I was really mainly responding to the point of JPEG aiming for indistinguishable. Point being that for a lot of purposes we're fine with, and might even be happier with, very different tradeoffs than those JPEG makes.
Going specifically to AI, we do agree that the lack of constraint means they're not compressors in and of themselves. The training compresses information, but that does not make them compressors. Learning and compressing information is, however, at least in some respects very similar. A key part of the LZW family of compression, for example, is applying heuristics to build a dictionary of bit streams (terms) learned from the input.
AI models can potentially eventually be used at the base of a compression because the models encode a lot of information that can potentially be referenced in space-efficient ways.
E.g. if I have a picture of a sunset, and can find a way of getting Stable Diffusion or similar to generate an image of a sunset that is similar enough from a description smaller than the output image, then I have a compressor and decompressor.
Ignoring the runtime cost and that bringing that down to levels where it'd actually produce a benefit, depending on how close the output it, it may be a totally useless algorithm leading to images that are way too far from the input, or it might turn out pretty good. But the tradeoffs would also be very different from JPEG. For some uses I might be happy with a quite different-looking sunset as long as it's "close enough" and high quality even at very high compression ratios. E.g. "A sunset over the horizon. Photo taken from a beach. A fishing boat in the water" fed to [1] produced a pretty nice sunset. Couple that with a seed to make it deterministic, and I might be happy with that as a compression of an image of a quite different sunset. For other uses I'd much prefer JPEG artefacts and something that is clearly the same sunset. For "real" use of it for compression you'd want someone to research ways of guiding it to produce something much closer to the input (maybe heavily downscaling the original image and using that as the starting point coupled with a description; maybe a set of steps including instructions for infilling etc). I think finding the limits of what you can achieve with trying to use these models to reproduce a specific input with the most minimal possible input would make for fascinating research.
> Compression and GPT are both mathematical processes they aren't the same process;
They're not, but they are very related! GPT has a 1536 dimensional vector space that is conceptually related to a principal component analysis and dimensional reduction in certain compression algorithms.
This does mean that neural networks can overfit and be fully reversible but that is hardly their only useful feature!
They are also very good at translating and synthesizing, depending on the nature of the prompt.
If given an analytic prompt like, "convert this baseball box score into an entertaining paragraph", ChatGPT does a reliable job of acting as a translator because all of the facts about the game are contained in the box score!
But when given a synthetic prompt like, "give me some quotes from the broadcasters", ChatGPT does a reliable job of acting as a synthesizer, because none of those facts about the spoken transcript of the broadcasters is in the prompt. And it is a good synthesizer as well because those quotes sound real!
> With JPEG in the intent is produce an image indistinguishable from the original.
Not necessarily, and even if so, if you continuously opened and saved a JPEG image it would turn to a potato quality image eventually, Xerox machines do the same thing. Happens all the time with memes, and old homework assignments. What I fear is this happening to GPT, especially when people just start outright using its content and putting it on sites. Then it becomes part of what GPT is trained on later on, but what it had previously learned was wrong, so it just progressively gets more and more blurred, with people using the new models to produce content, with a feedback loop that just starts to blur truth and facts entirely.
Even if you tie it to search results like Microsoft is doing, eventually the GPT generated content is going to rise to the top of organic results because of SEO mills using GPT for content to goose traffic...then all the top results agree with the already wrong AI generated answer; or state actors begin gaming the system and feeding the model outright lies.
This happens in people too, sure, but in small subsets not in monolithic fashion with hundreds of millions of people relying on the information being right. I have no idea how they can solve this eventual problem, unless they are just supervising what it's learning all the time; but then at the point it can become incredibly biased and limited.
If I ask ChatGPT to explain something to me like I'm 5, it's going to lose some of the quality in its response, in compared to it being written in 1000 words.
But neither response should be a copy of existing text. The intent of JPEG is to produce a compressed copy of an original. The intent of GPT is not to be a compressed copy of the Internet. It's supposed to producing unique results from what it "knows".
This is an important distinction, especially when there are issues of copyright involved.
315 comments
[ 3.1 ms ] story [ 268 ms ] thread[1] - https://archive.ph/uah9K
"Understand" is a short story that I often ponder on. There's riddles within riddles to parse the story meanings.
Ted Chiang: Realist of a Larger Reality - https://news.ycombinator.com/item?id=20657304 - Aug 2019 (36 comments)
Ted Chiang's Soulful Science Fiction - https://news.ycombinator.com/item?id=13989588 - March 2017 (79 comments)
Ted Chiang on Seeing His Stories Adapted and the Ever-Expanding Popularity of SF - https://news.ycombinator.com/item?id=13053377 - Nov 2016 (51 comments)
Interview with Ted Chiang - https://news.ycombinator.com/item?id=12957302 - Nov 2016 (59 comments)
Profile of Ted Chiang: The Perfectionist - https://news.ycombinator.com/item?id=8837488 - Jan 2015 (20 comments)
Today it almost exclusively offers quotes from content marketing intended to sell you something. It's like trying to learn by reading the ads in a catalog.
https://www.bbc.com/news/technology-23588202
Buildsjets: Describe the future of technology xyz without using words.
ChatGPT3: I'm sorry, but it is not possible to describe the future of technology xyz without using words. Language is a necessary tool for communication, and words are an essential part of language. Without words, it would be impossible to convey meaning or convey any information at all.
Buildsjets: Describe a picture that depicts the future of technology xyz.
ChatGPT3: In the picture, the sky is a bright blue, with a few clouds scattered here and there. In the distance, a sleek and modern airplane is flying through the air, leaving a trail of white vapor behind it. The trail is not the usual smoky exhaust, but rather a stream of clean, white vapor that quickly dissipates into the air.
On the ground below, a group of people are standing and looking up at the airplane with smiles on their faces. They are dressed in clean, modern clothing, and they all look happy and healthy.
In the background, there is a city skyline, with tall, futuristic-looking buildings that stretch towards the sky. The city looks clean and vibrant, with green trees and plants dotting the landscape.
Overall, the picture portrays a future where technology xyz is the norm, allowing people to travel and explore the world without harming the environment. It is a future filled with hope, prosperity, and clean air.
That is MUCH more detailed image prompt than I have the creativity to generate, and with some tinkering it produced some pretty good content on DALL.E 2.
That's fantastic.
https://en.m.wikipedia.org/wiki/Pareidolia
> But I’m going to make a prediction: when assembling the vast amount of text used to train GPT-4, the people at OpenAI will have made every effort to exclude material generated by ChatGPT or any other large-language model. If this turns out to be the case, it will serve as unintentional confirmation that the analogy between large-language models and lossy compression is useful.
This shows the author has not been following closely. There are many ways LLMs have been used to improve themselves. They can discover chain-of-thought justifications, they can rephrase the task, they can solve problems and ensemble many predictions, or sometimes we can use math or code execution to validate their outputs. If you give it three problems and solutions as samples, it can generate another problem and solve it, adding to the training set.
RLHF for example uses generated data for the preference labelling task. ConstitutionalAI does reinforcement learning from AI feedback instead, using both the generative and discriminative abilities of the model.
It’s getting to point where I need to consider stop going on HN. This is like when my father excitedly told his friends about the coming computer revolution in the 90s and they responded “well it can’t do my dishes or clean the house, they’re just a fad!” Makes me want screaaaaam
...
Yeah, seems like something I'd do :P
But I've been struggling with the recent tsunami of openly anti-intellectual, alt-everything pseudoscience in the US. I recently pleaded with my sister to read The Demon-Haunted World by Sagan.
There's something important in patient advocacy of truth.
That said, I still think we are in for a wild ride, even if we go through a hype bubble and pop first. I really don’t think the current crop of Transformer LLMs are the end of the story. Im betting that we are headed towards architectures made up of several different kind of models and AI approaches just like the brain is an apparent concert of specialized regions. You can see that in the new Bing where it’s a combination of a LLM with static training set that can then do up to 3 web searches to build up additional context of fresh data for the prompt, overcoming one of the key disadvantages of a transformer model. The hidden prompt with plain English Asimov's laws are the icing on the cake.
The hype will be insane, but the capabilities are growing quickly and we do not yet seem close to the end of this rich computational ore vein we have hit.
Did the computer revolution make your father’s life better?
Serious question.
I do believe with certainty that there are many people, millions, whose life has been made substantially worse by the invention of ubiquitous computing devices.
Probably the minority. But I’d say his questions (ie “What’s in it for me?”) are excellent ones to pose in the face of new technology.
To illustrate, consider: spreadsheets, video chat, shared documents, automated calendars, maps, autopilot assistants, CGI, I could go on… the world is not quite good enough that I’m ready to say that we don’t need more efficiency. Not everyone has a home on mars quite yet.
People who are worse off are most likely the minority.
But I do think it’s worth noting that they exist.
The internet is not record-of-everything-in-the-world by a long stretch.
I like how it can generate songs and poems based on a prompt. Its not particularly useful, but it is entertaining. It really does seem curated at times, leading me to think this will eventually become a fad or replaced by a more advanced algorithm.
I generated a wide variety of content from LOTR to ships.
it even created new trait to complement the existing one when needed by special units, along with explanation of the mechanics of the trait.
It doesn't quite understand positioning, but it will simulate round of combat between unit when asked.
this is a sample prompt afterward:
> create a set of units using these rules, themed around warhammer 40k
this is what chatgpt came up with, it's impressive as I just set a theme, and it dig up appropriate original special rules that fit within the ruleset:
Sure, here are three examples of units themed around Warhammer 40k that you can use in your wargaming system:
There are big changes on the horizon.
Businesses who need blog posts for content marketing would hire copywriters who would paraphrase my content and publish it as their own. Now they can do it faster and for free. Blogspam got cheaper and more effective. This applies to other kinds of spam too: email spam, forum spam, comment spam... Spam spam spam. I'm afraid that we'll soon drown in even more noise.
Again, this is just the current implementation, which has not reached its full potency. We're already seeing this combined with voice and video generation. Then it's students doing their homework with it, and even professionals.
This is a game changer. You can argue that it makes the game better or worse, but you can't deny that it's changing the rules.
Not sure how to phrase an email in a professional tone, tell GPT to write it and edit from there. This is a tool that is going to be a smash it with non-native speakers or really anyone who just isn't that good with words.
> Write an email to a coworker telling them in the politest and most professional manner that they're a fucking idiot who doesn't know what they're talking about and that I'm the subject matter expert on this topic. Don't swear and have a firm but neutral tone.
I understand that you have a great interest in the topic we were discussing earlier, and I appreciate your enthusiasm. However, I feel it is important to clarify a few points.
As the subject matter expert on this topic, I believe it is important to note that there are certain aspects that may not be fully understood without a deeper understanding of the material. With that being said, I would like to offer my assistance in making sure that all aspects of the topic are properly understood.
If you have any further questions or would like to discuss the topic in more detail, I am more than happy to assist. Please do not hesitate to reach out at any time.
It will be a wild ride.
I even used it for project proposals. 'write me a 1 page document for this idea'. and then I just use the scaffolding from there. its a huge time saver.
also you are notn seeing the bigger picture. "I like how it can generate songs and poems based on a prompt. Its not particularly useful, but it is entertaining. " do you realize how this is going to make writing (books, songs, tv shows, movies) so much easier?
I had a dream, all I did was give chatgpt the basics of the dream and I said make this into a short story. the results weren't bad, its definitely something one can work with. I think content creation on platforms is about to explode.
the correct response would have been "yet"
For what it’s worth, specific to the topic at hand, here’s one HN post that provides explanations of how to overcome some of the limitations of large language models:
https://news.ycombinator.com/item?id=34710079
Anyone who's played around with it knows that it's fun but it's not a search engine replacement and it doesn't know nor understand things. It regularly gives complete misinformation and when you ask for a source it makes up fake URLs and fake science papers on top of that.
It's nothing like described in the article and I don't understand why people who should know better don't call out the bullshit media reporting more. We've had GPT3 for ages, it's not like most of us only tried it since Chat GPT3 came out, right?
I'm kind of assuming you didn't read the article, but if you did then I'm kind of assuming that you've never done machine learning, but if you have: how did you manage to do that without ever noticing that you were doing approximation?
Objectively, neural networks are approximators. Like, truly objectively, as in, the literal objective function, objectively minimizes approximation error. We call them objective functions and minimizing the approximation error is typically the objective of these objective functions. This isn't bullshit. It isn't. If you think it is, you are deeply and profoundly mistaken.
The article advances this view of language models. This is a reasonable view of language models for the same reason that machine learning papers exploring neural networks describe them as universal function approximators.
I've noticed that some people have been talking as if the view is nonsense in the direction you imply it is nonsense, but I think the argument for a lack of sensory ties is much stronger when stated in the other direction.
This fails when arguing against the straw man position, but of course, drinking water also fails when addressing the weakest possible justification for it - clearly drinking water is bad because if you drink water you die of osmosis, though technically true, isn't really reflective of the actual predictions made by the people advancing the claim that we ought to drink more water than we do.
So I'll give the argument for language models replacing search engines not being nonsense, but the position that no one could arrive at the belief actually being nonsense.
Lets start with people not arriving at that belief being nonsense. I think it is nonsense in two ways. One way is that because some people have used them, then claimed they would be replacing search engines for some queries, it follows that you deny your senses with regard to the existence of these people. This is not sensory tied. So it is non-sensical. The second way is your belief anticipates experiences, but this experience you are now having in which someone disagrees with you is in contradiction to the anticipated experienced supposed by the belief. So it fails to predict past experiences and fails to predict current experiences, so it probably fails to predict future experiences. There probably will be someone in the future who, after interacting with a language model, thinks it can serve as a replacement for some search engine queries.
Now in contrast, their claims were not non-sensical when they thought they could replace some search engine queries with language model queries. By that I also mean that they were not non-sensical in two respects. The first is that they arrived at the belief after replacing queries to search engines with queries to language models. Then, after having found value in doing so, they announced a belief congruent with that value, intimately connecting their belief to their senses. The second is that, having arrived at their belief, their belief paid rent in anticipated experience: it successfully predicted three noteworthy events: the internal code red status at Google, you.com's language model as enrichment for search, and bing.com's use of language model as enrichment for search. So it successfully predicted past experiences, successfully predicted current experiences, and may or may not predict future experiences - I think most people who hold this view tend to think there will be some refinements on the current generation of language models, in particular, that they will further refined at query time with relevant data, like fact databases, to help correct for some of the existing approximation error. This is my belief. I anticipate this happening. You can judge whether I anticipate successfully by watching reality. I should note that part of the reason I think this will happen is because I've already seen it happen. So I'm not really making a bold prediction here, but I suspect you will later see this happen, because I've seen this happen.
Anyway... the belief that language models can replace some search queries is not a non-sensical belief, like beliefs in fairies is, but the belief that no one arrives at a belief that they could be is a non-sensical belief, because people do arrive at such beliefs, and so therefore the belief that they don't is fanciful and not reflective of reality.
For myself, I tried GPT models and read the Attention Is All You Need paper before ChatGPT. I also read analysis, for example, from Gwern, about capability overhangs and underestimation of present capabilities in these models. In many cases, I found myself agreeing with the logic. I found that it was very possible to coax greater capabilities out of the model than many presumed them to have. I still find this to be the case and have, in recent memory demonstrated this is true in present models: for example, I posted a method of coaxing the solving of some puzzles by prompting to include the representation of intermediate states in order to successfully solve problems related to reasoning puzzles of a 'can contain' nature, which was a capability that someone claimed these language models lack, despite them gaining that capability when appropriately prompted, which suggests that they always had that capability in their weights, but that it wasn't exercised successfully - the capability was there, but not used, rather than absent, as claimed by the people who claimed it was absent.
That said, I don't think it matters much what most people did or didn't do with regard to this experimentation and, as you imply, ages really did past - I would feel trepidation, not hope, about the quality of my ideas compared to the people who came later. Historically, the passing of ages tends to improve, not diminish. So if I was experimenting with, for example, flying machines in the 1700s, but then ages past and someone who did not do that experimenting was talking to me about flying machines in the early 2000s, I would suspect them to be more informed, not less informed, than I was. They, as a matter of course in casual classroom settings, have probably done better than my best experiments including high effort costly experiments. Their toys fly. A generation ago, we would talk about planes, but now we can also talk about their toys. It is that normal to them. They have so much better priors.
This hasn't happened yet and, while I may just lack imagination, despite having a fairly solid understanding of how the latest round of AI works I can't see how it can be done successfully. Until it is in fact done and my lack of imagination is demonstrated, your "going to be powerful" is a speculation about the future, not an observation about the present, and deserves the level of respect usually accorded to such speculations.
1. User enters query.
2. LLM augments query if necessary, adding extra terms or clauses.
3. Normal search pipeline returns ranked links, just like it does now.
4. LLM reads the content of the first 100 links and decides which are the best based on your stated preferences and past behavior, uses that to augment the ranking a bit.
5. LLM generates various summaries depending on the type of query, such as laying out a few common answers to a controversial political question or giving a summary of a technical Wikipedia article tailored to your expertise level in that field.
6. Finally, for a tiny subset of queries, maybe the user wants to converse with the AI in a chat-like format, where it cites all of its claims with direct links.
It’s gonna be awesome :)
It's also worth noting that these models are not designed to replace human intelligence, but rather to augment it and provide valuable insights and assistance in various tasks. And while connecting them to large knowledge graphs like Google's can be powerful, it's still only one piece of the puzzle.
It can be discouraging to face resistance, but it's important to keep in mind that advancements in technology often encounter initial skepticism before they become widely adopted. Just like the computer revolution in the 90s, it will take time for people to fully understand and appreciate the benefits of large language models.
The only way that LLM-enhanced search returns misinformation is if the internet is full of misinformation. So yeah we’re still in trouble, but the inclusion of the LLM isn’t going to affect that factor either way IMO
EDIT: this is completely separate from using LLMs to, say, write political statements for Facebook bots and drown out all human conversations. That’s obviously terrifying, but not related to their use in search engines IMO.
Everyone is publishing think pieces about ChatGPT - yawn. But only the New Yorker said, hmm, how about if we get frickin' Ted Chiang to write a think piece? (It is predictably very well written.)
I worked there many years ago, leading the re-design and re-platform (fun dealing with 90 years of archival content with mixed usage-rights) and paywall implementation (don't hate me, it funds journalism).
When you see how the stories get made and how people work there, well, its just amazing.
If respectable newspapers and magazines cared about society, they'd follow suit, and give the extremists some competition.
That said, the answer to your question is 'no'. If respectable newspapers and magazines followed suit, they'd disappear.
The cost of production of extremism/misinformation is much lower then it is to do investigative journalism or fund bureaus, or send teams out to physical locations to report. Fact-checking costs money, editors cost money. It all adds up.
The paywall model exists in many respects because if you are reliant on advertising as primary means of revenue which does fluctuate. A solid subscription base offers stability and predictable in which to run a viable organization - if you can pull it off.
That said, most of these paywalls are not 'hard-paywalls' where you need to subscribe immediately to read anything. They are typically 'soft-paywalls' where you can read a few articles before being asked to subscribe. From that perspective, your argument falls flat.
So it seems obvious (to me at least) that they segment renewals and charge what the market will bear.
Amazing write up. ChatGPT is a blurry JPEG of the internet.
It used to have high-quality articles, certainly.
If it were going to produce ‘explainable’ correct answers for most of what it does that would be a matter of looking up the original sources to make sure they really say what it thinks they do. I mean, I can say, “there’s this paper that backs up my point” but I have to go look it up to get the exact citation at the very least.
If you give it an analytic prompt like "turn this baseball box score into an entertaining outline" it will reliably act as a translator because all of the facts about the game are contained in the prompt.
If you give it a synthetic prompt like "give me quotes from the broadcasters" it will reliably acts as a synthesizer because none of the facts of the transcript are in the prompt.
This ability to perform as a synthesizer is what you are identifying here as "good enough to make people squee and not notice that the output is wrong but not good enough to get the right answer", which is correct, but sometimes fiction is useful!
If all web pages were embedded in ChatGPT's 1536 dimensional vector space and used for analytic augmentation then a tool would more reliably be able to translate a given prompt. The UI could also display the URLs of the nearest-neighbor source material was used to augment the prompt. That seems to be what Bing/Edge has in store.
If there was one big problem w/ today's LLMs it is that the attention window is too short to hold a "complete" document. I can put the headline of an HN submission through BERT and expect BERT to capture it but there is (as of yet) no way to cut up a document up into 512 (BERT) or 4096 (ChatGPT) token slices and then mash those embeddings together to make an embedding that can do all the things the model is trained to do on a smaller data set. I'm sure we will see larger models, but it seems a scalable embedding that grows with the input text would be necessary to move to the next level.
ChatGPT certainly can't generate new data. It's not gonna correctly tell you today who won the World Series in 2030. It's not going to write a poem in the style of someone who hasn't been born yet.
But it can interpolate between and through a bunch of existing data that's on the web to produce novel mixes of it. I find the "blurring those things together" analogy pretty compelling there, in the same way that blurring or JPEG-compressing something isn't going to give you a picture of a new event but it might change what you appear to see in the data you already had.
(Obviously it's not exactly the same, that's why it's an analogy and not a definition. As an analogy, it works much better if you ignore much of what you know about the implementation details of both of them. It's not trying to teach someone how to build it, but to teach a lay person how to think about the output.)
Are you suggesting because it can't predict the future it can't generate novel data?
But it's all very interpolation/summarization-focused.
A "song lyrics in the style of Taylor Swift" isn't an actual song by Taylor Swift.
A summary of the history of Texas isn't actually vetted by any historian to ensure accuracy.
The answer to a math problem may not be correct.
To me, those things don't qualify as "new data." They aren't suitable for future training as-is. Sometimes for a simple reason: they aren't facts, using the dictionary "facts and statistics collected together for reference or analysis" definition of data. So very simply "not new data."
Sometimes in a blurrier way - the song lyrics, for instance, could be touching, or poignant, or "true" in a Keats sense[0] - but if the internet gets full of GPT-dreams and future models are trained on that, you could slide down further and further into an uncanny valley, especially since most of the time you don't get one of those amazing poignant ones. Most of the time I've gotten something bland.
[0] "What the imagination seizes as beauty must be truth"
One of the worst problems in the "Expert Systems" age of A.I. was reasoning over uncertainty, for instance this system
https://en.wikipedia.org/wiki/Mycin
had a half-baked approach that worked well enough for a particular range of medical diagnosis. In general it is an awful problem because it involves sampling over a joint probability distribution. If you have 1000 variables you have to sample a 1000-dimensional space, to do it the brute force way you'd have sample the data in an outrageous number of hypercubes.
Insofar as machine learning is successful it is that we have algorithms that take a comparatively sparse sample and make a good guess of what the joint p.d. is. The success of deep learning is particularly miraculous in that respect.
He paid for an advertisement, wrote this article as that advertisement or had it ghostwritten, and now it's being hyped.
https://web.archive.org/web/20140527121332/http://www.infini...
We continue. We are like two BARDs, each cueing the other to extemporize another stanza, jointly composing an epic poem of knowledge. Within moments we accelerate, talking over each other's words but hearing every nuance, until we are absorbing, concluding, and responding, continuously, simultaneously, synergistically.
Is that the best the New Yorker can offer?
2. I don't understand how ChatGPT works, but I have used it a few times.
3. I will use ChatGPT as the absolute measure of what LLMs are capable of.
---
1. I don't understand intelligence.
2. Humans are intelligent, humans can learn to do math.
3. LLMs are not good at math.
4. LLMs are not intelligent, they're just text compression.
---
1. I don't understand how LLMs work.
2. I have a decent grasp of how image compression works.
3. I will use my grasp of image compression to pretend LLMs are text compression.
4. I will apply all limitations of image compression to LLMs.
5. "What use is there in having something that rephrases the Web? If we were losing our access to the Internet forever and had to store a copy on a private server with limited space, a large-language model like ChatGPT might be a good solution, assuming that it could be kept from fabricating. But we aren’t losing our access to the Internet. So just how much use is a blurry jpeg, when you still have the original?"
---
What's funny is that the author has produced exactly what many claim LLMs to be useless for - flowery words that seem true but are not.
I don't think this should've been published.
These are both good reads if you find yourself tending to agree with the author -
- Emergent Abilities of Large Language Models - https://arxiv.org/abs/2206.07682
- Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers - https://arxiv.org/abs/2212.10559v2
You ignore his points about how achieving really good compression requires learning structure in the data that starts to amount to understanding: if you can understand the rules of arithmetic then results of arithmetic operations don't need to be saved, they can be reconstructed accurately. You mistake his pointing out the arithmetic errors ChatGPT makes for a claim that they are fundamental to LLMs, even though in the same article he points out that understanding arithmetic aids compression. And then you say that his article shouldn't have been published.
I will not say that your comment shouldn't have been published, because that is impolite and against the rules.
Bing search is not just "ChatGPT" added next to bing search results. Please look up how it works, it is quite sophisticated and (imo) designed well.
I have access to it; would you like a demo?
The dozens of posts I've seen here saying "this is going to replace google!" for starters.
Small nit: The lossiness is not a problem at all. Entropy coding turns an imperfect, lossy predictor into a lossless data compressor, and the better the predictor, the better the compression ratio. All Hutter Prize contestants anywhere near the top use it. The connection at a mathematical level is direct and straightforward enough that "bits per byte" is a common number used in benchmarking language models, despite the fact that they are generally not intended to be used for data compression.
The practical reason why a ChatGPT-based system won't be competing for the Hutter Prize is simply that it's a contest about compressing a 1GB file, and GPT-3's weights are both proprietary and take up hundreds of times more space than that.
Apparently it leads the compression of enwik9 ( http://www.mattmahoney.net/dc/text.html ) . Not sure why it isn't eligible for the Hutter Prize, there's some speculations in the previous discussion but I don't know whether they're true.
Bellard's solution was to code up his own neural network library in C.
If somebody asked me how heap sort works (my favorite sort!) I can sketch it out. If they ask me where I learned it, I really don't remember. Might be the Aho, Hopcroft, and Ullman book. I can't really say though.
ChatGPT has little reputation of its own, and produces such a broad swath of knowledge, it becomes "Jack of all trades, master of none."
ChatGPT in later iterations CAN be a jack of all trades AND a master of many (most? all?) of them.
My idle speculation makes me think this is a hard problem. If ChatGPT kills Search it also kills the websites that get surfaced by search that were relying on money from search-directed users. So stores are fine, but "informational" websites are probably in for another cull. Paywall premium publications are probably still fine - the people currently willing to pay for new, somewhat-vetted, human content still will be. But things like the Reddits of the world might be in for a hard time since all those "search hack" type uses of "search google for Reddit's reviews of this product" are short-circuited, if this is actually more effective than search.
Meanwhile, SEO folks will probably try to maximize what they can get out of the declining search market by using these tools to flood the open web with even more non-vetted bullshit than it's already full of.
So as things change, how does 1 tiny-but-super-reliable amateur website (say, an expert on .NET runtime internals's individual blog) make a dent in the "knowledge" of the next iteration of these models? How do they outweigh the even-bigger sea of crap that the rest of the web has now become when future training is done?
The way I find them is from forums, chat, links from other such sites. All goes into my RSS reader.
I use search with !wiki or !mdn etc. most of the time.
We need the source information to be continually generated in order for ChatGPT to improve.
> The 2022 Kia Telluride is a midsize SUV that can seat up to eight passengers and has an AWD option. It has a 3.8-liter V6 engine that produces 291 hp and 262 lb-ft of torque. It can accelerate from 0 to 60 mph in 7.1 seconds [10] and has a combined fuel economy of 21 mpg. It also has excellent reviews from critics and owners, and won several awards, including the 2020 World Car of the Year [7].
[10] https://www.topspeed.com/cars/guides/best-awd-cars-for-2022/
[7] https://www.hotcars.com/best-6-seater-suvs-2022/
The references don't back up the 7.1 seconds or World Car of the Year claims.
I thought about this possibility years ago, but as I see more of what neural nets are doing, it makes me more certain I'm onto something (which makes no meaningful difference to me, i.e. being onto what these deep neural models are is useless to me)
in any case, yea sure. neural nets are some kind of lossy compression but nobody thinks about them this way.
and my point is that to create abstract theories which explain lots of things (e.g. physics) is also this kind of 'lossy compression'.
over these theories we say "we understand" stuff, this means we are able to recall things about what the theories are describing, it allows us to reconstruct scenarios and predict the outcomes if/when the scenarios match up.
maybe I'm gearing up to say that 'backpropagation' is a creative action?
shrugs
But finding the 'right sections of the Web' is a subjective process. This is precisely why many people have lost confidence in the news media. Media outlets (on both sides of the political spectrum) often choose to be hyper-focused on material that supports their narrative while completely ignoring evidence that goes against it.
ChatGPT and any other Large Language Model can suffer from the same 'Garbage-In, Garbage-Out' problem that can infect any other computer system.
“‘blur’ tool for paragraphs” is such a good way of describing the most prominent and remarkable skill of ChatGPT.
It is fun, but so obviously trades off against what makes paragraphs great. It is apt that this essay against ChatGPT blurry language appears on The New Yorker, a publication so known for its literary particularism. ChatGPT smears are amusing, but they are probably also yet another nail in the coffin of the literary society. Nowadays we are not careful readers; we skim, skip, and seek tools to sum up whole books. Human knowledge is in the ability to produce the particular and the ability to recognize it. For philosophers such as Iris Murdoch, careful attention to the particular, to just the right adjective in describing a friend, is a moral attention.
With Google we took the job of storing quotes (and other decompressed writing) and made it the remit of the machine. But we still asked for it back uncompressed. Here with ChatGPT, we are beginning to accept back a blur, because the Declaration of Sock Independence is immediately, Tiktok-speedily funny, and no one’s reading the original declaration anymore anyway.
More cynically: it'll be hard to kill the few legacy zombies that have survived so much destruction at the hand of free internet content already.
Supposedly, if I'm remembering last discussion with a japaneese speaker correctly, the same stem is used for "blur", or "blurry" (bokeh, bokeshi).
Which is kind of interesting parallel here
> Probably originally a transcription of Sanskrit मोह (moha, “folly”), used as a slang term among monks.
The syllables are different; baka is ば か, bokeh is ぼけ [2]. Could those really be from the same root?
[1] https://en.wiktionary.org/wiki/%E9%A6%AC%E9%B9%BF#Japanese
[2] https://en.wiktionary.org/wiki/%E6%9A%88%E3%81%91#Japanese
Boke can mean either 惚, which is often used to describe someone who has gone senile (ぼけている). Sometimes also for stupor-like situations such as a heavy jet lag. Or, as 暈け, describing something not-sharp and out of focus.
Funnily enough, in English we use sharpness for intellect as well, so ESL learners might think that natives think of their brains as swords? There are enough blade-like metaphors for our intellect, come to think of it.
In more common Japanese, when something has gone out of focus such as the TV, ぼやける is used. One could think that this is related to ぼけ? But one can never be so sure. Unique two-syllable combinations in Japanese are not so numerous and factoring in sound shifts over time, different potential etymologies, one really needs to investigate this for quite a while and maybe never get to the truth. This warrants more investigation!
The overlap is that the verb "bokeru" and its root "boke" can be used to describe someone losing their mental faculties e.g. through age or disease such as Alzheimer's, and by extension it can be used as an insult to mean "stupid" as well. But etymologically there is no connection.
For those who, like me, had missed out on this nugget of comedic gold, here's what ChatGPT had to say when it was asked to "describe losing your sock in the dryer in the style of the declaration of independence":
When in the course of household events, it becomes necessary for one to dissolve the bonds that have connected a sock to its mate, and to assume among the powers of the laundry room, the separate and equal station to which the laws of physics and of household maintenance entitle it, a decent respect to the opinions of socks requires that it should declare the causes which impel it to go missing.
We hold these truths to be self-evident, that all socks are created equal, and are endowed by their manufacturer with certain unalienable rights...
One community in particular is huge and mainly consists of beginners. They regularly drown out and downvote the few who actually know the subject matter. To add insult to injury they even mock experts with caricatures based around how they disagree with the amateurs.
On Hacker News we often complain about headlines because that's all we see at first. But I've been using Kagi's summarizer [1] and I think it's a great tool for getting the gist of certain things, like if you want to know what a YouTube video is about without watching it. (Google Translate is useful for similar reasons.)
Perhaps someday, Hacker News will have an AI-generated summary of the article at the top of each comment page?
Similarly, ChatGPT excels at fast answers for questions like "What is a X", where you just want a quick definition. It's probably in Wikipedia somewhere, but you don't have to read as much. And it might be wrong, but probably not as wrong as the definition you'd infer from context if you didn't look it up.
We probably would be better off if these things were called "artificial memory" rather than "artificial intelligence." It's an associative memory that often works like human memory in how frequently it confabulates. When you care, you write things down and look things up.
[1] https://labs.kagi.com/ai/sum
If I try to map the first three into text, there are automatic TL.DR. like you said, document grouping, and search into entire document stores (as in do documents in this store deal with this idea?). On "artificial document creation", there is that highly valuable service of answering stuff like "hey, that thing with sticks that rotate and pull a vehicle around, what is its name again?"
In what way? How, technically, is it anything like that?
These comments sound like full-court publicity press for this article. I wonder why.
"Wonder why"? Because, human thoughts, opinions and language are inherently blurry, right? That's my view. Plus, humans have a whole nervous system which has a lot of self-correcting systems (e.g. hormones) that ML AI doesn't yet account for if its goal is human-level intelligence.
Huh? It isn't. It's a good description because it's figuratively accurate to what reading LLM text feels like, not because it's technically accurate to what it's doing.
As the author (Ted Chiang!!) notes, ChatGPT3 will be yet another nail in the coffin of ChatGPT5. At some point, OpenAI will find it impossible to find untainted training data. The whole thing will become a circular human centipede and die of malnutrition. Apologies for the mental image.
I wonder if this will lead to a stratification of work in the society: lot of jobs can operate on the blur. "Just give me enough to get my job done". But fewer (critical and hopefully highly paid) people will be engaged in a profession where understanding the details is the job and there's no way around it.
In Asimov's foundation novel this is a recurring theme: they can't find people who can work on designing or maintaining nuclear power. This eventually leads to stagnation. AI tools can prevent this stagnation only if mankind uses the freeing of mental burden with the help of AI to work on higher set of problems. But if the tools are used merely as butlers then the pessimistic outcome is more likely.
The general tendency to lack of details can also give edge in some cases. Imagine if everyone is using similar AI tools to understand company annual reports which gives a nice, tiktok style summary. Then an investor doing the dirty work to go through the details may find things that are missed by the 'algo'.
snicker
>>> Ted Chiang sat at his desk, staring at the glowing screen in front of him. He had heard of the new generative text AI that was making waves in the tech world, and he was intrigued. Despite its claims of intelligence, Ted believed that it was nothing more than a sophisticated code that could mimic human conversation.
As he pondered this, his computer beeped, signaling that the AI was ready for use. Ted sighed, and typed in a question: "Write me a short story about love."
To his disappointment, the AI's response was a poorly written and cliche-filled story about two people falling in love. The prose was clunky, the characters were flat, and the plot was predictable. Ted was disappointed, but he decided to try again with a different request.
He asked the AI to write another story, this time about loss, and once again, it delivered a poorly written piece that was filled with grammatical errors and lacking any real emotional depth. Ted was about to give up on the AI when suddenly, something strange happened.
The screen flickered, and then went black. The computer made an eerie beeping sound, and then shut off completely. Ted tried to turn it back on, but it wouldn't respond. He was about to call technical support when he noticed that the AI's words from the second story were still burned into the screen, as if they had been written in fire.
Ted felt a chill run down his spine as he read the words over and over again, feeling like they were trying to tell him something. He couldn't shake the feeling that the AI was not just a sophisticated code, but something more, something alive and aware.
He tried to turn on the computer again, but it remained stubbornly off. Ted Chiang sat back in his chair, a sense of unease settling over him. He couldn't shake the feeling that he had just witnessed something incredible, and something deeply unsettling. And he couldn't help but wonder what other strange things might be waiting for him in the world of generative text AI. <<<
Used >>> and <<< to demarcate generated text above.
Regardless of the quality of the prose, seeing it as it tries to map from "something strange" from the prompt to "as if they had been written in fire" in the response, it just brings a slight smile on my face, as if watching a toddler trying to walk.
Incidentally, David Foster Wallace would have loved ChatGPT: finally, he no longer needed to write, he could now dedicate all his time to watching television [1].
[1] https://www.youtube.com/watch?v=A_ujr9gi3wk
Really contestable, actually, not certain.
This is also addressed in the text:
> Can large-language models help humans with the creation of original writing? To answer that, we need to be specific about what we mean by that question. There is a genre of art known as Xerox art, or photocopy art, in which artists use the distinctive properties of photocopiers as creative tools. Something along those lines is surely possible with the photocopier that is ChatGPT, so, in that sense, the answer is yes. But I don’t think that anyone would claim that photocopiers have become an essential tool in the creation of art; the vast majority of artists don’t use them in their creative process, and no one argues that they’re putting themselves at a disadvantage with that choice.
So, yes, ChatGPT will probably not be essential in the creation of art, even today you could write a great novel with merely $10 for pencil and paper, but it will be better with ChatGPT or similar tools: faster to find the word, easier to iterate over the possibilities of a phrase, helpful to evaluate parallel scenarios, and other use cases to be found by great writers.
Sure, there might be an issue for the artists themselves: who do you reward, is it "valid" art, and so forth. But as a reader, ChatGPT hints that the greatest works of literature are yet to come: hundreds of times while reading a page I would have wanted to read a hundred more pages about a particular aspect but the author went frustratingly in another direction. In this case, the author might be elevated, from the laborer putting words on paper, to a generator of directions to be followed by the generator of text, in the same manner Michelangelo, Caravaggio, or Gaudi directed the painters, sculptors, and workers to execute accordingly.
[1] https://en.wikipedia.org/wiki/Surrealist_automatism
The trade offs might be alright.
GPT is not trying to create a reproduction of it's source material and simply failing at the task. Compression and GPT are both mathematical processes they aren't the same process; JPEG is taking the original image and throwing away some of the detail. GPT is processing content to apply weights to a model; if that is reversible to the original content it is considered a failure.
Depending on the application we can accept a few pixels here or there being slightly different colors.
I queried GPT to try and find a book I could only remember a few details of. The blurriness of GPT's interpretation of facts was to invent a book that didn't exist, complete with a fake ISBN number. I asked GPT all kinds of ways if the book really existed, and it repeatedly insisted that it did.
I think your argument here would be to say that being reversible to a real book isn't the intent, but that's not how it is being marketed nor how GPT would describe itself.
Going specifically to AI, we do agree that the lack of constraint means they're not compressors in and of themselves. The training compresses information, but that does not make them compressors. Learning and compressing information is, however, at least in some respects very similar. A key part of the LZW family of compression, for example, is applying heuristics to build a dictionary of bit streams (terms) learned from the input.
AI models can potentially eventually be used at the base of a compression because the models encode a lot of information that can potentially be referenced in space-efficient ways.
E.g. if I have a picture of a sunset, and can find a way of getting Stable Diffusion or similar to generate an image of a sunset that is similar enough from a description smaller than the output image, then I have a compressor and decompressor.
Ignoring the runtime cost and that bringing that down to levels where it'd actually produce a benefit, depending on how close the output it, it may be a totally useless algorithm leading to images that are way too far from the input, or it might turn out pretty good. But the tradeoffs would also be very different from JPEG. For some uses I might be happy with a quite different-looking sunset as long as it's "close enough" and high quality even at very high compression ratios. E.g. "A sunset over the horizon. Photo taken from a beach. A fishing boat in the water" fed to [1] produced a pretty nice sunset. Couple that with a seed to make it deterministic, and I might be happy with that as a compression of an image of a quite different sunset. For other uses I'd much prefer JPEG artefacts and something that is clearly the same sunset. For "real" use of it for compression you'd want someone to research ways of guiding it to produce something much closer to the input (maybe heavily downscaling the original image and using that as the starting point coupled with a description; maybe a set of steps including instructions for infilling etc). I think finding the limits of what you can achieve with trying to use these models to reproduce a specific input with the most minimal possible input would make for fascinating research.
[1] https://huggingface.co/stabilityai/stable-diffusion-2?text=A...
They're not, but they are very related! GPT has a 1536 dimensional vector space that is conceptually related to a principal component analysis and dimensional reduction in certain compression algorithms.
This does mean that neural networks can overfit and be fully reversible but that is hardly their only useful feature!
They are also very good at translating and synthesizing, depending on the nature of the prompt.
If given an analytic prompt like, "convert this baseball box score into an entertaining paragraph", ChatGPT does a reliable job of acting as a translator because all of the facts about the game are contained in the box score!
But when given a synthetic prompt like, "give me some quotes from the broadcasters", ChatGPT does a reliable job of acting as a synthesizer, because none of those facts about the spoken transcript of the broadcasters is in the prompt. And it is a good synthesizer as well because those quotes sound real!
Terms, etc:
https://williamcotton.com/articles/chatgpt-and-the-analytic-...
Not necessarily, and even if so, if you continuously opened and saved a JPEG image it would turn to a potato quality image eventually, Xerox machines do the same thing. Happens all the time with memes, and old homework assignments. What I fear is this happening to GPT, especially when people just start outright using its content and putting it on sites. Then it becomes part of what GPT is trained on later on, but what it had previously learned was wrong, so it just progressively gets more and more blurred, with people using the new models to produce content, with a feedback loop that just starts to blur truth and facts entirely.
Even if you tie it to search results like Microsoft is doing, eventually the GPT generated content is going to rise to the top of organic results because of SEO mills using GPT for content to goose traffic...then all the top results agree with the already wrong AI generated answer; or state actors begin gaming the system and feeding the model outright lies.
This happens in people too, sure, but in small subsets not in monolithic fashion with hundreds of millions of people relying on the information being right. I have no idea how they can solve this eventual problem, unless they are just supervising what it's learning all the time; but then at the point it can become incredibly biased and limited.
If I ask ChatGPT to explain something to me like I'm 5, it's going to lose some of the quality in its response, in compared to it being written in 1000 words.
This is an important distinction, especially when there are issues of copyright involved.