I recently spoke with a cousin about their current project decommissioning an oil and gas platform over the next four years (ie. real engineering (in reverse?)).
Their comment was that ChatGPT can't write final reports but it's extremely useful for seeding brainstorming sessions with real human engineers.
It can generate a list of health, safety, and environmental concerns related to the project that can be distributed and act as a "starter dot point list" that engineers can agree to, expand on, reject as being just silly, and add to as tengential thoughts arise.
I'd argue that makes it useful in the context of real engineering brainstorming .. and that it's being used that way on multi million dollar projects in a billion dollar domain.
There's a lot of this with current AI - someone will write a piece titled something like "LLMs are not suitable for brainstorming" or "GPT4 is useless for programmers" or "AI image generation has no use cases" - and what they say in the body is that they're not perfect for brainstorming, GPT4 can't write an entire app perfectly from one prompt, and Midjourney doesn't let you take a picture from your mind's eye and put it on paper.
Seriously, it's the same argument that people give for "ChatGPT can't give me good code, I don't know why", just rephrased. The deluge of "GPT is not useful for X" articles meant to bait the average critic, despite it being used en masse for "subproblems in X-space" already...
They're asking the wrong type of work from it. If you need some boilerplate or a transformation, it's going to give you a fantastic template to work with. If you need it, on the other hand, to engineer out a highly-specific and nuanced solution with an esoteric codebase to a complex problem, maybe not so much. The former is wide, the latter is narrow. It's going to take maybe a bit more breakdown of the scope into proper subproblems before you'll get a good answer; and that's something you can do yourself, or have an agent perform across multiple queries maybe (though I'll admit, more work needs to be done for the whole multi-agent workflows to be truly useful).
Came here to say this. The article assumes that you're already an expert on what you're asking the LLM to help you brainstorm for. A better title would be "LLMs are not suitable for brainstorming in topics you know deeply"
I'll also add that there's a big difference between "give me 10 startup ideas" and having a conversation with it to explore different startup ideas (for example). Through conversation you can build off what each other say and explore the space.
I tried asking an LLM for assistance with Chef Policyfiles today (more specific than that of course), and the response was as if it was for a non-existent product rather than Chef. It even included a code block that it claimed I could run on repl.it if I had an account, which resembled a resource block in chef (policies are not a kind of resource).
If a coworker confidently gave me code that is not only wrong, but relies on things that don't exist, I wouldn't call it creative. If they tried to claim that no I'm wrong, it is real, I might even say they hallucinated it.
I have often inadvertently found myself brainstorming just by explaining something challenging I am working on to a colleague with no expertise in my area. Just explaining forces new perspectives and new ideas.
Similarly, LLMs are captive audiences for throwing out ideas and playing with them. The interaction creates something more than just talking to myself or a whiteboard.
Not all brainstorming partners need to be first order contributors to have a very positive impact. Different kinds of interactions can spark entirely different kinds of ideas.
Me: “I’d like to start a business harvesting cat shit to fight global warming. People would pay me to take their cat shit and bury it because maybe it has carbon in it or something IDK I’m not a scientist. I would give them carbon credits in the form of a cat-shit-sequestration crypto token my business created. I’d pick up the cat shit by bicycle to save carbon emissions. Please help me develop a business plan.”
Chargpt: “that’s a great idea for a business! Here are some suggestions…”
Not a real exchange, but god, it does feel that way sometimes. One reason you can’t trust the damn thing is it’s too positive.
I just used your prompt on GPT 4o appended with "Be brutally honest, if the idea is bad, feel free to let me know without any sugarcoating" as sibling comments have suggested, and it works pretty well and doesn't give false platitudes.
Tried putting that verbatim into Phind, got back an essay on how to get funding to start a business. In the conclusion was this gem: "While the idea of harvesting cat feces for carbon sequestration is innovative, success will depend on thorough research, strategic planning, and effective execution."
I imagine the first models that trained on HN comments resulted in staff members resigning and becoming hermit alcoholics after some brainstorming and "Show HN" test runs. To make something that didn't result in instant and persistent despair likely meant a lot of sentiment analysis on input data, and the result is a system that is extremely pleased every time we show up.
No, its better. The reason is the LLM will say or add things you wouldnt have said or added on your own. Wether right or wrong, what it adds, it adds novel input to your thought stream and helps prevent you getting stuck. Ive found myself surprised from time to time with questions an LLM asked, or the justifications provided when pushing for a different path.
If you've never talked with a highly opinionated LLM, I understand your scepticism, but there is certainly value if you use the right models creatively.
Continually asking the LLM to create exhaustive lists of responses to new ideas (big or small) as they first appear, remaining apropos to the running context, is wonderful.
Only one unexpected connection out of 20 or 50 is fantastic, when you can iterate quickly. Brainstorming is the tradeoff of accepting lots of "failure" ideas, in order to generate more serendipitous and creative ideas.
As I use it, the LLM accelerates, augments, and documents my brainstorming session.
--
For some reason, I find it easy to push LLMs to be helpful in ways other people often say "LLMs can't do that".
It is just a matter of setting up the right context, being the right guide, and iterating. They are often faster, aware of more concepts, and more versatile (in terms of fulfilling roles), than any single human.
Their biggest weakness is having a short context. With some work, multiple sessions can mitigate that. Like getting help on a problem from a series of people, where you have to communicate the progression of context to each.
The second weakness is their default to conventional responses. But that is easily overcome by iterative/patient pushing for more creativity responses. Within a session, you can see the model transitioning to more creative as originality goes up, and it gets increasingly "emotionally excited".
They start having "fun", and get more adventurous, as epiphanies emerging from either of you go up. This is not just a funny artifact of how we behave, but also a great barometer for achieving LLM "creative mode".
I’ll add this here for the fake internet points but also as a general PSA. Most people are not aware that “Brainstorming” although popularized as a process since its origins in the 1960s… is actually Step 2. Most people are unaware that Step 1 is called “Questionstorming”. At the heart of the process is leveraging divergent and convergent modes of thinking which is done both to generate questions (and select the most promising ones) and answers… you’re welcome :D
"continue" is now the most important word in the english language right after "No" right now.
This causes the LLM to orbit around some point in the latent space, but doesn't cause it to explore. You have to tell it to pick a direction and there is no better direction than a question.
> The reason [LLMs are not a good tool to do truly effective brainstorming] is LLMs are trained to follow existing patterns in the human-produced corpus, and not natively taught to “brainstorm”.
The problem with this argument is that people do the same thing, we’re not that great at brainstorming either. When we brainstorm in groups, we’re just bringing multiple points of view together. The more data LLMs are trained on, the more viewpoints it might be able to bring that you haven’t considered.
That said, LLMs and all NNs so far are built to interpolate, and they are bad and have unbounded error when extrapolating outside their training examples. That is a good reason to not expect today’s AI to come up with new ideas.
Setting the temperature to a higher value would emulate this, wouldn't it? Then the model would be more willing to deviate from the most likely next token.
Increasing temperature just makes more uncommon tokens more likely to appear. This can include both uncommon ideas and uncommon spelling and grammatical mistakes. It also won't make ideas that the LLM isn't capable of thinking appear, unless you crank the temperature way up and get lucky (like monkeys on typewriters lucky)
Not just “uncommon tokens”. In the case when generation does weighted random sampling, it autoregressively takes its own past tokens into account. It generates whole different sequences. So if it’s diversity of output you want, you will get it.
Right, so in rough outline an LLM-based brainstorming framework might generate multiple responses to a question with multiple different combinations of temperature, top_p, top_k, etc., to get a mix of dull-but-baseline (you don’t want to miss those options) and more off-the-beaten path responses, then take each one and send it back to the LLM with a prompt to evaluate “does this provide a coherent answer – however unconventional or bad of an idea – to the question asked?” to filter out the cases where it was asked for summer vacation ideas and responded with something like “Banana banana banana banana” (this request probably with default tuning.)
> and have unbounded error when extrapolating outside their training examples
I would say, try reading a conspiracy theory forum - or simply wait for one to get posted on HN, and then revisit this particular conclusion.
I don't think this is a unique failure of LLMs, except insofar as LLMs have less of a physical grounding in reality then humans do: but we have people out there who have convinced themselves they can project forcefields, and take that all the way through to getting punched in the face by a martial artist when it turns out that, no, they can't.
Nothing is really original. That's why you can make finite lists of the possible plots of books and movies. It's why TVtropes is a thing. Everything is just a mashup of things before.
Ah, yes, Everything that can be invented has been invented. - apocryphally attributed to Charles H. Duell, Commissioner of US Patent Office way back in 1889.
This is a myth. Just because you don't really see blinding insight every day doesn't mean it doesn't exist (and it's more spectacular when you do see it!)
Another reason that this seems to happen is that innovation and invention tend to happen in more esoteric and technical realms today, and so they're only really fully understood by experts in that field.
I happen to actually be a a computational biologist whose work could be considered "esoteric and technical" and whose papers are only fully understood by others in my field. But the fact is every paper of mine (and everybody else's) has 50 to 100 references. Because research is pretty much just reusing existing knowledge in the papers cited. The main source of "innovation", if it can be called that, is taking an idea from another field and applying it to your own. The idea of a scientist or inventor who comes up with an idea completely out the blue with no context is common in fiction, but that isn't how it works in reality.
> Nothing is really original. That's why you can make finite lists of the possible plots of books and movies. It's why TVtropes is a thing. Everything is just a mashup of things before.
That's a lot like claiming no one says anything new because we're also using more-or-less the same words we were using last century.
Aren’t independent LLMs just more data? Some AI researchers are arguing that more data is the primary thing that got us this far, and the primary thing that will improve AI from here. Here’s an example (and a very good talk whether you believe my summary) https://www.youtube.com/live/a13aqr07tJ4?si=FZO5m_XzrfpDhyQP
As a part-time generative artist for several decades, a user of Monte-Carlo methods on a daily basis, and author of some papers on the topic, I personally believe that randomness is not a good answer to anything creative. Randomness is boring, and average. Randomness only helps you when you have a high quality Markov model constraining what your RNG chooses between, and that’s more or less all LLMs actually are. Adding more randomness to creative works in general makes creative works muddy and lowers quality, it needs to be guided. Randomness is a useful tool, but is widely misunderstood IMO and not very effective at brainstorming style exploration; brainstorming is about solving problems in interesting ways, i.e., there is reasoning behind it, not slightly more random word salad.
You're right, most people do that, but that's because they haven't trained themselves to be inventors. This is a skill that definitely needs to be developed more, but finding truly creative (and sometimes backwards-seeming) solutions can sometimes mean thinking about problems in a completely different way.
However, all LLM's do this. That's one of the points of the article.
There was an actual peer-reviewed article from an institution yesterday showing, numerically, robustly, human evaluated, that LLMs are better at creative thinking than humans.
I don't think handwaving about exclusively outputting training data and random punditry via article really holds up here.
It's wrong for so many reasons, from the training data having more perspectives, to the "Fat Tony" test, empirically, we can go make it output novel combinations right now, to the CS test, we can have it emulate arbitrary Turing machines.
The methodology of this paper is seriously suspect.
First they have a very small sample size of around n = 150. Second, they don't represent average husband, they use people working on mechanical Turk for $9 an hour.
Finally, they don't have any standard measures of divergent thought - they just use 4 different types of question answering problems, and those are supposed to measure creativity. Whether they do or don't would require us to evaluate the questions.
It's important to be clear that this is saying that LLMs might be more creative than low paid workers on mechanical Turk, given no training time. That's not a very good representation of the average human, or the fact that humans need training.
So for example, an average doctor or scientist might still be more creative than LLMs.
Tldr; this is not a good paper to throw around if you want to convince people. It's truly awful in its design, and the way it's presented is not always honest.
No results for mechanical turk, n=151 is perfectly significant for what amounts to a taste test (I was quite taken aback to read that, 151 is low in very few contexts), I'm not "throwing it around", whatever that means in this context, and the extreme reaction and 100% focus on it belies that it was 1 of 3 short points I made, as well as that the research backs up the claim that was shut down, that is intuitively correct, I'd rather have a 1000 specialists at undergrad level ideating than me and Einstein and Picasso and Steve Jobs.
So, yeah, between that, and that both your average person and someone arguing from Turing tapes can show AI doing things not in the training data, I'm going with actual data thats being "thrown around" rather than the rando blog post that argues from "by definition it can't be creative because it isn't original because it only repeats training data" Call me when they're at N=152 on a blind test.
There is a difference between a verbatim answer and an answer in the shape of a verbatim answer. The latter is not much better if you actually want to do new things.
LLM’s degree of adherence to training patterns can be directly tuned, per request (up to whatever the maximum it is capable of.)
Can this be leveraged to build a framework around an LLM for brainstorming usefully? I’m not sure. But “Nope, because LLMs just directly adhere to existing patterns” is definitely not a useful answer to that question. It is the kind of answer someone who just parrots common patterns with surface-level knowledge of the subject area would come up with. (Huh, did a default-settings LLM write this article?)
> LLM’s degree of adherence to training patterns can be directly tuned, per request (up to whatever the maximum it is capable of.)
I think people often forget the behavior of instruction following is still based on SFT training data. This is exactly "adherence to training patterns". Prompt engineering is not elixir, it's just a way to utilize the patterns seen in training data.
I’m not talking about prompt engineering when I talk about per request tuning of how closely it follows established patterns, I’m talking about inference parameters (temperature, top_p, top_k, are common for most models and ways of calling them, some others may be available, too.)
There is a thread below discussing tuning temperature and why it's not tied to creativity. I think the point is temperature and other sampling parameters don't have any inherent mechanism to control the adherence to training pattern as a whole (all top-X candidates are all from training data). Deviating from the max probability in next token generation has no relation with thinking out of box. It does give more generation results that may be lucky to hit something, but I doubt the cost effectiveness
How will increasing temperature improve the arithmetic performance of the model with numbers it has never seen? When prompted the right way, the model has trouble sticking to the answer that is implied by its chain of thought process. For some strange reason it knows how to perform the task, but when it is time to output the answer, it decides to recalculate it from scratch using attention. Before you complain about me benchmarking arithmetic performance. Consider that I could have defined any algebraic structure and I could have made the exact same complaint. It's just that arithmetic is very easy to verify using a calculator. Dismissing the easy arithmetic also dismisses all operations on any algebra.
The obvious problem is that these logic and arithmetic operations don't really need exhaustive training examples. Training the LLM is supposed to teach it a method or process by which it arrives at the answer, not just the question answer pair itself, which tends to only lead to very good approximate results. That is ok when using language, since your exact words don't really matter if you can express yourself in another way, but when it comes to algebras, accuracy is key.
It really is strange that when one wishes for a more capable LLM that one is told how stupid one is for wanting such a thing. What exactly is so wrong about giving the model a price list and an example of an order at a restaurant and then asking how much the whole order costs and expecting the correct total sum? For some strange reason I'm getting numbers that would make sense if I generated the text in a vacuum without the price list, as if they are following some existing pattern...
> How will increasing temperature improve the arithmetic performance of the model with numbers it has never seen?
“Brainstorming” and “doing precise arithmetic” are wildly different tasks. It is a complete non-sequitur to ask how a suggestion about the former will help the latter.
I came here to say pretty much the same. It is so tiring to hear opinions like these from technically-minded people who clearly lack "better-than-average" creativity and lateral thinking skills. LLMs are garbage-in garbage-out in terms of brainstorming. If you aren't good at brainstorming without LLMs then they won't be much use. So many people are just seeing their own limits reflected back at them.
When I'm brainstorming math or cryptography or philosophy with an LLM it is only because of the connections and leaps that I introduce that leads to novel and amusing discoveries.
You have to develop successful patterns. Coaxing and condensing. Drilling into numbered lists. Asking for tables with progressively more columns and altered sorting. You have to know the spatial reasoning limits of the model. Asking for enumeration and disabling of social niceties. The number of skills needed is probably beyond most people's imaginations.
This is utterly wrong. The strength of LLMs is that they have very wide knowledge, so while your specialist knowledge might surpass them in a particular area it will know more than you across other topics.
When brainstorming this wide knowledge is what you want. The trick is (as always) better prompting to push it hard - so things like "consider parallels in similar situations in other fields" are useful.
Counterpoint: LLMs are very suitable for brainstorming if prompted appropriately
LLMs are going to give you the consensus reality (average opinion? average facts?) but you can easily steer it into offbeat, controversial and esoteric areas of it's training with the right prompts
I spent a good long while trying to come up with novel video-game game play ideas, exactly what one would call "brainstorming" for ideas, and quite frankly it was pretty awful.
ChatGPT more or less it converged on simply taking the current subject, adding a generic gameplay element to it and outputting it. It took half the ideas being thrown at it and included a rhythm mechanic... That's not that interesting, and I just couldn't get it to really think outside the box.
And here's my comment on another site to a pro novelist who suddenly discovered Claude wrote like a genius who posits that Anthropic deliberatly cripples Claude's writing ability because they don't want to scare writers suddenly, they want to ease them into it
I disagree. Your supernatural scenes triggered words an analysis from higher quality writers and commenters in the training data. If you were writing about bass fishing it would likely not impress you with it's writing. You can try something like this as an experiment. In other words it's good at some writing and bad at others, depending on the training data. I'd love to be proven wrong, like a gripping story about bass fishing might be interesting.
> What’s worse is when we ask topics that don’t have consensus currently, the LLMs won’t behave in a more creative and independent way (as we hoped), but more susceptible to issues like hallucination.
I mean, what is the difference between creativity and hallucination (honest question)?
---
Maybe this means AI would be better at the opposite - after you brainstorm creative out there ideas, AI can tell you if they have been tried in the past and what the consensus view is on why they failed, allowing you to adjust course.
> I mean, what is the difference between creativity and hallucination (honest question)?
Well, creative problem solving involves getting new ideas and perspectives by creating novel associations between things we know. Hallucinations are creating things we "know" because they sound like they could be right and basing "ideas" on them. In short, hallucinations are bullshit. Totally open creative problem solving isn't always perfect, but even the ideas that don't really work can reveal something about the concepts involved. And bullshit often takes creativity to create, but that doesn't make it useful as actual creative output. It's not like the second you move beyond empirically provable statements, everything has equal merit. If you're trying to creatively work to a useful end, whether or not the knowledge you're working with is fabricated is pretty consequential. Doing otherwise would be like trying to optimize your code based on utterly fake but plausible algorithms-- sounding right isn't right enough to be useful. IMO, being able to create lies that pass the smell test is LLMs' most dangerous proclivity, especially when they're presented as expertise-in-a-box.
Its not like humans always have good creative output. A lot of human creative output is essentially unworkable bullshit that gets discarded quite quickly.
I'm not saying llms are good at creativity (i certainly don't think they are) but i kind of feel like its a difference in quality not kind.
Like if you asked me to describe what it means for a human to be creative, i would probably write something quite similar to what you wrote above.
The main difference for my use case is what's usable about off-the-mark creative output to those who engage with it. With an LLM's hallucination, bullshit was made, bullshit was received, and that's that. If you figure out it's bullshit, the only thing you've likely gained from the exercise is having one less piece of bullshit to consider.
With humans creatively solving problems, what they say isn't 'output'-- it's externalizing a reasoning process. Being off-the-mark about something is a beneficial part of defining the bounds of an unknown solution because it's based on reality. It's a tangible idea that can be reasoned about and modified-- not a collection of output. It's not a point on a scatter plot, it's a waypoint towards a useful end.
I don't know a case study but I was personally told by 2 different founders who had 100MM+ exits that their startup ideas were formed (at least partially) from brainstorm sessions. One guy even pointed to me the library where they had the whiteboard session which led to the idea that they exited in the end.
I'm interpreting "brainstorming" to be any kind of general noodling on an issue to try to find new/novel/interesting solutions. In that case, I think every significant moment of true inspiration I had (of which there have been like, maybe 3 ever), they were always the result of seemingly random, completely unrelated things popping into my mind that, for whatever reason, clicked perfectly into the problem I was mulling over. To me, this means that manufacturing "true" inspiration doesn't require a tool that can deviate from "standard" human thinking patterns. I think it just means that you would want a tool that helps expose you to as many new and unknown fields/concepts/ideas in as little time as possible. So I think in that way LLMs are an amazing tool for helping one to brainstorm.
"LLMs are no good for [use case X]" often means "I aren't very good at using LLMs for [use case X]".
With many powerful tools - violins, say, or carpentry tools - we know that it takes a long time and a lot of learning to achieve competent performance, much less virtuoso performance. Someone who spent ten hours learning the violin and concluded "Violins sound terrible" wouldn't have diagnosed a problem with violins, but with their own mastery. I certainly think current LLMs have some big intrinsic weaknesses, but also that what they are is quite subtle.
Strongly disagree - brainstorming isn’t about asking someone to give you creative answers, it’s a team sport about triggering unique thoughts that neither person would have had on their own.
Although I might be biased, I love using AI for brainstorming and started building a tool for it https://youtu.be/t5gfETbUzy0
LLM might be mostly bad at it, but that doesn't make them unsuitable. They don't have a career to worry about. They don't care what peers or the boss thinks. Etc.
They can spit out ideas - LOTS of them - that real humans then use as a starting point to carry on with.
For more details see "Co-intelligence" by Ethan Mollick.
Author here. Was not expecting this quick post being picked up by HN - thanks for all the comments!
I want to acknowledge that the original title is inaccurate, as many of you have pointed out. It should be "(current) LLMs don't brainstorm novel things really well" rather than just brainstorming.
It's not intended to be a clickbait though - I was kind of mixing two definitions of brainstorming unintentionally. When we refer to the group activity that aims to collect all angles from participants (and common wisdoms), LLMs are really good, and it's something I do on a regular basis. However when it comes to the hope of reaching novel ideas that don't exist before (which some of us will consider what distinguishes brainstorming from group discussion or research study), I would say today's LLMs don't do well. I've seen such issue in business and arts domains, and also someone here mentioned similar experience in video game design.
I would argue that (so far) for any idea LLMs tell us, there exists at least one instance of a similar pattern in the training data (either exact or in a high level). If this is what you need, then great. But some problems require more than that. And I would argue that a lot of important innovations in history didn't follow this pattern. I'm aware of reports on LLMs helping research (e.g. the works shared by Terry Tao), but I don't think they contradict the point here. Will be super happy to be proven wrong though!
> Was not expecting this quick post being picked up by HN - thanks for all the comments!
You submitted the post?!
It got “picked up” so quickly because it is wrong. I think what your post does show is how effectively you can farm HN for supporting arguments.
You provide zero evidence for your claims, nor do you give us any transcripts of your attempts. Every post that matches the structure of yours is usually a summary of how the author has low skill in using LLMs.
Though I am absolutely delighted in the comments and the nih paper referenced was a joy to read.
I think effective brainstorming is actually a matter of prompt engineering, which is to say, breaking the model out of its box and into unexplored territory. See https://twitter.com/repligate for some extreme examples of this.
98 comments
[ 3.0 ms ] story [ 174 ms ] threadWas thinking if you can apply TRIZ method to invent Transformer before 2017 - hard to imagine it working on me
Great title, they baited and switched.
95% of the time I don't need effective brainstorming, I need a bunch of ideas, let me pick the best, and move on.
If its a real engineering problem, then I need
>truly effective brainstorming.
Their comment was that ChatGPT can't write final reports but it's extremely useful for seeding brainstorming sessions with real human engineers.
It can generate a list of health, safety, and environmental concerns related to the project that can be distributed and act as a "starter dot point list" that engineers can agree to, expand on, reject as being just silly, and add to as tengential thoughts arise.
I'd argue that makes it useful in the context of real engineering brainstorming .. and that it's being used that way on multi million dollar projects in a billion dollar domain.
They're asking the wrong type of work from it. If you need some boilerplate or a transformation, it's going to give you a fantastic template to work with. If you need it, on the other hand, to engineer out a highly-specific and nuanced solution with an esoteric codebase to a complex problem, maybe not so much. The former is wide, the latter is narrow. It's going to take maybe a bit more breakdown of the scope into proper subproblems before you'll get a good answer; and that's something you can do yourself, or have an agent perform across multiple queries maybe (though I'll admit, more work needs to be done for the whole multi-agent workflows to be truly useful).
I'll also add that there's a big difference between "give me 10 startup ideas" and having a conversation with it to explore different startup ideas (for example). Through conversation you can build off what each other say and explore the space.
What is creativity if not what people call hallucination? Horribly named phenomenon, btw.
If a coworker confidently gave me code that is not only wrong, but relies on things that don't exist, I wouldn't call it creative. If they tried to claim that no I'm wrong, it is real, I might even say they hallucinated it.
I have often inadvertently found myself brainstorming just by explaining something challenging I am working on to a colleague with no expertise in my area. Just explaining forces new perspectives and new ideas.
Similarly, LLMs are captive audiences for throwing out ideas and playing with them. The interaction creates something more than just talking to myself or a whiteboard.
Not all brainstorming partners need to be first order contributors to have a very positive impact. Different kinds of interactions can spark entirely different kinds of ideas.
Or even like "huh? explain"
Looking forward to ramping up the honesty parameter. Hoping OpenAI's new voice model trivializes this so I don't need to prompt engineer.
Chargpt: “that’s a great idea for a business! Here are some suggestions…”
Not a real exchange, but god, it does feel that way sometimes. One reason you can’t trust the damn thing is it’s too positive.
… it is still a little weird that the default is, like, psychotic levels of positivity, though.
If you've never talked with a highly opinionated LLM, I understand your scepticism, but there is certainly value if you use the right models creatively.
Continually asking the LLM to create exhaustive lists of responses to new ideas (big or small) as they first appear, remaining apropos to the running context, is wonderful.
Only one unexpected connection out of 20 or 50 is fantastic, when you can iterate quickly. Brainstorming is the tradeoff of accepting lots of "failure" ideas, in order to generate more serendipitous and creative ideas.
As I use it, the LLM accelerates, augments, and documents my brainstorming session.
--
For some reason, I find it easy to push LLMs to be helpful in ways other people often say "LLMs can't do that".
It is just a matter of setting up the right context, being the right guide, and iterating. They are often faster, aware of more concepts, and more versatile (in terms of fulfilling roles), than any single human.
Their biggest weakness is having a short context. With some work, multiple sessions can mitigate that. Like getting help on a problem from a series of people, where you have to communicate the progression of context to each.
The second weakness is their default to conventional responses. But that is easily overcome by iterative/patient pushing for more creativity responses. Within a session, you can see the model transitioning to more creative as originality goes up, and it gets increasingly "emotionally excited".
They start having "fun", and get more adventurous, as epiphanies emerging from either of you go up. This is not just a funny artifact of how we behave, but also a great barometer for achieving LLM "creative mode".
This causes the LLM to orbit around some point in the latent space, but doesn't cause it to explore. You have to tell it to pick a direction and there is no better direction than a question.
The problem with this argument is that people do the same thing, we’re not that great at brainstorming either. When we brainstorm in groups, we’re just bringing multiple points of view together. The more data LLMs are trained on, the more viewpoints it might be able to bring that you haven’t considered.
That said, LLMs and all NNs so far are built to interpolate, and they are bad and have unbounded error when extrapolating outside their training examples. That is a good reason to not expect today’s AI to come up with new ideas.
I would say, try reading a conspiracy theory forum - or simply wait for one to get posted on HN, and then revisit this particular conclusion.
I don't think this is a unique failure of LLMs, except insofar as LLMs have less of a physical grounding in reality then humans do: but we have people out there who have convinced themselves they can project forcefields, and take that all the way through to getting punched in the face by a martial artist when it turns out that, no, they can't.
how do you know?
This is a myth. Just because you don't really see blinding insight every day doesn't mean it doesn't exist (and it's more spectacular when you do see it!)
Another reason that this seems to happen is that innovation and invention tend to happen in more esoteric and technical realms today, and so they're only really fully understood by experts in that field.
"What has been will be again, what has been done will be done again; there is nothing new under the sun."
- Ecclesiastes 1:9 (circa 250 BC)
That's a lot like claiming no one says anything new because we're also using more-or-less the same words we were using last century.
I think you're missing a lot.
As a part-time generative artist for several decades, a user of Monte-Carlo methods on a daily basis, and author of some papers on the topic, I personally believe that randomness is not a good answer to anything creative. Randomness is boring, and average. Randomness only helps you when you have a high quality Markov model constraining what your RNG chooses between, and that’s more or less all LLMs actually are. Adding more randomness to creative works in general makes creative works muddy and lowers quality, it needs to be guided. Randomness is a useful tool, but is widely misunderstood IMO and not very effective at brainstorming style exploration; brainstorming is about solving problems in interesting ways, i.e., there is reasoning behind it, not slightly more random word salad.
You're right, most people do that, but that's because they haven't trained themselves to be inventors. This is a skill that definitely needs to be developed more, but finding truly creative (and sometimes backwards-seeming) solutions can sometimes mean thinking about problems in a completely different way.
However, all LLM's do this. That's one of the points of the article.
I don't think handwaving about exclusively outputting training data and random punditry via article really holds up here.
It's wrong for so many reasons, from the training data having more perspectives, to the "Fat Tony" test, empirically, we can go make it output novel combinations right now, to the CS test, we can have it emulate arbitrary Turing machines.
First they have a very small sample size of around n = 150. Second, they don't represent average husband, they use people working on mechanical Turk for $9 an hour.
Finally, they don't have any standard measures of divergent thought - they just use 4 different types of question answering problems, and those are supposed to measure creativity. Whether they do or don't would require us to evaluate the questions.
It's important to be clear that this is saying that LLMs might be more creative than low paid workers on mechanical Turk, given no training time. That's not a very good representation of the average human, or the fact that humans need training.
So for example, an average doctor or scientist might still be more creative than LLMs.
Tldr; this is not a good paper to throw around if you want to convince people. It's truly awful in its design, and the way it's presented is not always honest.
So, yeah, between that, and that both your average person and someone arguing from Turing tapes can show AI doing things not in the training data, I'm going with actual data thats being "thrown around" rather than the rando blog post that argues from "by definition it can't be creative because it isn't original because it only repeats training data" Call me when they're at N=152 on a blind test.
LLM’s degree of adherence to training patterns can be directly tuned, per request (up to whatever the maximum it is capable of.)
Can this be leveraged to build a framework around an LLM for brainstorming usefully? I’m not sure. But “Nope, because LLMs just directly adhere to existing patterns” is definitely not a useful answer to that question. It is the kind of answer someone who just parrots common patterns with surface-level knowledge of the subject area would come up with. (Huh, did a default-settings LLM write this article?)
I think people often forget the behavior of instruction following is still based on SFT training data. This is exactly "adherence to training patterns". Prompt engineering is not elixir, it's just a way to utilize the patterns seen in training data.
I’m not talking about prompt engineering when I talk about per request tuning of how closely it follows established patterns, I’m talking about inference parameters (temperature, top_p, top_k, are common for most models and ways of calling them, some others may be available, too.)
The obvious problem is that these logic and arithmetic operations don't really need exhaustive training examples. Training the LLM is supposed to teach it a method or process by which it arrives at the answer, not just the question answer pair itself, which tends to only lead to very good approximate results. That is ok when using language, since your exact words don't really matter if you can express yourself in another way, but when it comes to algebras, accuracy is key.
It really is strange that when one wishes for a more capable LLM that one is told how stupid one is for wanting such a thing. What exactly is so wrong about giving the model a price list and an example of an order at a restaurant and then asking how much the whole order costs and expecting the correct total sum? For some strange reason I'm getting numbers that would make sense if I generated the text in a vacuum without the price list, as if they are following some existing pattern...
“Brainstorming” and “doing precise arithmetic” are wildly different tasks. It is a complete non-sequitur to ask how a suggestion about the former will help the latter.
When I'm brainstorming math or cryptography or philosophy with an LLM it is only because of the connections and leaps that I introduce that leads to novel and amusing discoveries.
You have to develop successful patterns. Coaxing and condensing. Drilling into numbered lists. Asking for tables with progressively more columns and altered sorting. You have to know the spatial reasoning limits of the model. Asking for enumeration and disabling of social niceties. The number of skills needed is probably beyond most people's imaginations.
When brainstorming this wide knowledge is what you want. The trick is (as always) better prompting to push it hard - so things like "consider parallels in similar situations in other fields" are useful.
LLMs are going to give you the consensus reality (average opinion? average facts?) but you can easily steer it into offbeat, controversial and esoteric areas of it's training with the right prompts
ChatGPT more or less it converged on simply taking the current subject, adding a generic gameplay element to it and outputting it. It took half the ideas being thrown at it and included a rhythm mechanic... That's not that interesting, and I just couldn't get it to really think outside the box.
I put a clip up supporting why I think LLMs are good for brainstorming from Marc Andreeson about this topic in case your interested
"Marc Andreessen says with the right prompting, you can unlock the latent super genius in AI models"
https://www.youtube.com/watch?v=N2yN4IG8UYA
And here's my comment on another site to a pro novelist who suddenly discovered Claude wrote like a genius who posits that Anthropic deliberatly cripples Claude's writing ability because they don't want to scare writers suddenly, they want to ease them into it
I disagree. Your supernatural scenes triggered words an analysis from higher quality writers and commenters in the training data. If you were writing about bass fishing it would likely not impress you with it's writing. You can try something like this as an experiment. In other words it's good at some writing and bad at others, depending on the training data. I'd love to be proven wrong, like a gripping story about bass fishing might be interesting.
I mean, what is the difference between creativity and hallucination (honest question)?
---
Maybe this means AI would be better at the opposite - after you brainstorm creative out there ideas, AI can tell you if they have been tried in the past and what the consensus view is on why they failed, allowing you to adjust course.
Well, creative problem solving involves getting new ideas and perspectives by creating novel associations between things we know. Hallucinations are creating things we "know" because they sound like they could be right and basing "ideas" on them. In short, hallucinations are bullshit. Totally open creative problem solving isn't always perfect, but even the ideas that don't really work can reveal something about the concepts involved. And bullshit often takes creativity to create, but that doesn't make it useful as actual creative output. It's not like the second you move beyond empirically provable statements, everything has equal merit. If you're trying to creatively work to a useful end, whether or not the knowledge you're working with is fabricated is pretty consequential. Doing otherwise would be like trying to optimize your code based on utterly fake but plausible algorithms-- sounding right isn't right enough to be useful. IMO, being able to create lies that pass the smell test is LLMs' most dangerous proclivity, especially when they're presented as expertise-in-a-box.
I'm not saying llms are good at creativity (i certainly don't think they are) but i kind of feel like its a difference in quality not kind.
Like if you asked me to describe what it means for a human to be creative, i would probably write something quite similar to what you wrote above.
With humans creatively solving problems, what they say isn't 'output'-- it's externalizing a reasoning process. Being off-the-mark about something is a beneficial part of defining the bounds of an unknown solution because it's based on reality. It's a tangible idea that can be reasoned about and modified-- not a collection of output. It's not a point on a scatter plot, it's a waypoint towards a useful end.
How often is that a successful approach at all? I can’t think of a good case study off the top of my head.
The SPC also have a blog that proposes brainstorming from -1 to 1: https://blog.southparkcommons.com/how-to-go-from-minus-1-to-...
"LLMs are no good for [use case X]" often means "I aren't very good at using LLMs for [use case X]".
With many powerful tools - violins, say, or carpentry tools - we know that it takes a long time and a lot of learning to achieve competent performance, much less virtuoso performance. Someone who spent ten hours learning the violin and concluded "Violins sound terrible" wouldn't have diagnosed a problem with violins, but with their own mastery. I certainly think current LLMs have some big intrinsic weaknesses, but also that what they are is quite subtle.
I found that og gpt4 is better for brainstorming than gpt4 turbo, which in tern is better than gtp4o.
If you're just using the web based portal you don't get much of a choice which model to use or it's temperature.
Although I might be biased, I love using AI for brainstorming and started building a tool for it https://youtu.be/t5gfETbUzy0
LLM might be mostly bad at it, but that doesn't make them unsuitable. They don't have a career to worry about. They don't care what peers or the boss thinks. Etc.
They can spit out ideas - LOTS of them - that real humans then use as a starting point to carry on with.
For more details see "Co-intelligence" by Ethan Mollick.
I want to acknowledge that the original title is inaccurate, as many of you have pointed out. It should be "(current) LLMs don't brainstorm novel things really well" rather than just brainstorming.
It's not intended to be a clickbait though - I was kind of mixing two definitions of brainstorming unintentionally. When we refer to the group activity that aims to collect all angles from participants (and common wisdoms), LLMs are really good, and it's something I do on a regular basis. However when it comes to the hope of reaching novel ideas that don't exist before (which some of us will consider what distinguishes brainstorming from group discussion or research study), I would say today's LLMs don't do well. I've seen such issue in business and arts domains, and also someone here mentioned similar experience in video game design.
I would argue that (so far) for any idea LLMs tell us, there exists at least one instance of a similar pattern in the training data (either exact or in a high level). If this is what you need, then great. But some problems require more than that. And I would argue that a lot of important innovations in history didn't follow this pattern. I'm aware of reports on LLMs helping research (e.g. the works shared by Terry Tao), but I don't think they contradict the point here. Will be super happy to be proven wrong though!
You submitted the post?!
It got “picked up” so quickly because it is wrong. I think what your post does show is how effectively you can farm HN for supporting arguments.
You provide zero evidence for your claims, nor do you give us any transcripts of your attempts. Every post that matches the structure of yours is usually a summary of how the author has low skill in using LLMs.
Though I am absolutely delighted in the comments and the nih paper referenced was a joy to read.