Off topic but of all the Mooney images ever made, why a scary clown?
That aside, working with complex systems and constraints there often isn't an aha moment, there's just a decision to be made. As someone who loves that aha moment, I can get stuck trying to figure out perfect from good enough. Interesting to see there is indeed a positive emotion correlated with that aha moment that keeps people searching for solutions.
I wonder if there's a correlation between addiction and this aha moment. Like you get drunk and suddenly "aha!" those big unresolvable problems don't matter. The next morning they matter again until, aha, beer:30 hits.
I totally believe there's a correlation between addiction and the aha moment. You don't even need to "aha!" with facts; pairing emotion to a simple fact will suffice.
Try pairing a feeling to:
"So THAT'S how a mouse cursor moves"
"So THAT'S why revolving doors move clockwise!"
"So THAT'S why lights at night feel cosy!"
With a little practice, you can arbitrarily get 'aha' moments. I assume the good feeling is some sort of dopamine release where my brain is rewarding me for "figuring something out," even though I've kind if hijacked the mechanism.
tldr: There is a background, non-verbal process in your brain that has the advantage of a larger working set size than your foreground verbal thinking. It is able to observe and consider more stuff at once and find associations better than your conscious thought process.
But, it has several disadvantages. It takes time to do its processing. You can't will it into action. It communicates non-verbally with your foreground process. It doesn't work under pressure (thus the need for relaxed, unfocused time). The non-verbal understanding is difficult to deconstruct, generalize and reapply. It can lead to you solving a problem, not understanding how and not being able to solve a variant of the same problem.
So, the general recommendation is: If you have a complex problem to solve, first absorb as much information about the problem as your brain can hold. But, do not try to solve anything. Then, go take a break. A walk in a natural environment is preferable. Don’t think about the problem. Relax in a low stress environment. Let your background brain have a chance to chew on it and maybe bubble up some suggestions.
Thanks for the link and tl;dr, even though it's a quite short video (3:16). I found your explanation very interesting because I have intuitively felt like this is accurate, but never knew what the underlying process is. I have been following this for years already, first absorbing information about anything where I need to make a decision, and then just leave it to stew in the back of my mind. And after a while the answer just appears in my mind, without me really understanding where it comes from.
There's a Paul Graham essay I happen to have read just the other day about this topic. In it, he refers to this concept as "ambient thought", though I've found the title, "The Top Idea in Your Mind" to be a more sticky "motto" to remind me of the concept.
Richard Feynman suggested keeping a dozen favorite problems "constantly present in your mind” encouraging subconscious processing of challenges.
By keeping problems in a dormant state, the brain can work on them in the background, leading to unexpected solutions when a new piece of knowledge is encountered
[Spoiler]
Here are three words: pine, crab, sauce. There’s a fourth word that combines with each of the others to create another common word. What is it?
The neuroscience here hints at something that current AI systems still lack:
a direct, internal positive signal tied to closing a reasoning loop.
Transformers learn almost everything through language-like supervision. Wrong token = small penalty, right token = small reward. That’s great for pattern induction, but it means the model treats a correct chain-of-thought and a beautifully phrased but wrong chain-of-thought as almost the same kind of object—just sequences with slightly different likelihoods.
Human reasoning isn’t like that.
When a logic chain closes cleanly, the brain fires a strong internal reward. That “Aha” isn’t just emotion; it’s an endogenous learning signal saying: this structure is valid, keep this, reuse this. It’s effectively a structural correctness reward, orthogonal to surface language.
If AI ever gets a similar mechanism — a way to mark “self-consistent causal closure” as positively rewarded — we might finally bridge the gap between language-trained reasoning and true general learning. It would matter for:
fast abstraction formation
reliable logical inference
discovering new concepts rather than remixing old ones
Backprop gives us gradient-based correction, but it’s mostly negative feedback. There’s no analogue of the brain’s “internal positive jolt” when a new idea snaps together.
If AGI needs general learning, maybe the missing piece isn’t more scale — it’s this reward for closure.
Is there a single repo that has all of these "aha" images? I could see the clown right away, and the vines/plants in the 2nd example were what I thought first but organic shapes are harder to be sure about.
That also brings to mind that first exposure to this dataset affects the effectiveness of the rest of the dataset. If you're doing initial exposure, you'll definitely get the "aha" moment. But if all of the images in the dataset are of the same type, your brain quickly learns the pattern and the "aha" moment vanishes.
If they did their study on all of the images per test subject, the results after maybe the first 5 are basically useless for any definitive conclusions.
The little puzzle the article opens with is fun, but solving it is not what I'd associate with an "aha" moment.
To me it felt more like a brute force search, or like solving a Wordle puzzle.
I consider "aha" more creative, like recognizing that key insight that crystalizes a solution to a problem you're working on. (Or maybe a pattern or analogy that cleanly collapses a swath of the complexity).
I really wish we could move beyond fMRI for brain studies. We have no good models for any insights beyond “this region of the brain lights up.” It’s medieval. Neurophrenology. Change my mind.
I've been exploring this concept in LLMs for the last week or so, to see if I can RL train one into being inherently curious.
I haven't got any beyond my own working notes and some basic plots, but I've unceremoniously dumped them into a document here incase anyone else finds them interesting. If so I'd _love_ to chat with you. enjeyw @ google's email provder.
Aha moments cause massive firing in widespread brain regions (executive, memory, verbal and psyche). No surprise that some of it is caught with some detection device.
If the cerebral cortex is analogous to a neural network but aha moments or moments of insight depend on the structure of the amygdala plus the hippocampus is there an architecture that vaguely represents the structure between a cerebral cortex amygdala and hippocampus
23 comments
[ 5.5 ms ] story [ 50.9 ms ] threadThat aside, working with complex systems and constraints there often isn't an aha moment, there's just a decision to be made. As someone who loves that aha moment, I can get stuck trying to figure out perfect from good enough. Interesting to see there is indeed a positive emotion correlated with that aha moment that keeps people searching for solutions.
I wonder if there's a correlation between addiction and this aha moment. Like you get drunk and suddenly "aha!" those big unresolvable problems don't matter. The next morning they matter again until, aha, beer:30 hits.
Try pairing a feeling to: "So THAT'S how a mouse cursor moves"
"So THAT'S why revolving doors move clockwise!"
"So THAT'S why lights at night feel cosy!"
With a little practice, you can arbitrarily get 'aha' moments. I assume the good feeling is some sort of dopamine release where my brain is rewarding me for "figuring something out," even though I've kind if hijacked the mechanism.
https://www.youtube.com/watch?v=aB_4YU6UtCw
tldr: There is a background, non-verbal process in your brain that has the advantage of a larger working set size than your foreground verbal thinking. It is able to observe and consider more stuff at once and find associations better than your conscious thought process.
But, it has several disadvantages. It takes time to do its processing. You can't will it into action. It communicates non-verbally with your foreground process. It doesn't work under pressure (thus the need for relaxed, unfocused time). The non-verbal understanding is difficult to deconstruct, generalize and reapply. It can lead to you solving a problem, not understanding how and not being able to solve a variant of the same problem.
So, the general recommendation is: If you have a complex problem to solve, first absorb as much information about the problem as your brain can hold. But, do not try to solve anything. Then, go take a break. A walk in a natural environment is preferable. Don’t think about the problem. Relax in a low stress environment. Let your background brain have a chance to chew on it and maybe bubble up some suggestions.
->https://www.paulgraham.com/top.html
By keeping problems in a dormant state, the brain can work on them in the background, leading to unexpected solutions when a new piece of knowledge is encountered
YXBwbGU= (Use Base64 Decoding) [/Spoiler]
->https://www.quantamagazine.org/how-your-brain-creates-aha-mo...
Transformers learn almost everything through language-like supervision. Wrong token = small penalty, right token = small reward. That’s great for pattern induction, but it means the model treats a correct chain-of-thought and a beautifully phrased but wrong chain-of-thought as almost the same kind of object—just sequences with slightly different likelihoods.
Human reasoning isn’t like that. When a logic chain closes cleanly, the brain fires a strong internal reward. That “Aha” isn’t just emotion; it’s an endogenous learning signal saying: this structure is valid, keep this, reuse this. It’s effectively a structural correctness reward, orthogonal to surface language.
If AI ever gets a similar mechanism — a way to mark “self-consistent causal closure” as positively rewarded — we might finally bridge the gap between language-trained reasoning and true general learning. It would matter for:
fast abstraction formation
reliable logical inference
discovering new concepts rather than remixing old ones
Backprop gives us gradient-based correction, but it’s mostly negative feedback. There’s no analogue of the brain’s “internal positive jolt” when a new idea snaps together.
If AGI needs general learning, maybe the missing piece isn’t more scale — it’s this reward for closure.
That also brings to mind that first exposure to this dataset affects the effectiveness of the rest of the dataset. If you're doing initial exposure, you'll definitely get the "aha" moment. But if all of the images in the dataset are of the same type, your brain quickly learns the pattern and the "aha" moment vanishes.
If they did their study on all of the images per test subject, the results after maybe the first 5 are basically useless for any definitive conclusions.
To me it felt more like a brute force search, or like solving a Wordle puzzle.
I consider "aha" more creative, like recognizing that key insight that crystalizes a solution to a problem you're working on. (Or maybe a pattern or analogy that cleanly collapses a swath of the complexity).
This is the premise of a really good article I reccommend to anyone, the Seductions of Clarity by C. Thi Nguyen (https://philarchive.org/rec/NGUTSO-2)
Nicholson, Hannele B. M. (2007) DISFLUENCY IN DIALOGUE: ATTENTION, STRUCTURE AND FUNCTION PhD. thesis, University of Edinburgh, Edinburgh, Scotland, UK, available online: https://era.ed.ac.uk/bitstream/handle/1842/1763/Nicholson%20...
But "um" may not be quite the same as "aha" for English native speakers (and Japanese native speakers may use both "um" and "ahm" as disagrement).
I haven't got any beyond my own working notes and some basic plots, but I've unceremoniously dumped them into a document here incase anyone else finds them interesting. If so I'd _love_ to chat with you. enjeyw @ google's email provder.
https://thealephengine.substack.com/p/67e3786f-8e84-41bd-888...