It's because you're using Cloudflare DNS (1.1.1.1) which Internet Archive return an incorrect IP for. There's a bazillion threads about this on HN for more detail
Follow the link to the paper itself that titzer put in their comment. Towards the bottom, right above the Acknowledgements section, there's several WAV files with descriptions of each one.
Kind of surprising they didn't go with "Another Brick in the Wall, Part 2" instead. Part 1 is very low key and doesn't feature much of a recognizable melody. Musically, it doesn't really stand on its own, it only works as part of The Wall as a concept album.
Part 2 has a recognizable melody and it includes the iconic chorus "We don't need no education". If you've grown up in western culture, you've probably been exposed to Part 2 through cultural osmosis even if you're not a fan of Pink Floyd or classic rock.
Very interesting research that is 100% oversold in the popular media. Look at the original paper and listen to the reconstructed audio samples (at the end):
Can you identify which song has been reconstructed? I could not.
The basis for song "reconstruction" claims is the song-excerpt identification analysis (e.g. Fig 4) that divides the song into 38 5-second clips and then ranks how closely each identified excerpt is to the expected excerpt--effectively, how well does the reconstruction match 38 excerpts to 38 expected excerpts. This is not reconstructing the song, just matching puzzle parts.
A nice piece of neural science, completely misrepresented in the media.
> The basis for song "reconstruction" claims [in the popular media] is the song-excerpt identification analysis
Not correct for the Dutch state news, at least. They make the same "reconstruction" claims while showing the audio fragments that don't use any input data (for 1 and 29 patients), while mentioning that it's not very accurate but at any rate still recognizable. https://nos.nl/artikel/2486771-computer-herkent-pink-floyd-n...
Based on earlier neural decoding news items I was also expecting something completely overblown, but in this case I think the reporting is quite legit, and you might just be having a knee-jerk contrarian reaction.
my expectations were low based on this comment, so I found the reconstructions much better than expected.
Impossible to say if I'd be able to ID them without any context...
...but that leads immediately to the thought that a useful metric for these things might well be whether Shazam and similar "landmark identification" tools can.
Especially those which themselves are based on perceptual encoding models of some kind... the more human-perceptual-system based, the more interesting.
I.e.: if our machine models of audio perception, can consume the output of machine models of the output of those models, and reliably track that back to the originals,
the "fidelity" of the the lossy "encoding" ("recording") is becomes less interesting.
Especially from a behavioralist/instrumentalist view.
Which is the view underneath commercialization and e.g. surveillance.
Does the <data blob> sound/look like the original? Who cares, if the machine knows it's <thing>, we're good.
Vaguely related, in the long term it will be interesting to learn if the internal modeling (sic) of music in different people differs significantly as a function of e.g. musical training, proficiency, and the like. This would imply you could tell how good a musician someone is or might be, by inspecting the qualities of their internal model.
Which then makes me wonder about grokking and eidetic memory for audio. Anecdote: Hollywood composer (and sometime pop musician and collaborator with Amy Mann) Jon Brion used to have a regular night at bar in LA, which we attended ~25 years ago. He was regularly joined by other then-famous musicians like Rickie Lee Jones. Part of his shtick was to pick up suggestions from the audience of free-associate musically, performing fragments of various popular music (e.g. 80s pop).
It was truly remarkable, even uncanny and unsettling, how good his ability was to instantly riff out not just recognizable or faithful covers, but inflect them with a large number of the idiosyncratic micro-details of the original performance. At the time I understood this as the rare intersection of something like eidetic memory for music, and, a caricaturist's talent for emphasizing exactly those features which make the thing unique.
Which I suspect are exactly the sorts of landmarks systems like Shazam tend to abstract out; and what I anticipate work like this will reveal are exploited by our own perceptual memory systems.
Which, incidentally, seem to be distinct from "episodic memory" viz. the well established manner in which people far gone to various degraded memory and dementia conditions, can be "brought back" to themselves through flawless recall and performance/singing of songs they knew, even when they are otherwise "lost." C.f. https://www.openculture.com/2013/05/the_strange_day_when_bug... as well
>Algorithms already can construct pretty accurate models of a person’s preferences based on physical activities, including typing search terms or tapping to “like” a photo. While there are high hopes that reconstructing words and music from neural activity could enable applications that promote mental health and social connection, reading and interpreting thoughts is regarded as the next privacy frontier.
Uncannily, this reminds me of a popular German cartoon from 1819 titled "The Thinkers Club" (Der Denker-Club) [0] which had the slogan: How long will be thinking allowed?" (Wie lange möchte uns noch das Denken wohl noch erlaubt bleiben?)
It was meant to satirize the clamp down on liberal and nationalist ideas by restricting the freedom of assembly and press as a way to follow through the European Restauration after the Napoleonic Wars.
In the current asymmetry of giant tech corporations (in bed with government) I find the idea of further breaking any privacy walls whatsoever by trojan horsing it with mental health and social connection and declaring it a new frontier just totally backwards. I think as with liberal and nationalistic ideas back in the day we should value privacy not try just to let it go for ultimately what? Dissolve ourselves into AGI superseding Gaia: Le monde, c'est moi! But maybe I'm the old-fashioned one, here.
Has anyone tried quantizing some metric of brain activity (EEG etc.) into a set of tokens, and then training a Transformer model on it?
I wonder if hidden in that is some good prior for human-cognition-related activities, i.e. extend the token space to add human language tokens, train on that and see if it trains significantly faster than a randomly initialized model.
I and many other people can create new music in my head by imagining it. I wonder if this could allow me to turn my brain music into an album without needing a 104 part score written out, a full orchestra, and the philharmonic to play and record it live first?
26 comments
[ 3.2 ms ] story [ 45.0 ms ] thread(This comes from a thread I started on a different submission, see https://news.ycombinator.com/item?id=37142315 and https://news.ycombinator.com/item?id=37143776)
It would be interesting if they picked a hummable tune like Peter Cottontail.
Missed an opportunity to use Brain Damage instead.
I'd like to see them do this experiment with Brain Damage/Eclipse, climax of Echoes, or Celestial Voices.
Part 2 has a recognizable melody and it includes the iconic chorus "We don't need no education". If you've grown up in western culture, you've probably been exposed to Part 2 through cultural osmosis even if you're not a fan of Pink Floyd or classic rock.
“Hello? (Hello? Hello? Hello?) Is there anybody in there? Just nod if you can hear me Is there anyone home?”
https://www.science.org/content/article/hear-classic-pink-fl...
https://journals.plos.org/plosbiology/article?id=10.1371/jou...
Can you identify which song has been reconstructed? I could not.
The basis for song "reconstruction" claims is the song-excerpt identification analysis (e.g. Fig 4) that divides the song into 38 5-second clips and then ranks how closely each identified excerpt is to the expected excerpt--effectively, how well does the reconstruction match 38 excerpts to 38 expected excerpts. This is not reconstructing the song, just matching puzzle parts.
A nice piece of neural science, completely misrepresented in the media.
Pretty cool stuff even if it's early.
Not correct for the Dutch state news, at least. They make the same "reconstruction" claims while showing the audio fragments that don't use any input data (for 1 and 29 patients), while mentioning that it's not very accurate but at any rate still recognizable. https://nos.nl/artikel/2486771-computer-herkent-pink-floyd-n...
Based on earlier neural decoding news items I was also expecting something completely overblown, but in this case I think the reporting is quite legit, and you might just be having a knee-jerk contrarian reaction.
my expectations were low based on this comment, so I found the reconstructions much better than expected.
Impossible to say if I'd be able to ID them without any context...
...but that leads immediately to the thought that a useful metric for these things might well be whether Shazam and similar "landmark identification" tools can.
Especially those which themselves are based on perceptual encoding models of some kind... the more human-perceptual-system based, the more interesting.
I.e.: if our machine models of audio perception, can consume the output of machine models of the output of those models, and reliably track that back to the originals,
the "fidelity" of the the lossy "encoding" ("recording") is becomes less interesting.
Especially from a behavioralist/instrumentalist view.
Which is the view underneath commercialization and e.g. surveillance.
Does the <data blob> sound/look like the original? Who cares, if the machine knows it's <thing>, we're good.
Vaguely related, in the long term it will be interesting to learn if the internal modeling (sic) of music in different people differs significantly as a function of e.g. musical training, proficiency, and the like. This would imply you could tell how good a musician someone is or might be, by inspecting the qualities of their internal model.
Which then makes me wonder about grokking and eidetic memory for audio. Anecdote: Hollywood composer (and sometime pop musician and collaborator with Amy Mann) Jon Brion used to have a regular night at bar in LA, which we attended ~25 years ago. He was regularly joined by other then-famous musicians like Rickie Lee Jones. Part of his shtick was to pick up suggestions from the audience of free-associate musically, performing fragments of various popular music (e.g. 80s pop).
It was truly remarkable, even uncanny and unsettling, how good his ability was to instantly riff out not just recognizable or faithful covers, but inflect them with a large number of the idiosyncratic micro-details of the original performance. At the time I understood this as the rare intersection of something like eidetic memory for music, and, a caricaturist's talent for emphasizing exactly those features which make the thing unique.
Which I suspect are exactly the sorts of landmarks systems like Shazam tend to abstract out; and what I anticipate work like this will reveal are exploited by our own perceptual memory systems.
Which, incidentally, seem to be distinct from "episodic memory" viz. the well established manner in which people far gone to various degraded memory and dementia conditions, can be "brought back" to themselves through flawless recall and performance/singing of songs they knew, even when they are otherwise "lost." C.f. https://www.openculture.com/2013/05/the_strange_day_when_bug... as well
Uncannily, this reminds me of a popular German cartoon from 1819 titled "The Thinkers Club" (Der Denker-Club) [0] which had the slogan: How long will be thinking allowed?" (Wie lange möchte uns noch das Denken wohl noch erlaubt bleiben?)
It was meant to satirize the clamp down on liberal and nationalist ideas by restricting the freedom of assembly and press as a way to follow through the European Restauration after the Napoleonic Wars.
In the current asymmetry of giant tech corporations (in bed with government) I find the idea of further breaking any privacy walls whatsoever by trojan horsing it with mental health and social connection and declaring it a new frontier just totally backwards. I think as with liberal and nationalistic ideas back in the day we should value privacy not try just to let it go for ultimately what? Dissolve ourselves into AGI superseding Gaia: Le monde, c'est moi! But maybe I'm the old-fashioned one, here.
[0]https://en.m.wikipedia.org/wiki/Carlsbad_Decrees
I wonder if hidden in that is some good prior for human-cognition-related activities, i.e. extend the token space to add human language tokens, train on that and see if it trains significantly faster than a randomly initialized model.