For what it's worth, the presence of seemingly significant signal in the difference between the original and compressed tracks does not necessarily mean that significant sonic/perceptual loss has occurred. Operating correctly, the encoder is designed to cut not just sounds that the human ear cannot hear in general (e.g. sounds above 22kHZ) but also sounds which may not be perceptible in context (e.g. the quietest signals in a loud section). So if you find something beautiful about the ghost tracks (and I think there is something beautiful to find), don't immediately jump to concluding that mp3 is awful for cutting these sounds—they might be hardly perceptible when added to the mix.
Of course, at high-compression rates mp3 does begin to significantly degrade fidelity.
Edit: all of this is not to put down the project—I still think it's pretty cool as art and as a demonstration of the encoder, I just didn't want people to think that this was some sort of massive failing of mp3.
> Operating correctly, the encoder is designed to cut not just sounds that the human ear cannot hear in general (e.g. sounds above 22kHZ) but also sounds which may not be perceptible in context
This.
The idea is called "masking", and a well-designed encoder explicitly removes the most details from areas where they are masked by other, louder parts of the signal. See the presentation on CELT I gave at linux.conf.au in 2009, and in particular listen to the audio samples that go with slide 47: http://www.celt-codec.org/presentations/
Look into the sky at night during a new moon, and count the stars. Look again during the day. The stars are still there, but the atmospheric diffusion of blue light from the sun prevents you from discerning the fainter light of a more distant star.
Lower yourself to the bottom of a deep well during the day, and look up again. You can see the stars again, because the diffuse blue sunlight is much weaker there.
If you were to compress an image of the daytime sky in the same manner that an MP3 compresses audio, you could simply remove all that variation from faint stars and only keep the information from bright sunlight. Follow the ghost discovery technique by taking the difference between the original and the compressed image, and you will see an image resembling the night sky. But most people would not be able to discern much of a difference with the naked eye.
For audio, if you have a pianissimo piccolo solo, and someone plays a crash cymbal in the middle, you could remove some of the harmonics from the melody because humans are incapable of hearing them, and you can also remove that portion drowned out by the crash, because humans can't hear quiet sounds through louder noises.
Finding the "ghost" track is actually revealing all the sound in a piece of music that no one ever hears anyway, whether the track is compressed or not, simply due to the hardware limitations of the human auditory system.
From a technical POV I also question that contemporary encoders still optimize for those three tracks (which means any encoder but the ISO reference code and very close derivatives like 8Hz).
IIRC LAME which was used for this project, replaced the entire psychoacoustic modelling code (which is possible to do since MP3-the-format only cares about frequency bands and buckets, not about what the encoder does to them), and they likely tuned their new model to larger, more diverse sets of samples and ears.
Is it possible that some of the differences shown in this page come from phase changes rather than attenuation? If so, is such a phase change perceptible, even in principle?
Yes, the difference signal is present in both the residue and the mp3, not lost just shifted in phase so it doesn't cancel exactly any more. In general the phase shifts are not audible but you can construct some signals (like two impulses very close together) that fool MP3 so the encoded signal has the correct spectrum but the wrong phase to reconstruct the impulses so sounds different.
Even more importantly, there are many signals that sound identical to the human ear, yet are entirely different in waveform. (For instance, two sines shifted 90 degrees.) You can save space by representing all similar sounding waveforms as the same waveform. This might make for a giant delta when you subtract it from the original recording, even though nothing audible was lost.
Good point! I believe the author in this particular case was subtracting in the frequency domain, not in the time domain — subtracting spectrographs instead of waveforms. In that case, a 90° phase-shifted sine wave would indeed cancel out.
There is good reason to think that's different. Audio quality that's measured by subjects' ability to perceive, and report perception of a sound masks the difference between their ear being unable to detect a signal, and hence no signal comes over the auditory nerve and a situation where a soft sound might not be noticed in a loud passage but the ear responds to the signal.
Is this significant? One would probably have to do an FMRI study to answer that. In some applications, like understanding the words of someone speaking, by definition it isn't. But music is emotional and there might be aspects of music listening that are outside the ability to consciously report.
An obvious case of audition working when we can't consciously report what we hear is when we are sleeping, but experiments can detect the influence of those sounds.
I would appreciate a couple examples showing more typical bitrates at 256kbps and 320kbps. I like what it is, but there's this lossy difference between the argument being made and reality. Now, if I could get a copy of the intended argument, I could invert the phase with this site and show you the difference.
I'd refer you to the top-voted comment from @
daturkel. Because of the psychoacoustic models used for lossy audio compression (and, in particular, the "masking" effect) listening to these delta signals won't give you useful information about the compressor's accuracy.
That "sines" example was 320kbps in case you didn't notice. Judging by that I can imagine that less radical compression for other music pieces would make pretty similar ghosts, just, well, less radical. Not so hard to check it for yourself, anyway.
Because she is from Ohio? The instruments and scales she uses are pretty western European, which is probably a better way of sorting music than examining the person making it.
This is hardly innovative or creative. People on audio forums have been playing around with this technique for years. It is literally a mechanical process.
Exactly. OP doesn't have enough to complain about, so he's literally hyperfocusing on nearly imperceptible artifacts and amplifying them...
...in order to weakly raise a vague grumbling about "European" audio engineers.
Even though he has no evidence to support that claim.
Yes, Fraunhofer is a German company, with headquarters in Munich. But did OP know they employ 23,000 people, with centers in the U.S. as well as Asia?
He could have looked that up, but it's so much easier to just throw bullshit out there and hope it resonates with people ashamed of their ancestors.
MP3 is not one single technology that all came from Fraunhofer. It is an accumulation of several areas of technology, including types of filters, compression as well as the perceptual stuff. I once traced it through the research literature all the way back to voice compression protocols for the American F15 (16?) in the 1970s!
And if you're reading this far, you're probably totally obsessed like me and you might want to check out my associate Jonathan Sterne's book on the history of MP3 from a cultural perspective [0].
Sterne traces the history of perceptual encoding all the back to the early 1910s! It's not just some pop-tech designed in 9 months for a quick buck!
Finally, audio engineers is a term that generally means people who work with sound in a studio or recording context, who produce audio for projects with specs. MP3 was designed by scientists who produce work about audio, not audio engineers.
“Listening tests, primarily designed by and for western-european white men, and using the music they liked, were used to refine the encoder.”
Not to be oversensitive, but would it be okay to say something like this if Fraunhofer IIS were in... Japan or something, and they happened to find a lot of Japanese people who liked Japanese music for listening tests?
“Listening tests, primarily designed by and for east-asian yellow men, and using the music they liked, were used to refine the encoder.” Would probably come off a bit rougher, ね？
FWIW the tracks were interesting to listen to, and I've had fun doing this as well. I recommend trying it with the Opus codec. As for the sensibilities of the author, I hope they can get over their paranoia about audio codecs being an ethnocentric conspiracy to destroy "the sounds they didn't want us to hear".
In the United States, "yellow man" is considered pejorative, "white man" is not. In isolation this seems illogical as they are structurally analogous, but words do not have meaning in isolation. The meaning of a word entails its historical use by a society.
I dunno. I almost exclusively hear "white man" used as an insult. Not long-term historically certainly, but certainly now - it's almost always accompanied by the implication that white men are racist (and/or sexist).
Nice and interesting, but doesn't soundcloud use some compression itself? I mean, isn't it different from simply posting some FLAC file? I don't know, just wondering.
"Does SoundCloud convert my files to mono?
We don't modify original audio files that are available for download. For streams, we convert the original into MP3 Joint Stereo for streaming."
35 comments
[ 3.5 ms ] story [ 73.5 ms ] threadOf course, at high-compression rates mp3 does begin to significantly degrade fidelity.
Edit: all of this is not to put down the project—I still think it's pretty cool as art and as a demonstration of the encoder, I just didn't want people to think that this was some sort of massive failing of mp3.
This.
The idea is called "masking", and a well-designed encoder explicitly removes the most details from areas where they are masked by other, louder parts of the signal. See the presentation on CELT I gave at linux.conf.au in 2009, and in particular listen to the audio samples that go with slide 47: http://www.celt-codec.org/presentations/
Lower yourself to the bottom of a deep well during the day, and look up again. You can see the stars again, because the diffuse blue sunlight is much weaker there.
If you were to compress an image of the daytime sky in the same manner that an MP3 compresses audio, you could simply remove all that variation from faint stars and only keep the information from bright sunlight. Follow the ghost discovery technique by taking the difference between the original and the compressed image, and you will see an image resembling the night sky. But most people would not be able to discern much of a difference with the naked eye.
For audio, if you have a pianissimo piccolo solo, and someone plays a crash cymbal in the middle, you could remove some of the harmonics from the melody because humans are incapable of hearing them, and you can also remove that portion drowned out by the crash, because humans can't hear quiet sounds through louder noises.
Finding the "ghost" track is actually revealing all the sound in a piece of music that no one ever hears anyway, whether the track is compressed or not, simply due to the hardware limitations of the human auditory system.
IIRC LAME which was used for this project, replaced the entire psychoacoustic modelling code (which is possible to do since MP3-the-format only cares about frequency bands and buckets, not about what the encoder does to them), and they likely tuned their new model to larger, more diverse sets of samples and ears.
There is good reason to think that's different. Audio quality that's measured by subjects' ability to perceive, and report perception of a sound masks the difference between their ear being unable to detect a signal, and hence no signal comes over the auditory nerve and a situation where a soft sound might not be noticed in a loud passage but the ear responds to the signal.
Is this significant? One would probably have to do an FMRI study to answer that. In some applications, like understanding the words of someone speaking, by definition it isn't. But music is emotional and there might be aspects of music listening that are outside the ability to consciously report.
An obvious case of audition working when we can't consciously report what we hear is when we are sleeping, but experiments can detect the influence of those sounds.
Especially to compare different bitrates. I'm not even sure what bitrate this used, maybe 128 like the example files?
I find that a lot more interesting than anything learned, strangely.
...in order to weakly raise a vague grumbling about "European" audio engineers.
Even though he has no evidence to support that claim.
Yes, Fraunhofer is a German company, with headquarters in Munich. But did OP know they employ 23,000 people, with centers in the U.S. as well as Asia?
He could have looked that up, but it's so much easier to just throw bullshit out there and hope it resonates with people ashamed of their ancestors.
MP3 is not one single technology that all came from Fraunhofer. It is an accumulation of several areas of technology, including types of filters, compression as well as the perceptual stuff. I once traced it through the research literature all the way back to voice compression protocols for the American F15 (16?) in the 1970s!
And if you're reading this far, you're probably totally obsessed like me and you might want to check out my associate Jonathan Sterne's book on the history of MP3 from a cultural perspective [0].
[0] https://www.dukeupress.edu/MP3/
Sterne traces the history of perceptual encoding all the back to the early 1910s! It's not just some pop-tech designed in 9 months for a quick buck!
Finally, audio engineers is a term that generally means people who work with sound in a studio or recording context, who produce audio for projects with specs. MP3 was designed by scientists who produce work about audio, not audio engineers.
Not to be oversensitive, but would it be okay to say something like this if Fraunhofer IIS were in... Japan or something, and they happened to find a lot of Japanese people who liked Japanese music for listening tests?
“Listening tests, primarily designed by and for east-asian yellow men, and using the music they liked, were used to refine the encoder.” Would probably come off a bit rougher, ね？
FWIW the tracks were interesting to listen to, and I've had fun doing this as well. I recommend trying it with the Opus codec. As for the sensibilities of the author, I hope they can get over their paranoia about audio codecs being an ethnocentric conspiracy to destroy "the sounds they didn't want us to hear".