Just as interesting and surprising to me is that this 0-100Hz line is unexplained. Given that signal processing is the foundation of pretty much all digital technology, as well as the analog technologies that came before, I've kind of assumed every segment of a frequency plot be well studied and understood, with half a dozen of names to choose for (doublesine quefrency this, Kowalski-Shannon that...) and a heap of decades old textbooks about it.
I confirm, the low frequency range looks weird on every DFT plot I ever saw, particularly the audio ones. I just assumed it has something to do with ADC and is probably explained on Wikipedia. It's literally one of the last place on Earth when I'd expect to find unsolved mysteries.
It's not completely unexplained. Roughly speaking you can get that power spectrum in the limit when you are adding up many different events where the magnitude of the event is inversely proportional to its likelihood (and in practice, there is a limit to the magnitude of the events that will cause it to level off at some point, but for some processes this is not measurable even over decades). The main mystery in most cases is what exactly is the physical process that is causing it. For some electronics it looks like it is due to trapped charges sometimes tunneling around, but it doesn't explain every case of it in electronics let alone everything else. Convective thermal effects can also be a good candidate in a lot of systems, since turbulence also has 1/f noise properties.
(Also, the low frequency range on a DFT can look weird for reasons other than noise: it'll also tend to rise up if there's any longer-term structure to the signal as well, so you need to be careful interpreting them blindly if you're trying to measure noise)
Low frequencies are studied quite extensively, especially the mHz-10Hz region for noise characterization of solid state materials. 1/f is quite well studied depending on your field. In semiconductor physics, for example, one possible explanation is electrons trapped in defects or on charged islands and then slowly trickling down. Of course this explanation cannot be used in other fields where 1/f noise shows up as well. The problem is that no model gives a satisfying answer as to why it occurs therefore its unexplained. Lack of model doesn't mean that you can't engineer your way around 1/f noise for example the chopper amp works so well because it shifts away towards frequencies above the 1/f cutoff.
Compounding the mystery is that it crops up everywhere, not just electronic noise plots. But outside of a few systems such as electronic noise in a laboratory setting, it's phenomenally hard to measure. For one thing, to get into the 1/f domain, you have to measure things for a long time. And the noise measurement itself is noisy. And the number of things that you need to control, such as environmental conditions, increases.
So its existence is often largely treated as an empirical rule of thumb rather than having a specific physical cause.
The other thing to note is that the noise plot in a dataset is probably a curve fit.
Wikipedia has an article with some other information on pink noise (a more common name), including a random generator: https://en.wikipedia.org/wiki/Pink_noise. Some music generation algorithms use pink noise, as it (supposedly) strikes a better balance between randomness and predictability.
When I was learning DSP, I was surprised by the fact that generating pink (1/f) noise, sample by sample, is not mathematically easy at all.
One practical approach is passing white noise (every sample independent random) through a "pinking filter", which is a sum of a bunch of lowpass filters at different frequencies, so that the sum of their cutoff "knees" approximates the frequency curve of pink noise. Another approach is generating in chunks using Fourier transform from a desired shape. Given the ubiquity of 1/f noise, you'd think there would be a simpler and more direct algorithm, but no.
>"Here is something even more interesting. As you approach α = -1, the time domain approaches a shape of t-1, and the frequency domain approaches a flat magnitude with a zero phase. However, a flat magnitude and zero phase corresponds to a delta function, δ(t), in the time domain."
>"Indeed, Heaviside introduced the δ-function in his work on electromagnetism and electrical engineering.[14] In a 1963 interview, Dirac stated, "All electrical engineers are familiar with the idea of a pulse, and the δ-function is just a way of expressing a pulse mathematically."[15]"
Power law distributions are specializations of the more general Levy stable distributions [0] [1]. Levy stable distributions answer the following question:
Given that the sum of independent and identically distributed random variables that converge to a distribution, what is the distribution they converge to?
If you answered Gaussian, you'd be wrong. The correct answer is Levy stable. There was no condition on finite variance. When variance can be infinite, Levy stable, or power law tail distributions, is the result. When the variance is finite, a Gaussian is the limiting distribution and, consequently, a Gaussian distribution is part of the family of Levy stable distributions.
The stability quality is the reason why the Levy stable (aka power law tail) distributions show up all over the place. If you've ever heard that the reason why the Normal distribution is called "normal", because the sums of (finite variance) random variables converges to a Gaussian, the same reasoning applies to the Levy stable. In some sense, Levy stable distributions are more normal than the normal distribution. My opinion is that infinite variance is hard for people to wrap their heads around, so they reject the premise.
Unfortunately I don't have a good answer for what the article brings up about the Fourier transform, but I'm almost positive that this can be answered with Levy stable distributions in mind. I will say that the distribution is often characterized by it's characteristic function. A short perusal of Wikipedia talks about Levy stable distributions being closed under Fourier transforms, which is what the article is talking about.
OP is considering power laws in the frequency domain.
You are considering power laws in the heavy tail of a distribution.
Different things! But there are confounders that make discussion seem similar:
- questions about moments and convergence (OP: do we have finite energy in the Fourier domain; your comment: do the tails of the distribution fall off fast enough to have finite moments of order 1 or 2)
- questions about averaging (OP: in the time domain; your comment: as an expectation obtained by integrating a distribution, or as a closure property of the stable class of distributions)
17 comments
[ 0.30 ms ] story [ 6.4 ms ] threadTurns out this property is not unusual. There are many such pairs - there’s a reasonably well-known journal paper with a construction technique.
I confirm, the low frequency range looks weird on every DFT plot I ever saw, particularly the audio ones. I just assumed it has something to do with ADC and is probably explained on Wikipedia. It's literally one of the last place on Earth when I'd expect to find unsolved mysteries.
(Also, the low frequency range on a DFT can look weird for reasons other than noise: it'll also tend to rise up if there's any longer-term structure to the signal as well, so you need to be careful interpreting them blindly if you're trying to measure noise)
So its existence is often largely treated as an empirical rule of thumb rather than having a specific physical cause.
The other thing to note is that the noise plot in a dataset is probably a curve fit.
Surely there is some connection to entropy.
If you want to be boring you could call it the uniform distribution for log frequency.
One practical approach is passing white noise (every sample independent random) through a "pinking filter", which is a sum of a bunch of lowpass filters at different frequencies, so that the sum of their cutoff "knees" approximates the frequency curve of pink noise. Another approach is generating in chunks using Fourier transform from a desired shape. Given the ubiquity of 1/f noise, you'd think there would be a simpler and more direct algorithm, but no.
>"Here is something even more interesting. As you approach α = -1, the time domain approaches a shape of t-1, and the frequency domain approaches a flat magnitude with a zero phase. However, a flat magnitude and zero phase corresponds to a delta function, δ(t), in the time domain."
Related:
https://en.wikipedia.org/wiki/Dirac_delta_function
>"Indeed, Heaviside introduced the δ-function in his work on electromagnetism and electrical engineering.[14] In a 1963 interview, Dirac stated, "All electrical engineers are familiar with the idea of a pulse, and the δ-function is just a way of expressing a pulse mathematically."[15]"
Given that the sum of independent and identically distributed random variables that converge to a distribution, what is the distribution they converge to?
If you answered Gaussian, you'd be wrong. The correct answer is Levy stable. There was no condition on finite variance. When variance can be infinite, Levy stable, or power law tail distributions, is the result. When the variance is finite, a Gaussian is the limiting distribution and, consequently, a Gaussian distribution is part of the family of Levy stable distributions.
The stability quality is the reason why the Levy stable (aka power law tail) distributions show up all over the place. If you've ever heard that the reason why the Normal distribution is called "normal", because the sums of (finite variance) random variables converges to a Gaussian, the same reasoning applies to the Levy stable. In some sense, Levy stable distributions are more normal than the normal distribution. My opinion is that infinite variance is hard for people to wrap their heads around, so they reject the premise.
Unfortunately I don't have a good answer for what the article brings up about the Fourier transform, but I'm almost positive that this can be answered with Levy stable distributions in mind. I will say that the distribution is often characterized by it's characteristic function. A short perusal of Wikipedia talks about Levy stable distributions being closed under Fourier transforms, which is what the article is talking about.
[0] https://en.wikipedia.org/wiki/L%C3%A9vy_distribution
[1] https://en.wikipedia.org/wiki/Stable_distribution
OP is considering power laws in the frequency domain.
You are considering power laws in the heavy tail of a distribution.
Different things! But there are confounders that make discussion seem similar:
- questions about moments and convergence (OP: do we have finite energy in the Fourier domain; your comment: do the tails of the distribution fall off fast enough to have finite moments of order 1 or 2)
- questions about averaging (OP: in the time domain; your comment: as an expectation obtained by integrating a distribution, or as a closure property of the stable class of distributions)