In 1993, two unknown French engineers claimed to have found a coding scheme to provide virtually error-free communications at data rates and transmitting-power efficiencies well beyond what most experts thought possible. Nobody believed them and set out to find the error in their paper... There was no error.
30 years ago? What took so long for broad adoption? Patents?
edit
Article doesn't elaborate on why that might be, but it does note:
"an alternative that has been given a new lease on life is low-density parity check (LDPC) codes, invented in the early 1960s by Robert Gallager at MIT but largely forgotten since then...Now researchers have implemented LDPC codes so that they actually outperform turbo codes and get even closer to the Shannon limit...Another advantage, perhaps the biggest of all, is that the LDPC patents have expired, so companies can use them without having to pay for intellectual-property rights."
> 3. Discovery is laughed out of industry as "impractical", "academic", etc.
To be fair, sometimes academic inventions are impractical given the technical landscape du jour. They only become feasible as technology and society progresses, say 30 years.
Also, your observation does not mean any invention is meaningful (and will eventually be recognized as such). I for one, learned a bunch of "impractical", "academic" nonsense too in university.
But yes, I think as a whole, your 7 points observation as a whole is very much at play in the Turbo Codes case. Yes.
The specific issue with LDPC and Turbo Codes is that they both introduce latency, and require a ton of compute to decode.
If you don't care about latency and have an "unlimited" compute budget, then they're great and hence they were used very early for deep space communications. The speed-of-light delay means that there's minutes or even hours of latency anyway. And of course, the receiving station is a data centre with as much computer power available as you please.
In hand-held devices running on battery with latency constraints measured in milliseconds, it's an entirely different ballgame. Suddenly, there's real engineering problems to solve, and the trade-offs aren't always worth it.
Something I've told people when both 4G and 5G technologies rolled out is that these are enabled by the advances in the bleeding edge of silicon chip technology. Both rely on very heavyweight signal decoding algorithms that require dedicated "offload" circuitry almost as big as the main CPU!
Some anti-5G crazies think that their hand is getting hot from the "radiation", when in fact it is the decoding chip that's getting warm from the power it has to draw from the battery!
PS: This is why every new 4G/5G/6G/etc... tech always gets panned as impacting battery life. One or two silicon process shrinks later, this is not a problem. The radio power didn't change, but the decoding power did!
4.5 some of the less arrogant academics internalize the complaints about their solutions and iterate on it to make it 20% easier to use and/or 20% better in practice, making the cost of change more attractive.
Academics have different goals, metrics and incentives than industry, but I would seldom call academic work "arrogant".
In particular, iterating to make things easier may or may not be sufficient to get the industry to take a second look at the project, and there is a chance that it may consume enough of the academic's time to ruin their career in academia.
It's anecdata, but I have seen a few cases (e.g. Coccinelle) in which an academic essentially hides the fact that their contributions to e.g. an open-source project comes from academic research until after it has been battle-tested by industry.
Sometimes the academics aren't talking to each other, not out of spite, but just because they're in different fields.
I recall from a computer vision class that some of the early CV researchers wasted a lot of time redeveloping math techniques that had already been developed by the photogrammetry people literally decades before.
Wait, what exactly is metacompilation, and in what form is it used in the industry? I've tried searching the internet, but only found theoretical explanations I'm too tired to parse right now, and something about yaks and bisons.
And you're right, it may be a bit early to speak of it being used in the industry. Examples of subset of multi-staged compilation include `constexpr` in C++ and `comptime` in Zig. I also seem to remember seeing stuff in the GraalVM ecosystem but I can't find them right now.
While that does sort of happen, that's not the reason.
Article says it was used in satellite links and deep space networks before it was cool. Those applications had hit their technological limits and no one was making something new they could buy next year, so they had to do it themselves.
Contrast that with phone networks being able to rely on the next G coming out. Now we have 5G, but it's 5G with an asterisk, so it's time to look for a new approach.
Same with Moore's law. The semiconductor industry was really focused on smaller transistors for a long time until they started getting close to the physical limit. Then suddenly everyone is talking about chiplets and other ideas that had been around a long time but weren't mainstream.
Changing approach requires a lot of coordination and carries a lot of risk. It's easier to finish milking one cow for as long as possible before moving on to the next. The laughing is just because your advisor knows how long it will probably be before the general market has exhausted its old approach.
Yep, as in all things, we do the easy thing until it stops working. Only then is it worth putting more work into harder things.
I’m not deriding this approach, mind you. It’s efficient.
Edit: Also the incremental improvement over the current system needs not only to be big enough that it outweighs the cost of the changeover (including hidden costs like risks and opportunity cost), but also to be big enough to bother with in absolute terms. Things like zip, mp3 and jpg are good enough that even though there are now alternatives that are better by any reasonable metric, for most users it's just not worth the bother of changing over.
Not an expert, but I've encountered Reed Solomon error correction extensively in my career, and this seems to be replacing RS applications? I wouldn't be surprised if there was a fair amount of inertia to overcome given how widely used and how well RS worked for a very long time. RS was critical for things like ADSL data transmission over POTS and other similar large scale applications.
The wikipedia page for RS suggests some RS implementations are now "being replaced by more powerful turbo codes".
Just adding some extra color: Turbo codes did get instant traction in the industry once the academics actually "believed" them to be true. So the friction came from the academics and not the industry. Today, every communication system you are using are relying on Turbo codes in some shape or form.
Yes, wanted to share an interesting story of how great ideas can come from folks who are not the most popular names in a field. I feel the same will happen with AI now.
You get less overhead from the code. Actually, this article is from 2004, so if you use a 4G cell phone you actually already are using the code. 5G uses something different though.
37 comments
[ 2.6 ms ] story [ 88.9 ms ] thread1. Academics make discovery/invent something.
2. Academics attempt to convince industry to adopt discovery.
3. Discovery is laughed out of industry as "impractical", "academic", etc.
4. Academics teach discovery to their students.
5. Students get into industry.
6. Students get into position of tech leads, architects, etc.
7. Students demonstrate the use of discovery.
(by then, 30 years have elapsed)
e.g. garbage collectors, type systems, threads, distributed systems, message passing concurrency, actors, JITs, metacompilation, machine learning, functional programming...
To be fair, sometimes academic inventions are impractical given the technical landscape du jour. They only become feasible as technology and society progresses, say 30 years.
Also, your observation does not mean any invention is meaningful (and will eventually be recognized as such). I for one, learned a bunch of "impractical", "academic" nonsense too in university.
But yes, I think as a whole, your 7 points observation as a whole is very much at play in the Turbo Codes case. Yes.
If you don't care about latency and have an "unlimited" compute budget, then they're great and hence they were used very early for deep space communications. The speed-of-light delay means that there's minutes or even hours of latency anyway. And of course, the receiving station is a data centre with as much computer power available as you please.
In hand-held devices running on battery with latency constraints measured in milliseconds, it's an entirely different ballgame. Suddenly, there's real engineering problems to solve, and the trade-offs aren't always worth it.
Something I've told people when both 4G and 5G technologies rolled out is that these are enabled by the advances in the bleeding edge of silicon chip technology. Both rely on very heavyweight signal decoding algorithms that require dedicated "offload" circuitry almost as big as the main CPU!
Some anti-5G crazies think that their hand is getting hot from the "radiation", when in fact it is the decoding chip that's getting warm from the power it has to draw from the battery!
PS: This is why every new 4G/5G/6G/etc... tech always gets panned as impacting battery life. One or two silicon process shrinks later, this is not a problem. The radio power didn't change, but the decoding power did!
When I was an academic, I never encountered (or heard of) such situations, though.
In particular, iterating to make things easier may or may not be sufficient to get the industry to take a second look at the project, and there is a chance that it may consume enough of the academic's time to ruin their career in academia.
It's anecdata, but I have seen a few cases (e.g. Coccinelle) in which an academic essentially hides the fact that their contributions to e.g. an open-source project comes from academic research until after it has been battle-tested by industry.
I recall from a computer vision class that some of the early CV researchers wasted a lot of time redeveloping math techniques that had already been developed by the photogrammetry people literally decades before.
(In the Forth world it means a special Forth vocabulary the Forth system can use to bootstrap itself.)
And you're right, it may be a bit early to speak of it being used in the industry. Examples of subset of multi-staged compilation include `constexpr` in C++ and `comptime` in Zig. I also seem to remember seeing stuff in the GraalVM ecosystem but I can't find them right now.
Article says it was used in satellite links and deep space networks before it was cool. Those applications had hit their technological limits and no one was making something new they could buy next year, so they had to do it themselves.
Contrast that with phone networks being able to rely on the next G coming out. Now we have 5G, but it's 5G with an asterisk, so it's time to look for a new approach.
Same with Moore's law. The semiconductor industry was really focused on smaller transistors for a long time until they started getting close to the physical limit. Then suddenly everyone is talking about chiplets and other ideas that had been around a long time but weren't mainstream.
Changing approach requires a lot of coordination and carries a lot of risk. It's easier to finish milking one cow for as long as possible before moving on to the next. The laughing is just because your advisor knows how long it will probably be before the general market has exhausted its old approach.
I’m not deriding this approach, mind you. It’s efficient.
Edit: Also the incremental improvement over the current system needs not only to be big enough that it outweighs the cost of the changeover (including hidden costs like risks and opportunity cost), but also to be big enough to bother with in absolute terms. Things like zip, mp3 and jpg are good enough that even though there are now alternatives that are better by any reasonable metric, for most users it's just not worth the bother of changing over.
The wikipedia page for RS suggests some RS implementations are now "being replaced by more powerful turbo codes".
> https://en.wikipedia.org/wiki/Reed%E2%80%93Solomon_error_cor...
Edited to add an early link to more granular technical work: https://ipnpr.jpl.nasa.gov/progress_report/42-120/120D.pdf
How is perfect data transmission a make or break feature for lossy compression and transmission of audio and video?
A few big issues with Turbo codes: 1. Decoder complexity - often 4x or more the complexity of simpler codes.
2. Encoder delay- you need to have a really long interleaver at the transmitter to get the big dB of improvements.
3. These work best an pure additive white gaussian noise channels - most channels suffer from multipath, fading and other disturbances.
So while they are good far satellite broadcast system they don't always work as well in other applications.