Cool project. But because this is a derivative work of the original shine library, it almost certainly isn't legal for you to relicense it from LGPL to MIT.
> The GPL/LGPL isn’t IP in the same sense as Disney’s character portfolio or corporate patent trolling though.
Sure, these licensing terms are meant to protect against said evils. They would also be unnecessary in a world that discarded the idea of owning data. And they're still ridiculous—especially copyleft.
If you don’t control both the encoding and playback ends, you have to accommodate legacy decoders. Podcasts use constant bitrate MP3 because it’s the lowest common denominator. Even though iPods popularized AAC, there are millions of clip-on MP3 players and car head units that don’t decode AAC and might even choke on variable bitrate MP3. Spotify still hasn’t migrated from Ogg Vorbis to Opus.
You don’t need to care about it, but (1) it did change the world, (2) it’s a good format to start with if you want to understand perceptual audio encoding, and (3) it is (only because of inertia, really) still used to deliver almost all open podcasting content.
> it’s a good format to start with if you want to understand perceptual audio encoding
It honestly isn't, it's actually more complicated than later codecs like LC-AAC for no benefit because of the weird filterbank thing. Though, I would start with ADPCM.
You can hear that they sound different from lossless audio - it will be smeared out and start too early due to artifact called "pre-echo". Basically it's the same thing as what people call ringing or mosquito noise in image compression.
For some reason all the music on my phone/computer is in mp3. I've never really made that choice, it just happened. Some of it was recorded in 2023.
I'm considering creating an small project that (as a part of a bigger task) needs to be able to play users' sound files. I'd be happy to know that I don't need to support mp3s anymore, but so far I've failed to convince myself that it's a right choice.
This blog post really feels like it was written for the sake of writing something. A C library mechanically translated to Go does not magically become a Go library, and the whole thing feels like cargo-culting. ("libc calls bad", "unsafe bad")
I would like to ask, why was this blogpost written? What is the purpose?
While the Go community's dislike of CGO can be a bit unreasonable from time to time (in my humble opinion), the post does say that one couldn't build a CGO version of LAME binding, so turned to the (automated) source transpilation instead. This tells much about the "portability" of C in practice.
> This tells much about the "portability" of C in practice.
There are multiple products using LAME on Windows. You've mis-identified the product that isn't portable here.
IOW, LAME works on Windows in products written in C, C++, Python, etc. It's just that this particular Go library doesn't support Windows.
I fail to see how that is a C portability problem, when the C bits already work on Windows.
[EDIT: Looking at the Go LAME project that he references, it appears he didn't even log an issue about whatever it was that went wrong on his Windows build. I'm more inclined to believe that he did something wrong than that LAME doesn't work on Windows using the referenced library.]
Portability means many things beside from being cross-platform and compilable on some compilers for the target. Many cross-platform libraries are still cumbersome to compile on Windows, sometimes because they only provide Makefile and autoconf (sigh), sometimes because they have to fetch another dependencies, sometimes because those dependencies are actually system-dependent but can't be easily used [1]. Go has no such issue assuming CGO is not used, so this does relate to my point---source code portability is only a part of portability.
[1] For example, Rust `openssl` crate was notably annoying to compile for a long time, even on macOS (it took literally years to make the typical experience smooth enough). And yet it was an essential component for HTTPS! Today we would use the `native-tls` crate instead that avoids OpenSSL on Windows and macOS.
> source code portability is only a part of portability.
I broadly agree, but in this particular case:
1. LAME compiles just fine using multiple different compilers on Windows (including an older MSVC - newer ones will error out on the use of unsafe functions, fixed by passing a #define in a flag).
2. LAME has no dependencies to download.
3. LAME is successfully used from many other languages on Windows.
So, yeah, if it's not working for him but works for everyone else, you can't blame C for portability here.
OTOH, because LAME is written in C, you have even more options - don't compile it yourself. Download the compiled LAME .dll and just use it from Go.
Author didn't even try the easiest option, said easiest option being possible only because it was written in C.
It's not a "personal blog post" any more if they self-promote it here. That means they want to share it with us, so it must have some value. I am weird, I don't really care tbh.
> I would like to ask, why was this blogpost written? What is the purpose?
I think he's trying to show an alternative MP3 encoder to LAME for Go projects. It does seem that he has fallen foul of the Shine license though.
In his own words, he transformed Shine from C to Go mechanically, but he put the result under a license incompatible with Shine's original LGPL license.
Yeah, the blog author should fix the license. Not ok at all.
This is not the case of zero knowledge reverse engineering or something on those lines even.
The best open source MP3 encoder, which still makes LAME very important. If you need the absolute best MP3 encoder, ABX testing against Fraunhofer’s encoder tells the tale.
I'm interested in knowing more, but after a few minutes of searching, I can't seem to find a modern macOS implementation of the reference Fraunhofer codec. All my tools (Audition, xACT, ffmpeg, and even dbpoweramp) use various versions of LAME. Does Fraunhofer's encoder preserve above 20kHz at 320kbps?
As an aside, the docs attached to the repo referenced above has the first good explanation of "joint stereo" I've ever seen. I remember seeing the "joint stereo" button on the first encoders, and nobody ever said what that actually did.
This was an enjoyable light read. A window into solving a programming problem outside my day to day work, easy enough to follow, with a couple of fun concepts like "psychoacoustic model".
One of the first things I ever pirated was some mp3 library for vb6 (well...I also pirated vb6). I built an mp3 player called 3pm, so clever. It had a userbase of me and maybe 2 friends.
The author seems to have spent considerable effort on learning as little about it as possible. Most of this article talks about translating C to Go mechanically, without understanding the code. There is even a cameo of the Bullshit Eng—, er, ChatGPT.
In other words, the title is misleading clickbait.
> 16k lines of obscure C, riddled with pointer magic and arcane memory manipulations? No thanks!
I swear I'm not trying to sound like an uppity, badass Real Programmer here, but this seems... not too bad? 16 kloc of C for what is apparently the gold standard in lossy audio encoding is actually less than I would have guessed.
I'm not sure if it is the codec at issue there. I have only used Discord voice chat a few times being interviewed for podcasts and the quality at times is astoundingly good. That said, those doing the interviews are using higher end audio gear that most people would so that would go a long way to improving that.
Generally when I am grabbing audio off Youtube via NewPipe, I generally use the 50kbs OPUS stream if it is just talking/lectures. I cannot tell the difference. That said I once did a FLAC to 128KBit OPUS test on myself with music convinced I would be able to tell the difference. I could not. I just don't have hearing that is sensitive to codec loss it seems.
Gold standard as far as the library to use, from what I gather. AFAIK LAME is definitely the gold standard for open source mp3 encoding tools, it's what I always seem to find under the hood of any audio library I use.
To address the grandparent comment, I kinda tend to agree with the OP here. Yes, 16 KLOC is smaller than I would have expected, but that's definitely large enough to be a project that will take some time to grok. And especially for math-heavy c code, less KLOC tends to mean that there's arcane stuff going on. Definitely not out of the realm of approachable, but not a codebase you'll quickly get what's going on without some serious study. And given that the OP's first attempt to convert it to go was to feed it through GPT one function at a time, it may not be something they are ready to modify.
Using ChatGPT one function at a time kinda leads me to believe they aren't super-familiar with c, since pointers alone will definitely not make that approach work. In this case, the OP made the right call by finding a simpler original library to work off of, since production c code isn't necessarily great for learning from. (although I don't like using ChatGPT for a learning tool like this, it adds another layer of abstraction & obfuscation)
Edit: rereading this, it comes off as negative to the OP. Not what I intended, the article was well written and I'm all for exploring new things and documenting them. I actually like writeups like these, since documenting approaching new things is a valuable thing to do, otherwise it makes the discovery part of programming dissappear if the only things that are written up are expert articles.
Given that people have been sued for patent infringement for merely using office scanners [0], patent encumbrance is absolutely a chilling effect even for hobbyists.
Developers living in places with no software patents and with no plans to go on markets with software patents don't care. See ffmpeg or VLC for example.
> 16k lines of obscure C, riddled with pointer magic and arcane memory manipulations? No thanks!
I think this line sells it short:
It's battle-tested code. Like SQLite, forget about the language used to implement it; the fact that it runs on multiple architectures, used by multiple products, from multiple different programming languages, deployed to multiple platforms ... means that I have more confidence in the software than in a new project made in a safe language.
> humans can only hear frequencies between 20 Hz and 20 kHz
I can hear up to around 21.5kHz, which I know because I have a 48kHz DAC. However those frequencies are almost always just unnecessary to include in a signal anyway.
Not sure to both. I haven't measured. I used a 1kHz tone to calibrate the volume to a comfortable level and then raised the frequency until I couldn't tell whether the tone was playing or not.
> 16k lines of obscure C, riddled with pointer magic and arcane memory manipulations? No thanks
Given the popularity of the library I'd imagine it's quite well tested. I also fail to understand why he attempts to port libshine from C to Go when he could just port the header files from LAME as FFI bindings instead. That should also ensure comparability with windows, as I'm sure such a library us also buildable on windows, given that ffmpeg uses it and ffmpeg works on windows.
Go programmers usually avoid C bindings if at all possible.
Pure Go can be cross-compiled easily, but that goes out the window once you include C code. You can make it a bit easier by using Zig as your c compiler, but it's still a bit of a pain. If you want to expose your code as a Go package, the situation gets even more complicated.
> The basic encoding/decoding technology for MP3 was under a patent for all of the 90s and 00s. It wasn’t patent-free in the US until 2017. Open-source and hobbyist developers like myself are going to avoid anything involving patents.
Haha, yep. This is partly why there were so many recordings that sounded like ass back in the infancy of music sharing and partly why we cared about bitrate.
We have many more options today. Today's music sounds great at 96Kbps with AAC or Ogg Vorbis encoding, which is a big reason why streaming became, and continues to be, viable.
I was surprised he/she/they/xe/zi/zer/it thought 16K lines of C was too much to comprehend and re-write in Go. It may seem like a lot but if you're going to maintain something as complex as an encoder, it's sorta your job to know exactly how the encoding works. Also, 16K lines isn't too bad for a single person if you dedicate a day or two to it.
74 comments
[ 3.1 ms ] story [ 141 ms ] threadWhat a ridiculous concept. IP needs to die an ugly death.
Many people would like to see the death of the latter kind of IP but not necessarily the “enforced sharing” version that GPL invented.
Sure, these licensing terms are meant to protect against said evils. They would also be unnecessary in a world that discarded the idea of owning data. And they're still ridiculous—especially copyleft.
It honestly isn't, it's actually more complicated than later codecs like LC-AAC for no benefit because of the weird filterbank thing. Though, I would start with ADPCM.
eg: https://hydrogenaud.io/index.php/topic,77128.msg674479.html#...
I'm considering creating an small project that (as a part of a bigger task) needs to be able to play users' sound files. I'd be happy to know that I don't need to support mp3s anymore, but so far I've failed to convince myself that it's a right choice.
I would like to ask, why was this blogpost written? What is the purpose?
https://news.ycombinator.com/item?id=34625573
There are multiple products using LAME on Windows. You've mis-identified the product that isn't portable here.
IOW, LAME works on Windows in products written in C, C++, Python, etc. It's just that this particular Go library doesn't support Windows.
I fail to see how that is a C portability problem, when the C bits already work on Windows.
[EDIT: Looking at the Go LAME project that he references, it appears he didn't even log an issue about whatever it was that went wrong on his Windows build. I'm more inclined to believe that he did something wrong than that LAME doesn't work on Windows using the referenced library.]
[1] For example, Rust `openssl` crate was notably annoying to compile for a long time, even on macOS (it took literally years to make the typical experience smooth enough). And yet it was an essential component for HTTPS! Today we would use the `native-tls` crate instead that avoids OpenSSL on Windows and macOS.
I broadly agree, but in this particular case:
1. LAME compiles just fine using multiple different compilers on Windows (including an older MSVC - newer ones will error out on the use of unsafe functions, fixed by passing a #define in a flag).
2. LAME has no dependencies to download.
3. LAME is successfully used from many other languages on Windows.
So, yeah, if it's not working for him but works for everyone else, you can't blame C for portability here.
OTOH, because LAME is written in C, you have even more options - don't compile it yourself. Download the compiled LAME .dll and just use it from Go.
Author didn't even try the easiest option, said easiest option being possible only because it was written in C.
I think he's trying to show an alternative MP3 encoder to LAME for Go projects. It does seem that he has fallen foul of the Shine license though.
In his own words, he transformed Shine from C to Go mechanically, but he put the result under a license incompatible with Shine's original LGPL license.
(I wonder if he is reading this thread)
The best open source MP3 encoder, which still makes LAME very important. If you need the absolute best MP3 encoder, ABX testing against Fraunhofer’s encoder tells the tale.
https://podcastengineeringschool.com/itunes-uses-fraunhofer/
It's iTunes' AAC encoder that's good.
What's amusing is that speakers may have been optimizing themselves for LAME, since it's pretty much the dominant encoder.
Thank you for sharing.
It had AOL chat coms. Ha.
The author seems to have spent considerable effort on learning as little about it as possible. Most of this article talks about translating C to Go mechanically, without understanding the code. There is even a cameo of the Bullshit Eng—, er, ChatGPT.
In other words, the title is misleading clickbait.
I agree that the article content was a bit of a letdown after I read the article title.
I actually, really, did want to learn about MP3 Encoding.
I swear I'm not trying to sound like an uppity, badass Real Programmer here, but this seems... not too bad? 16 kloc of C for what is apparently the gold standard in lossy audio encoding is actually less than I would have guessed.
No, it does not.
Opus is the most efficient audio codec we've got. (For stereo anyway.)
At high bitrates there might be better tuned ones, but also at high bitrates it doesn't really matter what you use as long as it's not MP3.
Generally when I am grabbing audio off Youtube via NewPipe, I generally use the 50kbs OPUS stream if it is just talking/lectures. I cannot tell the difference. That said I once did a FLAC to 128KBit OPUS test on myself with music convinced I would be able to tell the difference. I could not. I just don't have hearing that is sensitive to codec loss it seems.
To address the grandparent comment, I kinda tend to agree with the OP here. Yes, 16 KLOC is smaller than I would have expected, but that's definitely large enough to be a project that will take some time to grok. And especially for math-heavy c code, less KLOC tends to mean that there's arcane stuff going on. Definitely not out of the realm of approachable, but not a codebase you'll quickly get what's going on without some serious study. And given that the OP's first attempt to convert it to go was to feed it through GPT one function at a time, it may not be something they are ready to modify.
Using ChatGPT one function at a time kinda leads me to believe they aren't super-familiar with c, since pointers alone will definitely not make that approach work. In this case, the OP made the right call by finding a simpler original library to work off of, since production c code isn't necessarily great for learning from. (although I don't like using ChatGPT for a learning tool like this, it adds another layer of abstraction & obfuscation)
Edit: rereading this, it comes off as negative to the OP. Not what I intended, the article was well written and I'm all for exploring new things and documenting them. I actually like writeups like these, since documenting approaching new things is a valuable thing to do, otherwise it makes the discovery part of programming dissappear if the only things that are written up are expert articles.
Do open-source and hobbyist developers really care about patents? I would rather think that's only the case when they plan to commercialize.
[0] https://arstechnica.com/tech-policy/2013/01/patent-trolls-wa...
For others, I do care and go to reasonable lengths to avoid infringing on any.
I think this line sells it short:
It's battle-tested code. Like SQLite, forget about the language used to implement it; the fact that it runs on multiple architectures, used by multiple products, from multiple different programming languages, deployed to multiple platforms ... means that I have more confidence in the software than in a new project made in a safe language.
I can hear up to around 21.5kHz, which I know because I have a 48kHz DAC. However those frequencies are almost always just unnecessary to include in a signal anyway.
Given the popularity of the library I'd imagine it's quite well tested. I also fail to understand why he attempts to port libshine from C to Go when he could just port the header files from LAME as FFI bindings instead. That should also ensure comparability with windows, as I'm sure such a library us also buildable on windows, given that ffmpeg uses it and ffmpeg works on windows.
Pure Go can be cross-compiled easily, but that goes out the window once you include C code. You can make it a bit easier by using Zig as your c compiler, but it's still a bit of a pain. If you want to expose your code as a Go package, the situation gets even more complicated.
Haha, yep. This is partly why there were so many recordings that sounded like ass back in the infancy of music sharing and partly why we cared about bitrate.
We have many more options today. Today's music sounds great at 96Kbps with AAC or Ogg Vorbis encoding, which is a big reason why streaming became, and continues to be, viable.