98 comments

[ 0.16 ms ] story [ 184 ms ] thread
jsmad lead developer here, shoot if you have any questions!
Here too, and please send us a tweet or so if you make something cool with it!
It seems to stutter if you change tabs, is this due to restrictions in Firefox background tabs and timers?

Edit: Awesome stuff, btw :D

The stuttering when in a background tab we think is related to timers yes, but we hope that properly buffering the audio will fix it.
jsnockert's reply to this is dead for some reason:

---

jensnockert 3 minutes ago | link [dead]

The stuttering when in a background tab we think is related to timers yes, but we hope that properly buffering the audio will fix it.

-----

The answer to this is complex, and I've spent the last night tuning the player so that it wouldn'd do that.

There are several scenarios here:

1) Either the MP3 data buffer doesn't fill fast enough (slow network transfer), so when it runs out of buffer, all hell breaks loose. This is because player.js is really naive, so more a demo bug than a jsmad bug.

2) Or Firefox fails to call the scheduled tasks (via setTimeout) for more than the length of a sound buffer, and then the audio device just stops. If the audio device stops updating for more than 500ms, player.js just resets the audio device and tries again.

3) And worst case: something goes really wrong, the rebuffer is called but audio is no longer being played. I have no idea what happens here, I'm laying the blame somewhere between Firefox 4 and audiolib.js (which is awesome btw).

So yeah, apparently browsers are really bad at scheduling CPU and IO-intensive concurrent things. It shows when streaming a large file - been listening to 1h+ mixes, which works, but stutters a lot in the beginning.

Just giving the mozilla devs a good benchmark to optimize for :)

From the Firefox 5RC1 release notes:

> Background tabs have setTimeout and setInterval clamped to 1000ms to improve performance

As far as I'm aware, Chrome does this also but I'm not sure what version it was introduced in.

If we could find a way to increase the audio buffer size in audiolib.js (the refill buffer it's giving us pretty small although we have a big preBufferSize) it'd probably be enough.
(comment deleted)
o/
Hey dude :) Long time no see. As you can see, I've moved on t greener pastures ;)

(I also happen to be the author of http://ooc-lang.org/, and that's where I know gmaster1440 from)

Does anyone have a good handle on what the GPL means in the context of a javascript library invoked by an html page rendered by a browser? I certainly don't!

nb, you say: "libmad has commercial license. As for jsmad, we're in a sort of grey legal area." I presume you mean that you would have problems with a commercial licence? Your work is clearly a derivative work of libmad, so it has to be GPLv2 licensed: any commercial license would have to be by agreement with the libmad developers I think.

Regardless: very cool stuff!

Yes, anything under the GPLv2 is obviously fine. But we cannot relicense it under a commercial/liberal license like libmad could (at least not without their permission).
That's my take too.

The remaining question then is just how far the reach of the GPL is in the context of a javascript library. Just the javascript that actually calls the code? Everything else called from the same html page? Everything else on the same site?

It's a shame the history of libmpg123 is so clouded: they've made a good effort to contact all the previous developers to get permission to LGPL the library, but they haven't been able to reach everyone so there are always going to be lingering doubts.

Yes, we would love to discuss commercial license with the Underbit/libmad guys. So if you're here, hello! Let's talk :)
Did you intentionally take a stab at Atwood's law?
I'll dig up the mathematician in me and answer to a generalized version of this question:

"Are you aware that this is probably one of the most inane use of Javascript and did you do it to piss people off?"

To both questions the answer is: Yes, absolutely.

I think this is the beginning of something beautiful.

On Chrome 13 if you enter a new ID track and hit enter it will keep playing the old song even if you start playing the new one or it will have that crkkkk repeating sound. :) Firefox 4 seems to have no problem but you have to sometimes hit refresh and then play to get the new track playing.

One cool feature would be to make a random ID track generator so you won't have to search/enter the codes for yourself.

Hope this feedback helps somehow.

Did I say how cool all this is?

Thanks for the feedback, jaquis!

Re: Chrome 13, I haven't had a chance to test this myself, but I suspect that dev.kill() doesn't work as expected then :/ if it's a Chrome bug that would really suck cause it wouldn't be the first one.. see http://code.google.com/p/chromium/issues/detail?id=82795 (seriously Google Chrome team, get your stuff straight!)

Re cool feature: actually, we had in mind using the Official.fm Simple API (see http://official.fm/developers/simple/ for more infos) to add a Search feature. As an official.fm dev, though I'm warning you: our search system is undergoing serious refactoring and the current system has lots of... room for improvement, put politely.

Thanks again for your time!

While I'm at it, I forgot to say this, but: if you have MP3s that don't play, please upload them on Mediafire and submit a Github bug report: https://github.com/nddrylliog/jsmad/issues - you'd be surprised how quick we can figure out, after collectively spending 300+ hours debugging the code :)

As a wise Keanu Reeves once said, "Woah."

Is there any upper limit you've found with bitrates, etc.? Also, what was the most difficult part in building this?

320kBit ones works fine (the max for MPEG1/2 streams.)

We did most of the work during and after Music Hackday Berlin (and it was continued at MHD Barcelona) and just staying awake was quite difficult, but porting much of the low-level bitfiddling and pointer-stuff to javascript was hard because of language differences, but also converting a fixed-point library to the floating-point that is available in javascript was a bit of trouble.

Been playing 320kbps Stereo flawlessly afaik. The bitrate is really not much of an issue.

Browsers have a much harder time dealing with binary XHRs, huge typed arrays, huge strings, and a real fucking lot of arithmetic operations.

I think the most difficult part was the debugging. Jens Nockert, Matthias Georgi & I finally used node.js and carefully diff'd the output from minimad.c (the canonical libmad example) and jsmad.

However, it quickly became obvious that comparing exact values wasn't going to cut it. Listening was also almost useless (try to debug when all you have is SCRRRCHHHHHH vs CRRRRRRKKKKKK). So we plotted graphs. With gnuplot. It was a matrix-like experience but quickly allowed us to see what was wrong.

Profiling the code via Firebug made us realize that most of the time (~28%) was spent in the synthetizer (synth.js). The I/O code is particularly expensive too, it's a mix of Uint8Arrays and javascript Strings, depending on whether you're using the local file version (filestream.js) or the http streaming version (ajaxstream.js)

One of the funniest part was when I had to code 36-bit precision integer routines in Javascript such as shifting, OR-ing, AND-ing.. I had to separate the numbers in a low word and a high word and then work with floats (which have a 54-bit mantissa in JS).

Really quite an experience!

*52 Bit.

And I agree, debugging it was a special experience.

I suspect that there's a fairly interesting blog post with samples of the gnuplot output waiting to be written about this.
That's a good idea. I think some of the graphs are still on the depo (in experiments/node/ or something).
If the site isn't loading for you (I was getting nginx errors for a bit before it finally loaded for me), here's the github repo: https://github.com/nddrylliog/jsmad
Yes, we're running this on a node.js+nginx VPS and you guys are MURDERING it.

That's great news though! Bear with us as we try to handle the traffic :) And reload from time to time until it works.

Edit: all static assets are now served by nginx, and the default track is cached on our server instead of being proxied by node.js all the way. Wish we had CORS at official.fm!

We just set another VPS up. Everything should work fin by now.
Doesnt work for me (no audio): Chrome 12 on Ubuntu 10.10 64-bit Youtube is playing fine.

Here's the Chrome debugger log:http://pastebin.com/MwqfD1K5

No, it does not work in Chrome yet, only Firefox 4+ (or Aurora etc). But it is on the roadmap.
Works fine for me on Chrome 13, OS X 10.6.7, however I do have Web Audio enabled in about:flags. Whether or not the Web Audio API's by-passing jsmad, I don't know.
Awesome! Good to know, I'll test it right now.

Edit: I cannot get it to work in Canary, even with Web Audio enabled. Hm...

Did you click the Relaunch Chrome button at the bottom of the about:flags page? I got it working with Chrome on OSX once I did that.
You are right, it does work. Thank you!
Confirmed, it works after restart.
Yes, as said in the README, the audio output doesn't work on Chrome. I have no idea why - audiolib.js should support it. Maybe we're just using it wrong?

If the audiolib.js dev is here or anyone else has a clue, it'd be very welcome.

Mozilla's audio API allows for more control than chrome's latest stable. If you're using the latest snapshot, it should work.
To get it working on Chrome 13+, enable 'Web Audio API' in about:flags, then click relaunch.

It should then work - more or less :)

It'd be neat to have a overview of the different 'modules' in play and what they do (either in the README or a separate wiki page). Great stuff!
Like an overview of all the different parts of the decoder?
We would've loved to have that too when porting the mad code.

Here's a quick overview:

bytestream is our general I/O abstract class - there's much room for improvement because it works with Strings. substream allows to have a 'view' of a slice of a bytestream.

filestream and ajaxstream are two implementations of bytestream, doing local File I/O (with the upcoming W3C File API) and XMLHttpRequest streaming. On Firefox 4.0+, ajaxstream takes advantage of mozBinaryResponse and uses Uint8Array(s) to 1) compute the amount of data read, and 2) implement getU8 (which is used a lot).

bit.js is a straight port of libmad's bit manipulation routines. Bit-level I/O is a must-have in an MP3 decoder because you have numbers stored on 7 bytes, 19 bytes, and other weird stuff like this (especially in the huffman decoding part).

huffman.js, imdct_s.js, rq_table.js are all tables used somewhere else in the process.

layer3.js is really the central part. it implements huffman decoding, the various IMDCT transforms, reordering, requantizing... all sorts of magic tricks I didn't fully grasp even while porting. Comparing the output was really our best weapon. The libmad guys are really the reference guys for this kind of question. We had to strip out all their fixed-point stuff because it's really of no use in Javascript, since JS only has floating point numbers - even if we can make them behave almost as integers, it's still as costly as FP arithmetic so we just went floating point all the way - hence, jsmad doesn't have 24-bit output like libmad has. libmad is wayyyy better at everything :)

synth.js is the second central part - it synthesizes the sound waves from the decoded MPEG Layer-III data. It's the most expensive part of the project, since it's just thousand meaningless arithmetic operations, something JavaScript doesn't excel at.

mad.js contains mostly constants (errors), a few utility functions

decoder.js isn't used anywhere, it's my attempt to port decoder.c from libmad, but it's unfinished.

player.js is our ill-fated attempt to play the decoded MP3 correctly - it's really naive (no seeking, stutter in some cases, no buffering of decoded frames, etc.)

Maybe it's a silly question, but how did you output sound after decoding the stream? I wasn't aware browser JavaScript had any APIs that work with sounds directly. There is Audio, but IIRC it just streams the file from the server.

I've looked at the sources, but couldn't fine the part that would answer my question.

You're talking about the HTML5 <audio> tag.

We're using audiolib.js, which abstracts over Mozilla's Audio Data API and Chrome's Web Audio API - both of which allow direct access to the sound device :)

(And no, it's a really good question! More people should know about this stuff!)

If imitation is the sincerest form of flattery, feature requests must be the next-sincerest. With that out of the way, I'd love to see:

* Standard playback widgets: volume, position slider, etc.

* A way to link songs, e.g. http://jsmad.org/play/114578

Fantastic work!

I just added the link song feature. The link above should now work ;) Enjoy!
For those wondering: romac is too modest. He's this awesome guy I'm building a startup with, and he's been kind enough to let me host the demo on our shared VPS zip.cat, and he's been sysadmining really hard for the last 2+ hours to keep our server from collapsing under the combined weight of the internets :)
That is both amazing and amazingly fast. Thank you!
...and I was working on the volume control feature while we got Slashdotted. Now fulltime on trying to get the server to handle the load. Sorry, will have to wait :(

(edit: typo)

Here's a nice dubstep/electro playlist to thank you guys for all the jsmad testing:

http://jsmad.org/play/176848 | N.E.R.D & Daft Punk - Hypnotize U (Nero Remix)

http://jsmad.org/play/266408 | Nero - Me & You

http://jsmad.org/play/201732 | Cassius - I <3 U So (Skream Remix)

http://jsmad.org/play/265379 | The Streets - Blinded By The Lights (Nero Remix)

http://jsmad.org/play/237774 | Ellie Goulding - Lights (Bassnectar Remix)

There's some awesome music on Official.fm :) I'm so thankful I get paid to develop stuff like jsmad over there.

I'm curious whether you came up with the logo after the fact, or whether you named it after the mag?

I've been surprised at how blatantly some recent open source projects reference pop culture in ways that are just begging for a takedown order - cool.io is another. I suppose it may be worth it if you're primarily focused on getting noticed in the short term (perhaps to get investment or a good job offer), but long term it seems to guarantee that the project will need to go through a rename with all the confusion that entails.

Edit: Forgot to say the technology really is amazing.

The library is named after libmad, which when ported to js became jsmad.

The page linked is just a demo for Music Hackday Barcelona, was not intended to get _this_ much publicity.

Ah, makes perfect sense. Thanks and good luck with the project!
Yeah, this is not the official logo btw - @antoinem is the one to contact about the Demo's visual look.
Really, a takedown order? Under what authority?
Oh, and by the way. Please join #jsmad on freenode (IRC) if you want to help us make the library awesome!
Thanks to all contributed to this clever hack. MP3 on all browsers, without flash, is a really worthy goal.

But I'm hearing sputtering, myself. Are you guys going to carry this to a production-capable system?

Yes, it is stuttering right now. Especially when it is in the background. We think this is a buffering issue, but yes. We hope this will be able to go into production soon(tm).
FWIW, I'd love to see support for icecast or shoutcast streams.
As long as they contain only MP3 files, that would be really really easy to do. If someone does it on Github, please ping me - we'd love to host it on jsmad.org
--------- http://www.soozone.com

http://www.soozone.com We accept pyapal payment. You do not miss it. We need your support and trust!!! free shipping (Jordan wholesale) Dear customers, thank you for your support of our company. Here, there's good news to tell you: The company recently launched a number of new fashion items! ! Fashionableand welcome everyone to come buy. (Jordan wholesale) http://www.soozone.com

Does nothing on Firefox 3.6.x
No, maybe we should add something to the page about not working on oldfox/chrome/ie/etc.
Time to upgrade :) The Audio Data API is brand new.
Awesome! But...

I'm not a JS guy. I can't contextualise this. How does this compare to HTML5 browser-based audio playing? Will this be a new way to make an audio button? Or does it go much deeper than that for JS apps?

From what I hear,the HTML5 <audio> tag implementation quality varies wildly. It hasn't been used much in production for desktop-targeted web apps. It is, however, quite popular on the iPlatforms, because you have no Flash there!

As far as I can tell, jsmad is not going to replace the HTML5 audio tag on mobile devices. However, it does constitute a nice legal workaround for MP3 in desktop browsers such as Firefox.

It gives access to the full decoded audio data, the code doing the playback (player.js) is actually just a tiny little part of jsmad! The HTML5 audio tag seems to also give access to the decoded data, but I haven't messed with it enough to know what's the level of control.

What would be really interesting though, would be something like streaming audio applications - completely flash-free, something that would be (imho) hard to achieve using the HTML5 <audio> tag.

My opinion is that it's wrong to try to add everything as browser built-ins. If the Javascript are powerful enough, why not use it to decode PDF/MP3/whatever? I'm guessing we won't see a video decoder for a few months more, but who knows? Things are moving fast.

So to answer your original question: it goes much deeper. For an audio button it would probably be overkill, but then again, why not?

At 50% cpu utilization this is very impractical outside tech demos.
Yes, please help optimize it.
The question is can it be optimized? My knowledge of this stuff is very limited but I assume javascript object wrappers for number types and lack of SSE/SSE2 instructions will be the biggest blockers to performance.
I doubt SSE2 would help much, at most 2x on arithmetic performance. Much time is spent fiddling with arrays, and the newer version of firefox the better that performance is. In addition there are many optimizations to do.

A Pentium 1 can decode MP3s in assembly, so in Javascript we probably need a few times more power, but not insanely much more.

Switching entirely from Strings to typed arrays would help a great deal imho. Right now the code is hybrid.
(comment deleted)
One of the main problems with the html5 audio tag isn't that it doesn't play audio, it is that it doesn't allow the control needed for things like gaming (playing different streams in different speakers, no latency guarantees, etc).
This is exactly where jsmad could shine, even on current browsers (on FF4/Chrome10+). Game usually use small sound effects, that can be decoded in a few ms and then stored in buffers. From there, it'd be very easy to do the mixing in pure JS using audiolib.js - which is a treat, btw! You'd get as many audio channels as your CPU can handle, mixed in realtime. If the Audio Data API / Web Audio API support more than 2 channels, we might even be looking at full 5.1 demos real soon.

This is so exciting!

I agree! And actually you can already do multichannel stuff, but currently that requires you to use either Audio API exclusively, because Chrome supports only 2 channels for javascript processing (but the internal effects are multi-channel), but Firefox supports only javascript processing.
Would have been great if it worked on mobile Safari!
When it supports the Webkit Audio APIs, it will work. But it might be too slow on a phone. Even on a laptop it is a serious CPU hog.
How does this work? I didn't think JavaScript could read a file on a byte level.
Magnets heh. In JavaScript you have basically two options:

The legacy way is dealing with binary data as an ASCII String - effectively a string of bytes. This works but it's hella slow - probably because browsers didn't think we'd be using 67MB strings :)

The new way is with Typed Arrays: using an Uint8Array is pretty awesome, see the [Mozilla Developer Docs](https://developer.mozilla.org/en/JavaScript_typed_arrays).

Mozilla Firefox since version 4.0 allows to do XMLHttpRequests (xhr) to be retrieved directly as an ArrayBuffer, which you can then create an Uint8Array view on. Since Gecko 6.0 (Firefox 6.0), mozResponseByteArray is deprecated and they've implemented the new standard way: see https://developer.mozilla.org/En/XMLHttpRequest/Using_XMLHtt... for more details.

As for reading local files, the [W3C File API Draft](http://dev.w3.org/2006/webapi/FileAPI/) is implemented in Firefox 4.0 and Chrome 11+ as far as I can tell, and it also allows us to retrieve a binary string.

No, but it can read a string by codepoint. Which is almost the same.
Can this be made to work in Opera?
I don't know, I would guess that some sort of Audio API might be keeping it from running (like in Webkit).
Hmm... 502 Bad Gateway. Can't wait to see this!
Sorry, yes. nddrylliog and romac are doing heroic efforts to keep the servers up. But we got slashdotted.
Firefox 4.0+ AND Chrome 13.0+ are supported.

However, Web Audio API doesn't seem enabled in Chrome 13 by default, just go to 'audio:flags', check "Web Audio", then click the "restart browser" button and it should work.

How did you find working with Firefox's AudioAPI and Chrome's Web Audio API, how do they compare? What drawbacks and advantages did you find working with each of them?
Actually, we're using audiolib.js, which abstracts over the Audio Data API and the Web Audio API, so your question would be better directed to the audiolib.js author :)
Couldn't get it to load in Firefox 4.x, it just says loading. Looks like a neat idea though, I'll give it a second look once the Slashdot load clears off.
First HTML5, now JS? Looks like everyone's trying to make Adobe obsolete.
And I'm sure hoping we succeed! They recently pulled the plug on Adobe Air Linux support ( http://blogs.adobe.com/open/2011/06/focusing-on-the-next-lin... ), which only makes them look worse in the eyes of devs like me.

I don't care if the year of the Linux desktop never comes - you just don't pull the plug one something like that. Adobe is becoming incredibly irrelevant to the consumer, while still strong to the content producer market - which I want to know nothing about anyway. So, all the best :)

Not sure if anyone has mentioned this yet, but I'm noticing speed/pitch differences in playback when I load a file from my disk versus playing in iTunes. Billy Joel sounds more like Janis Joplin.

Edit: Horrible bug report -- I'm in Chrome 12 with the Web Audio about:flag enabled on Mac OS X 1.6.7

Edit 2: Just tested in Chrome Dev 14 and the issue is unchanged.

Please get on this Grooveshark. I run Linux and your client runs so slowly.
When does mp3 playback fall out of patent protection? I know people keep digging up new patents for popular standards but assuming Mozilla and like-minded developers put some effort into patent avoidance you'd think they'd be able to include native support for some subset of mp3 playback without too much patent trouble by now.
Technical question: You're using goto in a do/while loop here :

https://github.com/nddrylliog/jsmad/blob/master/src/decoder....

How are you pulling this off? I ask because while goto is a reserved word in JavaScript I didn't think it did anything.

Good question :) If you watch the resources/network tab under Chrome/Firebug you'll notice that decoder.js isn't used in the demo. It's my unfinished attempt to port libmad's decoder.c, so it still contains a bit of C code, I'm not even sure it's valid Javascript at this point :)

In the demo, we're replicating some of its functionality in https://github.com/nddrylliog/jsmad/blob/master/src/player.j... instead, although in a less robust way.