Mobile Safari has pretty weak HTML5 Audio support. Apple limited it to user generated events, so for example you need to trigger play() from a touch event. In this case it is triggered by a self executing function that runs after onLoad and will not be followed by Mobile Safari. The demo could be updated to offer a start button and then it could work on Mobile Safari. They would also need to not use ogg as that is not supported by Mobile Safari either.
Audio Noob question: I understand 'beat detection' in music to be a really hard problem, are they successfully doing it here in javascript, or is this doing something else?
Beat detection usually work by taking the Fourier transform of the signal, and by observing the changes around 100Hz (the frequency where the kicks are, usually, may need tuning). By observing the changes in frequency, one can 'detect beats'.
The 'hard' part here is to compute the Fourier transform (which can be both a hard maths and performance problem for a person who knows nothing about signal processing), but quite a few implementations exist in javascript already (plus, if the author uses the new typed arrays, it brings a nice speed boost).
The library is leveraging FFT Corban Brook's dsp.js FFT -- while it's not beat detection in the sense that it's able to tell a track is 140 BPM, you can specify a minimum threshold of a frequency's amplitude to register as a beat, and then that decays until the frequency's amplitude surpasses both the minimum and decay rate -- all configurable
Uncaught TypeError: Cannot read property '0' of null, dancer.js:319
All I see are floating particles. The code looks sleek, though, and I'd love to see it working. I had a hacked up WebGL visualizer for a computer graphics class long ago, but it was nowhere near as thoughtfully written.
And of note, while this is using Three.js, it's just using a CanvasRenderer, since FF has problems with loading particles (without a system) in WebGL. Made a ticket for it since this will improve performance as well https://github.com/jsantell/dancer.js/issues/6
Out of the box, no, as this is frequency data rather than time-domain, but could possibly be configurable in Webkit browsers. For an easier solution, maybe something like: https://github.com/katspaugh/wavesurfer.js
Ok, I just tried it on my desktop, and it works perfectly in Chrome. On the Dev Alpha, like I said, I see rotating fuzzy balls, but not the beat synch'd animation like I see on Chrome.
At the top, I see the spectrum scrolling by, with it averaging about 20 FPS. I cannot click on toggle FFT spectrum, it doesn't recognize my tap. Like a limitation with the WebGL implementation.
Both Mozilla and Webkit implemented different audio (Audio Data/Web Audio) APIs -- wonder what BlackBerry Dev Alpha uses if any.. needs a better 'not supported' message
I'm not really up to speed on the differences between browsers, but the Dev Alpha does use Webkit. Unsure what build we have, and what audio APIs are actually supported. The fact that I hear the music and see the particles moving is a start at least.
I'll look into more webkit browsers -- I know on my Nexus S webkit browser, it has awful performance, but the audio plays. If the webkitAudioContext/AudioContext isn't found, it falls back to the Mozilla implementation, which is an extension of the non-fancy HTML5 audio, which means it will play audio, just be unable to read the frequency data from it -- and while the glowy balls move around independent of that data, there'll just be no boom and shaking due to absence of the frequency data :)
34 comments
[ 3.5 ms ] story [ 86.4 ms ] threadI'll have to check it out on my computer when I go back inside.
The 'hard' part here is to compute the Fourier transform (which can be both a hard maths and performance problem for a person who knows nothing about signal processing), but quite a few implementations exist in javascript already (plus, if the author uses the new typed arrays, it brings a nice speed boost).
Hard? Actually it is very easy on rock/dance/pop music.
It's only difficult in classical/jazz pieces.
I'll dig into the code for sure, i just started webgl and this kind of things is perfect to learn.
And of note, while this is using Three.js, it's just using a CanvasRenderer, since FF has problems with loading particles (without a system) in WebGL. Made a ticket for it since this will improve performance as well https://github.com/jsantell/dancer.js/issues/6
The music loads, the particles move, but not really in sync with the music. It just spins...
At the top, I see the spectrum scrolling by, with it averaging about 20 FPS. I cannot click on toggle FFT spectrum, it doesn't recognize my tap. Like a limitation with the WebGL implementation.
Now to build Chromium and try on vanilla Webkit..
If you need me to test/debug, give me a shout. I'd be happy to help.