From the article: "When using AirBending for pitch control, you can lock your gestures to specific musical scales and keys. This ensures every note you play is perfectly in tune with your composition"
Reminds me of the Moog Theremini - that was a fun bit of kit.
Not just a ‘theremin’ — that totally downplays the power of midi.
Someone else mentioned the mimu gloves, and I love the idea of a vision based controller - almost everyone has a phone, tablet or laptop with a camera, especially if theyre making music.
I also love that this could blur the lines of music playing and dance.
The Leap Motion Controller came out in 2014 already (11 years ago, wow!) and isn't very expensive. The SDK was lacking in the beginning if I recall correctly, but a webcam seems to be inferior. Technology isn't the limiting factor for a quite some time now. I'm sure many projects existed to translate gestures to MIDI, some less polished, some more polished[0][1].
Reminds me... I even used two PlayStation Eyes (EUR 5 each) with OpenCV and the EVM algorithm[2] on a ThinkPad X230 for a dance performance piece back in 2015. Movements rather than gestures and OSC instead of MIDI, but it worked great!
This is so fun. I built this into a live performance tool I am using this weekend. Thank you for sharing!
Minor UX feedback
- I find the internal scroll of setting up the presets limiting. Because the whole top half is fixed it is hard to see all my settings at once. Could either the whole panel scroll?
- In dark mode, the "Save" button is black on dark grey and looks disabled.
- The delete button is scarily prominent and no confimation. Can that be moved inside the preset panel so i dont accidently hit it while performing?
17 comments
[ 2.2 ms ] story [ 36.4 ms ] threadHere's someone playing a theremin who's reasonably good.[1]
[1] https://www.youtube.com/watch?v=K6KbEnGnymk
Using the gloves during an NPR Tiny Desk concert: https://www.youtube.com/watch?v=3QtklTXbKUQ&t=555s
Reminds me of the Moog Theremini - that was a fun bit of kit.
https://en.wikipedia.org/wiki/Theremini
I also love that this could blur the lines of music playing and dance.
Great job OP, thanks for sharing.
Reminds me... I even used two PlayStation Eyes (EUR 5 each) with OpenCV and the EVM algorithm[2] on a ThinkPad X230 for a dance performance piece back in 2015. Movements rather than gestures and OSC instead of MIDI, but it worked great!
[0]: https://midipaw.com/
[1]: https://uwyn.com/geco/
[2]: https://people.csail.mit.edu/mrub/evm/
This is not a theremin, it’s a programmable controller which you can tie to various midi inputs. Think “slider” or “dial”.
Do you have any suggestion for how to learn how to hook this up to Logic for anyone who hasn’t used midi before?
I wonder how this would perform under live stage lighting conditions, i.e coloured strong lights and high contrast.
I seem to only be able to map each axis to a single finger?
It seems like if it is detecting two fingers at a time I should be able to use the X and Y on each of them simultaneously?
(Prompt: "Make an in browser midi theremin using Mediapipe")
There's a Web MIDI API, so might be able to reproduce all the functionality in a way that works across devices and operating systems.
Minor UX feedback
- I find the internal scroll of setting up the presets limiting. Because the whole top half is fixed it is hard to see all my settings at once. Could either the whole panel scroll?
- In dark mode, the "Save" button is black on dark grey and looks disabled.
- The delete button is scarily prominent and no confimation. Can that be moved inside the preset panel so i dont accidently hit it while performing?