The first time I saw it, I immediately had an idea to play a drumkit and whistle a bassline at the same time, with another person on guitar and vocals, making a full band with two people. But then I was too lazy to try it.
There been some people doing the whole "one man band" thing with a back pack drum, guitar and harmonica. I bet someone crazy enough could do one instrument per limb + harmonica and maybe even a theremin with some remaining appendage too.
In live music, the music itself is not the focus. You don't need to play very well or play very complex stuff. The character you present is much more important: you've got to give people an excuse to worship your personality. And, well, blowing into a harmonica while carrying a backpack full of cutlery won't make you look like Jim Morrison to anyone. But two people with drums+whistling and guitar+vocals can be made to look graceful, I think.
In case you haven't seen it, Imitone's patent is US20170098112A1 (https://patents.google.com/patent/US20170098112A1/en). This one's from 2017 and is still active, and it's for a periodicity analysis system. Hopefully it's not directly relevant to your synth.
That patent is about a particular way of implementing a low-latency pitch tracking algorithm. My tool uses a different pitch tracking algorithm based on zero crossings. It's much simpler and far from novel.
A big difference is that Imitone works well with singing, whereas my algorithm only does a good job with whistling. On the other hand, my algorithm is lower latency when given a whistle as input.
I wouldn't expect MIDI to be a good fit, and when I experimented with it a few years ago I wasn't able to get anything I liked. Pitch and volume are continuous, and while this can be encoded in MIDI it's awkward. If you send too many updates you can overwhelm the synth, and if you slide a single note too far it will re-trigger the note. Also when the sound starts you know "there's whistling" before you have a great estimate of the pitch, but MIDI wants to know the initial tone (and many voices have tuned onsets).
This is the sort of thing that’s gonna accidentally activate some old forgotten MKULTRA code-key in some poor fella and bang!, they’ve Manchurian Candidated Sonic the Hedgehog
Reminds me of this thing where I was trying to make a three-formant speech synthesizer, like a really simplified DECtalk, track my own voice: https://x.com/me_irl/status/2075472573627093145
21 comments
[ 0.29 ms ] story [ 22.9 ms ] threadFYI, Imitone has a patent, I don't know what it covers or claims though.
A big difference is that Imitone works well with singing, whereas my algorithm only does a good job with whistling. On the other hand, my algorithm is lower latency when given a whistle as input.
I often find my improvised melodies are more interesting when I whistle them, so this would be a handy feature.
But if you want to try it, checking out https://github.com/jeffkaufman/whistle-synth and asking Claude or Codex to add MIDI support would probably work!