HN Explorer

20 comments

[ 5.9 ms ] story [ 59.7 ms ] thread
donatj
Starting in the early aughts and extending into the release of the Wiimote there were a handful of projects like this, largely for Mac. Never amounted to much more than a proof of concept most of the time but I always found the concept exciting. I had a lot of hope for Magic Leaps initial stand-alone hand tracking device before they pivoted to AR
NegativeK
At the end of the day, for my computers, my favorite hand gestures all involve the keyboard.
bee_rider
Yeah I keep a webcam pointed at my keyboard too. That way the computer knows I’m subtly flipping it off when I undo in vim.
dosinga
What? Not using LLMs or an expensive cloud based API?
brutal_chaos_
Quite pleasant, eh? I hope we see more projects without them.
anon1566541
> This project utilizes the hand gesture recognition model provided by the Google MediaPipe project.

The rest is just some glue.

thedanbob
Suggestion: middle finger mapped to "run last command with sudo".
ToDougie
I would prefer looking up towards the ceiling combined with a deep breath, followed by muttering a four-letter word.
aidenn0
For those who don't know, this works in many shells:

    sudo !!
bee_rider
In the screenshot they’ve mapped fist to “shutdown now” which seems to either indicate high confidence in the gesture recognition capabilities, or a good sense of humor about the whole thing.
Josh5
Some people want to live life on the edge. Expert mode would be a gesture that nukes your installation
sigmonsays
I love this, I can't tell if it's really true or not

I feel like a few gestures to lock or switch workspaces would be really ideal.

skeledrew
I worked on something very similar to this a couple years ago. My main issue with it why I didn't go beyond the PoC stage was that raising my hands up to the camera soon became a pain (I use a laptop), esp as I have it at eye level. It also had a hard time keeping up with my motions, so most of the time it was getting a blur.
mncharity
I had a overhead-ish camera on a stick, that flopped up from the laptop screen back, to point straight down at the keyboard. Alternately, a phone clip-on fish-eye lens was low-res but ok, iirc. Kludgy little mirrors were iffyer, but there's a commercial one for iPad I didn't try.

Blur wise, perhaps it was 30 fps? I was mostly ok with 60 fps. Would have liked 4k resolution for markers and precision though.

More problematic was integrating high-latency "wait for the camera frame and then for mediapipe" with low-latency "fast keypress response". Drove me towards replayable event streams, and thus custom apps. But I was using it for custom XR-ish apps anyway.

Running high-res mediapipe etal on multiple cameras kept a thinkpad doorstop laptop warm even before starting XR. Keyboard feel got a bit toasted. But TFA doing gestures-only fixed-poses low-res single-camera is far lighter.

skeledrew
It's a pretty old system; had it over 10 years now, so I didn't expect too much re the blur. I'll likely revisit after I've upgraded to something more modern. Maybe by then I'll have the camera position figured as well.

Love to get into the XR space myself.