Motions like they use in the fighter jet game or the shooting game (and some others) may not come naturally to everyone. In fact the one for continuous shooting specifically seems to be counter intuitive.
But the idea of bringing such gesture based interaction to just about any device is really great.
Motions like using a mouse or a keyboard are not a single bit more intuitive for flying planes or shootings guns. It just an issue of familiarisation with the interface as it was with the mouse, as it was the keyboard.
Beside that, it really doesn't matter as these were only two examples of a large variety of possible applications. If it doesn't fit the need - don't use it. There are other interfaces. There is no need for one interface to rule them all but for interfaces that really hits the spot for particular applications.
Regarding Leap: Looks really promising. Though i would prefere if it would be "hidden". Anyhow, can't wait to get my on it. Or over it.
As a concept this is pretty awesome and it removes the need to touch your screen so I guess you can move the sensor closer to a more relaxed place, I can see it getting tiring pretty quickly. Just like a standing desk though I guess you just get used to it over time.
Aside from end use issues, the tech behind this is very nice
you must be thinking of something else. kinect has two cameras, one for laser IR particle-filter type stuff, and then an rgb camera. it isn't stereo vision with two regular 2d cameras, however, doing displacement math. it is looking at the grid of dots projected and seeing where the dots moved in order to create a depth map.
Not to nitpick, but there is a stereo aspect to it. The IR projector is offset laterally from the IR camera, and this is critical to evaluating the depth, because the disparity of the IR dots is more extreme the greater the distance between the emitter and the camera.
It is actually exactly stereo, except one of the cameras is replaced with an IR projector; you basically do the standard stereo math as though the projector is a camera 'seeing' the image that is being projected.
Yeah, my guess is some kind of LIDAR type device and perhaps a mix of a couple of other things. A researcher friend of mine was musing over possible uses of LIDAR a couple of years ago and this was one of his end uses if my memory is correct.
I could of course be wrong, so if someone knows different
I thought LIDAR was prohibitively expensive technology for use in consumer applications, but I could be wrong. I remember reading about it when Radiohead did their "House of Cards" music video with LIDAR tech.
The first thing I thought after watching the video was how much money will they make with pre-orders on this site and how much would they have made with a kickstarter campaign?
I really can't imagine using this for a longer period of time. Maybe as an extension of keyboard and mouse/trackpad that you would use to scroll trough pages when researching something or stuff like that. You still need a keyboard to type as far as I can see.
That being said; I really like the idea and would love to know the tech behind it.
Pretty shitty. It comes across to me like they're trying to give a head-start to developers in the US; Not the best way to welcome the international developer community.
Dev kits go to anyone internationally. Pre-orders are domestic only (for the moment).
And we've got nothing against our northern neighbors, nor is it some grand favoritism conspiracy. Pure rollout logistics. Lots of people are getting confused on this point, we'll announce more shortly.
Hmm I've just pre-ordered two from Japan (and applied to the dev program too).
I believe the form included international locations, can you confirm it?
To me this looks amazing and although LEAP seem to be pushing for you to get rid of your mouse/keyboard, personally I think this is probably best as an addition to it. Imagine if you had one of these built into the keyboard.
You're typing an email need to add a location switch over to google maps, hands off keyboard as you manipulate it around to get a decent resolution, 'tap' the address bar to copy it, swipe left to switch back to the email program, tap again to paste and boom carry on typing.
You wouldn't need to be using it all the time for it to be extremely useful.
We keyboard jockeys sometimes forget how much faster something like this would make the less shortcut-key knowledgeable users!
As much as I would love toying with this, it really all about implementation.
There are a lot of issues to consider - what if I mean to swipe one thing but the system recognizes another? how is that handled? Does it calculate the position of my head and the perspective I'm seeing?
I agree with the above comment that this would ideally be an addition to our growing arsenal of HIDs, including keyboard, mouse, touch mouse, joystick, wacom tablets, and others are not intended to replace one or the other, most of them are complementary to other HIDs.
Imagine surgeons who can't touch a surface with his/her hands due to hygiene concerns using it to manipulate various images/scans of the patient on the table in front of him/her.
Well yes. If multiple solutions gain traction, it just proves that there is a market for this kind of thing. It's not about being first, it's about executing well. If Leap is more accurate for tracking movement, it could prove more suitable than the kinect.
The problem is one of digital vs analogue, and ballistic vs servo. On a keyboard, you either press or do not press a key, and assuming that your finger is aimed somewhere in the right box, you need no visual feedback to see where it is and which thing you're touching. You also get the tactile feedback of "yes I hit the key", "I hit the key but it was at the edge, better recalibrate", etc.
So while there are a basically continuous range of positions you can put your fingers into (and with fifty or so degrees of freedom), the differences between many similar positions are subtle and require feedback for you to see which of them you're in---looping in your visual system and slowing down the interaction considerably. Which is what you want for certain sorts of continuous-ish interactions, and not at all what you want for certain sorts of digital-ish interactions.
All of which is to say, this sounds suuuuuper cool, but it's not going to replace the keyboard.
Agreed. It reminds of when the iPhone introduced people to the idea of an on-screen touch keyboard. Many people said they didn't like it and would keep using their Blackberries. How's that working out for RIM these days?
The point is that if the technology works (which is the important question, IMO), then users will adapt and embrace it.
The touch screen keyboard is still inferior to a hardware keyboard for typing. There are other reasons that companies produce more touch screen only devices and people buy them. Style, cost, simplicity, larger screen etc.
Language is a powerful way to interact with systems. With the mouse, we got first a single primitive - point and click. That vocabulary expanded to drag, but still that's about it. With touch interfaces, we got swipe and pinch thrown in. The kinect expanded that with body gestures. Following that trend, it looks like leap vastly expands our visual interaction vocabulary, or at least has the potential to do that. Considering all the things human hands have made in history, it would indeed be a shame if our computers thought of us as one fingered two dimensional entities!
I can't see why there is so much negativity as to end use applications. I can see lots of potential for this - slideshow presentations, laptops (goodbye annoying trackpad), as well as a stylus-and-tablet replacement for designers, etc. etc.
I think the key however the key will be in the recognition of subtler gestures. If you can show me a man using two hands to type, then moving them not far from the keyboard to activate simple gestures for navigating a document, I'd be really sold that this is for everybody.
I think i'll need to demo this unit before I purchase it. I remember getting burned in the early nineties by the power glove's cool commercial: http://www.youtube.com/watch?v=93iDhnBcMGo
Forget the desktop. With the sensor being this small, I can imagine hanging this from your neck and have gesture sensing anywhere, SixthSense-style. ( http://www.pranavmistry.com/projects/sixthsense/ )
This is very cool technology. The company was formerly "Ocuspec". One break-through is using very inexpensive hardware (<$5 at RadioShack) to get that sub-mm resolution. At the other end of the spectrum, they can cover a football field (and more in the future). What they are showing now is just the beginning. Kudos to rolling out with an SDK. I can inly imagine what sorts of applications developers will dream up.
Not to be cynical, but this reminds me a little of Alan Kay's comment:
"""
By the way, Sketchpad was the first system where it was discovered that the light pen was a very bad input device. The blood runs out of your hand in about 20 seconds, and leaves it numb. And in spite of that it’s been re-invented at least 90 times in the last 25 years.
"""
I think there will be some things it's great at and some it's terrible at. Here's one for free: how about using this to create the Rosetta Stone of sign language? Or a portable sign-to-speech translator? (These are hard problems for other reasons, but this thing brings you a lot closer.)
Think of it like this: Alan Kay isn't wrong, but you could say the same thing about any input. The mouse is a very bad input device. It takes forever to use it for on-screen typing. The keyboard is a very bad input device. It can't tell how hard you're hitting each key when you do musical typing in GarageBand. The microphone is a very bad input device. Voice control is way slower than just clicking the menu item you want ...
If this thing is real, then within a couple of years there will be a dozen reasons to refuse to buy a computer without one.
(I like the idea of a pressure-sensitive keyboard. Could be useful for music. Probably hard to engineer in a way that would be useful and which wouldn't compromise button input. Blue-switch cherry keyboards are annoying to type on. Still, interesting idea.)
You're papering over the problems with relativism.
The keyboard is rapid to use and flexible. Although it has the possibility of physical health problems with RSI, with light pen interaction you have the certainty of pain and fatigue. Sign speakers don't need sign-to-speech - there are keyboards already. There's been multiple attempts at text to speech as well, and the result is more fiddly, less flexible and less rapid than you can get with a keyboard.
There could be an argument that keyboards are complicated and have a learning curve. But computers are the super-tool of our age. Why would you not learn use of a tool that gives you combines great power with flexibility.
I think we still have discussions about alternative user interfaces that hark back to the way humans interact with each other because most of the population are not yet expert keyboard users. This will change. Once the developed world is flush with expert keyboard users, user interfaces will go back to putting greater emphasis on them.
I don't think it's one or the other. Has the mouse replaced the keyboard? It's an additional way to interact.
Hell, I could have used it a little earlier today - I was bleaching my hair and realized my machine wasn't playing any music. Currently my options for remedying this are (1) poking at the media keys on my keyboard (and getting bleach all over them), or (2) picking up my Wacom stylus, going to the Dock to bring up iTunes, and hitting its play button, thus smearing bleach EVERYWHERE. With one of these sitting on my desk and wired into some global hotkeys, I'd have had the additional option of waving my hand in a particular way.
I'm also thinking I totally want to incorporate one of these into the media PC I'm working on based on a Raspberry Pi, a pico projector, and half of a rubber cat[1]. Slouch on the couch, wave my hands in precise gestures to control it instead of having to bring up a mouse somehow.
Using it for long, sustained periods? Nah. Using it now and they? Oh yeah.
[1] the whole thing is disguised as Nyan Cat, with the projector poking out of its snarling mouth and the plaque it's mounted on painted like a Pop-tart.
Painters are able to operate a brush for full working days in the studio without the elbow support that a desk supplies here. It seems if the benefit is great enough people can work up the supporting muscles through repetition.
In fact, if a painter (or user of Leap) were to continually rest their elbow on some support, they could compress the ulnar nerve and cause cubital tunnel syndrome. If you've ever hit your "funny bone," that's the ulnar nerve.
Heck, consider how light a painter's brush is in comparison to a fencer's epee (which only weighs about 1 lb at the most, but still, try holding one for an hour without having any experience!)
These devices should be used with the monitor turned into a drafting-table like angled work surface instead of mounted vertically in the usual way. If you can work touch/resting into the screen, you have a much more humane setup.
One change since that time is that there is way more flexibility in the placement of displays.
This also isn't a light pen. You do not need to hold it close to the display.
Finally, there is way more casual computer interaction. A light pen could be fine for short interactions (especially if you do not have to search for the pen first)
I want to play Modern Warfare 3 (or more likely BLOPS2) with that thing.
I've already started thinking about some gestures that could be used for this, but I'm wondering, how hard it's going to be on the hand(s)? I mean with the mouse and keyboard (supposing PC gaming) the hands are resting on the table 90% of the time, with this the hand(s) will be up in the air.
...unless someone puts a nice glass table on top of that thing so that my hands could rest... could this work?
> also the demo did not show a way to type information
I don't think people in the gesture interface market are looking for ways to replace the essential function of the keyboard. For all intents and purposes, it's probably the best way to input textual data into a machine.
Keyboards are the also the only way ( until now) to input textual data.not implementing that makes this at best a companion device to existing setup and not a replacement.
[edit]. voice to text transcriptions can be an alternative..
If you're an interface designer (I am), you should pre-order this thing. This will be a standard form of interaction in a couple of years and you should jump on it early and start figuring out the kinks. Too cool.
It really won't. Have you any idea how tiring waving your arms around like that would be all day?
There are so many use cases where it doesn't even work that would require a complete rethink of how anything is presented on in internet. For example how about the HN comments where it's pasted as code and it scrolls horizontally. Target that and scroll it with that system. As soon as you have a single use case where a mouse and keyboard is more effective you blow your value.
Hm, maybe. But that was my point with saying interface designers should hop on this: they design the interaction to be simple enough that it doesn't wear you out (at least, that's how I'd look at one facet of designing for this).
Moreover, it's not enough to just copy our current understanding of UI over to a form of input like this. Of course that's the natural inclination, but in reality, interfaces will change and adapt to things like this (meaning scrolling may not exist and a whole new form of pagination may be invented). The "how" is up to the designers.
More than anything, you really articulated my point by saying that "it really won't." You're right: as things stand in terms of interaction, this would become tiring. You just have to think of a way to make it not.
I'm familiar with the usual "arm-waving sucks" arguments against gesture-based inputs, but I was just wondering -- is there any reason this couldn't just replace the touchpad on laptops, maybe being integrated into the forward edge for a larger field of view?
They do claim sub-mm accuracy; maybe applications in the small are realistic.
So instead of arm-waving, think of rotating your hand just above the touchpad to rotate and object in 3d space, but briefly. And the touchpad would still work like a regular touchpad, but maybe you don't even need to touch it.
Sub-mm accuracy seems to imply that really subtle gestures could work.
Not being able to un-touch your pointing device on your laptop would really suck. It'd have to be mixed with a capacitive plate that knows when you touch it.
Agree that it can't provide the precision we need for arbitrary complex actions.
Disagree about the physiological concern. Provided your elbows are resting on the table, it would be easy to get used to. Humans would adapt and it would be healthier than our current much talked about static postures.
Looks like it'd be a nice augment to the usual keyboard & mouse/trackpad setup (& to a touchscreen, too).
For some tasks (e.g. changing to a diff browser tab five across from the current one) I can imagine that pointing at it would be the quickest and easiest way to switch to it.
I can imagine it'd get a bit tiring if you were relying on it too exclusively.
It'll be interesting to see how this pans out - but I can't see it being widely adopted in the home any time soon. People who want gesture based stuff will use Kinect.
However, I can see this being huge in the commercial market. I can easily imagine using something like this in a shop, or for presentations at work.
Googling for "kinect price" gives me numbers of about $150; this is priced at $70. Even if we assume this is a discounted price to lure in pre-orders, that still leaves room for it to cost less.
I would not think of getting a Kinect but my thought upon seeing the video was "I want to get one of these and use it to control the RasPi I'm sticking in some fake taxidermy along with a pico projector for a micro media PC."
I think you're spot on. Every mall in the developed world could have easy to use 3D maps and offer/advert displays - that's a huge market right there, whether or not the desktop ends up being one.
I've been searching all over for something like this as a potential mouse replacement to help with my finger tendinitis. I just pre-ordered. Also an interface designer, so I'm excited to see where this goes.
186 comments
[ 3.9 ms ] story [ 203 ms ] threadBut the idea of bringing such gesture based interaction to just about any device is really great.
Beside that, it really doesn't matter as these were only two examples of a large variety of possible applications. If it doesn't fit the need - don't use it. There are other interfaces. There is no need for one interface to rule them all but for interfaces that really hits the spot for particular applications.
Regarding Leap: Looks really promising. Though i would prefere if it would be "hidden". Anyhow, can't wait to get my on it. Or over it.
Aside from end use issues, the tech behind this is very nice
I guess some kind of infrared thing?
I could of course be wrong, so if someone knows different
You can get pretty impressive accuracy and precision at short distances, and the plane of depth points matches what their demo video shows: http://www.youtube.com/watch?v=_d6KuiuteIA&feature=playe...
That being said; I really like the idea and would love to know the tech behind it.
And we've got nothing against our northern neighbors, nor is it some grand favoritism conspiracy. Pure rollout logistics. Lots of people are getting confused on this point, we'll announce more shortly.
To me this looks amazing and although LEAP seem to be pushing for you to get rid of your mouse/keyboard, personally I think this is probably best as an addition to it. Imagine if you had one of these built into the keyboard.
You're typing an email need to add a location switch over to google maps, hands off keyboard as you manipulate it around to get a decent resolution, 'tap' the address bar to copy it, swipe left to switch back to the email program, tap again to paste and boom carry on typing.
You wouldn't need to be using it all the time for it to be extremely useful.
We keyboard jockeys sometimes forget how much faster something like this would make the less shortcut-key knowledgeable users!
There are a lot of issues to consider - what if I mean to swipe one thing but the system recognizes another? how is that handled? Does it calculate the position of my head and the perspective I'm seeing?
I agree with the above comment that this would ideally be an addition to our growing arsenal of HIDs, including keyboard, mouse, touch mouse, joystick, wacom tablets, and others are not intended to replace one or the other, most of them are complementary to other HIDs.
Now imagine if each position was mapped to a different shortcut...
How many unique positions could you map? (compare to keyboard).
I suspect that unless there is some form of support, this may be like drinking Pepsi: Sweet on the first sip, but after a while, not as great.
See: http://en.wikipedia.org/wiki/Stenotype
So while there are a basically continuous range of positions you can put your fingers into (and with fifty or so degrees of freedom), the differences between many similar positions are subtle and require feedback for you to see which of them you're in---looping in your visual system and slowing down the interaction considerably. Which is what you want for certain sorts of continuous-ish interactions, and not at all what you want for certain sorts of digital-ish interactions.
All of which is to say, this sounds suuuuuper cool, but it's not going to replace the keyboard.
The point is that if the technology works (which is the important question, IMO), then users will adapt and embrace it.
I think the key however the key will be in the recognition of subtler gestures. If you can show me a man using two hands to type, then moving them not far from the keyboard to activate simple gestures for navigating a document, I'd be really sold that this is for everybody.
http://www.microsoft.com/en-us/kinectforwindows/
""" By the way, Sketchpad was the first system where it was discovered that the light pen was a very bad input device. The blood runs out of your hand in about 20 seconds, and leaves it numb. And in spite of that it’s been re-invented at least 90 times in the last 25 years. """
from http://archive.org/details/AlanKeyD1987 around 7:10
Think of it like this: Alan Kay isn't wrong, but you could say the same thing about any input. The mouse is a very bad input device. It takes forever to use it for on-screen typing. The keyboard is a very bad input device. It can't tell how hard you're hitting each key when you do musical typing in GarageBand. The microphone is a very bad input device. Voice control is way slower than just clicking the menu item you want ...
If this thing is real, then within a couple of years there will be a dozen reasons to refuse to buy a computer without one.
You're papering over the problems with relativism.
The keyboard is rapid to use and flexible. Although it has the possibility of physical health problems with RSI, with light pen interaction you have the certainty of pain and fatigue. Sign speakers don't need sign-to-speech - there are keyboards already. There's been multiple attempts at text to speech as well, and the result is more fiddly, less flexible and less rapid than you can get with a keyboard.
There could be an argument that keyboards are complicated and have a learning curve. But computers are the super-tool of our age. Why would you not learn use of a tool that gives you combines great power with flexibility.
I think we still have discussions about alternative user interfaces that hark back to the way humans interact with each other because most of the population are not yet expert keyboard users. This will change. Once the developed world is flush with expert keyboard users, user interfaces will go back to putting greater emphasis on them.
Hell, I could have used it a little earlier today - I was bleaching my hair and realized my machine wasn't playing any music. Currently my options for remedying this are (1) poking at the media keys on my keyboard (and getting bleach all over them), or (2) picking up my Wacom stylus, going to the Dock to bring up iTunes, and hitting its play button, thus smearing bleach EVERYWHERE. With one of these sitting on my desk and wired into some global hotkeys, I'd have had the additional option of waving my hand in a particular way.
I'm also thinking I totally want to incorporate one of these into the media PC I'm working on based on a Raspberry Pi, a pico projector, and half of a rubber cat[1]. Slouch on the couch, wave my hands in precise gestures to control it instead of having to bring up a mouse somehow.
Using it for long, sustained periods? Nah. Using it now and they? Oh yeah.
[1] the whole thing is disguised as Nyan Cat, with the projector poking out of its snarling mouth and the plaque it's mounted on painted like a Pop-tart.
(sample size of one: myself)
Heck, consider how light a painter's brush is in comparison to a fencer's epee (which only weighs about 1 lb at the most, but still, try holding one for an hour without having any experience!)
This also isn't a light pen. You do not need to hold it close to the display.
Finally, there is way more casual computer interaction. A light pen could be fine for short interactions (especially if you do not have to search for the pen first)
I've already started thinking about some gestures that could be used for this, but I'm wondering, how hard it's going to be on the hand(s)? I mean with the mouse and keyboard (supposing PC gaming) the hands are resting on the table 90% of the time, with this the hand(s) will be up in the air.
...unless someone puts a nice glass table on top of that thing so that my hands could rest... could this work?
I don't think people in the gesture interface market are looking for ways to replace the essential function of the keyboard. For all intents and purposes, it's probably the best way to input textual data into a machine.
There are so many use cases where it doesn't even work that would require a complete rethink of how anything is presented on in internet. For example how about the HN comments where it's pasted as code and it scrolls horizontally. Target that and scroll it with that system. As soon as you have a single use case where a mouse and keyboard is more effective you blow your value.
Moreover, it's not enough to just copy our current understanding of UI over to a form of input like this. Of course that's the natural inclination, but in reality, interfaces will change and adapt to things like this (meaning scrolling may not exist and a whole new form of pagination may be invented). The "how" is up to the designers.
More than anything, you really articulated my point by saying that "it really won't." You're right: as things stand in terms of interaction, this would become tiring. You just have to think of a way to make it not.
They do claim sub-mm accuracy; maybe applications in the small are realistic.
So instead of arm-waving, think of rotating your hand just above the touchpad to rotate and object in 3d space, but briefly. And the touchpad would still work like a regular touchpad, but maybe you don't even need to touch it.
Sub-mm accuracy seems to imply that really subtle gestures could work.
Disagree about the physiological concern. Provided your elbows are resting on the table, it would be easy to get used to. Humans would adapt and it would be healthier than our current much talked about static postures.
Funding announcement: http://www.marketwatch.com/story/leap-motion-announces-1275-...
For some tasks (e.g. changing to a diff browser tab five across from the current one) I can imagine that pointing at it would be the quickest and easiest way to switch to it.
I can imagine it'd get a bit tiring if you were relying on it too exclusively.
However, I can see this being huge in the commercial market. I can easily imagine using something like this in a shop, or for presentations at work.
I would not think of getting a Kinect but my thought upon seeing the video was "I want to get one of these and use it to control the RasPi I'm sticking in some fake taxidermy along with a pico projector for a micro media PC."