Probably less useful than a google search, which shows explicit intent. If you followed me around all day long you’d have an overwhelming amount of noise.
If the gaze tracking resolution is reasonably decent, these really extend the utility of Tobii products for neuroscience/psych research (where i've used previous Tobii products before), but also really does open the door for things like gaze-driven HCI for people with very limited mobility. I've prototyped gaze-based T9 input and the like before, and it's probably doable.
TVs are massive, so they don't have to be this small and you can already buy TVs with built in cameras, since it can be used for video conferencing. Also, this type of tech can already be used to measure ad awareness in stores [0]. It's also starting to be implemented in cars, to check whether you're paying attention to the road while using 'autonomous' systems.
So... This is not really a type of tech that will enable your fears, it already exists, there just doesn't seem a business model for applying it in TVs yet.
Tobii can say whatever it wants. The license you accept only applies to their software, not to their hardware. You own that. You can use their gaming devices to do research all you want, as long as you use your own software with them.
“The license you accept only applies to their software, not to their hardware.”
As far as I know, their EULA has three groups: gamers, commercial, research.
You can buy a 3000€ research license that simply allows you to collect the eye tracking data on the basic 250€ devices…
Tobii can sue whoever they want, too. You sure you can afford better representation than theirs? If they're as assertive about their license terms as it sounds like they are, I'd think hard about it first, I suppose.
Does anyone know of a good example of eye-tracking being used for HCI where it surpasses keyboard/mouse?
When searching for it most of the use seems to be in foveated rendering, research, marketing or accessibility.
Something like this? Fast, but still a way to go before it's faster.
[Hands-Free Coding: How I develop software using dictation and eye-tracking]
https://news.ycombinator.com/item?id=24846887
"I'd say I probably work at about 50% of my normal speed*. Now, this doesn't mean that I produce 50% of the results; it just means I need to prioritize a little more ruthlessly."
I once gave Marvin Minsky a headache by showing a video tape of bright blinking PostScript graphics! I apologized, and we had a nice conversation about how eye tracking interfaces tend to do the same thing.
>Zero Bandwidth Video: SCAN 1981. Zero Bandwidth Video: SCAN is an excerpt from demos of Architecture Machine Group project circa 1980 an interactive portrait of Raleigh Perkins with various input devices to manipulate the image. The viewer pushes a joystick to make her look in various directions, Nicholas Negroponte demonstrates linking the portrait to an eye tracking device. SCAN was exhibited at SIGGRAPH Art Show 1989.
>The Incomplete Tyranny of Dynamic Stimuli: Gaze Similarity Predicts Response Similarity in Screen‐Captured Instructional Videos
>Although eye tracking has been used extensively to assess cognitions for static stimuli, recent research suggests that the link between gaze and cognition may be more tenuous for dynamic stimuli such as videos. Part of the difficulty in convincingly linking gaze with cognition is that in dynamic stimuli, gaze position is strongly influenced by exogenous cues such as object motion. However, tests of the gaze‐cognition link in dynamic stimuli have been done on only a limited range of stimuli often characterized by highly organized motion. Also, analyses of cognitive contrasts between participants have been mostly been limited to categorical contrasts among small numbers of participants that may have limited the power to observe more subtle influences. We, therefore, tested for cognitive influences on gaze for screen‐captured instructional videos, the contents of which participants were tested on. Between‐participant scanpath similarity predicted between‐participant similarity in responses on test questions, but with imperfect consistency across videos. We also observed that basic gaze parameters and measures of attention to centers of interest only inconsistently predicted learning, and that correlations between gaze and centers of interest defined by other‐participant gaze and cursor movement did not predict learning. It, therefore, appears that the search for eye movement indices of cognition during dynamic naturalistic stimuli may be fruitful, but we also agree that the tyranny of dynamic stimuli is real, and that links between eye movements and cognition are highly dependent on task and stimulus properties.
HDMI out? Makes me curious what tech they have on the glasses vs in the dongle/box… maybe it’s convenient to use hdmi cable’s twisted pairs to just take the csi/mipi from the cameras straight out and then do processing externally?
These aren't consumer products. They're specialized business products aimed at market researchers. In that context, €5,900 doesn't seem expensive at all to me, and I'd expect the Tobii glasses to be more.
Which proves my point exactly. The Tobii Eye Tracker V is a consumer-oriented product marketed to gamers. The market-research-oriented use cases that one would need for a "walking around pair of eye-tracking glasses" are totally different, and they're priced accordingly.
We're building one that's only 500 euros - kexxu.com
It uses a convolutional neural network to detect the pupil location, which is a lot cheaper in addition to running real time (for controlling IoT devices around the house). The website shows the 3d printed prototype, injection molded mass production coming in about two months, probl also on kickstarter. We'll open source our libraries in Python, JavaScript, C++, Arduino, Dart (Flutter) to connect to it with MQTT to all DIY projects that can benefit from eye control. In addition to gaze location, the scene camera can detect ARUCO markers and QR-codes. The battery lasts about 10hrs.
Email me at jschreuder@kexxu.com if you want early access.
+ 1 using their devices since a long time and know the founders.
It's stable and nice, yes the hardware of tobii is a bit better, yet the extensibility of tobii is terrible.
Pupil Labs have their code on github (you can also use it with none pupillabs cameras :). It's python and easy to change /extend.
here's the repo:
https://github.com/Pupil-Labs/Pupil
Check also the branches and who forked it ... there are a lot of experimental eyetracking and other algs implemented.
We already used them for a lot of research projects (implementing smooth pursuit interactions and similar).
Edit: added the github repo of the desktop software.
Has anyone tried to use these for a home flight simulator application? It seems like these or the pupil-labs equivalent could deeply improve immersion vs. TrackIR.
More immersive social VR. Personally really looking forward to this because "the eyes are a mirror to the soul" is very apparent when you see well made avatars with eye tracking.
The possibility of dynamic foveated rendering, which is probably how Valve could realistically use an APU in their rumored Deckard.
And of course intrusive ads and data collection.
Morbidly curious to see how long it takes before platforms start using ML algorithms to create personalized ads that are tailored to make you look at them for as long as possible, one would hope such a thing would kill a platform.
Does that account for the fact that it's decoupled from visible light? Afaik the iris contracts in response to visible light, so here could get exposed to more IR proportionally. (OTOH either way in absolute terms the amount you get from sunlight is orders of magnitude more than you'd get from a few LEDs.)
Generally with near-infrared emitters, the main issue is tissue heating up. Of course this depends on the tissue, the specific wavelength(s), and the angular distribution of the emitter.
OSRAM has an excellent datasheet about this, AN090 . If you're curious about how this kind of thing is safely engineered, have a look!
“The most common eye disease associated with near-infrared radiation is cataracts. Prolonged exposure to IR radiation causes a gradual but irreversible opacity of the lens. Other forms of damage to the eye from IR exposure include scotoma, which is a loss of vision due to the damage to the retina. Even low-level IR absorption can cause symptoms such as redness of the eye, swelling, or hemorrhaging.”
I'm not sure about the energy and wavelength difference compared to the glasses.
But even if it has a positive effect from short-term exposure, it may also have bad effect from too much exposure.
Out of curiosity, what is the gaming use case for eyetracking? I'm in research, and we use Tobii eyetracking during experiments, but I never thought about its use in games.
In flight simulators or driving games, it gives more range of motion by tracking head position and gaze. The really good implementations allow you to do more head tracking, or more eye tracking, depending on your preference. FS2020 needed some tweaking in this department, when I turned my head to look behind me IRL, the game camera would spin like mad. The responsiveness is pretty shocking to say the least.
In shooters, there's an added "snap to aim" feature that is borderline cheating, it snaps your aim to where you may be looking. I've only seen it in non-competitive and first person games though. For competitive shooters, the tracking can be very useful to know you weren't looking in the right place at the right time, if you care about that stuff. There is an overlay you can enable which shows where your gaze is calculated in real time.
In third person games, gaze can be used to reveal information in the HUD (hide HUD elements unless I look at them) and to aim throwable objects. More gimmicky in these cases, but simply having more field of view without having to move your mouse/arm is useful IMO.
Anyone know the status of Google Glass for consumers? Last I heard they nixed the consumer version because of poor market reception (lots of people started calling users "glassholes" and banning them from bathrooms and stuff because they never knew when they were recording), but surely at some point they'll be cheaper and more available?
66 comments
[ 4.0 ms ] story [ 131 ms ] threadhttps://www.youtube.com/watch?v=uSQApGLbgNg
So... This is not really a type of tech that will enable your fears, it already exists, there just doesn't seem a business model for applying it in TVs yet.
0: https://www.unravelresearch.com/en/blog/eye-tracking-in-adve...
“The license you accept only applies to their software, not to their hardware.”
As far as I know, their EULA has three groups: gamers, commercial, research. You can buy a 3000€ research license that simply allows you to collect the eye tracking data on the basic 250€ devices…
"I'd say I probably work at about 50% of my normal speed*. Now, this doesn't mean that I produce 50% of the results; it just means I need to prioritize a little more ruthlessly."
https://web.media.mit.edu/~minsky/papers/Telepresence.html
>Telepresence, by Marvin Minsky, Omni Magazine, June 1980
https://www.youtube.com/watch?v=cyU79dFbH0Q
>Zero Bandwidth Video: SCAN 1981. Zero Bandwidth Video: SCAN is an excerpt from demos of Architecture Machine Group project circa 1980 an interactive portrait of Raleigh Perkins with various input devices to manipulate the image. The viewer pushes a joystick to make her look in various directions, Nicholas Negroponte demonstrates linking the portrait to an eye tracking device. SCAN was exhibited at SIGGRAPH Art Show 1989.
https://www.researchgate.net/publication/352739339_The_Incom...
>The Incomplete Tyranny of Dynamic Stimuli: Gaze Similarity Predicts Response Similarity in Screen‐Captured Instructional Videos
>Although eye tracking has been used extensively to assess cognitions for static stimuli, recent research suggests that the link between gaze and cognition may be more tenuous for dynamic stimuli such as videos. Part of the difficulty in convincingly linking gaze with cognition is that in dynamic stimuli, gaze position is strongly influenced by exogenous cues such as object motion. However, tests of the gaze‐cognition link in dynamic stimuli have been done on only a limited range of stimuli often characterized by highly organized motion. Also, analyses of cognitive contrasts between participants have been mostly been limited to categorical contrasts among small numbers of participants that may have limited the power to observe more subtle influences. We, therefore, tested for cognitive influences on gaze for screen‐captured instructional videos, the contents of which participants were tested on. Between‐participant scanpath similarity predicted between‐participant similarity in responses on test questions, but with imperfect consistency across videos. We also observed that basic gaze parameters and measures of attention to centers of interest only inconsistently predicted learning, and that correlations between gaze and centers of interest defined by other‐participant gaze and cursor movement did not predict learning. It, therefore, appears that the search for eye movement indices of cognition during dynamic naturalistic stimuli may be fruitful, but we also agree that the tyranny of dynamic stimuli is real, and that links between eye movements and cognition are highly dependent on task and stimulus properties.
[0]: https://pupil-labs.com/
How much are the Tobii glasses if Pupil Labs' are almost 6 thousand Euro?!
It uses a convolutional neural network to detect the pupil location, which is a lot cheaper in addition to running real time (for controlling IoT devices around the house). The website shows the 3d printed prototype, injection molded mass production coming in about two months, probl also on kickstarter. We'll open source our libraries in Python, JavaScript, C++, Arduino, Dart (Flutter) to connect to it with MQTT to all DIY projects that can benefit from eye control. In addition to gaze location, the scene camera can detect ARUCO markers and QR-codes. The battery lasts about 10hrs.
Email me at jschreuder@kexxu.com if you want early access.
Pupil Labs have their code on github (you can also use it with none pupillabs cameras :). It's python and easy to change /extend. here's the repo: https://github.com/Pupil-Labs/Pupil
Check also the branches and who forked it ... there are a lot of experimental eyetracking and other algs implemented.
We already used them for a lot of research projects (implementing smooth pursuit interactions and similar).
Edit: added the github repo of the desktop software.
The possibility of dynamic foveated rendering, which is probably how Valve could realistically use an APU in their rumored Deckard.
And of course intrusive ads and data collection.
Morbidly curious to see how long it takes before platforms start using ML algorithms to create personalized ads that are tailored to make you look at them for as long as possible, one would hope such a thing would kill a platform.
Honest question: is there any reason to be concerned about such near and constant IR illumination directed at one's eyes?
OSRAM has an excellent datasheet about this, AN090 . If you're curious about how this kind of thing is safely engineered, have a look!
https://ehs.lbl.gov/resource/documents/radiation-protection/...
https://www.newscientist.com/article/2298723-red-light-thera...
I'm not sure about the energy and wavelength difference compared to the glasses. But even if it has a positive effect from short-term exposure, it may also have bad effect from too much exposure.
In shooters, there's an added "snap to aim" feature that is borderline cheating, it snaps your aim to where you may be looking. I've only seen it in non-competitive and first person games though. For competitive shooters, the tracking can be very useful to know you weren't looking in the right place at the right time, if you care about that stuff. There is an overlay you can enable which shows where your gaze is calculated in real time.
In third person games, gaze can be used to reveal information in the HUD (hide HUD elements unless I look at them) and to aim throwable objects. More gimmicky in these cases, but simply having more field of view without having to move your mouse/arm is useful IMO.
For those of us with RSI’s like mouse-shoulder it would be a godsend.