There are a bunch of hardware out there to get gaze vectors, the problem is that they rely on a generalised model of the eye. With fine tuning you can go from >5 degree of error, to <1-2
Also donning the AVP headset might not be possible for someone with advanced ALS. A fixed outside-in apparatus (probably attached to the patient’s wheelchair) would make more sense.
Except Apple has stated they will encrypt that information and not supply it to apps to avoid targeting or fingerprinting. And second how would that help someone with ALS?
> And second how would that help someone with ALS?
They won't share information on where you look, but they will share info on where you 'click', which is used to navigate apps. IIRC you are supposed to use your fingers to do this, and other actions, but I imagine that Apple's accessibility team will have an alternate mode for people with motor limitations. It could be a long-blink, or rapid blinking, for example.
Is the lack of mentioning Apple deliberate? It seems like they've already poured a lot of R&D into this for the Vision Pro, which might be exactly the kind of thing the friend needs.
I used to work on it and have spent tons of time in the headset. The eye tracking is next-level and it's really the only platform that exists with eye tracking as a primary input method. I'm pretty confident it will greatly improve your friend's quality of life.
Because of that, I'm also sure that eye tracking will go mainstream in other areas once the Vision Pro is released once everyone else catches on to it as a great input method.
Doesn't it rely on external cameras to see users hands and use that as the "click" inputs? Seems like that negates usage for ALS cases.
Also, I'm not an ALS expert, but if the only muscular control is in the eyes, then lack of control in the head/neck probably breaks some assumptions about how the vision headset works (just a guess though).
It does not require using one's hands to click, it supports various input hardware (keyboard, mouse, switch, etc.). If someone has control of basically any muscle, it can use a switch input. The Vision Pro also has Dwell Control, activating things by keeping your gaze on it long enough, but I don't know whether it can currently be solely operated using nothing but one's eyes.
The level of eye tracking performance for general population interactions is really only possible when you control the illumination like in a VR headset. A Vision Pro might work for the friend in question. More generally this requires the full vr display to make it work. See through AR or just plane glasses will not be nearly as good, and I think that will cap the general acceptance.
This is pretty much exactly why I vehemently disagree with Apple's decision to draw such a firm line in the sand preventing devs from accessing the eye/gaze data directly. I'm part of an academic spin-off start-up that specializes in analyzing gaze and movement data. Locking the gaze information outside of the app sandbox severely hampers the ability to quickly iterate design and UI patterns that could be game changing for accessibility. Hopefully they make accommodations moving forward for these circumstances.
The issue is doubly close to my heart because my father has ALS and is nearly at the point where eye-tracking will be his only means of communicating effectively with the world. While existing Tobii systems work well enough, typing with your eyes is still exhausting to do.
Ultimately I don't think a platform like the vision pro is suitable for ALS patients, especially later term. They cannot support the weight of the headset and/or fatigue will set in rapidly. Many (including my father) also require use of a ventilator, accompanied with a mask that can seal effectively enough to support the positive pressure necessary to inflate their lungs. Unless the form factor for HMD's minimalizes significantly, it will likely interfere with the respirator's efficacy.
I don’t know much about those medical conditions, but it doesn’t take that much imagination to understand that access to eye-gaze data would pretty much give developers mind-reading abilities against whoever is wearing the headset. As the platform matures it will probably be a whole discussion around how it works, who gets access to it and for what reason. I could imagine Apple putting their weight behind developing all sorts of wild disability features.
Developers control what goes in front of the user and where, we'll still be able to tell plenty about a user's decision making process given that and how their head and hands navigate the space. There are plenty of companies that specialize in this as their entire product offering, assessing fitness for duty, alertness, attention mapping, etc. Plenty of published research on the matter as well.
The supposed security of blackboxing the eye data itself is illusory and functionally just for marketing.
Is it doing something fundamentally different from what everyone else is doing (infrared light source, do some flavour of pupil segmentation and pose estimation)?
Is the eye-tracking performance/accuracy step change on Apple's headset purely just a software/algo change? Or is it actually using a new principle/apparatus for eye-tracking?
I don't think there's anything revolutionary, just a lot of parts working very well in tandem:
- Multiple cameras per eye, and at a very short distance from your eye
- The screen is fixed relative to the cameras for all devices, there's no worry about that half of the equation getting off calibration or differing for every customer
- OS is built around eye tracking, which means there won't be any actions that are unnaturally hard to perform with eye tracking
Agreed. It would be great if the hardware was more affordable/accessible as well. That's potentially a barrier to entry worth addressing for devs who might otherwise be interested in tackling the problem, but don't have the quality eye tracking hardware to start. A Tobii-like hardware devkit could be a starting point.
Not just multiple companies, multiple approaches. Eye tracking is exhausting, and that's pretty fundamental to the modality - dwell time requires significant control to be usable as a click, and even able bodied people find it exhausting. Some folks have tried doing eye tracking and using something else (EMG for example) as the click, but it doesn't work consistently for the population. ALS is also progressive, and people lose their eye control. Blackrock Neurotech has been working on a brain implant, with spinal cord injury as a first target population (because they're less fragile, among other reasons), and it works for current research patients, but medical devices take a lot of time, money and work to get cleared in the US. The implant itself is cleared, but the FDA wants the entire system to be cleared too.
1) its eye tracking isn't good enough for this kind of application.
2) direct access to the gaze vector is disabled
3) its really intrusive
4) its heavy.
5) it doesn't exist(in consumer world) yet.
The goal is to enable someone who has motor control issues, be able to communicate directly with the outside world. Shoving a heavy skimask that totally obscures the outside world on their face directly stops that.
Not only that, but you'll need to create and keep up to date the software needed to make a communicator. Apple are many thing, but it's new platforms are not stable, rapid os updates will break things.
Is anyone speedrunning this? I can't find anything with a quick look.
I'd consider an approach like the human powered helicopter parable.
I'd create a model for the eyes / face. So a screen like a phone is put in front of a camera with a model face and controllable eyes that you use software to control. Maybe skip the screen in front of a camera and link straight into the video feed.
It knows the limits of the eyes (different models for different people and situations) can measure fatigue etc.
That's an interesting idea. How do you see it being beneficial to machine learning models, other than (I assume) it could work more efficiently within less foveal regions? Perhaps cases where you want the vision to emulate human vision?
Running a fast network on sparse data then calling one optimized for a task on a subset seems like a good optimization and dealing with video we are probably going to need them.
After you parse what an object is, tracking it doesn't take anywhere near the effort of original segmentation. No need to re-evaluate until something changes.
Maybe even use activations to turn on and off networks. "Oh text better load ocr into memory"
And it does inform a lot of our built world
It's strange to think that when watching a movie only 10% is in focus.
Eye movement does provide a lot of information to other people and I think the physical movement produces feedback for velocities and things too. Mimicking biology is often a good bet.
Aw, that's nice of pg to want something better for his friend. As cynical as we are about technology, new developments can be so fantastic for accessibility and better quality of life.
And if that were where the story ended we'd have an honest feel-good going. New developments -could- be fantastic for accessibility and QoL, but without exception they just end up getting sucked into a marketing surveillance suite.
Lithium extraction is also very bad for the environment. Requires insane amounts of water, requires often times destroying forests or other natural areas that are on top of lithium, etc.
Sure I use lithium ion batteries too, in the developed country I live in, I can't go and live in a cave. But I won't buy a SUV that weighs 2 tons to sit alone in it most of the time and have it parked 95% of the time in a garage.
The "what have you done" attack is a bit too easy and, really, this is not about me.
We should strive to be sober, to buy repairable goods whenever possible, to educate ourselves on environmental impact of things, and perhaps more importantly to teach this to our children.
This thread is about eye tracking. You went from talking about the congo for some reason to now shifting to "lithium takes a lot of water".
But I won't buy a SUV that weighs 2 tons to sit alone in it most of the time and have it parked 95% of the time in a garage.
What are you even talking about? This whole thread is about eye tracking.
We should strive to be sober, to buy repairable goods whenever possible, to educate ourselves on environmental impact of things, and perhaps more importantly to teach this to our children
It's all related. High tech has a lot of negative impacts which are rarely if ever evaluated and more importantly compensated. The reason is they bring in money.
The entire supply chain and lifecycle of a product should be evaluated to understand if it is a net positive or a net negative. Not only the finished product: "wow eye tracking how cool let's deploy everywhere regardless of how the materials for that tech are mined and how the device will be recycled once obsolete".
To be clear, in your mind when someone talks about wanting to fund certain types of software, it is someone's duty to talk about lithium batteries using lots of water?
It seems you want to keep asking the same questions over and over phrased differently and in a way that tries to discredit me or make me sound dumb, so forgive me but this discussion has now reached its end.
Someone should do this, but for the love of god, DO NOT take any venture capital to do it. No matter how well-intentioned the VCs are at the start, eventually your eye tracking startup will 100% be used for advertising as investors in your Series D need to make an exit or take you public.
As we are talking about a company to help people with medical problems.
Second Sight was giving patients artificial eyes. When they ran out of funding, they closed shop. The patients lost support system for their eyes. If anything goes wrong with their artificial eye, there is no one to repair or fix it. They just have to carry a piece of useless metal junk in their head.
Not really a huge PG fan, but this is what billionaires should be doing: see where a need is exists and put some of your insane wealth towards making an improvement. This why I respect Elon even though I don’t really like him; he puts his money to use, in a very public manner.
I'm not arguing that Elon doesn't put his money to use publicly, but I'm interested how you think he does. Do you mean he uses his wealth privately in a way that benefits the public, and/or he does it in public view?
I fail to see how Tesla fixes an "important problem". They're big electric SUV with huge batteries that require tons of mining and destroy the environment. They're largely non repairable. The full self driving feature seems to be vapor ware.
The space stuff looked interesting until he started sending all kinds of things into space to become an ISP, polluting space the same way Tesla pollutes the earth.
The real issues are still here: poverty, lack of access to clean water, etc. How does Musk fix that?
This had nothing to do with conservative talking points. It has to do with current capitalism always looking for money but letting the real societal issues ongoing because fixing them doesn't pay enough dividends.
Simplified, with current capitalism, doing the right thing doesn't pay off. We have more than enough know-how and technology to fix real issues but we'd rather put rich people in overweight electric cars.
Yes, full self driving could help save lifes. Space exploration helps us better understand our world. But often times, current capitalism encourages the use of technology and know-how for things that benefit a minority at the expense of the most vulnerable.
It’s not Musk’s responsibility to fix societal problems, it’s the government’s. He sucks as a person but the companies he’s in charge of are quite useful and I’m happy they exist. I’m watching my government fail every single day at addressing basic issues like homelessness despite near infinite money. They would fail at fast satellite internet and electric cars too.
I don’t entirely disagree with your point of view, I just question what the hell we pay taxes for if we (the people) are still responsible for all the societal issues. If I have to dedicate a portion of my day’s effort to chipping away at these things then why is the government entitled to a significant portion of my money? It doesn’t make sense to me.
YC was investing in ways the traditional VCs weren’t when it started, and coding HN was a part of it. I doubt I’m the only one who had having a few HN tech support emails from PG.
Yes, and? There is no plausible near future where anyone else (except maybe his heirs) directs how that wealth is used. Better that it go towards socially useful investments than megayachts or land hoarding.
Another route might also be sub-vocalization[1], like TTS for your thoughts. I recently picked up some cheap toys to get started trying to emulate the results[2].
I guess I'm just interested in more information. Do people who aren't NASA have this working? I am just learning this sci-fi feature from Ender's universe is possibly a reality, and if I can check it out, I want to.
TBH, other than the NASA resources, I'm building it from scratch. Goal would be to record the signal while prompting a user to say/think a subset of words. Afterwards the HOPE is that I will be able to train a simple- functional- model based on those inputs.
I agree! My PhD thesis is on this topic [1]. We’ve also done a very limited pilot test on a patient with ALS, with above random chance. Actual results may vary heavily on individual disease progression—the more motor recruitment that’s intact, the better.
EEG recording is an alternative that would outlast the potential disease-related degradation of eye movements. Manny Donchin gave a brown bag at UIUC about the possibilities of using this approach to support communication by ALS patients many years ago. It's clever: they use the P300 marker to index attention/intention. I do not recall whether he and his colleagues ever commercialized the tech. I believe that this publication is representative: https://doi.org/10.1016/j.clinph.2005.06.027
This! Eye tracking is slow and not good - but does that just mean we need to “faster horse” it or is there another option for bridging the communication gap for people with ALS and similar diseases? I have to believe there are better answers with other tech - likely EEG (+ AI).
My family is one of the unlucky ones that has genes for ALS so I’ve watched enough family members struggle. (I’m lucky, selfishly, because I dodged the gene but I still care deeply about this).
I had this thought, but then I thought about if my friend was struggling with a problem had had practical but imperfect solutions, would I better serve them by funding highly feasible solutions that they're already familiar with, or experimental moonshots that are more likely to fail, will take longer to implement, and my friend may not even care for at all...
I did a PhD in brain-computer interfaces, including EEG and implanted electrodes. BCI research to a big extent focuses on helping paralyzed individuals regain communication.
Unfortunately, EEG (including P300) doesn’t provide sufficient signal-to-noise ratio to support good communication speeds outside of the lab with Faraday cages and days/weeks of de-noising including removing eye-movement artifacts in the recordings. This is a physical limit due to attenuation of brain’s electrical fields outside of the skull, which is hard to overcome. For example, all commercial “mind-reading” toys are actually working based off head and eye muscle signals.
Implanted electrodes provide better signal but are many iterations away from becoming viable commercially. Signal degrades over months as the brain builds scar tissue around electrodes and the brain surgery is obviously pretty dangerous. Iteration cycles are very slow because of the need for government approval for testing in humans (for a good reason).
If I wanted to help a paralyzed friend, who could only move his/her eyes, I would definitely focus on the eye-tracking tech. It hands-down beat all BCIs I’ve heard of.
++ with mere MA in cog sci and psychology. If you really wanted to get EEG to work for typing, maybe you could train someone to map thinking about specific kinds of things to the keyboard, but that would be an extremely weird experience. Eyeball is going to have the best signal about eyeball-related motor cortex we can access.
I responded to the thread but Senseye has been working on this for a while now. Originally they were working with the US Air Force to help with improving pilot training - fatigue etc.. inference with retinal reading
They have generally struggled to find funding for their eye tracking focused work, and have recently had to pivot away from the really exciting but hard to fund stuff into PTSD screening (which is important too).
I can connect you with the founder if desired via the email in my bio
I worked on eye tracking hardware for Microsoft HoloLens. Several AR headsets offer decent eye tracking, including Hololens 2 and Magic Leap's ML2. I think Tobii's eye tracking glasses are probably better as a stand-alone solution though: https://www.tobii.com/products/eye-trackers/wearables/tobii-...
Agreed, the eye tracking itself is really a mostly-solved problem (Tobii are indeed leaders in the area). It's how it's used that matters - and as mentioned above, it's likely that it's the usability/interface that needs work.
PS VR2 uses Tobii's tech to do eye tracking, it is mostly being used for foveated rendering but some games also use it for gameplay, one for example allows you to shoot at enemies with your gaze.
I don’t know about the Tobii glasses, but I have a Tobii head/eye tracker that attaches to my monitor, and it works incredibly well and was only $200 or so. I’d be surprised if this isn’t an essentially solved problem at this point.
Not sure, but it would be cool to do something that didn’t require cameras, like EMG to detect eye motion from muscle activation. It would be hard to get the necessary accuracy though.
> A friend of mine has ALS and can only move his eyes. He has an eye-controlled keyboard, but it's not very good. Can you make him a better one?
When I worked for one of the big game engines I got contacted by the makers of the tech that Stephen Hawking used to communicate, which includes an eye tracker:
I believe Paul Graham can Google or use AI and already knows about the companies and links you posted. His post was a call to action to connect with people working on yet to be discovered innovations and inspire those and others quietly working to come forward and connect with him.
How about a library that starts loading a link when you look at it with intent. Or maybe with BCI integration that detects the moment you decide you want to access it.
Or how about a UI that automatically adapts to your eye movement and access patterns to minimize the amount of eye movement required to complete your most common tasks by rearranging the UI elements.
I have a new approach of doing ML, where autodiff is replaced with something better. Magically a lot of things fall into place. This approach should make problems like this relatively straight forward.
If you can show your new approach works, sure. Usually this is done via papers in ML conferences, but if you have reproducible results on Github I'll take a look.
Adhawk, adhawk.io, has the only all day ultralight eye tracking wearable I'm aware of, all MEMS based with ultra high scan rates, 500Hz+ and research grade accuracy. For ALS u likely need something light and frictionless, wearing a hot and heavy headset all day probably doesn't work.
I've been working on the menuing side [1] based on crossing Fitt's Law with Huffman trees. But, don't know the constraints for ALS.
Hopefully, whomever takes this on doesn't take the standard Accessibility approach, which is adding an extra layer of complexity on an existing UI.
A good friend, Gordon Fuller, found out he was going blind. So, he co-founded one of the first VR startups in the 90's. Why? For wayfinding.
What we came up with is a concept of Universal design. Start over from first principles. Seeing Gordon use an Accessible UI is painful to watch, it takes three times as many steps to navigate and confirm. So, what is the factor? 0.3 X?
Imagine if we could refactor all apps with a LLM, and then couple it with an auto compete menu. Within that menu is personal history of all your past transversals.
What would be the result? A 10X? Would my sister in a wheelchair be able to use it? Would love to find out!
235 comments
[ 4.0 ms ] story [ 121 ms ] threadBut maybe I’m wrong, it might be a purely software based problem that can be solved quicker.
I agree entirely!
There are a bunch of hardware out there to get gaze vectors, the problem is that they rely on a generalised model of the eye. With fine tuning you can go from >5 degree of error, to <1-2
VR Eye-Tracking Add-on Droolon F1 for Cosmos（Basic Version） https://a.co/d/asAAZwT
You can use it with any headset cause the data it provides is independent of the headset. Not sure how accurate that one is in particular.
Eye tracking data is not particularly complex nor super sensitive in nature. Once the sensor is calibrated it just sends over two vectors.
They won't share information on where you look, but they will share info on where you 'click', which is used to navigate apps. IIRC you are supposed to use your fingers to do this, and other actions, but I imagine that Apple's accessibility team will have an alternate mode for people with motor limitations. It could be a long-blink, or rapid blinking, for example.
I think that would be my major hesitation but I don't have a lot of experience evaluating patents.
Apples Eye Tracking Patent: https://patents.google.com/patent/US20180113508A1/en
Because of that, I'm also sure that eye tracking will go mainstream in other areas once the Vision Pro is released once everyone else catches on to it as a great input method.
Also, I'm not an ALS expert, but if the only muscular control is in the eyes, then lack of control in the head/neck probably breaks some assumptions about how the vision headset works (just a guess though).
The issue is doubly close to my heart because my father has ALS and is nearly at the point where eye-tracking will be his only means of communicating effectively with the world. While existing Tobii systems work well enough, typing with your eyes is still exhausting to do.
Ultimately I don't think a platform like the vision pro is suitable for ALS patients, especially later term. They cannot support the weight of the headset and/or fatigue will set in rapidly. Many (including my father) also require use of a ventilator, accompanied with a mask that can seal effectively enough to support the positive pressure necessary to inflate their lungs. Unless the form factor for HMD's minimalizes significantly, it will likely interfere with the respirator's efficacy.
The supposed security of blackboxing the eye data itself is illusory and functionally just for marketing.
Is the eye-tracking performance/accuracy step change on Apple's headset purely just a software/algo change? Or is it actually using a new principle/apparatus for eye-tracking?
- Multiple cameras per eye, and at a very short distance from your eye
- The screen is fixed relative to the cameras for all devices, there's no worry about that half of the equation getting off calibration or differing for every customer
- OS is built around eye tracking, which means there won't be any actions that are unnaturally hard to perform with eye tracking
1) its eye tracking isn't good enough for this kind of application.
2) direct access to the gaze vector is disabled
3) its really intrusive
4) its heavy.
5) it doesn't exist(in consumer world) yet.
The goal is to enable someone who has motor control issues, be able to communicate directly with the outside world. Shoving a heavy skimask that totally obscures the outside world on their face directly stops that.
Not only that, but you'll need to create and keep up to date the software needed to make a communicator. Apple are many thing, but it's new platforms are not stable, rapid os updates will break things.
It'd be good to know what rate we need to beat and some other metrics.
I'd consider an approach like the human powered helicopter parable.
I'd create a model for the eyes / face. So a screen like a phone is put in front of a camera with a model face and controllable eyes that you use software to control. Maybe skip the screen in front of a camera and link straight into the video feed.
It knows the limits of the eyes (different models for different people and situations) can measure fatigue etc.
You could run billions of simulations....
https://thinksmartbox.com/products/eye-gaze/
I once interviewed at this company. Unfortunately didn't get the job but very impressed nonetheless.
The solution actually works pretty well, especially when calibrated to a single individual.
After you parse what an object is, tracking it doesn't take anywhere near the effort of original segmentation. No need to re-evaluate until something changes.
Maybe even use activations to turn on and off networks. "Oh text better load ocr into memory"
And it does inform a lot of our built world It's strange to think that when watching a movie only 10% is in focus.
Eye movement does provide a lot of information to other people and I think the physical movement produces feedback for velocities and things too. Mimicking biology is often a good bet.
You are talking about cobalt and that is only used in lithium ion batteries. You can avoid cobalt by using lithium iron phosphate batteries.
There are active efforts to develop new lithium ion chemistries to avoid cobalt and there are even commercial sodium ion batteries now.
You bring this in to a completely irrelevant conversation, what have you done to help solve it?
Sure I use lithium ion batteries too, in the developed country I live in, I can't go and live in a cave. But I won't buy a SUV that weighs 2 tons to sit alone in it most of the time and have it parked 95% of the time in a garage.
The "what have you done" attack is a bit too easy and, really, this is not about me.
We should strive to be sober, to buy repairable goods whenever possible, to educate ourselves on environmental impact of things, and perhaps more importantly to teach this to our children.
But I won't buy a SUV that weighs 2 tons to sit alone in it most of the time and have it parked 95% of the time in a garage.
What are you even talking about? This whole thread is about eye tracking.
We should strive to be sober, to buy repairable goods whenever possible, to educate ourselves on environmental impact of things, and perhaps more importantly to teach this to our children
Are you doing that?
The entire supply chain and lifecycle of a product should be evaluated to understand if it is a net positive or a net negative. Not only the finished product: "wow eye tracking how cool let's deploy everywhere regardless of how the materials for that tech are mined and how the device will be recycled once obsolete".
[1] https://patents.google.com/patent/US8246454B2/en
[2] https://i.imgur.com/dgGvgKF.png
Second Sight was giving patients artificial eyes. When they ran out of funding, they closed shop. The patients lost support system for their eyes. If anything goes wrong with their artificial eye, there is no one to repair or fix it. They just have to carry a piece of useless metal junk in their head.
https://spectrum.ieee.org/bionic-eye-obsolete
can you elaborate?
The space stuff looked interesting until he started sending all kinds of things into space to become an ISP, polluting space the same way Tesla pollutes the earth.
The real issues are still here: poverty, lack of access to clean water, etc. How does Musk fix that?
We can work on many problems at once. Tesla also makes a car.
Simplified, with current capitalism, doing the right thing doesn't pay off. We have more than enough know-how and technology to fix real issues but we'd rather put rich people in overweight electric cars.
Yes, full self driving could help save lifes. Space exploration helps us better understand our world. But often times, current capitalism encourages the use of technology and know-how for things that benefit a minority at the expense of the most vulnerable.
I think people do have a responsibility and in a democratic state, government can only be as good as the people it governs.
YC was investing in ways the traditional VCs weren’t when it started, and coding HN was a part of it. I doubt I’m the only one who had having a few HN tech support emails from PG.
1. https://www.nasa.gov/centers/ames/news/releases/2004/subvoca...
2. https://github.com/kitschpatrol/Brain
Got any jumping off points?
EDIT: User `tbenst` linked their thesis above[1].
1. https://neuroscience.stanford.edu/research/funded-research/s...
[1] https://neuroscience.stanford.edu/research/funded-research/s...
https://en.wikipedia.org/wiki/EyeWriter
https://github.com/eyewriter/eyewriter
https://www.instructables.com/The-EyeWriter-20/
https://www.moma.org/collection/works/145518
My family is one of the unlucky ones that has genes for ALS so I’ve watched enough family members struggle. (I’m lucky, selfishly, because I dodged the gene but I still care deeply about this).
Unfortunately, EEG (including P300) doesn’t provide sufficient signal-to-noise ratio to support good communication speeds outside of the lab with Faraday cages and days/weeks of de-noising including removing eye-movement artifacts in the recordings. This is a physical limit due to attenuation of brain’s electrical fields outside of the skull, which is hard to overcome. For example, all commercial “mind-reading” toys are actually working based off head and eye muscle signals.
Implanted electrodes provide better signal but are many iterations away from becoming viable commercially. Signal degrades over months as the brain builds scar tissue around electrodes and the brain surgery is obviously pretty dangerous. Iteration cycles are very slow because of the need for government approval for testing in humans (for a good reason).
If I wanted to help a paralyzed friend, who could only move his/her eyes, I would definitely focus on the eye-tracking tech. It hands-down beat all BCIs I’ve heard of.
https://senseye.co/
They have generally struggled to find funding for their eye tracking focused work, and have recently had to pivot away from the really exciting but hard to fund stuff into PTSD screening (which is important too).
I can connect you with the founder if desired via the email in my bio
Whilst it plays an unskippable and unblockable ad (thanks weiapi!)
Seems like all the solutions out there are some flavour or variation of this.
When I worked for one of the big game engines I got contacted by the makers of the tech that Stephen Hawking used to communicate, which includes an eye tracker:
https://www.businessinsider.com/an-eye-tracking-interface-he...
Both apple and Facebook acquired eye tracking companies to kickstart their own development.
Here are some Top-lists
https://imotions.com/blog/insights/trend/top-eye-tracking-ha... https://valentinazezelj.medium.com/top-10-eye-tracking-compa...
Its also an active research field, this is one of the bigger conferences: https://etra.acm.org/2023/
I believe Paul Graham can Google or use AI and already knows about the companies and links you posted. His post was a call to action to connect with people working on yet to be discovered innovations and inspire those and others quietly working to come forward and connect with him.
Or how about a UI that automatically adapts to your eye movement and access patterns to minimize the amount of eye movement required to complete your most common tasks by rearranging the UI elements.
Interested in hearing more?
If you can show your new approach works, sure. Usually this is done via papers in ML conferences, but if you have reproducible results on Github I'll take a look.
Hopefully, whomever takes this on doesn't take the standard Accessibility approach, which is adding an extra layer of complexity on an existing UI.
A good friend, Gordon Fuller, found out he was going blind. So, he co-founded one of the first VR startups in the 90's. Why? For wayfinding.
What we came up with is a concept of Universal design. Start over from first principles. Seeing Gordon use an Accessible UI is painful to watch, it takes three times as many steps to navigate and confirm. So, what is the factor? 0.3 X?
Imagine if we could refactor all apps with a LLM, and then couple it with an auto compete menu. Within that menu is personal history of all your past transversals.
What would be the result? A 10X? Would my sister in a wheelchair be able to use it? Would love to find out!
[1] https://github.com/musesum/DeepMenu