> If it notices that the driver's head has turned, it adjusts the HUD elements being projected in less than 300 milliseconds to compensate.
Assuming the road isn't just a flat straight road all the time, there will be uphill and downhill drives. Turns, curves, and twists. All of which affect your FOV. The YouTube trailer didn't show the actual windshield, but a view recorded from a mounted camera.
I wonder how this will compare to a bad FPS gaming experience.
I've been saying it forever since I got a HUD in my car.
This is the way. HUDs are an absurdly awesome coolness and safety feature. I don't care about how the dashboard looks because I don't want to look away from the road! Seeing this is extremely impressive and I hope that we can get AR technology to see widespread adoption in cars ASAP. The fact that we have to wait until 2024 (and for us Toyota or Lexus drivers 2028-2030) is just embarrassing. Why isn't this coming in the next year?
Tesla's refuse to add them and the aftermarket options suck, so I can't buy one either which is extremely sad.
i have missed many turns where the phone says “in 500 feet go right” and i have no idea how long 500 feet is and there’s many places to go right. it would be so nice to have a line drawn on the road
Maybe an unpopular opinion: I don't want this. I don't want any of this.
Don't block my windshield with your "helpful" alerts. I don't care, I can drive without them, that's how I (and millions of others) learned to drive.
I want a physical key.
I want the engine not to cut off when I'm stopped at a traffic light or stop sign.
I don't want more cameras, whether facing the occupants, or facing the road.
Cars are already "good enough", and have been for years. Why do we need to keep adding software and other janky "features" that create more failure points and more expensive repairs?
No, cars haven't been good enough for years. There's tons of fuel savings technologies being developed and added to reduce carbon emissions and pollution as much as possible (widespread turbocharging, start-stop, mild hybrid and full hybrid, smaller displacement, particulate filters). Not to talk about safety, with the forward-collision warning etc.
You can always get a 20 year old Toyota, and that will be more-or-less what you described.
But if you want to buy a modern car and drive it on public roads, you have to abide by the rules.
Yes, I get that. And in every car I've seen with it, I could turn that automatic behaviour off. (Apart from hybrids, where that's the whole point of having a hybrid)
Notice the bright yellow icon representing the bicycle. The risk is if some bicycles are not recognized (not showing icon), people who depend on the icons might be oblivious that there are bikes there.
Like when I write email and some parts in bold, people tend to read only the bold ones.
My first reaction was that it'll make it much harder to make eye contact with bicyclists. Making eye contact makes it much easier to predict where the bicyclist is heading, so I'm not sure that specific label is a good idea.
Well, if I'm in an intersection or something, and there's a bicyclist coming from the other direction, I usually... dunno... look at them? They usually look back. Somehow makes it much easier to read the other person and see where they are going.
I do the same when I'm riding a bike near an intersection and there's a car coming in the opposite direction.
I would put a high quality HUD in my weekend car in an instant, if I could find one. I love the way they can communicate information efficiently. Still haven't found anything I would remotely call acceptable and available though.
This would be interesting in busy environments or when the driver is less alert. Humans can only track about 4 objects at best (unless you are really lucky and can do 5) so instead of not seeing an incoming pedestrian or cyclist a big bright orange label would be way better since something else has already done the filtering for you.
Can't wait enough for HUDs to become mainstream for cycling too. Current offerings are way too expensive and immature. The way I envision these should go is helmet mounted visor with a replaceable, helmet mounted battery; standard lenses you can source from various vendors; swappable HUD modules; manufacturers could offer complete package (helmet, computer/projector, lens, battery) lineups and employ their regular pricing strategies. Not sure why nobody came up with such a system yet, all technologies are already there.
A better alternative to a small bike head unit screens, same benefits as for cars: navigation overlay over real world instead of a map; increased safety because you don't have to look down away from road. Some cyclists would appreciate a cleaner cockpit without phone/computer.
Hard to see how this will be a real issue. Ads make sense on articles because the most they could possibly charge you is a few dollars a month.
If they haven't made phones that are cheaper if you agree to watch an ad once a day before using it - then the dynamics for cars would be even worse for that.
- Netflix has constant product placement in their show and show ad for their other show on their main page
- Amazon Prime Video show you an ad for other show when you start playing one
- A lot of "smart TV" have ad in their UI at this point
- Windows has try to force ad in the OS UI several time and comes with preloaded junk
- Most Android phone comes with preloaded junk (which is basically ad)
And the list goes on. Companies have noticed that you can start to put ad even in paying product and secure a stable source of income and get away with it.
I am the only one who wants interface to the car comparable to the race cars? More knobs and dials instead of fancy touchscreens! Unless the car is really self driving and I don’t need to look at the road. 99,9% of the time I drive on familiar routes. On the highway I need a device that shows traffic jams ahead, not a fancy navigation.
I mean unless the HUD is in a helmet, then things like head position and driver's height will create rather large mismatches between what is the intended projection and actual road features, no?
Looks like it's not ready yet. Otherwise they could've made a more convincing demo from the driver's point of view, where we can see how things are positioned in relative to the wind shield, the engine hood, etc.
I would like to have a simple an clean HUD. No flashy warnings. And I would like to have a lot of knobs and dials.
Not a friend of touch screens in a car at all.
> If it notices that the driver's head has turned, it adjusts the HUD elements being projected in less than 300 milliseconds to compensate.
It is very slow.
People feel discomfort when different "overlays" lag by as little as few tens of milliseconds.
People feel uncomfortable working with touchscreens because they feel "unnatural" when they push the button, and it only reacts, and plays animation half a second later.
What people say what feels most "fake" about touchscreens is stuttering, and lagginess.
I believe you really need to go to sub-millisecond levels of latency for it to feel "glued" to the real world.
I had a touchscreen project in the musical instrument space, with a pressure sensitive touchscreen, where button animation had to match the touchscreen pressure.
Even with all tricks, lag on Android was making it completely unusable, not to say Android's audio API is completely not suited for serious audio work. In the end we went bare metal with it.
The lowest latency we got was 18ms by using the touchscreen interrupt to set the sensor value as a pointer to bitmap image of a button animation frame.
Yet, some more experienced musicians said that it was still "felt off."
I'm thinking to go full FPGA next time I need to do any latency sensitive HMI with kinetic-screen interaction.
Hey folks, Phiar founder here (YC S18), we are the AI and AR tech behind this Panasonic AR HUD.
Just wanted to help clarify that 300ms means "delay", not refresh rate (FPS). We run our entire AI and AR at under 33 ms per frame, and we can get to about 60 FPS of optimized throughput. The 300ms of delay mostly comes from the hardware display process: it takes time for the camera to receives an image, and passes that image through many different pieces before getting to the processor that we can use, and then it takes again some time to be rendered. With that said 300ms is the upper-bound and typically the delay is less than that. Further, you may not have noticed but there's similar delays for iPhone AR games, as well as to all other AR headsets, and that delay is there just for the videos to display (no AR, no AI).
We also recognize that distraction is a risk, and therefore UI/UX design is highly important here. Ultimately, we believe that overlaying on a live view in a clean and effective way is better than people reading directions off of a synthetic 2D map on a different screen that takes your eyes completely away from the frontal view.
Definitely appreciate all the feedback and discussions though, we will continue to improve and do our best to deliver a product that people want!
- BTW we are totally hiring: software engineers, PMs, sales, AI/AR scientists, and more!
52 comments
[ 2.7 ms ] story [ 110 ms ] threadAssuming the road isn't just a flat straight road all the time, there will be uphill and downhill drives. Turns, curves, and twists. All of which affect your FOV. The YouTube trailer didn't show the actual windshield, but a view recorded from a mounted camera.
I wonder how this will compare to a bad FPS gaming experience.
I imagine this will make you want to vomit faster than anything else you've experienced.
This is the way. HUDs are an absurdly awesome coolness and safety feature. I don't care about how the dashboard looks because I don't want to look away from the road! Seeing this is extremely impressive and I hope that we can get AR technology to see widespread adoption in cars ASAP. The fact that we have to wait until 2024 (and for us Toyota or Lexus drivers 2028-2030) is just embarrassing. Why isn't this coming in the next year?
Tesla's refuse to add them and the aftermarket options suck, so I can't buy one either which is extremely sad.
https://www.youtube.com/watch?v=uQT5pRs0yCQ
Don't block my windshield with your "helpful" alerts. I don't care, I can drive without them, that's how I (and millions of others) learned to drive.
I want a physical key.
I want the engine not to cut off when I'm stopped at a traffic light or stop sign.
I don't want more cameras, whether facing the occupants, or facing the road.
Cars are already "good enough", and have been for years. Why do we need to keep adding software and other janky "features" that create more failure points and more expensive repairs?
Because I like them and I'll pay more for the car than you will. And there are more people like me.
You can always get a 20 year old Toyota, and that will be more-or-less what you described. But if you want to buy a modern car and drive it on public roads, you have to abide by the rules.
Are there any cars where you can't turn it off? I thought the switch is pretty standard.
> I don't want more cameras
Why wouldn't you want a camera on the back? I get some other points, but this one is pure improvement of safety with no drawbacks I can think of.
Like when I write email and some parts in bold, people tend to read only the bold ones.
I do the same when I'm riding a bike near an intersection and there's a car coming in the opposite direction.
Watch "Hyperreality", if you haven't seen it, for where this leads.
[1] https://vimeo.com/166807261
No, ads exists mostly where people are not paying. Vast majority of paid products and subscription services are ad-free.
Just like journalism.
If they haven't made phones that are cheaper if you agree to watch an ad once a day before using it - then the dynamics for cars would be even worse for that.
- Netflix has constant product placement in their show and show ad for their other show on their main page
- Amazon Prime Video show you an ad for other show when you start playing one
- A lot of "smart TV" have ad in their UI at this point
- Windows has try to force ad in the OS UI several time and comes with preloaded junk
- Most Android phone comes with preloaded junk (which is basically ad)
And the list goes on. Companies have noticed that you can start to put ad even in paying product and secure a stable source of income and get away with it.
Edit: typos.
I mean unless the HUD is in a helmet, then things like head position and driver's height will create rather large mismatches between what is the intended projection and actual road features, no?
The first time I saw HUD navigation was when playing Need For Speed 9. It'd be awesome if that becomes the reality.
It is very slow.
People feel discomfort when different "overlays" lag by as little as few tens of milliseconds.
People feel uncomfortable working with touchscreens because they feel "unnatural" when they push the button, and it only reacts, and plays animation half a second later.
What people say what feels most "fake" about touchscreens is stuttering, and lagginess.
I believe you really need to go to sub-millisecond levels of latency for it to feel "glued" to the real world.
I had a touchscreen project in the musical instrument space, with a pressure sensitive touchscreen, where button animation had to match the touchscreen pressure.
Even with all tricks, lag on Android was making it completely unusable, not to say Android's audio API is completely not suited for serious audio work. In the end we went bare metal with it.
The lowest latency we got was 18ms by using the touchscreen interrupt to set the sensor value as a pointer to bitmap image of a button animation frame.
Yet, some more experienced musicians said that it was still "felt off."
I'm thinking to go full FPGA next time I need to do any latency sensitive HMI with kinetic-screen interaction.
Maybe only put up satnav directions. Like “take next turn right”. 3fps is enough for that.
But if you’re gonna draw lines and highlight cars and bicycles at 3 FPS you’re just gonna make everyone vomit in their own car.
Just wanted to help clarify that 300ms means "delay", not refresh rate (FPS). We run our entire AI and AR at under 33 ms per frame, and we can get to about 60 FPS of optimized throughput. The 300ms of delay mostly comes from the hardware display process: it takes time for the camera to receives an image, and passes that image through many different pieces before getting to the processor that we can use, and then it takes again some time to be rendered. With that said 300ms is the upper-bound and typically the delay is less than that. Further, you may not have noticed but there's similar delays for iPhone AR games, as well as to all other AR headsets, and that delay is there just for the videos to display (no AR, no AI).
We also recognize that distraction is a risk, and therefore UI/UX design is highly important here. Ultimately, we believe that overlaying on a live view in a clean and effective way is better than people reading directions off of a synthetic 2D map on a different screen that takes your eyes completely away from the frontal view.
Definitely appreciate all the feedback and discussions though, we will continue to improve and do our best to deliver a product that people want!
- BTW we are totally hiring: software engineers, PMs, sales, AI/AR scientists, and more!
https://www.phiar.net/#careers