> Given how ambitious the hardware package is, the software paradigm is surprisingly conservative.
I actually see this as a bit less surprising. After all, if you change the hardware in a big way, and you change the software in a big way, users will have a harder time adjusting to the new platform. Instead, they're making a big leap on the hardware side, keeping legacy apps and concepts, and then will presumably iterate to more 'native' experiences that were previously impossible/unimaginable/unintuitive.
Text is still a large part of the interfaces we use.
We are all highly trained to read text, it seems basic but it is in practice quite abstract.
Text is still best read on a flat surface.
The great innovation I can see with this new Apple device is eye tracking, they have not invented it, but they might have perfected it enough to be useable.
Yeah, if their eye tracking plus foveated rendering works as advertised, it could be a huge step forward. I'm really curious how responsive the gesture controls will be too, it was really cool seeing the finger pinches(?) being used as an input method. I wonder if it's specifically designed just for that thing or if it's all built out to track any arbitrary hand gesture accurately. And I wonder what the language/api for describing hand gestures would even look like.
Eye tracking is almost certainly more accurate and faster. To the point where we make it an entire game to see who can who can move the mouse to what their eyes are already looking at fastest, in the form of shooter games.
Interesting point — if you no longer need to use a joystick to aim your weapon, how will controllers evolve? Will the second joystick be used for some other function, or will it be replaced by a different type of input method?
It would be funny if controllers evolved to be more like the single-joystick models that we had decades ago, with the joystick on the left and rows of buttons on the right. History doesn't repeat itself, but perhaps it'll rhyme?
One joystick has to exist since there are many videogames where the character is not in focus (barely), thinking of hack and slash with top down camera (the recent diablo 4),the majority of the focus is on where the attacks are going.
In top down bullet hell shooters the player had to aim and shoot in one direction as it walks in another (potentially opposite), so one stick is still needed
Agreed — I'm thinking we'd have just one joystick, to control movement. The second joystick could evolve into some new interface, or devolve into a row of buttons.
PSVR2 already has eye tracking. And unlike the Apple Vision it also has controllers with a stick. Perhaps there are already such games which let you aim with your eyes (assuming you are shooting at something), while you move around with one stick, and the second stick isn't used at all.
In the demo, I was hoping to see some app/screen/display anchored to certain walls or being fixed in a 3D spot. That way I can walk to a place to see that "screen".
Isn’t that an intractable problem without LiDAR? I suppose the naive solution would be to fix the screens in spots as you suggest as opposed to pinning onto real life locations.
edit: ah, right the vision pro does have lidar. I look forward to seeing this pinning if it's implemented. I'm still skeptical it would work well - I've used ARKit and it's not really that accurate when you're moving around.
why do you guys say this? Amazon's "View in your room" feature does a decent job of placing a furniture in AR. We can even walk around/towards it. And that tech is just Apple ARKit right?
The Quest has been doing this for years with the guardian setup and now with their option to bring your desk and couch into VR. I think the Pro has more advanced features but I haven’t tried it. You do have to assist it by drawing our boundaries but it’s pretty easy to do
In the demo there is a scene where a guy displays some iceberg photo on a virtual screen and then moves closer to it, with the screen anchored to some 3D point in the real world.
They kept doing that, like the woman sitting on the sofa watching a film leans down towards her popcorn[1] and the screen doesn't move, or the woman lying in bed watching the ceiling change into a blue sky[2] instead of just a blue sky where she was looking.
I think they'll inevitably introduce this down the track. The idea that you can set up a workspace as fixed to a particular place. Sit on the couch, and by default it has media apps and a big screen. Put it on in your office, and you can cover the walls with reference drawings and appropriate apps that are all where you left them.
Brave's reader mode wasn't shown as an option, though my BeeLine Reader extension was able to parse the text when I invoked its Clean Mode. I was a little surprised, since the BeeLine extension didn't detect/color the text inline, like it usually does.
> But it does put an enormous amount of pressure on the eye tracking. As far as I can tell so far, the role of precise 2D control has been shifted to the eyes.
I've been researching eye tracking for my own project for the past year. I have a Tobii eye tracker which is probably the best eye tracking device for consumers currently (or the only one really). It's much more accurate than trying to repurpose a webcam.
The problem with eye tracking in general is what's called the "midas touch" problem. Everything you look at is potentially a target. If you were to simply connect your mouse pointer to your gaze, for example, any sort of hover effect on a web page would be activated simply by glancing at it. [1]
Additionally, our eyes are constantly making small movements call saccades [2]. If you track eye movement perfectly, the target will wobble all over the screen like mad. The ways to alleviate this are by expanding the target visually so that the small movements are contained within a "bubble" or by delaying the targeting slightly so the movements can be smoothed out. But this naturally causes inaccuracy and latency. [3] Even then, you can easily get a headache from the effort of trying to fixate your eyes on a small target (trust me). Though Apple is making an effort to predict eye movements to give the user the impression of lower latency and improve accuracy, it's an imperfect solution. Simply put, gazing as an interface will always suffer from latency and unnatural physical effort. Until computers can read our minds, that isn't going to change.
Apple decided to incorporate desktop and mobile apps into the device, so it seems this was really their only choice, as they need the equivalent of a pointer or finger to activate on-screen elements. They could do this with hand tracking, but then there's the issue of accuracy as well as clicking, tapping, dragging or swiping - plus the effort of holding your arms up for extended periods.
I think it's odd that they decided that voice should not be part of the UI. My preference would be hand tracking a virtual mouse/trackpad (smaller and more familiar movements) plus a simple, "tap" or "swipe" spoken aloud, with the current system for "quiet" operation. But Apple is Apple, and they insist on one way to do things. They have a video of using Safari and it doesn't look particularly efficient/practical to me [4].
But who knows - I haven't tried it yet, maybe Apple's engineers nailed it. I have my doubts.
From reading/listening to reports of people who were able to demo the device, I think Apple may have nailed it, or come close. Everyone I've seen has absolutely raved about how accurate and intuitive the eye tracking + hand gesture feels.
I just went to echo this. Most of the reviews I’ve watched or read have raved about the eye tracking, regardless of if the reviewer was a fan of VR in general. It feels really cool that we might finally have a new computing paradigm. There is so much potential here and I can’t wait to see where things go.
Entirely serious question: is this something that can be trained? That is, after a long enough time of use and strengthening of the appropriate muscles and neural pathways, the headaches and such go away?
I think for some people, it won't be a problem. Much the same way some people have no issues spending hours using VR goggles without nausea. There are many people with accessibility issues who manage it, for example, and geeks who just like the idea of living in the future. In my experience, your eyes just get tired, even after you get used to it.
I think unless Apple has really added some sort of magic, it will be an issue for most people to use Vision Pro for anything but passive activities, or with a physical trackpad.
Surely you are already moving your eyes to look at click/tap targets anyway. Why would there be any additional muscle strain or neural pathways needed? Clearly moving our eyes doesn’t cause problems because we move them all the time as it is.
Maybe that's why some people prefer keyboard shortcuts; I don't move my eyes around the keyboard when typing, or when pressing Ctrl+S or Ctrl-C/Ctrl-V or in Windows I can Alt-Tab repeatedly and the highlight moves and I can looking around at the available windows while watching the selected one change with my peripheral vision. I can keep typing in a textbox for a bit while looking away, but with a virtual keyboard or speech to text which might be getting it wrong that's less of an option.
Compare with a smartphone, I can move to tap the 'back' button in Safari without moving my eyes to the back button. Looking "at" the phone generally is enough to see where my finger is, and say clicking on links I can look at the link and move my hand to it and move my eyes away while my hand is moving.
Having to coordinate looking and clicking - while I haven't tried it - I can imagine that feels more load-bearing, more effortful, more constraining, more annoying.
Moving your eyes doesn’t cause headaches. Eye strain certainly can. If the resolution is high enough and the lag issues are taken care of there’s a good chance that it can be used for longer periods of time.
The people who designed it, built it, and tested it have used it for an extended period of time. It would be un Apple like if they decide to ship a product with such a well known potential defect in the user experience.
The eye tracking as the interface is what stood out to me as the most radical choice in the device.
I'm doubtful we really know how good it is yet. Without a few hours/days of hard use its just not going to be obvious the degree to which they've overcome all these long standing obstacles via "magic".
It's hard to think voice is anywhere near ready for this application, I still find it highly frustrating after decades of tries.
PSVR2 uses it for foveated rendering where the eye tracking doesn't really have to be that precise. It's done for performance optimization, not input controls.
> If you were to simply connect your mouse pointer to your gaze, for example, any sort of hover effect on a web page would be activated simply by glancing at it.
I’m curious about how they deal with apparently not doing this, in terms of how do you know what button is going to be clicked if you click? The product page under privacy says that websites and apps don’t get eye tracking data, only knowing when you click, so I guess that means a website won’t display its active/hover effects on a button you’re looking at. Because if it could then it could just link that to analytics in JavaScript and defeat that privacy protection. So if you don’t have that, how do I actually know what’s going to happen when I click?
Sure, for a well built standards compliant website making appropriate use of HTML elements. There are a lot of websites that do their own thing, use lots of custom Javascript, where standard browser features like tabbing through their elements don't work, and they don't work well with existing accessibility tools. I'm skeptical that an OS or browser level feature is going to be able to make sense of them. That's their fault, but it's the reality of a huge part of the web.
You have those problems mostly because your Tobii is external device much more far away - you won't get good enough accuracy comparing to any eye tracking that is just a camera few cm far away from your eyes.
With Apple solution (or any VR with eye tracking):
1. You have very accurate head 6d location (gyroscope, compass, accelerometer + lidar for translation) at 100+ fps minimum and very computationally cheaply without much latency
2. Cameras pointing at each eye from distance of few centimeters. In this situation Eye region of interest occupy full image resolution. Probably even VGA resolution (or less) is enough to have very accurate precision in this case
3. Because headset covers your face you don't have any problem with different lightning condition, cluttered background, whats more they use IR floodlight to have consistent lighting of pupils
4. Possibly they even use similar Truedepth IR dot pattern to get 3d depthmap of your pupils similar like truedepth sensor works with difference that here they can put thos camera more far away and have it more accurate disparity map
5. Who knows maybe they even use lidar instead of truedepth for even lower latency
6. Because sensors are so close they can have very accurate reconstruction of your eyes and distance between them - most people don't have completely symmetric eye/face
Tobii is a sensor that is just like 0.5m far away from your face. Has to track your head, estimate orientation and distance, and detected eye ROI will be a tiny crop of their image sensor even if they camera has 4K resolution (which I doubt they use such resolution). At that distance if you want to get accuracy of 1 cm of what you are looking at screen you need to detect pupil movement of ~1 degree angle. If you want even better accuracy without gitter like 1mm then you need to detect movement of pupils of 0.1 degree angle.
I'm disappointed even though it's entirely predictable that VisionOS is built on the iOS/iPadOS foundation rather than OSX. I guess we'll see how "walled in" it is but it's hard to see any reason Apple isn't going to be just as constraining and controlling about what happens on this OS as they are on iOS, if not more so. Which ultimately means I'll be very reluctant to ever adopt it in any meaningful way as my primary computing device.
Definitely pros and cons. I remember reading that because of the tight integration with iOS this allowed them to achieve best in class latency for iPad + pencil that couldn't be achieved on any other platform. Having followed Oculus / Quest development that low latency is not optional in this context and every millisecond counts so I can see why they would go this route.
On the other hand, the closed ecosystem is definitely cause for concern. Fingers crossed that WebXR support comes out from behind a feature flag to allow for progressive (spatial) web apps.
A 2018 Dell XPS that has a Wacom digitiser and OneNote gives me lower latency and better inking experience than an iPad and pencil 2. I used them side by side for long-form writing for a while and the iPad was left in the dust.
You might be right it's more marketing than substance, but FWIW this is the article I was thinking of praising low latency of iPad + stylus: https://danluu.com/term-latency/
Ultimately its more about the openness of the OS. If they built on iPadOS but they made a reasonable experience for freely sideloading apps, I'd be OK with it. Or even if they support WebXR really well, I might (just) find a way to be OK with that. But if they lock this down and position themselves as rentseekers to take a cut from every piece of software that runs there regardless of how much they contributed to it - I'm out.
Part of (my) definition of a platform is that when you buy it you are fully paying for the cost and contribution the platform provider made when you buy it. If there's a residual financial (or other) obligation after that then you don't have a platform, you have a partnership. I know there's other definitions of platform but that's the one that's important to me.
I don't see why that should be. Obviously you have a right to do whatever deals you like and not buy this product, but it seems to me that one reason Apple is investing so much in this is because they expect to recoup a huge reward over time, not just from device sales.
I guess you can always buy an android based headset that allows sideloading but presumably it won't have the same level of investment.
Right - and the Quest Pro isn’t well compared to what Apple is marketing here. I couldn’t accept a job working with a VR company because I couldn’t get past the nausea the Quest Pro induces in me.
I think the point is that Meta’s investment is weak compared to Apple’s full court press. It really is laughable.
> I couldn’t accept a job working with a VR company because I couldn’t get past the nausea the Quest Pro induces
Ouch. I'm sorry to hear that, I feel sorry for folks like you that are sensitive to nausea. There's so much potential in this tech but I can see a whole new class of disadvantaged people coming who aren't able to fully utilise it.
From a developer POV it's about the difference with the other platforms. If Apple provides a platform that bring the dev 50% more profit than the status quo, sure taking a cut of that extra profit could make sense.
If that same dev would have made the same profits on other competing platforms, Apple is just fragmenting the market and it's take should be limited to basically management fees.
I hope this new platform brings newer applications and innovations, but also expect competition to adjust, making it a situation where Apple is just one of the many platforms.
It's more complex than that...the whole limitation on promoting your existing membership services, the rules enforced on pricing and passing the 30% to the users etc. makes it a much muddier picture.
iOS sure brings some value to the table. How much actually ? who knows...
For comparison Youtube is also a platform regularily criticized by its participating creators, but you're not seeing high profile lawsuits or the EU slapping them fines at every turn.
I'm not sure where you're taking your figure of twice as much, but I assume it's the AppStore vs Google Play break down ?
That number is the total revenue for the whole store (there's a lot to say about it, since Google is also not the only Store for android, in particular in China and Korea), and it matters very few for any individual developper.
It's not because the total amount of in-App purchase for all games is twice as much on iOS that my new ssh shell app will make twice as much on iOS than on another Store, for instance.
Most US developers are not producing versions for Chinese app stores. That’s a red herring.
Of course individual products have their own idiosyncrasies, but is it in fact the case that on average, an app on iOS will make more than the same app on Android.
well i remember seeing you can bring your macbook or other devices into the space as a virtual window/screen/monitor (whatever they're calling it) so I already thought I can run vscode and any other apps at first at version 1 release. but I know what you mean.
Yeh this was my main thought. However good the hardware is doesn't matter if the software can't fully utilise it. The idea of virtual displays (the most obvious immediate benefit of Vision imo) for MacOS seems like a huge benefit, but for iPadOS its downgraded to pretty cool. I've never felt any particular need to add multiple displays to my iPad, and whilst a big display would be nice, I wouldn't describe it as groundbreaking (especially as others such as XReal are already doing this in a much smaller form factor).
This might have made sense with the iPad but I can't see how anyone is going to write software that 'fully utilizes' the system in a way that Apple doesn't support without billions of dollars of investment.
I also wonder if it made more sense from a security perspective. In the early iOS days, one of the big problems for desktop apps was unrestricted access to your data. IOS improved this with application sandboxing by default, and it eventually made its way into OS X.
But we all know the difference between implementing something brand new versus retrofitting something into an existing system. So my guess is iOS had more secure defaults for a system like vision.
It's a bit odd if you think about it: a desktop OS effectively creates a virtual 3d space for windows to live (move them in X and Y, or change their depth relative to each other), and the iPad or iPhone treats the surface as just a simple 2d plane with no depth or really even XY axes (you could sort of argue that the app switcher is an X axis).
So they're effectively recreating a lot of the elements of the macOS w/r/t window positioning, resizing, and depth, but building all on top of the iPadOS paradigm. If you consider the input methods are probably closer to a touch interface than a MKB interface then it kinda makes sense.
Personally I have zero interest whatsoever in living inside Apple's "only what we allow you to do" world, and the fact that they seem to be expanding it to what seems like a very powerful, desktop replacement computing device is kinda gross to me. But from an interface standpoint I think it's really interesting and actually makes some sense.
> I'm disappointed even though it's entirely predictable that VisionOS is built on the iOS/iPadOS foundation rather than OSX.
They’re the same foundation, iOS is a fork of OS X and was originally announced as such. I’d be shocked if there isn’t a well maintained internal build of iOS which keeps parity with macOS windowing etc.
The difference in restrictions between the platforms is entirely arbitrary. If this device appeals to Mac users or would-be Mac users, I’d be shocked if it doesn’t at least eventually allow a Mac-like environment.
It certainly isn't arbitrary, macOS has swap and doesn't kill apps and iOS doesn't have swap and does kill apps. This makes everything very different, and one makes it easier to do real-time rendering than the other.
I guess I should clarify. It’s arbitrary in that the restrictions are policy-based, not limitations of the underlying technology. There’s no technical reason iOS couldn’t have swap, it’s not available because trade off decisions were made to enable use cases on much more limited devices.
It's different-ish. iPadOS is still a more limited/controlled OS and the swap policy is also more limited. You can see it in the open source kernel where it's called "app swap", so it doesn't swap out the OS, but on macOS it can.
"Limited" makes it, well, limited, but it has performance advantages: fewer unexpected page faults mean audio/video processes won't unexpectedly drop a frame as often, and there's fewer disk writes which is good for the lifespan of the hardware it runs on.
Btw iOS actually has a very limited swap policy called "freezer" that only applies to suspended background apps, and only when it's in the performance budget.
The xnu source code on github. The rest of it… well you just did read about it. There's an older book called Mac OS X Internals that covers some of it.
Also it’s locked to Siri which is dogshit and falling further behind by the second. Apples speech to text is atrocious, the OpenAI tools blow it out of the water, as does Google’s.
No matter how fancy the visuals, it’s hard to have a pseudo-hands free voice interface that doesn’t uh, work?
I would not be so sure, mainly because of ecosystem lock. I sincerely doubt a double-digit percentage of Apple employees have touched and android phone or used OpenAI APIs. Big problem with ecosystems is you don’t know what you’re missing.
The regression in autocorrect is also nuts. It seems to aggressively modify grammar to the point of regularly changing the meaning of what I’m writing. It’s like a CS undergrad project where they went too hard on Markov chains.
I also have a fairly expansive vocabulary and talk about esoteric technical matters on my phone frequently. As far as I can tell Siri was trained on 8th-grade level text and constantly dumbs down my text and distorts the meaning.
Ugh, so true. I'm both a software developer and an avid reader of literature & philosophy, so my day to day word stream is both extremely varied, and frequently allusive with cryptic phrases and idioms. Not to mention the fact I do actually care about correct punctuation. Autocorrect and voice typing have long since given up on me.
> I sincerely doubt a double-digit percentage of Apple employees have touched and android phone or used OpenAI APIs
I strongly doubt you're right. Apple is full of geeks; just because they tend to be tight-lipped doesn't mean they aren't paying attention to all the same stuff we are.
Double digit? I live walking distance to Apple HQ and there’s not exactly a lot of people in the shops and cafes with Android phones and you practically never see someone pulling out Google pay at a register.
Whereas I’d wager more than half of Google employees use iPhone, and a lot of Google staff work exclusively on iOS and Mac apps. Apple software runs in their hardware, period. It makes them too insular and unable to see their shortcomings.
The current voice transcription engine at OpenAI is using Whisper-1[0] which is open source and runnable locally, if you wanted to keep it all on-device. I run it locally for various things and it works pretty damn well.
Yeah it kind of sucks because it seems like won't be able to use something like Hammerspoon to control windows and other UI elements the same way you can on Mac.
But it does put an enormous amount of pressure
on the eye tracking. As far as I can tell so
far, the role of precise 2D control has been
shifted to the eyes.
I've got one good eye and one bad eye. The bad eye is legally blind, has an off-center iris and is kind of lazy w.r.t. tracking.
I'm extremely curious to know how Vision Pro deals with this. One certainly hopes there's some kind of "single eye" mode; certainly seems possible with relatively small effort and the % of the population who'd benefit seems fairly significant.
Eye tracking most certainly sounds like the way to go, relative to hand-waving.
The Minority Report movie probably set the industry back by a decade or two. Waving your hands around to control stuff seems logical but is quickly exhausting.
Track record wise, apple is one of the best in terms of serving accessibility. So I’d bet greater than 50% odds that they’re thinking about lazy eye or one eye or derivatives there of.
Well, they have pretty consistently invested in and showcased accessibility, famously ROI-be-damned (to paraphrase Tim Cook). So, it’s not a stretch to expect (but not assume) they will have the perspicacity to do the same here.
I agree not to drink the apple copium, but they do excel in accessibility by a large degree.
Honestly, I think a large part of the industry taking into account accessibility in a more serious ways comes from the competitive advantage apple has created.
Doesn’t mean accessibility hasn’t been done in the past. By no means. But it’s a market segment they have on lockdown
FWIW, I emailed a fairly senior accessibility person at Apple yesterday with a question about the headset and got a response back today, even in the midst of WWDC. And I'm a relative nobody — just a bootstrapped solo founder who works in the accessibility field.
It would have been much less exciting to see Tom Cruise sitting on a couch, hands in his lap, gently flicking his fingers to scroll through crime scene footage. IIRC he talked about how tired his arms got during filming.
I can't speak for working in VR but I think us desk jockeys tend to underestimate what other people are doing for work. The lack of movement is what does thought workers in.
There are of course many jobs that are much more taxing on the body overall than waving one's hands at a Minority Report style computer interface. (I used to work at a restaurant! I've done actual work, I swear!)
However, waving your hands in the air for 8-10 hours a day feels a bit unnatural relative to "actual" physical work. I don't have a real scientific basis for this statement.
But there are some physics at play IMO. In boxing it's generally accepted that a missed punch consumes about twice as much energy as one that connects -- your body must do the work to accelerate your arm, and then it must do a roughly equal amount of work to decelerate it. I think there are some parallels to waving one's hands in the air for 40-50 hours per week relative to "actual" physical labor.
More to the point: MR-style interfaces (at least as typically implemented/depicted) don't really offer tangible advantages to using a good touch pad. They trade tiny finger movements on a trackpad for huge sweeping arm movements that accomplish the same things.
I think if MR-style interfaces had somehow been invented before mice and trackpads, we'd be celebrating mice and trackpads as absolute miracles of efficiency.
I actually prefer keyboard alone due to the concise control, I find mice pretty clumsy. But there are things that it can do that I can't achieve on a keyboard and same for VR, perhaps we'll end up with blended work stations.
On the note of tiring out, I have been strength training for 12ish years now and I still get tired quickly when holding my arms above my head to work on my car. I think because they are a smaller muscle group, they can be saturated easily. I don't have the same issue with repetitive motions, it's just holding the arm in a similar position that does it.
Maybe if AR gestures take into account full motions rather than holding the arm in a similar position for too long, it might not tire so easily.
I have VR already and I will say it's an exhausting experience in general, I can flat screen game for hours, but in VR I want out in less than an hour. I think it's the full focus it forces on you, and the split world spatial reasoning going on.
You're talking like I didn't need to go to physiotherapy and occupational therapy to deal with my wrist issues and the lack of muscle tone in my glutes.
I distinctly remember they said they had hired scientists to discuss what the future would look like and I always thought "yeah, right, so in the future we got those holo interfaces and computers ultra connected but people in the same office still copy files from one computer to the other using some kind of plug able storage".
But now that I am also in the future I still use USB drive to move files at the office because. Guess they were on something after all :].
The fallacies
The network is reliable;
Latency is zero;
Bandwidth is infinite;
The network is secure;
Topology doesn't change;
There is one administrator;
Transport cost is zero;
The network is homogeneous.
I rarely use USB media, instead just copying things around using the network. But last week I needed to copy 200GB to my son's laptop, and ended up putting a 256GB SD card in the laptop and copying to it over night (it was started at 9pm anyway).
Never underestimate the bandwidth of a station wagon full of mag tape.
I have pendulum nystagmus since birth. My eyes saccade and shake very rapidly, even when I'm focused on something. Folks who talk with me certainly notice.
I really hope I won't be locked out of using vision pro just because its input modality isn't compatible with my body...
That was my thinking. They don't always nail down accessibility issues right away, but I think they're fairly highly regarded in terms of getting to things eventually.
(I say "I think" because I have not actually used their accessibility features)
With all of the new paradigms in VisionOS it seems perhaps unavoidable that accessibility features might take some time to catch up. Engineers and designers have to wrap their minds around the new stuff.
Maybe. The eyes are a sort of the canary in the coal mine for lots of neurological issues. Sometimes people don’t notice problems because the brain adapts well, and accommodation may be more challenging than one might think.
I have a loved one having issues due to a brain cancer. Optical seizures were the first warning sign and went unnoticed for some time. It would be amazing if VisionOS could potentially detect some conditions or help folks with epilepsy just as the watch has for some cardiac events.
Their accessibility features are quite fascinating.
Sound recognition for example, or Assistive Touch on the watch. Not needing those I have no idea wether or not they’re actually useful, but it sure is impressive and seems well-thought.
Considering there is a "calibration" step. I'm going to guess that as long as the "bad" eye behaves relatively predictably, it should be able to ignore the bad input and put appropriate weight on the "good" eye.
It will also be interesting how the outward display will display the "bad" eye input including missing eyes, lazy eyes, nystagmus, etc...
"And for people who prefer a different way to navigate content, Pointer Control lets them select their index finger, wrist, or head as an alternative pointer."
This reads to me like they've got you covered (at least in principle).
They have a training process at the beginning that learns your eyes. It asks you to ‘point’ to various points in space. I’d imagine it would be able to detect a dominant eye with that, and prefer it over a slower or non-tracking eye.
Let’s see how it works before we start making all these assumptions. Apple has the best track record in the industry when it comes to accessibility, hands down.
> Eye tracking most certainly sounds like the way to go, relative to hand-waving.
I suppose Apple could change my mind but I've never been a fan of eyes as input. Touch typing or any equivalent where you can look at one thing and do another seems impossible now.
Apple don't need you to defend them here. They presented what they presented, and whether we're interested is based on that. It's a legitimate concern and a valid point of discussion.
The interaction paradigm you describe (where it's literally impossible to look at one window while actively using another) makes so little sense that we can almost certainly rule it out. Certainly no press members who demo'd the unit are describing the interface that way.
That's not "defending Apple", that's just... gosh, I can't think of any word besides common sense.
From all of the presentations and press accounts, it seems clear that eye tracking is something of a direct replacement for the mouse/trackpad cursor -- in fact it seems you can use a trackpad or presumably a mouse as well. Looking at an app or "window" seems equivalent to mousing over it, not equivalent to clicking to activate that window.
> The interaction paradigm you describe (where it's literally impossible to look at one window while actively using another) makes so little sense that we can almost certainly rule it out.
I think the implication is that the window you are looking at is focussed automatically. Which wouldn't work for say, typing up a document to a text editor while reading from a reference doc. That's just the beginning, I use a mouse-to-focus window manager and that happens all the time with all sorts of things.
I don't disagree with your other points AT ALL, but given how repeatedly proven that common sense is entirely a fiction, if any argument falls back on it you have to be immediately suspect of it.
I hear that now and then, but looking at the wiki on it, the “criticism” paragraph is only like 3 lines with no real refutation, out of a hefty lengthy article.
What’s your motivation for saying that? Personally I am suspect of people say there isn’t common sense.
Here is the definition at the top: Common sense (or simply sense) is sound, practical judgment concerning everyday matters, or a basic ability to perceive, understand, and judge in a manner that is shared by (i.e., "common to") nearly all people.
One can certainly say there is common sense. For instance no one likes pain, people like pleasure. People care about entertainment, not boredom. People tend to learn on the whole as they grow up. I also think on the whole have a perception of what is wrong and right, even if there is variance between cultures. The golden rule of reciprocity seems to be near universal (like: don’t kill, don’t steal and so on)
Your basis for discussion is that you read the wiki article on common sense? Then you cite a dictionary definition? Suffice to say that is pseudointellectual and inappropiate.
And then you obviate the concept by rendering your own gross ignorance: people don’t like pain? Today you learn about masochism. People need entertainment? Monasticism, asceticism, Buddhism, nihilism, et al. People learn as they age? Quite unfortunately many studies show many people lose a great deal after their teens. And your invocation of the idea that cultures share morality such as a prohibition on murder (this has nothing to do with “reciprocity” by the by) is truly divorced from basic observable reality.
100% of a farming town/community are going to hold "don't grab an unknown fence, as it might be electric" to be common sense. Yet there's going to be a sizable portion of the planet to whom that WON'T be obvious, and they'll try it.
100% of New Yorkers might know not to park your car overnight in a particular area, yet people new to the city will happily do so not knowing they may be putting themselves or their property in danger.
If you have to put bounds on "common sense", or contextualise it at all, then it isn't common, it's just knowledge shared by a group of people of unknown size.
You say it in your final sentence - "near universal". If it's not universal then what's the point? Where is the magical cutoff point of the population where "common knowledge" is no longer common? 95%? 50%? If so, why?
Such speculation is ludicrous because it's going to change drastically from area to area and change even more so depending one what you're talking about. And if such a definition is so fluid, when you're positing that it is in fact universal, then it's pretty clearly false.
"Common sense" and "common knowledge" are just shortcuts we use so we don't have to explain the entire contents of history in one go to explain one other concept. This is very useful. But relying on it existing _ALL_ the time is the work of a dangerous lunatic that has clearly not encountered a broad enough segment of humanity.
Thank you for your thoughtful reply. I think I might gonna read up a bit more on the topic, it’s quite interesting.
I think my personal stance is more on the idea that there is a lot we share as a species (from words, to symbols and human basics) and that could be grouped as common sense. But perhaps my grasp of english fails me here.
Looking at the etymology it also seemed to stem from “community” , what is common among a community. I would wager things were very common for his peers when Thomas Payne wrote his book “Common Sense” , a world to him that was largely very the same in style and substance.
Perhaps that concept simply can longer apply in a pluralistic global world or in a modern sense like you point out with your examples.
"Gorilla arm" is a term engineers coined about 30 years ago to describe what happens when people try to use these interfaces for an extended period of time. It's the touchscreen equivalent of carpal-tunnel syndrome. According to the New Hacker's Dictionary, "the arm begins to feel sore, cramped and oversized -- the operator looks like a gorilla while using the touchscreen and feels like one afterwards."
I’m with you on this one, but I also have nystagmus in both eyes. I’ve been curious but also somewhat apprehensive about stuff starting to rely on eye tracking and whether these edge cases will get handled.
Probably best for me not to be an early-adopter in this case :)
Edit to add: Can confirm that Apple stuff is amazing with accessibility, so I’m not surprised there are alternative input methods. My main thinking towards the future is around how nystagmus and foveated rendering will interact. That’s a giant and fascinating research topic I imagine.
Strabismus and amblyopia are common enough (4% of the population) that one would hope these types of systems would make sensible provisions to accommodate people with these conditions.
Lazy eyes, I think, are probably popular among software developers (I got one, and most devs I know have eyes issues). I think that's why Apple got lenses too. I think (hope) they got us covered since we'll be the first to use, develop on, and afford these devices.
> The Minority Report movie probably set the industry back by a decade or two. Waving your hands around to control stuff seems logical but is quickly exhausting.
The bad idea had waves even before that: Remember the gorilla arm.
I remember the gorilla arm but I found out that occasionally raising a hand from the keyboard and tapping the screen is not a noticeable effort and it's faster than aiming at the touchpad or (of course) reaching for the mouse. I discovered it mostly by using a Samsung tablet with both a Dex interface over Android and with an Ubuntu Unity one (the defunct Dex for Linux.)
That's maybe because the screen is small and low. Raising an arm at or above the head is a different matter. If you're cycling with a heart rate monitor you discover that you add some beats when you raise your hands (say, go from 150 to 155) and gain some if you lower your chest close to the handlebar. This doesn't take into account increased or reduced drag because I measured it at home on trainers.
I've noticed the same thing when I ride in my basement. If I come up off of the hoods, grab a water bottle and take a drink my heart rate bumps up ~5bpm. It'll climb a bit more if I eat something like a granola bar or a gel.
What's interesting is that unlike steady state to a hard sprint, the heart rate rise is almost immediate. There's some delay - 20 to 45 seconds - if I change it up and do a sprint interval before my heart rate reflects that effort.
> The Minority Report movie probably set the industry back by a decade or two. Waving your hands around to control stuff seems logical but is quickly exhausting.
There was an operations center I worked at that had a large touch screen (think cnn wall), that always sat in the corner of the area collecting dust, but whenever some higher up would visit it would get pulled out and used just to provide a wow factor. It was always the unlucky one that ended up having to use it during this time. The fatigue was horrible.
I tried the HoloLens ages ago and honestly the only upgrade would be the display & how often the inputs worked. Microsoft was demoing the HoloLens @ some hackathon and had a program where you could flip pages in a virtual book. On top of the book looking super low quality / illegible, I could never get the controls to work. The only thing that will make the vision pro seem like a worthwhile purchase would be if the controls worked flawlessly. I already have faith in the displays & picture.
I have a divergent squint on both eyes when tired/exhausted. I yet to try VR with eye tracking. Furthermore, I think the future would be some kind of brain reading implant/scanner.
Thanks, that looks very useful! I wrote a script to export Markdown from Joplin notes (https://gitlab.com/stavros/notes/), but the UI in this site is better.
If you got time and energy in the near future, please, consider slightly enhancing it according to the average web surfers expectations. Particular irritants were the broken scrolling (bound to a container), the broken footnote links and the forbidden content behind links. Additionally the classic escape hatch, Safari Reader, didn’t work.
This is a bit of a rant, but I bought a Quest Pro last year, and I feel it was a total waste of money. It's actually embarrassing how much of an unpolished piece of garbage it feels like. Sure, it's fun as a PCVR headset, but as a productivity tool or as a multimedia consuming device, or as a social tool, it completely misses the mark.
And I'd say that 80% of its failure can be attributed to bad software. It's buggy, I constantly have to re-calibrate my work/play area, the hand tracking is janky, and even though it supports (incredible) eye tracking, almost nothing takes actual advantage of the hardware.
It's insane that a billion-dollar company like Meta actually felt proud to release such a steaming pile of trash. Do they seriously expect VR enthusiasts to build an entire operating system for them? Much like the reports we're getting about the Vision Pro, the eye tracking in the Quest Pro feels like magic, but nothing uses it—basically it's irrelevant to navigating the operating system and barely any games use foveated rendering. It's infuriating.
If I was Zuck, I'd fire all my product managers. I say good for Apple. I'll probably be selling my Quest Pro and buying the Vision Pro.
Fitness and gaming wasn't demoed much yesterday, but are going to be insane on the Vision Pro - so I guess it was just too obvious to demo. Or is it?
Also Meta could catch up if it was just software. But how big of a deal is the M2+R1?
Anyway the metaverse was already being retracted, else this Vision Pro could have hammered Meta.
Clearly Occulus has to evolve to match VisionPro. Controller has to be optional. It has to have better AR. Meta has to be the more "open" alternative to Apple's ecosystem barriers.
> Also Meta could catch up if it was just software.
This was Microsoft's warcry for years: who cares if Apple has better software? We have better hardware. Well guess what, software matters and it matters more than hardware. If you have both, you have literally a generationally-defining product (e.g. the iPhone), but even if you just have the software, you'll still eat your competitor's lunch. Why does everyone have MacBook Pros these days? Prior to Apple Silicon, it certainly wasn't the hardware.
Meta's about to get a rude awakening because "we can fix the software later." As if technical debt doesn't exist and as if you can so easily get engineers that eagerly want to clean up someone else's mess.
>Why does everyone have MacBook Pros these days? Prior to Apple Silicon, it certainly wasn't the hardware.
Oh, I don't know about that. Admittedly there were some awful choices made between 2015 and the emergence of M1 architecture, but Apple's reputation for making well engineered hardware goes at least as far back as to when they started milling Macs out of aluminum. I'm thinking back specifically to the PowerMac G5 in 2006. It was designed well, felt solid, and when you opened it up, it continued to look well made. I recently popped open my 2015 Macbook Pro because I'm finally having hard drive issues, and for as much as I have railed against Apple fanboyism over the decades, it looked so nice inside I wanted to take a picture. (Why?! Who cares!? What would I even do with that picture?)
I got tons of mileage out of various Dell desktops and self-built machines over the last couple of decades. It was better bang for the buck, I didn't have to fuss over proprietary connectors and a locked down operating system. When the company I worked for made a big shift away from (it's okay to chuckle) Coldfusion and MS SQL Server to Ruby on Rails and MongoDB in 2013, though they offered to let me continue running on Windows, I asked for a Macbook because that's what everyone else was using. Might as well learn. Didn't much like OSX but the hardware was better than any Windows laptop I'd used. Later I bumped up to a 2015 MBP, which I only just retired this year.
While I was Windows at home and MacOS at work, last month I snagged a Macbook Pro w/ 64gb of RAM from B&H for $2400 - the first Apple computer I've ever paid for myself - and it's replaced my Dell desktop and that 2015 Macbook that my old job let me keep.
I effectively skipped the bad Macbooks, so my perspective is tinted by that. But even during the bad years, even my friends who continued to operate in the MS domain were buying Macbooks and dual booting them into Windows.
I had access to a Surface tablet through work early on. It was really nice! Just like a Zune. The Surface Pros looked pretty great too, but anecdotally, I never saw them outside the context of visiting a business that was a strictly MS shop.
I still like Windows as an OS better. But Apple's new architecture (which importantly doesn't have me carrying dongles around, or even needing them at home) was compelling enough for me to take the plunge.
I haven't liked the usability of the entire Macbook line though, and It's very upsetting that so many laptop makers have decided to just copy it.
There's lots of little details which are well made, but bad decisions: the screen hinges tend to be a little too floppy, the trackpad is oversized, removing the physical buttons makes it hard to use, moving the power button from a separate area to the "eject" button location on the keyboard (and making it look like a regular key), everything they've done with keyboards (flat, overlay spaced keys with no surface indentation and less and less travel). The ridiculously sharp edges of the body chassis which cut into your when you rest your hands there (seen at least one video of someone just filing a bevel into the machine to fix it).
They look great, but that's all they've ever seemed to me: in usability details they've always felt terrible (even MagSafe, which seems like it should be great, feels great, and yet has seemed pretty lacklustre when I've had to use it with a work machine).
Two thoughts here. The first is that this comments is totally valid and mostly the inverse of everything I think, really shows that we all value different things so thank god for a free market.
The two things I agree on for the 2016-2022 laptops are the sharp edges and terrible keyboards. Maybe take a look at the new laptops next time you get a chance, the edges on my MBAir and 16MBPro from 2022 are waaaay more comfortable than my previous two MacBooks and the keyboard feels much nicer too.
Back to point one though, to me the trackpads are well sized, the hinges are smooth and just right and the power button being where it is is a none issue, I even quite liked the touchbar.
> This was Microsoft's warcry for years: who cares if Apple has better software? We have better hardware.
What on earth are you talking about? Microsoft is a software company first. They rode the Windows advantage for more than a decade and didn't have anything to do with the hardware, because they aren't a hardware company. Windows ran (and was preinstalled) on everything, from the lowest tier of trash to the highest end machines, and Microsoft didn't care about the quality of the hardware, because they owned the platform that everyone used for computers. Office is another software product that still doesn't have any real competitors and is one of Microsoft's biggest cash cows.
On the flip side, when exactly did Apple have better software than Microsoft? Their office suite is barely usable for anything outside of the most basic use cases. Apple beat Microsoft in mobile, and Microsoft kept its head in the sand and assumed the Windows advantage would live forever and didn't wake up until Ballmer was gone, after Apple had started eating their lunch in the laptop space, but the assumption was that Windows was so good people wouldn't switch, not that Apple had bad hardware and good software, if only you could run it somewhere else.
Microsoft is named for being SOFTware for MICROcomputers, they don't have a blind spot around software. This is some Soviet revisionism or something.
> Meta's about to get a rude awakening because "we can fix the software later." As if technical debt doesn't exist and as if you can so easily get engineers that eagerly want to clean up someone else's mess.
I don't know, Meta is probably the largest employer of PHP developers in the world, I bet they can find people willing to do some other masochistic shit too.
> If you have both, you have literally a generationally-defining product (e.g. the iPhone), but even if you just have the software, you'll still eat your competitor's lunch.
The Vision Pro is really a first-product distraction for the Apple fans. This only further validates the market for XR devices and Meta's Quest's lineup will be the cheaper alternative for those who don't have iPhones or Macs (Since Android phones still outnumber iPhones) [0].
I'm only interested in the next iteration when it gets smaller and cheaper and when Apple announces a direct competitor to the cheaper Quest. Probably 'Apple Vision'. Then it will get its 'iPhone' moment, but it is not going to be the Vision Pro.
> Meta's about to get a rude awakening because "we can fix the software later." As
if technical debt doesn't exist and as if you can so easily get engineers that eagerly want to clean up someone else's mess.
Meta is fine. They will just copy Apple and make it cheaper and at worse case, become the Android of XR headsets and glasses.
> I guess it was just too obvious to demo. Or is it?
Gaming can't be good, because there's no controllers. Even if the hand and eye tracking is absolutely S-tier, you're never getting something like button input from swinging your empty hands around, twitching your fingers. Just the occlusion from the back of your hand would be enough enough for that to be impossible.
Fitness might be more doable, but even then I think you're going to be very limited with hand tracking only. I think there's a reason why Apple skipped over those segments almost entirely.
EDIT: I meant "VR" controllers, as in things that are tracked in 3d space. Of course you can use a game pad or MKB, but that kinda misses the point of being in VR in the first place.
At 1:31:50 in the Keynote video they mention “Bluetooth accessories like Magic Trackpad and Magic Keyboard” and show those on a table and a person typing on the keyboard. I don’t believe the cursor of the trackpad is shown (but I’m on my phone and my eyes aren’t as good as they used to be!). I bet it’ll work like on iPadOS.
Why do you say there are no controllers? I was reading about bluetooth keyboards and interacting with a laptop, so why wouldn't a bluetooth controller be feasible (say, before launch)?
I mean it's not impossible, but Apple would have to design and manufacture one. If they're planning on it it's weird that they wouldn't mention it.
VR controllers aren't simple the way a game pad is. They need a full tracking solution, which means sensors with base stations, cameras with some form of vision based tracking, or the headset needs to be able to track them based on some reference points (infrared lights or something). And that tracking needs to be fast, precise, and correctly positioned relative to the headset.
Using the Index controllers with the Lighthouse base stations is kinda plausible from a technical standpoint, but that would mean Apple would need to allow the headset to work with SteamVR, and that seems very unlikely at this point.
If camera based hand tracking works already, it seems like a Bluetooth controller wouldn't be that hard to add in after the fact. Didn't PSVR just use a single camera and a glowing orb?
I agree that keyboard and mouse is superior, but as a grown-up with a family, it's rather infeasible. The Steam Deck has been great; I actually play a game for 30 minutes maybe twice a week. I'm hesitant to even try keyboard and mouse on some games because it will spoil the experience with a controller. When I have the rare luxury of a free evening alone at my desktop, I'm more likely to play VR.
1) Console controls are not a suitable alternative to proper VR controllers
2) What makes you think the PS5 controller was connected to the headset, and not the headset being used as a TV showing the output of a PS5?
Re: 2, they showed it being used to play Apple Arcade games and since they already have support for that it’s unlikely that they have deep philosophical objections.
Re: 1, it’ll be interesting to see how well the camera system works in practice. All of the reviews are quite positive about precise movements so it might be that they’re throwing hardware at this problem but I’d also be surprised if they were not very carefully tracking performance – if the accuracy isn’t there, it’d take an unusually un-Apple like product manager to risk a billion dollar investment rather than enable that class of device.
Using controllers isn't about accuracy, it's about occlusion and haptics.
Using only gestures means no haptic feedback and it gets de-synced as soon as the camera can't see what you're doing.
Take Apples gestures for example (e.g. tap your thumb and index finger together to click). As soon as your hand is rotated in such a way that your own fingers are hidden from the camera, you actions stop being applied. This will also happen should one hand cross the other, or resting your arms down while standing, or reaching up for something, or even resting on a couch with your knee up.
I hate VR controllers. I'd prefer to play VR games without them. I'd prefer they do hand and eye tracking to determine what I'm doing, like look at a zombie and and go pew pew pew with my finger guns.
The only VR headset I've tried that can do hand tracking (Oculus Quest) sucks at it and games there didn't pick up on it as an input at all, as far as I know.
However, just because VR controllers won't exist for the Apple Vision at release it doesn't mean they never will. Perhaps Apple will release some later, or it'll be a third party release. Heck, someone might unironically release a Nintendo Power Glove style controller that works incredibly well with Apple Vision.
My family went to a Sandbox VR store and played a zombie game. In it they gave you “guns” that the cameras could track. I put guns in quotes because when you remove the headset you see that you were basically holding a stick with reflectors on it, but in VR it sure felt gunish.
That’s a unique take. Especially for guns in VR which feel incredibly natural with a controller that is basically a pistol grip with a trigger. For people really into Pavlov they even sell kits to mount the controllers together into a rifle configuration with attachments for a shoulder strap and buttstock. I’ve never heard of a successful shooter game that uses finger guns.
It supports Bluetooth input. I wonder if the visual tracking system that tracks the hands is so good that it could accurately track controllers. Then third party motion controllers could be relatively cheap and plentiful.
Assuming it could run Virtual Desktop, there’s your Half Life Alyx in ultra high res.
having a virtual screen that is way bigger than the space you are in is a huge selling point for me. I'm more interested in this than VR stuff honestly.
if it was under 1k usd I would probably pick one up for work. maybe give it a few more years.
I for one will be sticking to controllers and forgoing eye tracking. Hand input is irritating and limited, I hate using pinch gestures to click on things and the article makes a solid point that the lack of haptic feedback just makes it feel like you're flailing around (which you are). Eye tracking is nice to have in social interactions but I don't see the point otherwise.
> so I guess it was just too obvious to demo. Or is it?
I certainly don't think so? My guess going in was that fitness would be one of the main plugs — especially because high-end fitness devices already cost a fair amount, and that would have made their tail end, "How much would a computer plus a screen plus umm... it's $3,5000!" go a lot more smoothly.
I do think that's likely high on the road map, particularly given such a large investment in Fitness+, but also can see them being very wary about encouraging any high-energy movements with a $3,500 device. I think it's the pricing and dev-kit-ish nature of the v1 more than too obvious.
But Vision Pro does seem to imply a future Vision Sport.
Could be that they're leaving it (and gaming/etc) for developers. They largely showed core stuff only at this point.
The platform launch was very locked down. For incremental hardware upgrades, they put hardware in the hands of devs by either shipping out early or bringing devs to their offices to work on prototypes in a secure environment.
Sweating profusely in an electronics device... I've ruined ski goggles just sweating in them and they're meant for that. Also just stumbling with these on or slipping and there you go cracked $3500 screen.
Honestly, how often do you bang your head hard into a wall/floor? Sure, I'm a tall guy, roughly 200cm, and I sometimes bang my head in a overhead doorstock. But that is typically some old cottage or similar, where I would probably not bring a XR headset anyway.
I was taking while being tired from exercise. While skiing biking etc often. While doing vr, I've broken a monitor and gone through a wall and broken several controllers. While just waking around I forgot my shed had a low door and clipped my head last week first time in years. That's why I wear a effing helmet. You're blind on your stationary bike and pop off wrong. Done. Foot stays clipped in, faceplant.
The Quest Pro has actual VR games and proper VR controllers though. I doubt that such games are possible or probable with eye pointing and finger clicking alone. Apple also didn't include any VR games into their presentation.
People generally agree that VR games are miles ahead of ordinary games in terms of immersion. The adoption problem seems to be that there is no affordable VR console with a lot of big exclusive games. The Meta Quest comes close to a decent price point, but there aren't many publishers making elaborate games for it. A chicken/egg situation with regards to adoption and game investment.
You can safely treat “reviewer on YouTube” as “stranger at a bar” in terms of accuracy. They get paid to get clicks, not to deliver accurate or expert advice, and that heavily favors “why don’t they just…?” comments glossing over challenging engineering decisions.
Yeah. Also, the "why don't they just" (use plastic/carbon) came from me, not him. Apple is long known for using metal in places where it unnecessarily increases weight, because it makes a product feel more premium.
How many industrial design projects has he done, though? He doesn’t have any particular expertise in the area so while it’s fair to ask why they picked the trade-offs they did, it’s pretty far-fetched to think one of the best industrial design teams in the world forgot about a primary success factor, especially given how much other effort they made to hit weight targets. Absent someone who’s actually worked in the field saying it’s unnecessary, I’d give the benefit of the doubt and assume that they used aluminum instead of plastic for a good reason.
There are other headsets without metal frame, so we pretty much know this is not necessary. And we also know with certainty that metal absolutely isn't necessary for laptop cases, but Apple still uses metal. The explanation is easy: aesthetics, feel, "haptics".
There are no comparable plastic laptops with similar performance, portability/thiness, battery life, and quietness — metal is rigid, and an excellent heat conductor that contributes to those goals.
Yes, and Apple also sold plastic laptops when they were still on Intel. Apple has repeatedly attempted to sell products with a cheaper chassis (iPhone 5C, Plastic Unibody Macbook) yet their customers preferred metal enclosures.
We don't really know why Apple chose Glass + Aluminum for their headset, but we do know for a fact that:
- 1 Their customers have historically chosen metal and aluminum for their products, and Apple is in the business of building what their consumers prefer.
- 2 Glass and Aluminum conduct heat better than plastic. The 2hr battery suggests thermals are a great consideration where the materials chosen were not just for aesthetics. Perhaps plastic could've worked, but see point #1.
Have you tried a Quest? I found the SW on that fixed all neaseau I experienced on earlier headsets and I think the Quest 2 was even better but I don’t recall at this point.
It's very varied by person. My friend used a index on my 4090 dedicated (All other monitors to the 2080ti). Running at full frames on google earth. He puked his guts out 20 minutes later and took about 40 total minutes to recover. I was watching the stats the whole time. He just suddenly went urp right in the middle of mt rainier.
I had just showed him WA state and no ill effects, but I didn't have issues on a pre-release vive prototype or release occulus OG while doing barrel rolls and back flips or movement vluprs.
I have a 3090 with HP Reverb G2 and very high tolerance for motion sickness. I’ve noticed in Elite Dangerous that lower framerates cause less motion sickness. The stuttering between frames makes my eyes/brain not treat the visual motion as “real”.
So I crank up the details to super high and enjoy the awesome rendering detail and save myself from motion sickness.
Elite Dangerous is the only VR app that gives me any motion sickness at all and is also my favorite app in VR so far.
I think everyone can agree that artificial locomotion can be motion sickness-inducing. Which you can quite easily do a lot in Google Earths. The faster the worse.
Incidentally Google Earth has comfort options exactly due to that reason, such as the option to narrow the field of view when moving around. I wonder if that option was enabled?
No, it varies hugely person to person. Plenty of us can do completely disconnected motion in VR and not get any nausea. I could do barrel rolls and "Sliding" movement in VR no problem from day one. For the people who are marginal, software and framerate improvements can help, but those "comfort" settings work by killing immersion. There's also a small portion of people that will never be able to play VR without nausea.
Ok. But Index is made by Valve and you're using SteamVR. I've always had motion sickness issues with Valve. For whatever reason, Quest's rendering stack has always not made me motion sick. Even with Oculus Link which has what people consider "crazy" latencies, it's not which means that it's ASW and other last rendering corrections that are hiding better all the things that would cause motion sickness.
Same experience here. For stationary position games I found it fine, but anything with any kind of walking handled by the controllers it felt awful.
Interestingly I tried out F1 2022 in PCVR mode with it and was surprised to find that didn't make me feel ill. A lot of it comes down to the refresh rate and quality of tracking as its the lag that causes a lot of the nausea.
I think that has more to do with mental framing. Your brain is used to being "in a car" and having weird motion. You also usually have the inside of the car being stationary in your field of vision, meaning your brain can look at that and feel fine that you aren't moving.
There's no plauible way to allow you to move around the virtual space without moving in the real world and not give many people motion sickness, since by definition that is exactly what triggers it. The better the illusion of immersion, the worse sickness you'll get when your eyes tell you you're moving but your body and inner ear say you're not.
I don't think that price can always be outweighed by other factors. Successful game consoles are always pretty cheap. Only productivity devices can ask for a high price.
Maybe, but focusing on the "exclusive" part -- the one part that is consumer hostile -- seems weird. If there was a standard for VR games, or if a game existed in both Oculus and Vision, it'd be a good thing. Exclusivity is not a good thing.
What does the word "exclusive" mean in this context? What would be a non-exclusive VR game that can be played on a VR headset? What are some examples?
"Exclusive" in games consoles is a loaded word that means "exclusive to this platform and not available anywhere else".
If the intended meaning is what you suggest, then I'd recommend removing the word "exclusive" and just saying "there is no affordable VR console with lots of big games".
>What would be a non-exclusive VR game that can be played on a VR headset?
VRChat, Superhot, Simpleplanes, Flight Sim, Ultrawings, Phantasmagoria, Htiman, Nearly all VR compatible racing games, Rec Room, Euro Truck Simulator, Most VR compatible horror games like Phantasmagoria, etc etc etc.
I do not need to "look it up", I own and play Superhot on the Oculus for example.
What in your knee-jerk reply would you say contributes to the conversation about whether "VR exclusives" are needed for a given platform to succeed?
I'm going to help you by providing the context for this conversation (not written by me, but in the initial comment I was replying to), which is:
> "The adoption problem seems to be that there is no affordable VR console with a lot of big exclusive games."
There you go. What do you think "big exclusive games" means in this context? Do you think it means "exclusive" as in "Nintendo exclusive" or does it mean "exclusive" as in "it's designed for VR first, regardless of headset brand"? Or something else maybe?
Isn't that the same as saying "VR videogames"? Assuming we mean good quality, and not a minimum effort port.
I was really impressed by the Oculus "First Contact" (or whatever the one with the robot is called). I wonder by there are so few games that feel like it.
Beat Saber is pretty cool, but what killed it in my family is that you cannot see what the other person is seeing, so it's a bummer when you're not donning the headset. Unlike with a regular videogame, where you can watch the screen even if you're not playing.
An affordable VR console with a lot of big games would be sufficient for VR mass market success, but likely some of those big titles would be automatically exclusive for that console, namely for first party titles. Those would first be made just to get the console off the ground. Sony made "Horizon Call of the Mountain" probably just to sell more headsets, not because they expected to make money with it. The install base is likely too low for that. A third party publisher wouldn't have an incentive to make such a big game. Valve probably also didn't earn anything with Half Life Alyx, they wanted to make their Steam market bigger.
The problem with Valve is that their platform is PCs, and a gaming PC + VR headset is too expensive for most people, who often don't already own a proper PC. The problem with Sony is similar: The PSVR 2 is not a standalone console, but just an add-on to a PS5, and PS5+PSVR2 is too expensive for people who are just interested in VR. It's like trying to sell a Game Boy which requires you to already own an SNES for it to function. It would be a flop.
The Meta Quest 2 is in a better position, it's standalone and rather affordable. But Meta can't support it with big games like Sony or Valve could. Meta doesn't have the expertise and probably also not the will to develop expensive AAA games in order too push Quest sales. They are more interested in the metaverse than in becoming a games company.
Nintendo could perhaps make a successful VR console, but it would require massive investments in AAA titles, which would be very risky for them, given their size. And Sony isn't interested in a standalone system.
Microsoft would be more likely. They have huge cash reserves and can afford to lose large amounts of money until they become profitable, as they've shown with the first XBox. But I they haven't shown any ambitions so far to invest billions into an "XBox VR" console and corresponding AAA games.
How much do gamers actually value immersion though? Seems to me the satisfaction of solving puzzles or executing strategy and the sort are the real drivers of satisfaction with games. Immersion is nice, but unless the interaction mode provides something novel to enable better ways to play it’s really just a novelty.
In some cases they use it well, like Beat Saber. But how much is that worth to people?
The problem is that the Quest 2 is what everyone bought and it's arguably not a compelling enough experience to encourage people to put it on instead of playing traditional PC/console games. Most people serious about PC gaming eventually upgrade to an Index because it's still the best option at a great price if you have the room for a full-room setup - and these people definitely play at least a few times a month.
I agree that so far that is where we've been, but imagine a Sim City or Civilization game on the Vision Pro. THAT would be a ton of fun. I'm sure Sid Meier could make something interesting on it. Frankly, I would love it if Apple would throw money at him to make one of his games on it.
One problem to solve is that probably we don't want people to stand for hours and having to walk through the city or the world. They might not have the space at home to do it. They could sit on an office chair and rotate themselves 360°. If they spend 3k on the visor they can spend $100 on a chair. However you don't even have to do that kind of exercise on a screen.
The point is that most games would probably need a special VR controller (not a normal one!), unless they can be played point&click style. But Apple won't include them, so the respective games won't be made in the first place. There aren't even smartphone games which require external controllers, despite the install base of smartphones being orders of magnitude higher than what the sales numbers for the Vision Pro headset will be.
Moreover, making elaborate games like civilization wouldn't be economically feasible even if they could be appropriately controlled by default. The price for the headset is so high that not many people will buy it. Then elaborate games can't sell enough copies in order to be profitable. For example, it seems unlikely that even a success like Half Life Alyx made its development cost back. The hardware install base is too small.
So let's wait again 40 years. 2024 to 2064. But Moore's law will not last that long. The prices per transistor aren't exponentially decreasing anymore.
You can walk up to the walls of the room you are playing in (or bed, or chairs or flower pots or a ball you left on the floor, ouch) but I guess that if this sticks somebody will build a 360° walking pad specifically for games. Meanwhile they can artificially limit how far a player can walk in a straight line. Mazes, small vehicles, etc
>imagine a Sim City or Civilization game on the Vision Pro. THAT would be a ton of fun.
Why? No, seriously, why? People say this as if it is a given, but how would VR improve a game serious that is built around strategy as it's main selling point. How does VR improve Civ as a game? In fact, there are very few genres and types of games that the immersion VR brings is worth the limitations: Weird controls, isolation, headaches, wearing a mask, expense, physical exertion etc.
I'm so tired of people who don't seem to know anything about either video gaming or VR gaming parroting this empty hype of how VR magically makes all games better.
VR is a peripheral, not an advancement in rendering technology.
The same way a huge TV makes a college football game more fun.
It's a city. I live in a real city. SimCity is this flat set of graphics on a screen. I'd much rather be able to view my city that I created and see what it looks like on a human level.
I get your point, and many things would not be good.
But for me at least, SimCity would be. So would Call of Duty (and I know that because I've played 1st person shooters in VR before on multiple occasions).
I'd attribute part of that to there being so few games available that you've got a captive audience and limited content.
Meta was proudly boasting about how the Quest 3 will have "Over 500 titles" - thats not something to boast about, it's just highlighting how incredibly poor the platform has been from a developer adoption standpoint.
I’ve played VR games. They can be a lot of fun and very immersive.
But there are clearly still problems to solve. And some of them do involve hardware capability. Basically for the mass market they’re just not there yet.
The apple headset certainly appears to have significantly more power and capabilities in many ways. It will be interesting to see what pops up.
Though obviously a future version with a lower price will be necessary for it to truly go main stream.
VR games? Sure gaming, but I didn't know VR games specifically made so much money.
Either way, if I was gaming, playing a new single player 2D (Last of Us, etc) game on a MASSIVE screen on the Vision Pro seems so exciting. Beat Saber and throwing my hands around? Not so much.
Have you tried a VR "Big screen" experience? It's a massive gimmick. It's not special. The quality of what you are looking at is lower than using a nice monitor, and sitting in some empty environment that hosts this big screen is dumb.
It's impossible for me to try anything because it doesnt exist right now. There is no VR experience that has high resolution. If the resolution was better, it would be great. Don't care about the "environment" Just project it into my living room like a home theatre.
Even with my racing sim rig, I find myself using my curved monitor more than the Quest 2. It's just such a pain to re-calibrate and update the software and tweak my graphics settings every time I want to hop in that I usually just don't bother.
> VR gaming is a flop. Lots of people bought headsets, played with them for a day and now they're all gathering dust.
A flop indeed. /s From [0]
"Standalone VR headset Quest 2 from Meta (formerly Oculus under Facebook) has outsold Microsoft's Xbox Series X/S consoles, according to the latest information from the International Data Corporation (IDC)."
> Apple is right to ignore it.
Incorrect.
Apple didn't ignore it because they cannot ignore it. Apple is targeting both gaming and for work just like how the Meta Quest Pro has done so.
I have to agree overall. I enjoyed my Meta Quest 2 but it's sat unused for a while now. There were a few games that really were awesome (Beat Saber, Walkabout Mini Golf, and Breakroom) but the last 2 really only shine with other friends and coordinating a time we can all play isn't the easiest. This doesn't even touch on the motion sickness aspect. The Q2 further hampered by the piss-poor UI/UX on the Q2, it really is astounding how bad it is. Once you are in a game it gets better but joining a party or joining an in-progress game is a roll of the dice. When it works it's almost magical but the failure rate is way too high.
The resolution on the Q2 is also just way too low for certain games, I think a MtG-type game would be super cool on the Q2 (and the games I tried in this vein had /massive/ potential that was thrown away completely due to blurry text). In the ends the Q2 has way too many downsides and not enough upsides to keep me using it. I will be getting a Vision Pro at launch though, the resolution alone almost would sell it for me but the emphasis on AR and productivity are the clincher.
But isn't it still the most succesful application of VR?
I don't see another application of VR / AR that will prevent the device eventually gathering dust once novelty has faded. Certainly nothing in the presentation from Apple.
I’m sure people can come up with games or interfaces that would work just fine.
But they also showed full support for pairing standard Xbox and PlayStation controllers and playing with those the way you can on any other Apple platform (except the watch, of course).
They've had developer units but not a publicly released one as far as I know.
When Apple TV was being developed they were handing them out to devs for free like they had millions they were trying to get rid of. I was a lowly single app creator and they happily sent me one.
For the Intel to M1 migration they did something similar but required them to be returned this time, and limited which devs they sent them out to.
The first Intel Mac and the first ARM Mac were DTKs: Developer Transition Kits.
You’d buy them from Apple at a reduced price, could use them to get your software running, and then had to give them back when the real hardware was released.
There was nothing similar for the watch, iPad, or iPhone though.
> Outside US iPhones are nowhere nearly as popular. In many markets Android phones have about 90% of the market share
In my opinion, with the possible exception of South Korea, that is purely a matter of apple's global pricing vs local purchasing power and a huge number of those android users would have an iPhone if they could afford it.
You can get a surprising amount of information about cognitive processes from eye tracking. I worked on the industrial version of HP Reverb G2 which included a PPG sensor and Tobii eye tracking. Was really an amazing product which suffered from having to use third party platforms like Windows Mixed Reality and OpenXR which didn’t have first class support for what was possible.
One notable takeaway is that eye tracking of saccades is loosely correlated with cognitive load. In fact, “cognitive load” was a product that would be licensed to developers to use in their apps so that difficulty levels could be dynamically adjusted to keep users in a sweet spot for learning.
Especially when combined with a PPG sensor which can in theory detect if a man gets aroused (HP never investigated developing a model for that, but some comments from friends who were DoD-aligned contractors expressed alternating concern or interest in it, because they knew from their own career and independent research what applications might be possible with those technologies)
…it could be used to infer things which someone might prefer to keep private (I.e. which categories of things they have an emotional response to).
Apple definitely is aware of this and is heading off potential issues by not only not providing that data to third party app developers but also even to Apple itself. Apple does have a strong pre-existing framework for dealing with sensitive health data, so they could have decided to capture similar data and treat it the same way they deal with ECG data. HP did not have a HIPAA compatible framework so they threw away 99% of the PPG data and distilled it down to just “heart rate” which was the only data source available to app developers from the PPG sensor.
I imagine there will be some developers that try to have analytics fire when certain events happen, correlated with eye-hovering. Hopefully Apple will catch these tricks before they can go live. The potential value of knowing where someone's eyes are (whether they saw an advertisement, for example) could be very high, so I envision a cat-and-mouse game.
Apple addressed that in the keynote. There’ll be an invisible canvas over the window content, and the app never learns where you look (nor Apple, ie it is not further stored or processed). Only when you click (ie tap your thumb and index together) does the app get an event.
(Downside is that one loses the capability for hovering, but strikes me as the correct trade off.)
Sure, that works for certain contexts. But what about a game, where your vision is used to aim a weapon? The app has to know where you are looking in order to know where to shoot. If it's an online game, then the information cannot be isolated on the device since you're shooting at other players.
Now imagine the game has advertisements, or loot boxes, or something else that the game company wants to know if you've looked at. How does Apple prevent them from learning that, if they're able to know what they need to know for the game to function as a regular FPS?
Shooting is typically an action you take, not just a matter of the direction you're pointing in - so this fits the model well: the game gets information about motion to swing the camera, and gets the exact position when you press a button, but doesn't get to know exactly what you're looking at if you're not pressing the button.
The question will be how easy it is to reconstruct that information from the last exact position + movement info...
This doesn't hold true for FPS games. You absolutely need to tell the game where you are aiming so it can draw crosshairs, even for on-rails shooters favored by VR.
most probably you are right. But I could imagine crosshairs are not needed anymore with highly accurate eye tracking, you would just intuitively know where you are aiming anyways.
I expect fps will still require a controller/kbm rather than relying on the eye tracking/gesture input. For one thing, I don’t know how else you would move and perform actions at the same time.
There aren't usually any on-screen crosshairs in VR games. You just use the crosshairs that are on the weapon, which is tracked through controllers. I expect there to be trackable controllers available for Vision too. It's also possible to use hand tracking, but that's not optimal.
I got one last year also, but was disappointed by software selection. My children loved the hardware but wanted more software. At their behest, I tried to write my own software for it, but went in circles trying to figure out what SDK to use.
I wish someone on the SDK team there could observe me as I tried to search for this information and failed because it would be super-informative on how to (and how not to) succeed on creating a wider community of apps...by making SDKs easily available.
<All opinions are completely my own personal opinions>
This is on brand for meta. Look at fb. It didn't come with instructions, but that didn't stop Cambridge Analytica from happening. Look at React. It didn't have instructions and yet it blew up. I bet Meta expected people to pick up the slack, that the best of what was there while they killed time polishing it.
Zuck was probably part of the problem... a room full of POs pushing for stupid shit while building on a rocky foundation.
It’s kind of funny that meta stock didn’t drop on the vision pro announcement. What does that say about the market’s assessment of meta’s vr/ar prospects?
If Vision Pro will succeed Meta could be seen as the cheap-ass alternative. 3500USD vs 500USD is like Ferrari vs. Kia. Not everyone can afford the former.
If Vision Pro doesn't succeed, the market will likely move on from VR completely, leaving Apple and Meta in the same place. Money spent, product introduced, tried, didn't work. Move on.
Apple has the stubbornness and funds to drive this for a decade, and they will. Apple Vision Air AR glasses are coming even if they don't sell a single Pro unit for five years.
maybe the odds of meta dropping the whole vr thing have increased? then they can stop lighting piles of money on fire because zuck read snow crash as a teenager? OTOH, maybe apple will show meta how a headset can actually be done and they can mimic their way to product market fit?
Not because Meta had a better product, but Apple putting its might into this product category means that "Meta" is vindicated in investing here -- that there exists / will exist a viable market for this technology in coming years.
All the damage done by the poor quality 3D renders of AR/VR worlds from Metaverse -- that made a layperson laugh at the concept of AR/VR -- is probably partly offset by the positive press and marketing budgets that Apple will bring to the table.
> It's insane that a billion-dollar company like Meta actually felt proud to release such a steaming pile of trash
Let's be real. You can have all the money in the world, but that doesn't guarantee you access to talent. It doesn't guarantee that you made insightful technical decisions that hamstring you later in the product cycle. It doesn't free you from short-sighted managers or employees who just care about making it to the next vesting cycle.
When you're a billion-dollar(haha, not so significant these days) company, you have a tremendous pressure to show results. You have tremendous pressure to show innovation and be first to market. It has no bearing on the quality of the products you release in the short-to-medium term.
It’s a long game. In both desktop and mobile, a viable and commercially successful alternative emerged even though Apple had the edge in sophistication and innovation.
A lot of people probably don’t remember what Android looked like in 2007. Using the Android pre-release SDK next to an iPhone, you’d come to a similar conclusion as yours: how does Google release a mobile OS so behind the curve? Should they fire all their product managers?
Same story with Windows in 1986. It was difficult to see how the graceless file manager from Microsoft could compete with Apple’s holistic GUI.
Give it four years. I think Meta intends to be the Windows and Android of this third wave of UIs. Validation of the market by Apple is exactly what they’ve been waiting for.
i dont understand this mentality, obviously the intent is to improve the experience over time, creating a VR platform itself is an ambitious task. Are there no happy customers of the Quest?
metas strategy is to release a cheaper consumer product and improve it iteratively. Hate to break it to you but day1 apple vision pro will have software bugs too.
I echo the awe at the hardware and the disappointment at the paradigm. I wish the keynote had three extra demos:
- Something whimsical to inspire developers. Give me files represented as physical blocks that I can pile, or a task manager that shows processes as plants around me. Close apps by scrunching their screen like paper. A globe with my geotagged pictures.
- A game, any game. Beat Saber was the killer app for me to get my Vive, and Valve already spent tens of millions of dollars to create a triple-A VR game. Neither seem compatible with the Vision Pro input methods. Apple could at least play their strengths, like an AR hide-and-seek, or a horror game with eye tracking.
- Content creation or professional work. The last thing we need is another passive device to watch 3D TikTok. Show someone customizing their "home screen" environment to look like a fantasy potion shop; or a mechanic looking at a 20x magnified broken part; or an SRE watching their kubernetes cluster as a floating 3D graph; or a VFX artist in the scene scrubbing forward and backward to adjust an effect.
It feels like inventing the first smartphone, camera apps and all, but it doesn't make calls and the only input is tapping on icons.
I used Eagle Mode[1][2] for a number of years, and it was non-ironically the best file manager I've ever used. I only stopped because it required dozens of very out-of-date packages for its preview features.
But the point of whimsical demos is inspiration, like a concept car, not complete realism.
VisionOS uses old school UI like blocks and doesn't do anything new in terms of just reusing design paradigms from ipadOS/macOS. It makes sense why people would think of it just as a computer they strap to their face.
Half of their marketing was about how it's a essentially a macbook that you strap to your face...
It takes time for developers to get the device and begin writing software for it. As time marches on, those things that we all find value in get bought by Apple or Google and become the default apps for things. If someone then decides to try something crazy, cool. If not, someone else later will make a competing device that tries. If it wins, cool. If it doesn’t, someone will make it way later still (like Apple recreated what General Magic was working on way later).
By far the most disappointing part of the keynote was when they finally do show someone playing a game, and it's a basketball game played with a regular controller on a flat, virtual TV. The TV is even smaller than an average real-world TV.
It's typical of Apple to so completely ignore gaming, but it's disappointing that they're continuing that trend even with VR.
Realistically, that's probably the best we are getting for a while. VR gaming has had a difficult time. The physical space requirements are immense and the gameplay types have been limited.
A lot of value could be had playing flat screen games this way. Imagine sitting on a plane with a controller in your lap, but it feels like you are sitting in a private loungeroom or some zen environment playing a game on a large TV.
I didn't expect them to show a first-person shooter, but there are so many possibilities for casual games that actual utilize VR/AR in some meaningful way.
The fact that what they came up with was "game in a rectangle, but this time floating in the air" makes me think that either there are major limitations to what the AR platform is capable of or that they're intentionally trying to tell people "gaming isn't a priority here".
Because gamers, or even consumers, is not the target demographic for a $3500 device. They want to brand as a device for enthusiasts (but Apple everything!) or professional (work video chat, artists, graphic designers).
you mean the demographic that regularly buys enthusiast tier 2000 dollar pcs and 1500 dollar VR headsets is not a demo that apple wants to buy their enthusiast tier 3500 dollar device?
apple's anti-gamer slant has always stunk to me of some exec being salty about what went down with nvidia and hating the idea of ppl playing games.
their slapfight with epic games is another example of them not giving 2 shits about the gaming market and alienating a highly growing userbase.hell they didn't mention live streaming at all and that seems like a usecase this thing is tailor built to be good at considering how many cameras and screens it has!
meanwhile they want people to buy into apple arcade...
>- Something whimsical to inspire developers. Give me files represented as physical blocks that I can pile, or a task manager that shows processes as plants around me. Close apps by scrunching their screen like paper. A globe with my geotagged pictures.
I respect Apple for not doing this. These are demos that are kind of fun for 20 minutes but you would never want to actually use a device like this: https://youtu.be/z4FGzE4endQ?t=59
His comments on anchoring virtual tools in physical space reminds me of the work Microsoft was doing with Hololens and Windows Mixed Reality. If I recall correctly, there were APIs for anchoring things in space.
I recall stumbling upon those too in the Win 10 mixed reality API docs. There were a lot of interesting things in there that never really saw the light of day/any significant use
While I do enjoy me some MKBHD, I'd like the opinion of someone who's not fully bought into the Apple ecosystem already. This is a good perspective to have, but it's an incomplete perspective.
And I'd like to know how realistic this thing is for "not apple users". It's a VR headset, can I use it with Windows to play MS Flight Simulator or Star Citizen? Can I use it on Linux to get wall-to-wall emavs? More real world testing is required.
It’s not a VR headset, it is a computer with its own OS and M2 processor. I don’t think that VR or AR was mentioned a single time in the Apple event. They say this is the first device for spatial computing and that this is equivalent to the first Mac as far as changing OS UI goes.
This will be another part of the Apple system along with Macs, iPhones, iPads, etc. You will never be able to use this as just a monitor equivalent for any other OS.
Right, so: it's just another VR headset, just beefier. Have you not been paying attention to Meta's Quest and the like? Modern VR headsets are computers, they run their own OS, with applications/games loaded onto the device itself, and they can be used as a VR display client for other operating systems.
This is an M2 computer strapped to your head: it can bloody well do VR, let's see what folks do with this device that Apple never asked them to, but they do anyway because Apple doesn't get to tell them how to use their overpriced hardware (especially if the moment you buy it, the buyback is $50 if you're lucky)
Quest Pro owner[1]: it's "a total waste of money", "embarassing how much of an unpolished piece of garbage it feels like", it's "a steaming pile of trash", "infuriating", "buggy", "janky".
MKBHD on the Vision Pro: better image passthrough, better responsiveness, better hand tracking, better eye tracking, better image quality than any other headset he's ever used.
Someone can't say "let's wait until people have them for review before speculating" and then describe them as "overpriced" at the same time, those two are mutually exclusive.
They most certainly aren't, given Apple's track record for adding about 200% markup on every single product they've made in the last 20 years. This product is overpriced until proven otherwise even if it does exactly what they claim and it beats the competition by a country mile. If it's not $1000 over an already healthy profit margin, that'd be an Apple first in a very long time.
200% markup compared to what? The price they paid for the stuff? This product is not overpriced until any competitor makes anything like as good, and sells it significantly cheaper (people who pay for branding get value from branding). Which we won't know until ~2024 when this thing goes on sale.
Why? It's only been out for a day, you telling me everyone's already had their run at hacking it? That seems like pretending you know what folks will do with this thing.
This reminds me of people who gripe about sports rumours in forums about rumoured player signings. If you're not interested in speculation about hardware not yet freely in the hands of users, why not leave it to people who find that discussion interesting?
There are reports now from dozens of people who've had similar 30 minute sessions using some of the features. That's enough to get an idea of some facets, guess at what's not polished, what's not ready at all. But they're all easy to ignore if you want to wait until next year.
I think the big usecase of the Vision Pro is just a larger screen and immersion. That's what it looks like Apple highlights in the demos.
Imagine developing with the ultimate 10x setup: several files, documentation, and debuggers open at once, with a zen nature background. Or making music, or painting, or even writing a research paper.
Imagine playing a video game but the environment wraps around you. Imagine watching movies in a virtual theater with a simulated 100-inch TV (one of Apple's demos). Even reading and browsing the web can be improved with an ambient environment and extra space for more stuff.
Is it worth $3500? For end-users probably not, but if it genuinely makes professionals and hobbyists substantially more productive, it will be worth $3500 or even way more. How much money would you spend to write, code, create faster?
Of course, this assumes the VR actually performs, and whether a bigger screen and immersion actually makes people work faster. As of now it seems VR is still a gimmick which impresses people at first, but doesn't provide much outside of niche experiences; which, if this holds for the Vision Pro, makes it very much not worth the price.
If you want that immersion, I think $3500 is totally reasonable. I paid more than that for my two additional monitors.
The problem for me is that I like monitors better. I like the feeling of real space around me. I cannot conceive of wearing googles on my head all day. I just don’t want to.
And I definitely don’t want to be separated from other humans by googles. It just not attractive to me.
> Imagine developing with the ultimate 10x setup: several files, documentation, and debuggers open at once, with a zen nature background. Or making music, or painting, or even writing a research paper.
I used to believe this was the ideal setup, hell I even implemented it with monitors, but since I've integrated LLMs into my development process I find all I need are two side by side windows and now with CopilotX those are both in VSCode. I occasionally venture out for documentation but even that will make its way into the editor at some point. Using Edge and the Bing sidebar I can even save myself the trouble of reading through all of the documentation on a page by just asking Bing to retrieve the info I want from the page I'm on.
Doing all that in VR would be cool but I don't think it's going to increase my productivity nearly as much as LLMs have.
Disclaimer: I'm bullish on VR and I love gaming and socializing in VR
> but since I've integrated LLMs into my development process I find all I need are two side by side windows
This irrelevant for those of us not doing cookie-cutter-been-done-1000000-times-before stuff. In such cases all the LLMs in the world are useless, but many pages of docs open at once are useful. Go tell your LLM to write a ARMv6M assembly MIPS-II emulator. I'll wait. Go on.
I'm sorry that workflow doesn't work for you. Must be hard not getting to use modern tooling. If you care to, maybe try it out for some rubber ducking or inspirational uses, it's not limited to strictly technical things.
The pixel density appears to be about half of a no-frills monitor at a usual distance.
There's plenty for movies and games but just not enough to emulate your 2-4 working windows at a reasonable angular size.
> I think the big usecase of the Vision Pro is just a larger screen and immersion
No one has come up with anything better yet. That’s one of the big problems with VR that I’ve seen. Do you have games on one hand, and a virtual environment showing mostly 2D stuff on the other.
I think that’s one of the reasons Apple is releasing this when they are. I sort of suspect they’ve gone about as far as they can. They don’t have a killer app, but that’s ok.
They’re putting it out there for developers, and we’ll see. Someone will come up with something compelling.
VisiCalc, Lotus 123, WordPerfect, Photoshop, PageMaker, Angry Birds, the web… none of them came from the platform vendor. So what will someone come up with that will make everyone rush to order the Vision 2 when it’s announced for $1500?
Regarding the last part, I was wondering if an AR UI would maybe start a trend back to a more 3D/skeuomorphic design (in the sense of NextStep/Windows 95), which then would also have a comeback on the desktop. That's mostly just wishful thinking on my part, though. ;)
From years of observing first-hand how hard it is to develop software, I’m shocked that Apple continually makes products that fit their aesthetic (eg. The crown and headband evoking the Apple Watch) and have coherent and mostly-baked software on launch like this. Is there a book about how they’re able to operate at scale and deliver such consistent things? Do they just have an insanely good product team? How do they document requirements? It’s fascinating to me, because I think it’s their real competitive advantage - they get to the finish line with product that would make sense to a single, invested, super smart person, instead of a pile of inconsistencies and incoherencies that contradict each other as you move through the application. The only other product I feel is delivered at Apple’s level is CloudFlare, and I think that’s why they’ve grown so fast.
Hypothesis: Apple is a hardware company that also does software, and that ethos allows them to ship more complex products over much longer time scales than their competitors.
I agree. Sometimes it feels Apple ships software too slowly. For example we have to wait for iOS 17 to be released to get a Whisper level dictation on the iPhone while Whisper has been out for almost a year. But maybe shipping at the pace of hardware is a good thing. Maybe we shouldn't deploy our sites on each commit
You could say the same about Dyson or Oxo or Bodum. Having a single esthetic and coherent product line is a choice.
On the face of it, it sounds like a company would want coherency, but in practice it means leaving a lot of money on the table for every single of your products. Any consistent design decisions means anyone who's not pleased by it has no option in your lineup to go to. That's of course a gamble Apple is willing to take.
For instance, Apple consistently uses glass and metal in their product design, the only exception I can think of being airpods and their mouse. I personally prefered the plastic for weight and comfort, so that set me out of the market for their headphones and I went to Sony for instance.
Same for laptops with touchscreens, if you care any bit about them Apple is a no go to you. Same for so many things.
PS: on inconsitencies and incoherencies, Windows is the king of the kitchen sink approach, and it's consistently stayed ahead of Apple in numbers. Another instance on how being opiniated isn't only a positive aspect from a business perspective.
Your point is valid, but that ... diversity? ... comes at a _very_ steep price. Not only for the manufacturer, also for the consumer! With Apple I _know_ a couple of things about any future purchase, without ever having used, held or felt the particular product. Namely: 1) Build quality will be significantly higher than competitors, 2) It will hurt my feelings to look at the bill after checkout, 3) It will work as advertised (or slightly better).
Yes there also will be quirks I'll be unhappy with, which I _could_ (given enough time and experimentations) find a better solution for. But fundamentally I'll be getting a solid product, that works well enough. It is _that_ promise, which fundamentally allows Apple to do an online store and do a mostly directly-to-customer retail operation.
If you increase diversity, you make this "buy any product and you'll be fine" thing so much more complicated, loosing more sales than the niches even contain.
I'd guess almost all people just want it to work :TM:, at least that's me, and that's me deciding the phone of my mum
I totally see this point, in particular as an opportunistic Apple customer myself. Apple doesn't need to do marketing regarding some aspects of their products (e.g. build quality) and can focus their messaging on more specific parts.
On the other hand, that doesn't seem to be what most consumers long for. Apple had two runaway successes with the iPod and then the iPhone, but to note, the iPhone is not the best selling phone brand nor dominates the international market. Same way Apple is not the #1 computer maker.
So yes, focus and consistency brings tremendous value, but there are many other ways to get that level of value, and many customers will have different priorities.
(I also don't think any brands need to be #1, being profitable should be enough, but I am more and more interested by how Windows has stayed at the top for so many years. In particular Apple progressively feels closer to Sony when it was at its peak: superbe products, strong ecosystem binging, coupled with also a heavy dose of strategy tax on how to benefit from what the other companies are doing)
From everything I've heard, Apple's org structure is a lot more focused and UX-driven from the top down than basically any other large tech company. That means you get a lot less compromising and in-fighting.
The other side is how Apple makes money compared to other companies. Meta makes money mostly by selling you (the user) to advertisers (the real customers). Same with Google (70%ish ad revenue). Microsoft is a hot mess of competing B2B fiefdoms (the vast majority of their revenue comes from businesses, not consumers). Amazon only makes a few products directly for consumers, and mostly makes money from being a marketplace + AWS. That leaves Apple fairly uniquely positioned as a premium consumer hardware company that also sells consumer software services to enhance the experience and... drive more hardware sales.
Apple builds an insanely strong core from which any and all other teams can (maybe must) build on top of and use to power their team's platform.
You have a core messages team for example - off of which Mac, iPhone, and Apple Watch teams integrate. Do that for each core service or product (photos, safari, notes, music, etc).
The Amazon structure is such that everything flows from the person above you, work is not shared between teams.
One is compositional (Apple). The other is inheritance (Amazon).
If you look at it closely, any first level node with a child node at Apple can be bypassed by the central boss. The boss is connected to every other node.
Creative Selection by Ken Kocienda is the closest, modern look inside Apple's product development that I've found. Focus is on the development of the software for the original iPhone and is a great book.
Yes! this book is a first-person account of product development at Apple, showing how both the design and the features of a system tool keep progressing under the selective pressure of iterative review sessions, with creative and specific challenges provided by managers at multiple levels, all the way to the top (Steve Jobs at the time of the book's tales). It's a great book, very well written, the only one I've read that provides insight into (some of) Apple's creative process. When they say that Steve's DNA has profoundly shaped the company, I imagine this is one aspect of what they mean.
It's Steve Jobs' legacy and a function of the authoritarian, highly controlled process he created. A democratic self-directed environment like Google or design-by-committee like Microsoft could never produce such a product line. Even an Intel with billions more in funding cannot recreate TSMC factories.
If I were to speculate I'd say it's because they don't ship things until they are finished. I'm sure a lot of the polished launches we have seen have been late by 6, 12 or 24 months. If the market isn't ready, the product doesn't ship. If the technology isn't ready (say, a high enough resolution VR display) to create a wow product then the product doesn't ship. I think it takes a great organization to not "accidentally" ship bad things. You need to prevent bonuses, egos, sunk costs, and self-deception from shipping something bad prematurely.
Reading through his notes, I couldn’t help but think “I don’t know this guy, but it seems like he has spent years thinking deeply about interfaces. Also, he sounds ex-Apple.” It made me smile when I got towards the end and my suspicions were confirmed.
Andy’s thoughts were infinitely deeper than anything else I’ve read about the device+interface so far. In particular, I liked his observations that so far it’s mostly a 2D plane, with exceptions like the Breathe app and a few demos, which might just be good for demos.
Seeing the heart in 3D reminded a lot of what we saw 10 years ago around what book publishers were trying to do with the iPad. Cool demos but limited real world use.
Here was one particularly thoughtful section:
“unlike the physical version, a virtual infospace could contend with much more material than could actually fit in my field of view, because the computational medium affords dynamic filtering, searching, and navigation interactions (see Softspace for one attempt). And you could swap between persistent room-scale infospaces for different projects. I suspect that visionOS’s windowing system is not at all up to this task.“
Reading through it, I was struck by the contrast of how broken the web page was with what the author was saying about UX. The author seems experienced in the field, but the site felt completely broken. I don’t know if it’s a device or browser issue, but reading on my iPad, if I tried to use my right thumb to scroll, the page acted as if I had reached the end of the article, even though it was clear there was more because the last visible line was only half-visible. (And this was after having to manually zoom in because all the text was shoved to one side and tiny.) Scrolling with my left thumb worked about 50% of the time. The rest of the time it worked as poorly as scrolling with my right thumb. I gave up after about 5 paragraphs.
that site is not really suitable for mass consumption, iirc it used to even have a caveat that said it really only existed to make certain leaf nodes sharable. it's basically a personal zettelkasten, but this update was also sent to his patreons as well, where the writing just ended up being whatever your email client made of it.
If you visit his top level website, you can see that it uses a columnar layout. I agree that it compromises the experience if a single article is linked like this, especially on smaller screens, but it isn't broken, this is how it is intended. It just doesn't cater to all formfactors.
Did we see the same website? The page I visited appeared to just be straight text. I’m on a iPhone with safari. It felt like a totally normal blog post by someone who doesn’t care for the current js heavy web.
> What if programs live in places, live in physical objects in your space? For instance, you might place all kinds of computational objects in your kitchen: timers above your stove; knife work reference overlays above your cutting board; a representation of your fridge’s contents; a catalog of recipes organized by season; etc.
I'd prefer an OS that spans multiple devices. Something how computers behave in “The Expanse” where someone can flick stuff from their mobile device to another system for a better workspace. If I'm typing this on my workstation, but want to switch environment and bring something light like a tablet or just my phone, I could send the tab and it would retain its state. The closest to something like this is a server-client architecture like Logitech Media Server. A paradigm like this would be more useful than immersion.
This is also why I like Vim. Buffer is a separate concept from File, adding an extra flexibility – in terms of layout – to how I edit to complete my current task. I want a similar flexibility with applications, at least lightweight – state wise – ones.
CloudKit and Handoff are close, but very brittle. Apps like Bear, Reeder, Anybox, Things 3, make switching devices seamless. Another good example is handoff between the HomePod and the iPhone.
> Apps like Bear, Reeder, Anybox, Things 3, make switching devices seamless
I think Apple's whole point is that they provide the APIs so that apps can do this themselves. Which, yeah, is a bit disappointing; I'd love to have a 100% fidelity, flick-to-another-device type of Continuity built into the OS, but the APIs are at least a starting point.
Sorry for hijacking this thread for a question about personal curiosity but does anyone know of an affordable way of measuring pupil distance?
I want to measure my pupil distance while browsing social media. The reasoning is my reading in "Thinking Fast and Slow" that pupils dilate when we see something we find interesting / something we like / when we are thinking and vice-versa. I want to put it to the test and it seems like finally the consumer tech is close to making it possible.
Does quest support pupil tracking? I did some cursory research but couldn't find any reference for it. Industry pupil tracking headsets are way too expensive; My last hope is that someone will jailbreak vision pro...
I was hoping to see more paradigm shifting user interface ideas. I think those exist inside Apple Park, but are waiting for their time. It's telling that this was announced at the developers conference. The focus now is on the platform, and I think mind-blowing user experiences will come later. (At least I hope)
698 comments
[ 0.24 ms ] story [ 352 ms ] threadI actually see this as a bit less surprising. After all, if you change the hardware in a big way, and you change the software in a big way, users will have a harder time adjusting to the new platform. Instead, they're making a big leap on the hardware side, keeping legacy apps and concepts, and then will presumably iterate to more 'native' experiences that were previously impossible/unimaginable/unintuitive.
We are all highly trained to read text, it seems basic but it is in practice quite abstract.
Text is still best read on a flat surface.
The great innovation I can see with this new Apple device is eye tracking, they have not invented it, but they might have perfected it enough to be useable.
Eyes could be better than a mouse.
That lacks other features of course and must be tethered to a PlayStation 5.
But eye tracking + foveated rendering seems like it’s going to become table stakes in the next few years.
It would be funny if controllers evolved to be more like the single-joystick models that we had decades ago, with the joystick on the left and rows of buttons on the right. History doesn't repeat itself, but perhaps it'll rhyme?
In top down bullet hell shooters the player had to aim and shoot in one direction as it walks in another (potentially opposite), so one stick is still needed
edit: ah, right the vision pro does have lidar. I look forward to seeing this pinning if it's implemented. I'm still skeptical it would work well - I've used ARKit and it's not really that accurate when you're moving around.
And some might argue you just switch apps to find that "screen".
Personally I would put something interesting on the ceiling and look up at it.
[1] https://youtu.be/TX9qSaGXFyg?t=180
[2] Same video, 5min 12s: https://youtu.be/TX9qSaGXFyg?t=312
I've been researching eye tracking for my own project for the past year. I have a Tobii eye tracker which is probably the best eye tracking device for consumers currently (or the only one really). It's much more accurate than trying to repurpose a webcam.
The problem with eye tracking in general is what's called the "midas touch" problem. Everything you look at is potentially a target. If you were to simply connect your mouse pointer to your gaze, for example, any sort of hover effect on a web page would be activated simply by glancing at it. [1]
Additionally, our eyes are constantly making small movements call saccades [2]. If you track eye movement perfectly, the target will wobble all over the screen like mad. The ways to alleviate this are by expanding the target visually so that the small movements are contained within a "bubble" or by delaying the targeting slightly so the movements can be smoothed out. But this naturally causes inaccuracy and latency. [3] Even then, you can easily get a headache from the effort of trying to fixate your eyes on a small target (trust me). Though Apple is making an effort to predict eye movements to give the user the impression of lower latency and improve accuracy, it's an imperfect solution. Simply put, gazing as an interface will always suffer from latency and unnatural physical effort. Until computers can read our minds, that isn't going to change.
Apple decided to incorporate desktop and mobile apps into the device, so it seems this was really their only choice, as they need the equivalent of a pointer or finger to activate on-screen elements. They could do this with hand tracking, but then there's the issue of accuracy as well as clicking, tapping, dragging or swiping - plus the effort of holding your arms up for extended periods.
I think it's odd that they decided that voice should not be part of the UI. My preference would be hand tracking a virtual mouse/trackpad (smaller and more familiar movements) plus a simple, "tap" or "swipe" spoken aloud, with the current system for "quiet" operation. But Apple is Apple, and they insist on one way to do things. They have a video of using Safari and it doesn't look particularly efficient/practical to me [4].
But who knows - I haven't tried it yet, maybe Apple's engineers nailed it. I have my doubts.
1. https://uxdesign.cc/the-midas-touch-effect-the-most-unknown-...
2. https://en.m.wikipedia.org/wiki/Saccade
3. https://help.tobii.com/hc/en-us/articles/210245345-How-to-se...
4. https://developer.apple.com/videos/play/wwdc2023/10279/
I think unless Apple has really added some sort of magic, it will be an issue for most people to use Vision Pro for anything but passive activities, or with a physical trackpad.
Compare with a smartphone, I can move to tap the 'back' button in Safari without moving my eyes to the back button. Looking "at" the phone generally is enough to see where my finger is, and say clicking on links I can look at the link and move my hand to it and move my eyes away while my hand is moving.
Having to coordinate looking and clicking - while I haven't tried it - I can imagine that feels more load-bearing, more effortful, more constraining, more annoying.
I'm doubtful we really know how good it is yet. Without a few hours/days of hard use its just not going to be obvious the degree to which they've overcome all these long standing obstacles via "magic".
It's hard to think voice is anywhere near ready for this application, I still find it highly frustrating after decades of tries.
I’m curious about how they deal with apparently not doing this, in terms of how do you know what button is going to be clicked if you click? The product page under privacy says that websites and apps don’t get eye tracking data, only knowing when you click, so I guess that means a website won’t display its active/hover effects on a button you’re looking at. Because if it could then it could just link that to analytics in JavaScript and defeat that privacy protection. So if you don’t have that, how do I actually know what’s going to happen when I click?
With Apple solution (or any VR with eye tracking):
1. You have very accurate head 6d location (gyroscope, compass, accelerometer + lidar for translation) at 100+ fps minimum and very computationally cheaply without much latency
2. Cameras pointing at each eye from distance of few centimeters. In this situation Eye region of interest occupy full image resolution. Probably even VGA resolution (or less) is enough to have very accurate precision in this case
3. Because headset covers your face you don't have any problem with different lightning condition, cluttered background, whats more they use IR floodlight to have consistent lighting of pupils
4. Possibly they even use similar Truedepth IR dot pattern to get 3d depthmap of your pupils similar like truedepth sensor works with difference that here they can put thos camera more far away and have it more accurate disparity map
5. Who knows maybe they even use lidar instead of truedepth for even lower latency
6. Because sensors are so close they can have very accurate reconstruction of your eyes and distance between them - most people don't have completely symmetric eye/face
Tobii is a sensor that is just like 0.5m far away from your face. Has to track your head, estimate orientation and distance, and detected eye ROI will be a tiny crop of their image sensor even if they camera has 4K resolution (which I doubt they use such resolution). At that distance if you want to get accuracy of 1 cm of what you are looking at screen you need to detect pupil movement of ~1 degree angle. If you want even better accuracy without gitter like 1mm then you need to detect movement of pupils of 0.1 degree angle.
I'm disappointed even though it's entirely predictable that VisionOS is built on the iOS/iPadOS foundation rather than OSX. I guess we'll see how "walled in" it is but it's hard to see any reason Apple isn't going to be just as constraining and controlling about what happens on this OS as they are on iOS, if not more so. Which ultimately means I'll be very reluctant to ever adopt it in any meaningful way as my primary computing device.
On the other hand, the closed ecosystem is definitely cause for concern. Fingers crossed that WebXR support comes out from behind a feature flag to allow for progressive (spatial) web apps.
Hmm. Marketing horse-puckey.
A 2018 Dell XPS that has a Wacom digitiser and OneNote gives me lower latency and better inking experience than an iPad and pencil 2. I used them side by side for long-form writing for a while and the iPad was left in the dust.
Part of (my) definition of a platform is that when you buy it you are fully paying for the cost and contribution the platform provider made when you buy it. If there's a residual financial (or other) obligation after that then you don't have a platform, you have a partnership. I know there's other definitions of platform but that's the one that's important to me.
I guess you can always buy an android based headset that allows sideloading but presumably it won't have the same level of investment.
As far as this goes ...
> it won't have the same level of investment.
Meta is constantly mocked for how much they have invested. You can argue about how much it paid off, but they certainly invested.
I think the point is that Meta’s investment is weak compared to Apple’s full court press. It really is laughable.
Ouch. I'm sorry to hear that, I feel sorry for folks like you that are sensitive to nausea. There's so much potential in this tech but I can see a whole new class of disadvantaged people coming who aren't able to fully utilise it.
If that same dev would have made the same profits on other competing platforms, Apple is just fragmenting the market and it's take should be limited to basically management fees.
I hope this new platform brings newer applications and innovations, but also expect competition to adjust, making it a situation where Apple is just one of the many platforms.
iOS sure brings some value to the table. How much actually ? who knows...
For comparison Youtube is also a platform regularily criticized by its participating creators, but you're not seeing high profile lawsuits or the EU slapping them fines at every turn.
What you’ve said doesn’t change this at all.
That number is the total revenue for the whole store (there's a lot to say about it, since Google is also not the only Store for android, in particular in China and Korea), and it matters very few for any individual developper.
It's not because the total amount of in-App purchase for all games is twice as much on iOS that my new ssh shell app will make twice as much on iOS than on another Store, for instance.
Of course individual products have their own idiosyncrasies, but is it in fact the case that on average, an app on iOS will make more than the same app on Android.
But we all know the difference between implementing something brand new versus retrofitting something into an existing system. So my guess is iOS had more secure defaults for a system like vision.
So they're effectively recreating a lot of the elements of the macOS w/r/t window positioning, resizing, and depth, but building all on top of the iPadOS paradigm. If you consider the input methods are probably closer to a touch interface than a MKB interface then it kinda makes sense.
Personally I have zero interest whatsoever in living inside Apple's "only what we allow you to do" world, and the fact that they seem to be expanding it to what seems like a very powerful, desktop replacement computing device is kinda gross to me. But from an interface standpoint I think it's really interesting and actually makes some sense.
They’re the same foundation, iOS is a fork of OS X and was originally announced as such. I’d be shocked if there isn’t a well maintained internal build of iOS which keeps parity with macOS windowing etc.
The difference in restrictions between the platforms is entirely arbitrary. If this device appeals to Mac users or would-be Mac users, I’d be shocked if it doesn’t at least eventually allow a Mac-like environment.
"Limited" makes it, well, limited, but it has performance advantages: fewer unexpected page faults mean audio/video processes won't unexpectedly drop a frame as often, and there's fewer disk writes which is good for the lifespan of the hardware it runs on.
Btw iOS actually has a very limited swap policy called "freezer" that only applies to suspended background apps, and only when it's in the performance budget.
No matter how fancy the visuals, it’s hard to have a pseudo-hands free voice interface that doesn’t uh, work?
I switch between iPhone and Pixel and Apple/Google products and each has a lot going for it. But leaving Assistant for Siri is downright jarring.
Why Google made the shitshow that is Bard when they could have just iterated on Assistant I’ll never know. Such a solid product.
It would be really nice if there was a dictionary you could add technical words to.
I strongly doubt you're right. Apple is full of geeks; just because they tend to be tight-lipped doesn't mean they aren't paying attention to all the same stuff we are.
Whereas I’d wager more than half of Google employees use iPhone, and a lot of Google staff work exclusively on iOS and Mac apps. Apple software runs in their hardware, period. It makes them too insular and unable to see their shortcomings.
Hell it was bad as a voice assistant now it's gonna get lapped by other companies releasing personalized AI assistants.
[0] https://github.com/openai/whisper
No it’s not. Try doing literally anything without an active internet connection and she’ll say “I’m having trouble connecting”
Though I do remember reading you can connect it to your mac, so who knows. You’ll just need 2 $2000+ devices instead of one.
I'm extremely curious to know how Vision Pro deals with this. One certainly hopes there's some kind of "single eye" mode; certainly seems possible with relatively small effort and the % of the population who'd benefit seems fairly significant.
Eye tracking most certainly sounds like the way to go, relative to hand-waving.
The Minority Report movie probably set the industry back by a decade or two. Waving your hands around to control stuff seems logical but is quickly exhausting.
Honestly, I think a large part of the industry taking into account accessibility in a more serious ways comes from the competitive advantage apple has created.
Doesn’t mean accessibility hasn’t been done in the past. By no means. But it’s a market segment they have on lockdown
EDIT: found it — he didn't talk about it, but it was reported that he had to frequently rest his arms: https://medium.com/@LeapMotion/taking-motion-control-ergonom...
Artistically it was a great choice, and they certainly weren't intending to inspire a bunch of wrong-thinking UI choices in the real world.
However, waving your hands in the air for 8-10 hours a day feels a bit unnatural relative to "actual" physical work. I don't have a real scientific basis for this statement.
But there are some physics at play IMO. In boxing it's generally accepted that a missed punch consumes about twice as much energy as one that connects -- your body must do the work to accelerate your arm, and then it must do a roughly equal amount of work to decelerate it. I think there are some parallels to waving one's hands in the air for 40-50 hours per week relative to "actual" physical labor.
More to the point: MR-style interfaces (at least as typically implemented/depicted) don't really offer tangible advantages to using a good touch pad. They trade tiny finger movements on a trackpad for huge sweeping arm movements that accomplish the same things.
I think if MR-style interfaces had somehow been invented before mice and trackpads, we'd be celebrating mice and trackpads as absolute miracles of efficiency.
On the note of tiring out, I have been strength training for 12ish years now and I still get tired quickly when holding my arms above my head to work on my car. I think because they are a smaller muscle group, they can be saturated easily. I don't have the same issue with repetitive motions, it's just holding the arm in a similar position that does it.
Maybe if AR gestures take into account full motions rather than holding the arm in a similar position for too long, it might not tire so easily.
I have VR already and I will say it's an exhausting experience in general, I can flat screen game for hours, but in VR I want out in less than an hour. I think it's the full focus it forces on you, and the split world spatial reasoning going on.
The aches and pains were there, but just in different spots. Yes my core was stronger, but holy hell were my feet exhausted.
But now that I am also in the future I still use USB drive to move files at the office because. Guess they were on something after all :].
Never underestimate the bandwidth of a station wagon full of mag tape.
I really hope I won't be locked out of using vision pro just because its input modality isn't compatible with my body...
(I say "I think" because I have not actually used their accessibility features)
With all of the new paradigms in VisionOS it seems perhaps unavoidable that accessibility features might take some time to catch up. Engineers and designers have to wrap their minds around the new stuff.
I have a loved one having issues due to a brain cancer. Optical seizures were the first warning sign and went unnoticed for some time. It would be amazing if VisionOS could potentially detect some conditions or help folks with epilepsy just as the watch has for some cardiac events.
Sound recognition for example, or Assistive Touch on the watch. Not needing those I have no idea wether or not they’re actually useful, but it sure is impressive and seems well-thought.
It will also be interesting how the outward display will display the "bad" eye input including missing eyes, lazy eyes, nystagmus, etc...
"And for people who prefer a different way to navigate content, Pointer Control lets them select their index finger, wrist, or head as an alternative pointer."
This reads to me like they've got you covered (at least in principle).
I suppose Apple could change my mind but I've never been a fan of eyes as input. Touch typing or any equivalent where you can look at one thing and do another seems impossible now.
That's not "defending Apple", that's just... gosh, I can't think of any word besides common sense.
From all of the presentations and press accounts, it seems clear that eye tracking is something of a direct replacement for the mouse/trackpad cursor -- in fact it seems you can use a trackpad or presumably a mouse as well. Looking at an app or "window" seems equivalent to mousing over it, not equivalent to clicking to activate that window.
I think the implication is that the window you are looking at is focussed automatically. Which wouldn't work for say, typing up a document to a text editor while reading from a reference doc. That's just the beginning, I use a mouse-to-focus window manager and that happens all the time with all sorts of things.
What’s your motivation for saying that? Personally I am suspect of people say there isn’t common sense.
Here is the definition at the top: Common sense (or simply sense) is sound, practical judgment concerning everyday matters, or a basic ability to perceive, understand, and judge in a manner that is shared by (i.e., "common to") nearly all people.
One can certainly say there is common sense. For instance no one likes pain, people like pleasure. People care about entertainment, not boredom. People tend to learn on the whole as they grow up. I also think on the whole have a perception of what is wrong and right, even if there is variance between cultures. The golden rule of reciprocity seems to be near universal (like: don’t kill, don’t steal and so on)
Lots of things we share as a species.
And then you obviate the concept by rendering your own gross ignorance: people don’t like pain? Today you learn about masochism. People need entertainment? Monasticism, asceticism, Buddhism, nihilism, et al. People learn as they age? Quite unfortunately many studies show many people lose a great deal after their teens. And your invocation of the idea that cultures share morality such as a prohibition on murder (this has nothing to do with “reciprocity” by the by) is truly divorced from basic observable reality.
100% of a farming town/community are going to hold "don't grab an unknown fence, as it might be electric" to be common sense. Yet there's going to be a sizable portion of the planet to whom that WON'T be obvious, and they'll try it.
100% of New Yorkers might know not to park your car overnight in a particular area, yet people new to the city will happily do so not knowing they may be putting themselves or their property in danger.
If you have to put bounds on "common sense", or contextualise it at all, then it isn't common, it's just knowledge shared by a group of people of unknown size.
You say it in your final sentence - "near universal". If it's not universal then what's the point? Where is the magical cutoff point of the population where "common knowledge" is no longer common? 95%? 50%? If so, why?
Such speculation is ludicrous because it's going to change drastically from area to area and change even more so depending one what you're talking about. And if such a definition is so fluid, when you're positing that it is in fact universal, then it's pretty clearly false.
"Common sense" and "common knowledge" are just shortcuts we use so we don't have to explain the entire contents of history in one go to explain one other concept. This is very useful. But relying on it existing _ALL_ the time is the work of a dangerous lunatic that has clearly not encountered a broad enough segment of humanity.
I think my personal stance is more on the idea that there is a lot we share as a species (from words, to symbols and human basics) and that could be grouped as common sense. But perhaps my grasp of english fails me here.
Looking at the etymology it also seemed to stem from “community” , what is common among a community. I would wager things were very common for his peers when Thomas Payne wrote his book “Common Sense” , a world to him that was largely very the same in style and substance.
Perhaps that concept simply can longer apply in a pluralistic global world or in a modern sense like you point out with your examples.
https://www.wired.com/2010/10/gorilla-arm-multitouch/
Probably best for me not to be an early-adopter in this case :)
Edit to add: Can confirm that Apple stuff is amazing with accessibility, so I’m not surprised there are alternative input methods. My main thinking towards the future is around how nystagmus and foveated rendering will interact. That’s a giant and fascinating research topic I imagine.
https://www.childrenshospital.org/conditions/strabismus-and-...
The bad idea had waves even before that: Remember the gorilla arm.
http://www.catb.org/jargon/html/G/gorilla-arm.html
That's maybe because the screen is small and low. Raising an arm at or above the head is a different matter. If you're cycling with a heart rate monitor you discover that you add some beats when you raise your hands (say, go from 150 to 155) and gain some if you lower your chest close to the handlebar. This doesn't take into account increased or reduced drag because I measured it at home on trainers.
What's interesting is that unlike steady state to a hard sprint, the heart rate rise is almost immediate. There's some delay - 20 to 45 seconds - if I change it up and do a sprint interval before my heart rate reflects that effort.
There was an operations center I worked at that had a large touch screen (think cnn wall), that always sat in the corner of the area collecting dust, but whenever some higher up would visit it would get pulled out and used just to provide a wow factor. It was always the unlucky one that ended up having to use it during this time. The fatigue was horrible.
That said, you can have something identical to this with Obsidian with a few plug-ins and their hosting service.
https://obsidian.md/
https://github.com/deathau/sliding-panes-obsidian
And I'd say that 80% of its failure can be attributed to bad software. It's buggy, I constantly have to re-calibrate my work/play area, the hand tracking is janky, and even though it supports (incredible) eye tracking, almost nothing takes actual advantage of the hardware.
It's insane that a billion-dollar company like Meta actually felt proud to release such a steaming pile of trash. Do they seriously expect VR enthusiasts to build an entire operating system for them? Much like the reports we're getting about the Vision Pro, the eye tracking in the Quest Pro feels like magic, but nothing uses it—basically it's irrelevant to navigating the operating system and barely any games use foveated rendering. It's infuriating.
If I was Zuck, I'd fire all my product managers. I say good for Apple. I'll probably be selling my Quest Pro and buying the Vision Pro.
Also Meta could catch up if it was just software. But how big of a deal is the M2+R1?
Anyway the metaverse was already being retracted, else this Vision Pro could have hammered Meta.
Clearly Occulus has to evolve to match VisionPro. Controller has to be optional. It has to have better AR. Meta has to be the more "open" alternative to Apple's ecosystem barriers.
This was Microsoft's warcry for years: who cares if Apple has better software? We have better hardware. Well guess what, software matters and it matters more than hardware. If you have both, you have literally a generationally-defining product (e.g. the iPhone), but even if you just have the software, you'll still eat your competitor's lunch. Why does everyone have MacBook Pros these days? Prior to Apple Silicon, it certainly wasn't the hardware.
Meta's about to get a rude awakening because "we can fix the software later." As if technical debt doesn't exist and as if you can so easily get engineers that eagerly want to clean up someone else's mess.
Oh, I don't know about that. Admittedly there were some awful choices made between 2015 and the emergence of M1 architecture, but Apple's reputation for making well engineered hardware goes at least as far back as to when they started milling Macs out of aluminum. I'm thinking back specifically to the PowerMac G5 in 2006. It was designed well, felt solid, and when you opened it up, it continued to look well made. I recently popped open my 2015 Macbook Pro because I'm finally having hard drive issues, and for as much as I have railed against Apple fanboyism over the decades, it looked so nice inside I wanted to take a picture. (Why?! Who cares!? What would I even do with that picture?)
I got tons of mileage out of various Dell desktops and self-built machines over the last couple of decades. It was better bang for the buck, I didn't have to fuss over proprietary connectors and a locked down operating system. When the company I worked for made a big shift away from (it's okay to chuckle) Coldfusion and MS SQL Server to Ruby on Rails and MongoDB in 2013, though they offered to let me continue running on Windows, I asked for a Macbook because that's what everyone else was using. Might as well learn. Didn't much like OSX but the hardware was better than any Windows laptop I'd used. Later I bumped up to a 2015 MBP, which I only just retired this year.
While I was Windows at home and MacOS at work, last month I snagged a Macbook Pro w/ 64gb of RAM from B&H for $2400 - the first Apple computer I've ever paid for myself - and it's replaced my Dell desktop and that 2015 Macbook that my old job let me keep.
I effectively skipped the bad Macbooks, so my perspective is tinted by that. But even during the bad years, even my friends who continued to operate in the MS domain were buying Macbooks and dual booting them into Windows.
I had access to a Surface tablet through work early on. It was really nice! Just like a Zune. The Surface Pros looked pretty great too, but anecdotally, I never saw them outside the context of visiting a business that was a strictly MS shop.
I still like Windows as an OS better. But Apple's new architecture (which importantly doesn't have me carrying dongles around, or even needing them at home) was compelling enough for me to take the plunge.
There's lots of little details which are well made, but bad decisions: the screen hinges tend to be a little too floppy, the trackpad is oversized, removing the physical buttons makes it hard to use, moving the power button from a separate area to the "eject" button location on the keyboard (and making it look like a regular key), everything they've done with keyboards (flat, overlay spaced keys with no surface indentation and less and less travel). The ridiculously sharp edges of the body chassis which cut into your when you rest your hands there (seen at least one video of someone just filing a bevel into the machine to fix it).
They look great, but that's all they've ever seemed to me: in usability details they've always felt terrible (even MagSafe, which seems like it should be great, feels great, and yet has seemed pretty lacklustre when I've had to use it with a work machine).
The two things I agree on for the 2016-2022 laptops are the sharp edges and terrible keyboards. Maybe take a look at the new laptops next time you get a chance, the edges on my MBAir and 16MBPro from 2022 are waaaay more comfortable than my previous two MacBooks and the keyboard feels much nicer too.
Back to point one though, to me the trackpads are well sized, the hinges are smooth and just right and the power button being where it is is a none issue, I even quite liked the touchbar.
What on earth are you talking about? Microsoft is a software company first. They rode the Windows advantage for more than a decade and didn't have anything to do with the hardware, because they aren't a hardware company. Windows ran (and was preinstalled) on everything, from the lowest tier of trash to the highest end machines, and Microsoft didn't care about the quality of the hardware, because they owned the platform that everyone used for computers. Office is another software product that still doesn't have any real competitors and is one of Microsoft's biggest cash cows.
On the flip side, when exactly did Apple have better software than Microsoft? Their office suite is barely usable for anything outside of the most basic use cases. Apple beat Microsoft in mobile, and Microsoft kept its head in the sand and assumed the Windows advantage would live forever and didn't wake up until Ballmer was gone, after Apple had started eating their lunch in the laptop space, but the assumption was that Windows was so good people wouldn't switch, not that Apple had bad hardware and good software, if only you could run it somewhere else.
Microsoft is named for being SOFTware for MICROcomputers, they don't have a blind spot around software. This is some Soviet revisionism or something.
> Meta's about to get a rude awakening because "we can fix the software later." As if technical debt doesn't exist and as if you can so easily get engineers that eagerly want to clean up someone else's mess.
I don't know, Meta is probably the largest employer of PHP developers in the world, I bet they can find people willing to do some other masochistic shit too.
The Vision Pro is really a first-product distraction for the Apple fans. This only further validates the market for XR devices and Meta's Quest's lineup will be the cheaper alternative for those who don't have iPhones or Macs (Since Android phones still outnumber iPhones) [0].
I'm only interested in the next iteration when it gets smaller and cheaper and when Apple announces a direct competitor to the cheaper Quest. Probably 'Apple Vision'. Then it will get its 'iPhone' moment, but it is not going to be the Vision Pro.
> Meta's about to get a rude awakening because "we can fix the software later." As if technical debt doesn't exist and as if you can so easily get engineers that eagerly want to clean up someone else's mess.
Meta is fine. They will just copy Apple and make it cheaper and at worse case, become the Android of XR headsets and glasses.
[0] https://www.statista.com/statistics/272698/global-market-sha...
It’s not like it was better built or had a better screen than cheaper alternatives or anything.
Gaming can't be good, because there's no controllers. Even if the hand and eye tracking is absolutely S-tier, you're never getting something like button input from swinging your empty hands around, twitching your fingers. Just the occlusion from the back of your hand would be enough enough for that to be impossible.
Fitness might be more doable, but even then I think you're going to be very limited with hand tracking only. I think there's a reason why Apple skipped over those segments almost entirely.
EDIT: I meant "VR" controllers, as in things that are tracked in 3d space. Of course you can use a game pad or MKB, but that kinda misses the point of being in VR in the first place.
There are a LOT of WWDC sessions on visionOS; hopefully it’s shown off in one of them. EDIT: Trackpad usage shown here https://developer.apple.com/wwdc23/10076?time=914
VR controllers aren't simple the way a game pad is. They need a full tracking solution, which means sensors with base stations, cameras with some form of vision based tracking, or the headset needs to be able to track them based on some reference points (infrared lights or something). And that tracking needs to be fast, precise, and correctly positioned relative to the headset.
Using the Index controllers with the Lighthouse base stations is kinda plausible from a technical standpoint, but that would mean Apple would need to allow the headset to work with SteamVR, and that seems very unlikely at this point.
Re: 1, it’ll be interesting to see how well the camera system works in practice. All of the reviews are quite positive about precise movements so it might be that they’re throwing hardware at this problem but I’d also be surprised if they were not very carefully tracking performance – if the accuracy isn’t there, it’d take an unusually un-Apple like product manager to risk a billion dollar investment rather than enable that class of device.
PSX games with a PS5 controller on a theatre screen? My inner 8 years old is going to loose his mind.
But that doesn’t solve the issue for the mainstream.
Using only gestures means no haptic feedback and it gets de-synced as soon as the camera can't see what you're doing.
Take Apples gestures for example (e.g. tap your thumb and index finger together to click). As soon as your hand is rotated in such a way that your own fingers are hidden from the camera, you actions stop being applied. This will also happen should one hand cross the other, or resting your arms down while standing, or reaching up for something, or even resting on a couch with your knee up.
The only VR headset I've tried that can do hand tracking (Oculus Quest) sucks at it and games there didn't pick up on it as an input at all, as far as I know.
However, just because VR controllers won't exist for the Apple Vision at release it doesn't mean they never will. Perhaps Apple will release some later, or it'll be a third party release. Heck, someone might unironically release a Nintendo Power Glove style controller that works incredibly well with Apple Vision.
Because none exist.
Sensitive hands and wrists. Pain and fatigue using them.
Having to swap out batteries. Recharging AAs is a hassle, but I don't like using disposable ones.
More clutter to keep track of, store and organise. Kids misplace them frequently.
More crap to have to keep clean, sweaty and oily hands leaving residue on them.
Would rather just have a single device, the headset, that I put on and not have to deal with a bunch of other accessories.
Assuming it could run Virtual Desktop, there’s your Half Life Alyx in ultra high res.
if it was under 1k usd I would probably pick one up for work. maybe give it a few more years.
Why? When I go to an imax cinema I am impressed with the immersion, I don't feel the need to jump around and act out the scenes in front of me.
I certainly don't think so? My guess going in was that fitness would be one of the main plugs — especially because high-end fitness devices already cost a fair amount, and that would have made their tail end, "How much would a computer plus a screen plus umm... it's $3,5000!" go a lot more smoothly.
I do think that's likely high on the road map, particularly given such a large investment in Fitness+, but also can see them being very wary about encouraging any high-energy movements with a $3,500 device. I think it's the pricing and dev-kit-ish nature of the v1 more than too obvious.
But Vision Pro does seem to imply a future Vision Sport.
The platform launch was very locked down. For incremental hardware upgrades, they put hardware in the hands of devs by either shipping out early or bringing devs to their offices to work on prototypes in a secure environment.
Jesus set up the room boundaries, it's really not that hard.
Basically, a Game Boy or Switch for VR.
Is this the same team who came up with a phone that bends in the pocket?
- 1 Their customers have historically chosen metal and aluminum for their products, and Apple is in the business of building what their consumers prefer.
- 2 Glass and Aluminum conduct heat better than plastic. The 2hr battery suggests thermals are a great consideration where the materials chosen were not just for aesthetics. Perhaps plastic could've worked, but see point #1.
I had just showed him WA state and no ill effects, but I didn't have issues on a pre-release vive prototype or release occulus OG while doing barrel rolls and back flips or movement vluprs.
So I crank up the details to super high and enjoy the awesome rendering detail and save myself from motion sickness.
Elite Dangerous is the only VR app that gives me any motion sickness at all and is also my favorite app in VR so far.
Incidentally Google Earth has comfort options exactly due to that reason, such as the option to narrow the field of view when moving around. I wonder if that option was enabled?
Interestingly I tried out F1 2022 in PCVR mode with it and was surprised to find that didn't make me feel ill. A lot of it comes down to the refresh rate and quality of tracking as its the lag that causes a lot of the nausea.
What is even more impressive is when a VR version of a big, popular game is the best version of that game.
Affordability is somewhat relative. If something is truly compelling, people will spend a lot of money to get it.
"Exclusive" in games consoles is a loaded word that means "exclusive to this platform and not available anywhere else".
If the intended meaning is what you suggest, then I'd recommend removing the word "exclusive" and just saying "there is no affordable VR console with lots of big games".
VRChat, Superhot, Simpleplanes, Flight Sim, Ultrawings, Phantasmagoria, Htiman, Nearly all VR compatible racing games, Rec Room, Euro Truck Simulator, Most VR compatible horror games like Phantasmagoria, etc etc etc.
Did you even try to look it up?
I do not need to "look it up", I own and play Superhot on the Oculus for example.
What in your knee-jerk reply would you say contributes to the conversation about whether "VR exclusives" are needed for a given platform to succeed?
I'm going to help you by providing the context for this conversation (not written by me, but in the initial comment I was replying to), which is:
> "The adoption problem seems to be that there is no affordable VR console with a lot of big exclusive games."
There you go. What do you think "big exclusive games" means in this context? Do you think it means "exclusive" as in "Nintendo exclusive" or does it mean "exclusive" as in "it's designed for VR first, regardless of headset brand"? Or something else maybe?
something that people will look at and think “I need VR so I can play that”. The closest so far is half-life alyx, but that’s really not cutting it.
I was really impressed by the Oculus "First Contact" (or whatever the one with the robot is called). I wonder by there are so few games that feel like it.
Beat Saber is pretty cool, but what killed it in my family is that you cannot see what the other person is seeing, so it's a bummer when you're not donning the headset. Unlike with a regular videogame, where you can watch the screen even if you're not playing.
The problem with Valve is that their platform is PCs, and a gaming PC + VR headset is too expensive for most people, who often don't already own a proper PC. The problem with Sony is similar: The PSVR 2 is not a standalone console, but just an add-on to a PS5, and PS5+PSVR2 is too expensive for people who are just interested in VR. It's like trying to sell a Game Boy which requires you to already own an SNES for it to function. It would be a flop.
The Meta Quest 2 is in a better position, it's standalone and rather affordable. But Meta can't support it with big games like Sony or Valve could. Meta doesn't have the expertise and probably also not the will to develop expensive AAA games in order too push Quest sales. They are more interested in the metaverse than in becoming a games company.
Nintendo could perhaps make a successful VR console, but it would require massive investments in AAA titles, which would be very risky for them, given their size. And Sony isn't interested in a standalone system.
Microsoft would be more likely. They have huge cash reserves and can afford to lose large amounts of money until they become profitable, as they've shown with the first XBox. But I they haven't shown any ambitions so far to invest billions into an "XBox VR" console and corresponding AAA games.
In some cases they use it well, like Beat Saber. But how much is that worth to people?
I get why they didn't want to go the controller route, but they'll surely be very aware that it greatly limits what can be done with games.
Remember, the iPad doesn’t come with a controller, but you can use a Pencil with it if you want.
Same thing will likely happen with the Vision products.
Moreover, making elaborate games like civilization wouldn't be economically feasible even if they could be appropriately controlled by default. The price for the headset is so high that not many people will buy it. Then elaborate games can't sell enough copies in order to be profitable. For example, it seems unlikely that even a success like Half Life Alyx made its development cost back. The hardware install base is too small.
I think Apple is playing the long game here. Moore's law and manufacturing improvements will cause the price to go down over time.
Remember, the first Macintosh cost $3,000 in 1984. Today most Macs cost less than 2/3 of that price before adjusting for inflation.
My point was that it would be like walking around Madurodam while playing a game.
And standing would be optional. Apple showed most people sitting down, so I’m presuming you could do that.
But you should be able to do VR while sitting down and "walk" using gestures or a controller.
[0] - https://www.kat-vr.com/
Why? No, seriously, why? People say this as if it is a given, but how would VR improve a game serious that is built around strategy as it's main selling point. How does VR improve Civ as a game? In fact, there are very few genres and types of games that the immersion VR brings is worth the limitations: Weird controls, isolation, headaches, wearing a mask, expense, physical exertion etc.
I'm so tired of people who don't seem to know anything about either video gaming or VR gaming parroting this empty hype of how VR magically makes all games better.
VR is a peripheral, not an advancement in rendering technology.
It's a city. I live in a real city. SimCity is this flat set of graphics on a screen. I'd much rather be able to view my city that I created and see what it looks like on a human level.
I get your point, and many things would not be good.
But for me at least, SimCity would be. So would Call of Duty (and I know that because I've played 1st person shooters in VR before on multiple occasions).
But that's just me.
Meta was proudly boasting about how the Quest 3 will have "Over 500 titles" - thats not something to boast about, it's just highlighting how incredibly poor the platform has been from a developer adoption standpoint.
But there are clearly still problems to solve. And some of them do involve hardware capability. Basically for the mass market they’re just not there yet.
The apple headset certainly appears to have significantly more power and capabilities in many ways. It will be interesting to see what pops up.
Though obviously a future version with a lower price will be necessary for it to truly go main stream.
Either way, if I was gaming, playing a new single player 2D (Last of Us, etc) game on a MASSIVE screen on the Vision Pro seems so exciting. Beat Saber and throwing my hands around? Not so much.
A flop indeed. /s From [0]
"Standalone VR headset Quest 2 from Meta (formerly Oculus under Facebook) has outsold Microsoft's Xbox Series X/S consoles, according to the latest information from the International Data Corporation (IDC)."
> Apple is right to ignore it.
Incorrect.
Apple didn't ignore it because they cannot ignore it. Apple is targeting both gaming and for work just like how the Meta Quest Pro has done so.
[0] https://www.thevirtualreport.biz/data-and-research/65297/que...
The resolution on the Q2 is also just way too low for certain games, I think a MtG-type game would be super cool on the Q2 (and the games I tried in this vein had /massive/ potential that was thrown away completely due to blurry text). In the ends the Q2 has way too many downsides and not enough upsides to keep me using it. I will be getting a Vision Pro at launch though, the resolution alone almost would sell it for me but the emphasis on AR and productivity are the clincher.
But isn't it still the most succesful application of VR?
I don't see another application of VR / AR that will prevent the device eventually gathering dust once novelty has faded. Certainly nothing in the presentation from Apple.
But they also showed full support for pairing standard Xbox and PlayStation controllers and playing with those the way you can on any other Apple platform (except the watch, of course).
Can you name a better product in the price-bracket?
Five times the price, yeah. Today. This really feels like a product aimed at developers right now not the mass market yet.
Which fits. The iPhone took a few years to really take off. The Apple Watch took a few years to really takeoff.
When Apple TV was being developed they were handing them out to devs for free like they had millions they were trying to get rid of. I was a lowly single app creator and they happily sent me one.
For the Intel to M1 migration they did something similar but required them to be returned this time, and limited which devs they sent them out to.
The first Intel Mac and the first ARM Mac were DTKs: Developer Transition Kits.
You’d buy them from Apple at a reduced price, could use them to get your software running, and then had to give them back when the real hardware was released.
There was nothing similar for the watch, iPad, or iPhone though.
Not todays market, which has evolved.
In my opinion, with the possible exception of South Korea, that is purely a matter of apple's global pricing vs local purchasing power and a huge number of those android users would have an iPhone if they could afford it.
I don't think that's true. Samsung flagship phones cost the same as iphones (at least in the UK) and they sell very well here.
Interestingly, the Quest 2 seemed less buggy when I had one.
>Eye input is not shared with Apple, third-party apps, or websites. Only your final selections are transmitted when you tap your fingers together.
Along with this small video: https://www.apple.com/105/media/us/apple-vision-pro/2023/7e2...
Which is great and important for me to see, but I'm unsure how that jives with features such as foveated rendering.
One notable takeaway is that eye tracking of saccades is loosely correlated with cognitive load. In fact, “cognitive load” was a product that would be licensed to developers to use in their apps so that difficulty levels could be dynamically adjusted to keep users in a sweet spot for learning.
Especially when combined with a PPG sensor which can in theory detect if a man gets aroused (HP never investigated developing a model for that, but some comments from friends who were DoD-aligned contractors expressed alternating concern or interest in it, because they knew from their own career and independent research what applications might be possible with those technologies)
…it could be used to infer things which someone might prefer to keep private (I.e. which categories of things they have an emotional response to).
Apple definitely is aware of this and is heading off potential issues by not only not providing that data to third party app developers but also even to Apple itself. Apple does have a strong pre-existing framework for dealing with sensitive health data, so they could have decided to capture similar data and treat it the same way they deal with ECG data. HP did not have a HIPAA compatible framework so they threw away 99% of the PPG data and distilled it down to just “heart rate” which was the only data source available to app developers from the PPG sensor.
(Downside is that one loses the capability for hovering, but strikes me as the correct trade off.)
Now imagine the game has advertisements, or loot boxes, or something else that the game company wants to know if you've looked at. How does Apple prevent them from learning that, if they're able to know what they need to know for the game to function as a regular FPS?
The question will be how easy it is to reconstruct that information from the last exact position + movement info...
I wish someone on the SDK team there could observe me as I tried to search for this information and failed because it would be super-informative on how to (and how not to) succeed on creating a wider community of apps...by making SDKs easily available.
<All opinions are completely my own personal opinions>
Zuck was probably part of the problem... a room full of POs pushing for stupid shit while building on a rocky foundation.
If Vision Pro will succeed Meta could be seen as the cheap-ass alternative. 3500USD vs 500USD is like Ferrari vs. Kia. Not everyone can afford the former.
If Vision Pro doesn't succeed, the market will likely move on from VR completely, leaving Apple and Meta in the same place. Money spent, product introduced, tried, didn't work. Move on.
Not because Meta had a better product, but Apple putting its might into this product category means that "Meta" is vindicated in investing here -- that there exists / will exist a viable market for this technology in coming years.
All the damage done by the poor quality 3D renders of AR/VR worlds from Metaverse -- that made a layperson laugh at the concept of AR/VR -- is probably partly offset by the positive press and marketing budgets that Apple will bring to the table.
Let's be real. You can have all the money in the world, but that doesn't guarantee you access to talent. It doesn't guarantee that you made insightful technical decisions that hamstring you later in the product cycle. It doesn't free you from short-sighted managers or employees who just care about making it to the next vesting cycle.
When you're a billion-dollar(haha, not so significant these days) company, you have a tremendous pressure to show results. You have tremendous pressure to show innovation and be first to market. It has no bearing on the quality of the products you release in the short-to-medium term.
A lot of people probably don’t remember what Android looked like in 2007. Using the Android pre-release SDK next to an iPhone, you’d come to a similar conclusion as yours: how does Google release a mobile OS so behind the curve? Should they fire all their product managers?
Same story with Windows in 1986. It was difficult to see how the graceless file manager from Microsoft could compete with Apple’s holistic GUI.
Give it four years. I think Meta intends to be the Windows and Android of this third wave of UIs. Validation of the market by Apple is exactly what they’ve been waiting for.
metas strategy is to release a cheaper consumer product and improve it iteratively. Hate to break it to you but day1 apple vision pro will have software bugs too.
- Something whimsical to inspire developers. Give me files represented as physical blocks that I can pile, or a task manager that shows processes as plants around me. Close apps by scrunching their screen like paper. A globe with my geotagged pictures.
- A game, any game. Beat Saber was the killer app for me to get my Vive, and Valve already spent tens of millions of dollars to create a triple-A VR game. Neither seem compatible with the Vision Pro input methods. Apple could at least play their strengths, like an AR hide-and-seek, or a horror game with eye tracking.
- Content creation or professional work. The last thing we need is another passive device to watch 3D TikTok. Show someone customizing their "home screen" environment to look like a fantasy potion shop; or a mechanic looking at a 20x magnified broken part; or an SRE watching their kubernetes cluster as a floating 3D graph; or a VFX artist in the scene scrubbing forward and backward to adjust an effect.
It feels like inventing the first smartphone, camera apps and all, but it doesn't make calls and the only input is tapping on icons.
TikTok on your face likely has better market potential than the dreadful Horizon Worlds.
We finally have the technology to bring https://www.cs.unm.edu/~dlchao/flake/doom/chi/chi.html to life
But the point of whimsical demos is inspiration, like a concept car, not complete realism.
[1] https://eaglemode.sourceforge.net/ [2] https://www.youtube.com/watch?v=G6yPQKt3mBA
Half of their marketing was about how it's a essentially a macbook that you strap to your face...
By far the most disappointing part of the keynote was when they finally do show someone playing a game, and it's a basketball game played with a regular controller on a flat, virtual TV. The TV is even smaller than an average real-world TV.
It's typical of Apple to so completely ignore gaming, but it's disappointing that they're continuing that trend even with VR.
A lot of value could be had playing flat screen games this way. Imagine sitting on a plane with a controller in your lap, but it feels like you are sitting in a private loungeroom or some zen environment playing a game on a large TV.
The fact that what they came up with was "game in a rectangle, but this time floating in the air" makes me think that either there are major limitations to what the AR platform is capable of or that they're intentionally trying to tell people "gaming isn't a priority here".
apple's anti-gamer slant has always stunk to me of some exec being salty about what went down with nvidia and hating the idea of ppl playing games. their slapfight with epic games is another example of them not giving 2 shits about the gaming market and alienating a highly growing userbase.hell they didn't mention live streaming at all and that seems like a usecase this thing is tailor built to be good at considering how many cameras and screens it has!
meanwhile they want people to buy into apple arcade...
I respect Apple for not doing this. These are demos that are kind of fun for 20 minutes but you would never want to actually use a device like this: https://youtu.be/z4FGzE4endQ?t=59
EDIT: Found the API docs. There's some really interesting classes in there! I remember handling coordinates to be confusing/a pain though haha. https://learn.microsoft.com/en-us/uwp/api/windows.perception...
And I'd like to know how realistic this thing is for "not apple users". It's a VR headset, can I use it with Windows to play MS Flight Simulator or Star Citizen? Can I use it on Linux to get wall-to-wall emavs? More real world testing is required.
This will be another part of the Apple system along with Macs, iPhones, iPads, etc. You will never be able to use this as just a monitor equivalent for any other OS.
This is an M2 computer strapped to your head: it can bloody well do VR, let's see what folks do with this device that Apple never asked them to, but they do anyway because Apple doesn't get to tell them how to use their overpriced hardware (especially if the moment you buy it, the buyback is $50 if you're lucky)
MKBHD on the Vision Pro: better image passthrough, better responsiveness, better hand tracking, better eye tracking, better image quality than any other headset he's ever used.
You: "overpriced hardware".
No, better hardware.
[1] https://news.ycombinator.com/item?id=36220459
There are reports now from dozens of people who've had similar 30 minute sessions using some of the features. That's enough to get an idea of some facets, guess at what's not polished, what's not ready at all. But they're all easy to ignore if you want to wait until next year.
Imagine developing with the ultimate 10x setup: several files, documentation, and debuggers open at once, with a zen nature background. Or making music, or painting, or even writing a research paper.
Imagine playing a video game but the environment wraps around you. Imagine watching movies in a virtual theater with a simulated 100-inch TV (one of Apple's demos). Even reading and browsing the web can be improved with an ambient environment and extra space for more stuff.
Is it worth $3500? For end-users probably not, but if it genuinely makes professionals and hobbyists substantially more productive, it will be worth $3500 or even way more. How much money would you spend to write, code, create faster?
Of course, this assumes the VR actually performs, and whether a bigger screen and immersion actually makes people work faster. As of now it seems VR is still a gimmick which impresses people at first, but doesn't provide much outside of niche experiences; which, if this holds for the Vision Pro, makes it very much not worth the price.
The problem for me is that I like monitors better. I like the feeling of real space around me. I cannot conceive of wearing googles on my head all day. I just don’t want to.
And I definitely don’t want to be separated from other humans by googles. It just not attractive to me.
I used to believe this was the ideal setup, hell I even implemented it with monitors, but since I've integrated LLMs into my development process I find all I need are two side by side windows and now with CopilotX those are both in VSCode. I occasionally venture out for documentation but even that will make its way into the editor at some point. Using Edge and the Bing sidebar I can even save myself the trouble of reading through all of the documentation on a page by just asking Bing to retrieve the info I want from the page I'm on.
Doing all that in VR would be cool but I don't think it's going to increase my productivity nearly as much as LLMs have.
Disclaimer: I'm bullish on VR and I love gaming and socializing in VR
This irrelevant for those of us not doing cookie-cutter-been-done-1000000-times-before stuff. In such cases all the LLMs in the world are useless, but many pages of docs open at once are useful. Go tell your LLM to write a ARMv6M assembly MIPS-II emulator. I'll wait. Go on.
No one has come up with anything better yet. That’s one of the big problems with VR that I’ve seen. Do you have games on one hand, and a virtual environment showing mostly 2D stuff on the other.
I think that’s one of the reasons Apple is releasing this when they are. I sort of suspect they’ve gone about as far as they can. They don’t have a killer app, but that’s ok.
They’re putting it out there for developers, and we’ll see. Someone will come up with something compelling.
VisiCalc, Lotus 123, WordPerfect, Photoshop, PageMaker, Angry Birds, the web… none of them came from the platform vendor. So what will someone come up with that will make everyone rush to order the Vision 2 when it’s announced for $1500?
On the face of it, it sounds like a company would want coherency, but in practice it means leaving a lot of money on the table for every single of your products. Any consistent design decisions means anyone who's not pleased by it has no option in your lineup to go to. That's of course a gamble Apple is willing to take.
For instance, Apple consistently uses glass and metal in their product design, the only exception I can think of being airpods and their mouse. I personally prefered the plastic for weight and comfort, so that set me out of the market for their headphones and I went to Sony for instance.
Same for laptops with touchscreens, if you care any bit about them Apple is a no go to you. Same for so many things.
PS: on inconsitencies and incoherencies, Windows is the king of the kitchen sink approach, and it's consistently stayed ahead of Apple in numbers. Another instance on how being opiniated isn't only a positive aspect from a business perspective.
Yes there also will be quirks I'll be unhappy with, which I _could_ (given enough time and experimentations) find a better solution for. But fundamentally I'll be getting a solid product, that works well enough. It is _that_ promise, which fundamentally allows Apple to do an online store and do a mostly directly-to-customer retail operation.
If you increase diversity, you make this "buy any product and you'll be fine" thing so much more complicated, loosing more sales than the niches even contain. I'd guess almost all people just want it to work :TM:, at least that's me, and that's me deciding the phone of my mum
On the other hand, that doesn't seem to be what most consumers long for. Apple had two runaway successes with the iPod and then the iPhone, but to note, the iPhone is not the best selling phone brand nor dominates the international market. Same way Apple is not the #1 computer maker.
So yes, focus and consistency brings tremendous value, but there are many other ways to get that level of value, and many customers will have different priorities.
(I also don't think any brands need to be #1, being profitable should be enough, but I am more and more interested by how Windows has stayed at the top for so many years. In particular Apple progressively feels closer to Sony when it was at its peak: superbe products, strong ecosystem binging, coupled with also a heavy dose of strategy tax on how to benefit from what the other companies are doing)
From everything I've heard, Apple's org structure is a lot more focused and UX-driven from the top down than basically any other large tech company. That means you get a lot less compromising and in-fighting.
The other side is how Apple makes money compared to other companies. Meta makes money mostly by selling you (the user) to advertisers (the real customers). Same with Google (70%ish ad revenue). Microsoft is a hot mess of competing B2B fiefdoms (the vast majority of their revenue comes from businesses, not consumers). Amazon only makes a few products directly for consumers, and mostly makes money from being a marketplace + AWS. That leaves Apple fairly uniquely positioned as a premium consumer hardware company that also sells consumer software services to enhance the experience and... drive more hardware sales.
Apple builds an insanely strong core from which any and all other teams can (maybe must) build on top of and use to power their team's platform.
You have a core messages team for example - off of which Mac, iPhone, and Apple Watch teams integrate. Do that for each core service or product (photos, safari, notes, music, etc).
The Amazon structure is such that everything flows from the person above you, work is not shared between teams.
One is compositional (Apple). The other is inheritance (Amazon).
Andy’s thoughts were infinitely deeper than anything else I’ve read about the device+interface so far. In particular, I liked his observations that so far it’s mostly a 2D plane, with exceptions like the Breathe app and a few demos, which might just be good for demos.
Seeing the heart in 3D reminded a lot of what we saw 10 years ago around what book publishers were trying to do with the iPad. Cool demos but limited real world use.
Here was one particularly thoughtful section:
“unlike the physical version, a virtual infospace could contend with much more material than could actually fit in my field of view, because the computational medium affords dynamic filtering, searching, and navigation interactions (see Softspace for one attempt). And you could swap between persistent room-scale infospaces for different projects. I suspect that visionOS’s windowing system is not at all up to this task.“
I'd prefer an OS that spans multiple devices. Something how computers behave in “The Expanse” where someone can flick stuff from their mobile device to another system for a better workspace. If I'm typing this on my workstation, but want to switch environment and bring something light like a tablet or just my phone, I could send the tab and it would retain its state. The closest to something like this is a server-client architecture like Logitech Media Server. A paradigm like this would be more useful than immersion.
This is also why I like Vim. Buffer is a separate concept from File, adding an extra flexibility – in terms of layout – to how I edit to complete my current task. I want a similar flexibility with applications, at least lightweight – state wise – ones.
CloudKit and Handoff are close, but very brittle. Apps like Bear, Reeder, Anybox, Things 3, make switching devices seamless. Another good example is handoff between the HomePod and the iPhone.
I think Apple's whole point is that they provide the APIs so that apps can do this themselves. Which, yeah, is a bit disappointing; I'd love to have a 100% fidelity, flick-to-another-device type of Continuity built into the OS, but the APIs are at least a starting point.
I want to measure my pupil distance while browsing social media. The reasoning is my reading in "Thinking Fast and Slow" that pupils dilate when we see something we find interesting / something we like / when we are thinking and vice-versa. I want to put it to the test and it seems like finally the consumer tech is close to making it possible.
Does quest support pupil tracking? I did some cursory research but couldn't find any reference for it. Industry pupil tracking headsets are way too expensive; My last hope is that someone will jailbreak vision pro...
Gary has a great TED talk (Mar 2010) on this radical but intuitive spatial sorting interface. Foveation and hand movements are perfect user links.
https://www.youtube.com/watch?v=LT_x9s67yWA
Combined with a Vision Pro—OMG.
? =! Vision Pro