As long as it's still gimped by not having a true productive OS like macOS, ie making everything go through the app store so as to not cannibalize Mac sales, it will be relegated as simply another form factor for the iPad. The iPad could've been a true Surface competitor, where I could run VSCode on it and compile applications natively, but instead it's just become a media consumption device for most people.
> but instead it's just become a media consumption device for most people.
I don't think that is true. I think 'most people' don't need a computer for more than what an iPad can do, and that your use case is more exceptional than common.
Computers as we think of them, with a somewhat permissive operating system that let's you execute whatever you want are probably going to decrease in relevance to somewhere akin to how they were in the 90s -- incredibly powerful devices useful to some people who need them for particular reasons or just like using them. Everyone else will be fine with whatever the equivalent of the smartphone/tablet OS is.
Sure, but I still see most people using iPads for media while having a MacBook for "real work," which is exactly what Apple wants, to not cannibalize their own products.
I'm actually not seeing that and the market numbers don't agree with ya on that. Many less "pro" consumers (not coders or the heavy engineering skew you find here on HN) in my circles have moved away from their traditional Macs as their iPads coupled with their keyboard case of choice or apple pencil (or both) do a lot of their on-the-go.
I've had people at work tell me their kid doesn't want a MacBook because their phone works fine for writing papers. My brain still can't process.
I've never made my iPad work for me on a business trip except for something very short where I won't really be writing. I also don't draw and, while I could probably do photo editing on it well enough, I wait until I get home and use old-school Lightroom.
I'd agree with this, with one little carve out... when I was traveling internationally, it was super helpful to have an iPad and be able to slip in a sim card for a local carrier. That plus slightly smaller size made the iPad a better option than carrying my laptop in those specific circumstances.
Fair point. I don't have an iPad with cellular. Honestly not sure when/if I'll get a new iPad--it's about 6 years old--and I have a long recreational international trip coming up; haven't decided if I'll bring a laptop or not.
Apps like Procreate have sold a lot of iPads to artists who use them for “real work”, as has the Sidecar capability which lets an iPad and Apple Pencil act like a Cintiq with a better screen that’s not desk-bound.
It’s true that they’re limited but that doesn’t mean that there isn’t work that’s well-suited for them.
Or, the computer manufacturers will try their best to stuff the Permissiveness Genie back into the bottle. It seems with every macOS release, Apple is determined to make more things previous Macs could do either impossible, hidden behind permissions or hidden behind default-off settings.
Well, you can use your mac's screen inside the vision pro, but I've read the resolution for that is weirdly low, QHD.
Meta isn't really opening up their headset, since the whole point of that adventure for them was to have their 'iphone' type kingdom.
Our best bet might just be Microsoft but they gave up on mixed reality windows, and Idk if they're really gearing up to jump into this product space.
I'm fascinated in the idea of Vision pro, not sure I'm ready to shell out for what is basically just a really great movie watching experience. I can see the movie pretty well on my 4k tv, and I'm not sure the improvement is worth the cash.
Side loading apps on the quest is pretty simple using the aptly named SideQuest, it only requires a one time 15 minute set up. Definitely a lot easier than trying to do this on an iPhone/iPad/iQuest.
Meta Quest also has full support for WebXR which lets you bypass all the app store BS. You can use this framework to make worlds visible on most XR headsets as well as desktop and mobile
My understanding is that Apple is making excuses about being slow to support WebXR.
As much as the media has benn fixated about the failed Horizon Worlds (which is a very interesting story that I've never seen elaborated from a software dev's perspective) you might not know that the successful model for Quest is the same as publishing on other game consoles. The market for single-player games for MQ is really healthy but multiplayer games are really thin on the ground which makes the "metaverse" idea look pretty dodgy. I mean, there is the multiplayer VRChat but you're going to have to rub shoulders with weaboos and furries which keeps other obnoxious people away.
> My understanding is that Apple is making excuses about being slow to support WebXR.
While this is true for the desktop and iOS/iPadOS Safari, their VisionOS version of Safari comes with full WebXR support. Which was confirmed explicitly in this Apple’s tech talk[0] and by the fact that it is indeed supported in the VisionOS simulator (which has been available for quite a while already).
> I've read the resolution for that is weirdly low, QHD
This is something I've been wondering about. The Vision Pro's displays are 4K per eye, for the entire field of view. The monitor I'm sitting in front of is 5K, and takes up quite a bit less than my full field of view.
Surely the virtual Mac screen (and everything, I guess) is gonna be substantially lower resolution than traditional high-DPI screens at normal viewing distances?
It’s going to be presented as a 5k Retina display to your Mac, with a 2550x1600 at 2x rendering, downscaled somewhat to a bit over 4k of so. Therefore it will appear kinda like a retina imac. 27 display but not quite as sharp.
If the headset manages to be 4000 pixels wide, but that is filling say 110 degree FOV, then if your virtual display is 44 degree FOV, it's only going to be represented by about 1600 pixels wide. 4000/(110/44) = 1600.
I wish we knew what the actual resolution of this headset is and what the FOV is.
They say it's 23 MP total, so if the screens were square that would be 3391x3391.
If the FOV is 100 and our virtual screen was 50, then the screen would be 1695 pixels across.
And finally you lose some effective resolution because of the nature of the virtual screen being not aligned perfectly with the headset's panel. Since your head would be slightly askew in relation to the virtual monitor due to head tracking.
I think your math is correct, and it will be very limited compared to a real-life retina screen, but I think there's also a perceived resolution increase from having binocular vision and moving.
If you hold your head steady while looking through a window with a plastic screen on it, things outside are obstructed. If you move your head slightly back and forth and focus in the distance, the screen pretty much disappears.
Your brain can do some motion smoothing to determine the "actual" content, even if it's sampled. I'm not sure how you could quantify it, and it only helps a relatively small amount, but it's there.
But you can see your Mac (running macOS) on it. In that sense, it’s “just” a monitor, but far from Apple’s most expensive monitor.
Another comment mentions that you’re confined to the host computer’s “screen” and can’t break applications away from that rectangle. But you could imagine that being a possibility in the not-too-distant future.
I don't think it's worried about cannibalizing mac sales. If it does that it just means Apple gets to sell computers at a substantial premium with a moat against other manufacturers.
I think it's worried about not being able to apply a 30% tax on third party software.
I feel like this is a gut-level response based on history with the iPad, but doesn't translate to the Vision Pro.
I think it's meant to be used with a Mac for most productivity use cases. That's how I intend to use mine: VSCode, terminals, compiling all happening on my nearby laptop with the Vision Pro as "just" a 4K monitor, and then extra apps like Slack, Zoom, Safari, Mail, music, etc, floating around as native visionOS windows.
In addition, it can be used as an iPad-like media consumption device, e.g., on an airplane, but I see that as an additional (and for me only supplementary) use case.
You don’t need a large FOV for movie viewing. Even if it were only 60°, you’d be able see a 2 meter wide screen at a distance of 2 meters (and that’s measuring that distance to the edges of that screen; its center would be about 15% closer)
I've got an HTC Vive which is also ~110degrees. Although you do initially get the 'looking through ski goggles' feeling when you first put it on, that feeling VERY quickly goes away when you start doing anything inside. It's plenty immersive. Sure, more FoV would be nicer, but it's not like looking through a telescope or something.
Right, your brain really does not give a crap about the very edge of your vision. Instead of spending thousands on way more screen and optics real estate that you will barely notice because you barely notice ANYTHING outside of your focal point, we'd probably be just fine with a couple RGB LEDs shining just colors at your periphery.
That sounds like a way that may actually work (and don't cost too many penny). Just put dozens of programmable LEDS at the edge of your view. And shining correspond colored light when someone actually use a flashlight to point at you. That might even be used as a safety measure without block your view.
The Verge's review says the FOV feels noticeably smaller than the Quest 3 which is 110°:
"The displays have other limitations: the field of view isn’t huge, and the essential nature of looking at tiny displays through lenses makes that field of view feel even smaller. Apple won’t tell me the exact number, but the Vision Pro’s field of view is certainly smaller than the Quest 3’s 110 horizontal degrees. That means there are fairly large black borders around what you’re seeing, a bit like you’re looking through binoculars."
I don't think it's even possible to get a fully immersive FOV in that form factor with the current SOTA optics, the few headsets which do have ultra-wide FOVs are enormous bricks.
The widest FOV headset you can currently buy is the XTAL 3 at 180 degrees, and it's huge, despite being a PC-tethered design that doesn't need to make space for a SoC or battery.
Yeah, but XTAL 3 MR has zero standalone capabilities and requires being tethered to a rather beefy PC to function at all.
From their own page, the very minimum requirements for the PC are 32GB RAM and an RTX2060 GPU. And that’s a minimum, so I am not sure how well it functions with those specs.
That battery pack is nothing in terms of weight compared to the type of a rig needed to support XTAL 3 MR headset. I can put the Vision Pro battery pack in my pocket, but I would struggle to fit the rig for XTAL in a backpack.
Just the lightest GPU listed in the minimum requirements (RTX2060) is around 950g[0], while Vision Pro battery pack is ~350g. Add to that a motherboard (being able to support that GPU and a i7/i9 CPU and 32GB of RAM), a CPU fan, and a 500W (probably could get away with less, but i decided to be safe) power supply being around 3.9lb(~1.7kg)[1].
Even without accounting for the motherboard, desktop case, and CPU fan weight at all (which can vary significantly and are pretty heavy compared to the rest of the components) and counting them as a zero, you have already crossed into the 3.6kg territory. With those accounted for, you are crossing into the 5kg territory easily. And if you decide to go beyond minimum specs for GPU, I would expect it to be even worse. And the size of that monstrosity would be rather significant too.
I think comparing that to the weight and size of the Vision Pro battery pack is not that good of a comparison at all.
P.S. I am aware that there is a noticeable variation in weight of GPUs depending on the manufacturer and the specific variant for any given model, as well as for motherboard/cpu fan/power supply. However, that would not change the overall conclusion at all, as there is exactly zero way you would be able to get the build below 3kg. And I am being extremely generous here, as my estimate is very likely below even the lowest real-life lower-bound estimate.
No, but remote desktop solutions will be a popular app category for AVP. The developer of Virtual Desktop has announced that he intends to support visionOS, for example.
One of the aspects of the device that has been under-realized is that when mirroring your desktop/laptop display to the AVP, you can't break out its applications into different areas. You can't pull them away from the desktop window.
This is one of those things that Apple never claimed was supported, and yet there's something about that behavior that feels like such a natural intuitive implication to the technology that a lot of people feel alarmed or even cheated when they realize it's not possible (yet). It's been funny to watch the various discussion threads as people pop up talking about their shocked realization and disappointed feelings.
Update: I did realize when watching the WSJ video that the "mirrored" display actually appeared to have greater "resolution" (more pixels in height and width) than what she had on her laptop. So that's something.
I wouldn’t be surprised if this came in a visionOS update. On a shorter timescale it could also come in the form of third party apps, because there’s no technical limitations preventing a server app from cutting out windows on a desktop OS and sending them over a wire to a visionOS client.
There are third party apps that do this already on the Quest. I believe they can replicate Mac screens, they definitely can replicate Windows PC screens into the VR space. If Apple doesn't provide a 1st party solution I suspect someone else will soon.
On a shorter timescale it could also come in the form of third party apps
If Apple approves it, of course. This is one of my major concerns; there's a lot of potentially useful functionality that could be implemented, but you have to jump through the app store hoops and hope that Apple doesn't decide that it conflicts with their idea of what you should be allowed to do.
Functionally speaking these apps would be scarcely distinguishable from the plethora of screen streaming apps that exist on the App Store already, like Screens and Moonlight. Of course Apple could reject these apps anyway but it seems unlikely.
I agree that this should be considered long term, however... you are able to snap VisionOS / iPadOS apps anywhere around your Macbook view AND you are able to control those very apps with your Macbook trackpad.
So even though you have a sequestered Mac output alongside Vision apps, you can use the same controls for all them simultaneously. This should help in the interim.
Yeah when I found this out, it resolved my concerns. Most of my apps will have a native Vision release (email, web browser, slack, etc.) and my actual monitor screen will only need more professional software (e.g. Photoshop, Illustrator, InDesign).
It seems very reasonable this will be a future feature. I've long suspected iPad OS' stage manager feature shipping so half baked was really more of getting the platform ready to support multiple apps and easier manipulation (from a developer perspective) of the double buffered "window" textures - given Vision Pro is based on iPad OS.
With Stage Manager on macOS now, it feels like they have all the primitives in place to "transpose" macOS stage manager windows textures to Vision OS/ the iPad OS foundation.
Though this will be tricky to get right for all apps. Will be interesting to see if it's a macOS App store only feature/ API, opt-in, or some other option
They shipped iPad Stage Manager half-baked, to get iPad developers ready for double-buffered windows, so they could eventually ship the visionOS macOS integration half-baked? Doesn't sound right at all to my ears, even though I'm stoked for my order!
EDIT: -5* doesn't make sense, this is the most polite way you can point out that getting macOS apps windowed on visionOS has ~0 to do with double-buffered windows on iPad OS. n.b. I didn't use half-baked, OP did.
No, I'm saying they shipped iPad stage manager half baked for their own uses/ to refine for AVP. I'm positing that a major reason for macOS stage manager's existence is as a transport layer/ "texture formatter"
I don't think it's half-baked. I think it's lightly toasted. :-)
I use iPad Pro as a kind of sidecar daily driver, in the magnetic dock magic keyboard w/ trackpad.
As I type this, the screen shows a traditional MacOS style dock across bottom, four Stage Manager window clusters I can tap with a thumb on the left, and Safari plus Messages taking 2/3 and 1/2 of screen respectively.
There's more app and pixel real estate than most Windows laptops, and bringing screen sets to the foreground or swapping them back to the side is so natural I almost feel like giving up that space on my Mac as well.
The big thing I saw happen from apps over the past two version of iOS is app devs realizing their windows will not always be full screen or 1/2 screen size, but arbitrary size.
By now, most iPad apps of any serious nature are effectively window size independent, making them play well with others in stage manager. It's easy to see how that would make them play well with the headset one day.
They can already do this with the desktop composition software they use today. All the windows are virtualized onto backing layers that you can draw anywhere and add effects to. It’s how window shadows work, and how certain window effects are done.
So Apple could have done this, but did not. Why? (Speculation and leaks welcome)
- [Profit on basic innovation] Did they want to wait and see how their customers would adopt VisionOS's native free-floating windows, so as to avoid cognitive overload by commingling with MacOS windows?
- [Benevolence to fellow competitors] Did they not want to takeover the existing market of virtualized VR desktops?
I'm reasonably certain it's a combination bandwidth and tech issue.
The Vision Pro is effectively using AirPlay to mirror the whole screen. If you used AirPlay to mirror each window as a whole screen, you'd run out of bandwidth pretty quickly.
The windowing system in MacOS, Quartz Compositor, also isn't built to stream window information. Right now it has a big built in assumption that any windows its displaying are on a screen it also controls. It was probably too big a lift across teams to also re-write the graphics stack for MacOS for the launch of the Vision Pro. Hopefully they get it working in the future, but neither of these problems are easy to solve.
> Ensemble is currently "pre-alpha": it's really more of a demo at this point. There's a lot of things I need to work on and until then it is unlikely I will be taking any code contributions.... The code is definitely not designed for general-purpose use yet, so don't expect much of it :)
Some of the code is factored out into individual libraries. For example the networking and serialization code is separate and it is likely that screen recording will get pulled out too at some point.
Sorry but almost 10 years ago I could do this on Xorg where the worst problem is that compositors cannot redirect input (so you had to kludge new events from scratch). I cannot imagine it would take more than _half an hour_ for someone with macOS display compositor experience to implement it.
It probably actually is a quick initial implementation, but like everything else, it needs to be planned, ticketed, developed, tested, and slated for release.
Compositor work isn't tremendously hard when the graphics primitives are already done.
UI state handling and input is a majority of the work from there. I've implemented this work before. A large portion of my background is in compositors and UI primitives.
What's really cool about having window backing layer handles, is you can do all sorts of crazy fun stuff in the office and show off to your coworkers or tech demo that basically will never make it to production because it's totally impractical.
The best example of that which actually did end up in a final product in my opinion was Windows Flip 3D. Totally fun implementation, I'm sure.
And the reason Xorg is now dying in favor of a system that doesn’t have this capability is because the architecture that enabled it, while cool at the time, severely limited the graphics performance and capability of the applications.
Exactly that. A local only path that provides high performance.
Remember the original comment was the claim that Xorg could easily do remote windows years ago. This is only true using the low performance path that modern operating systems have universally rejected.
No, the original comment is that by writing an Xorg compositor I could project separate windows into separate surfaces at arbitrary positions in the 3D world. There's no need for network transparency and you can very well do with an HDMI cable (actually, I was using an HDMI cable -- this was 10 years ago).
Network transparency is exactly what people want. If you're going to say just tether it to your computer with a wire then you just aren't understanding the product.
Apple will always prioritize critical scenarios over nice to have. None of these things are technically difficult, it’s just time. I’m willing to believe they released too early, but at some point you have to start learning from real users.
Literally nobody has done it. It's beyond ridiculous that you can't already show or duplicate an application window on any display you want and allow it to be controlled from anywhere it is visible.
Searching for ways to do this lead one into extremely niche software ecosystems. Please is there any collaboration app out there that makes it seamless to toss windows around like everyone actually wants?
Nearly 20 years ago there was an OS X screensaver that would capture all the window buffers and then float them around the screen as they rotated various ways. Another app would save a snapshot of all windows as Photoshop layers with appropriate transparency (before blur was added). X11 was always network transparent, though I don’t think windows could easily be moved from one client to another.
Even classic Mac OS, which wasn’t designed to be “rootless” could be run that way could have its windows mixed with OS X windows in the classic days.
As others have pointed out, there is already a PoC app that is doing it, so it seems if Apple wants to, it’s completely in their power. However, does this match their vision? (pun intended) Time will tell.
Isn't this how ordinary spanning monitors works? It might be slightly awkward with AR goggles since the relative orientation of the displays will be constantly changing as your head moves, and what happens to a window you have half on and half off of the Macbook's screen when you look away? Or do you want to have the application jump between devices, like appearing on your fridge when you go for a drink? With the old X11 protocol and a daemon in the middle this was possible but the use cases were extremely limited and the security issues made it a pain in the ass to actually use. With distros moving away from X11 this is only going to get harder, and you have to ask yourself how much you really want it.
This would mean the goggles would be basically just a dumb display for the Mac. It would also be weird to try to move an AR app onto your Mac.
Yeah, I don't think "mirroring" is quite the right term. It's effectively a 4K monitor for the laptop, with the laptop screen going black. Most (all?) Mac laptops don't have a 4K screen, so you have more screen real estate than "mirroring" would make you think.
But this is sufficient for many use cases (or at least, mine). I pre-ordered one with the idea that my main work will be on the 4K monitor, with most of my superfluous apps floating around as native visionOS apps. That's mail, a web browser, and zoom, which all have apps now, and Slack, which I could just use Safari for but may have a native app in the future.
It's more like 1080p monitor. The virtual monitor only covers a small part of the VisionPro's display. You can compensate a bit for a lack of resolution by making the virtual screen bigger or by leaning in, but none of that gives you a 4k display.
To really take proper advantage of the VR environment you really need the ability to pull out apps into their own windows, as than you can move lesser used apps into your peripheral vision and leave only the important stuff right in front of you. You also miss out on the verticality that VR offers when you are stuck with a virtual 16:9 screen.
The screen real-estate is the same as for a 1440p screen. From The Verge’s review:
“There is a lot of very complicated display scaling going on behind the scenes here, but the easiest way to think about it is that you’re basically getting a 27-inch Retina display, like you’d find on an iMac or Studio Display. Your Mac thinks it’s connected to a 5K display with a resolution of 5120 x 2880, and it runs macOS at a 2:1 logical resolution of 2560 x 1440, just like a 5K display. (You can pick other resolutions, but the device warns you that they’ll be lower quality.) That virtual display is then streamed as a 4K 3560 x 2880 video to the Vision Pro, where you can just make it as big as you want. The upshot of all of this is that 4K content runs at a native 4K resolution — it has all the pixels to do it, just like an iMac — but you have a grand total of 2560 x 1440 to place windows in, regardless of how big you make the Mac display in space, and you’re not seeing a pixel-perfect 5K image.”
Discontent over this implementation detail shows users are fully sold on the basic idea. Like, if the main complaint about the first Fords was the colour range.
I think it's a more important feature than just a cosmetic color. Imagine if you bought a truck to haul cargo, but were then told it can only transport one type of cargo at a time. That would suck.
I don't know about you, but there's nothing easy about having to boot up an IDE every 7 days to sign and re-install an app you depend on. It's just one more thing to worry about and mentally keep on top of, the opposite to Apple's convenience and smooth UX ethos.
The visionOS side of this, which is the one that requires App Review, is a glorified VNC app. They will probably review it thoroughly but it doesn't seem like it breaks any rules.
The largest portable MacBook Pro 16.2 inch has a 3456-by-2234 native resolution at 254 pixels per inch which by default is halved . So I don’t know what she means exactly about 4k but there are enough pixels to do a portable 4k display.
The inability to break out Mac windows curbed a lot of my enthusiasm for the AVP. I hope Apple will eventually add it, but I'm not going to spend $3500 on that hope.
It will be less of an issue for me if we start seeing native builds of popular IDEs like xcode, intelli-j/goland, etc for vision pro (and other apps for other people say photoshop). I think of the 'projected screen' feature more like a compatibility layer like Rosetta 2. You use it until you get a native build then it stops being a thing you bother with.
That will never happen. Nobody will make real software for iOS because nobody wants to pay a 30% fee for the privilege of randomly having their whole business shut down when an app reviewer is in a bad mood.
Are you referring to IDEs? The iPad is not a productive machine. It’s for consumption. The VisionOS is for productivity which is very different from anything on iOS so I imagine it’s going to be available on it.
I don't see why you couldn't port an IDE to ipad os.. filesystem is a bit tricky with the way icloud files work.. but otherwise it's just a thing.
I think mostly people don't (for high end IDEs) because they expect a keyboard and pointing device that isn't a touchscreen and those are faily rare per ipads sold.
The ipad pro certainly is seeing some more beefy productivity apps it seems.
Streaming an arbitrary collection of windows instead of a single finished, composited framebuffer increases the bandwidth requirements by at least an order of magnitude. That's never going to work well over WiFi.
As long as the total number of pixels is less, I don't see what that has to be true, at least bandwidth wise? Compute wise, the vision might have to do slightly more to separate the buffer and composite them into the AR view in different places, but the bandwidth should be direction proportional to the number/size of each window. If I can fit all the windows on a 4K screen, then I don't see why the software can't split that and lay it out separate in my view instead of in a single rectangle.
Some of the windows will be obscured by others. If you stream a 100 windows to visionOS, it's possible to lay them out so that none of them cover each other and you have to render them all. On a flat screen there is a limit to how many pixels you need to paint.
Just compress the stream. Total pixels increase the vram on the device but popping out a static window shouldn't take any more than a trivial amount of streaming bandwidth.
Everything is already compressed. Uncompressed 1080p is 3Gbit/s, which is already well beyond what's actually achievable with WiFi. Allowing an unbounded number of window surfaces to be streamed invites people to use it in a way that current technology simply cannot provide the kind of slick experience that Apple needs from this product.
Seeing as you have the eye tracking you could probably greatly reduce the FPS/bitrate of windows that are out of gaze. This kind of foveated rendering seems to already be used in their ecosystem so I assume its sufficiently slick for Apple. Apps will sleep when not being gazed and such.
This allows you to restrict full bitrate to a single window with a maximum resolution and keep the user experience high and astonishment low.
Use the touted eye tracking feature and compress that particular applications stream inversely to how much the user is focusing on it within the Vision Pro.
Does foveated rendering work with the extra latency of a round trip over WiFi? I can certainly see this working fine on a head tracking scale, but I'm less sure about eye tracking. I also wonder if existing video compression hardware can actually make use of such outside feedback to adapt bitrate and quality with the necessary flexibility.
Further, I wish they added support to make multiple virtual monitors from macOS Workspaces, like what happens today when you attach another monitor. Switching workspaces can be bound to keys in the Keyboard Settings. Moving windows to other workspaces is easy to do with third-party apps like Amethyst.
It feels like the Vision Pro would definitely be a great replacement for people who (want to) buy multiple expensive monitors, but it doesn't fully reach that potential today, and mostly because of software? Although rendering 3 or 4 virtual workspaces through ad-hoc Wifi at 4K 60fps+ low-latency would certainly be a huge challenge.
I do this sometimes on my meta quest. Go into desktop VR and pull up a couple desktop views so I can see things happen in real time on different “screens”.
I guess there is something to the macpro being able to handle the output for one screen at a time. If it has to render 4k outputs for 10 different screens simultaneously, performance is going to suffer.
On Windows of all places (95ish to MEish) there was a remote tool called radmin and it had something that I wish companies had embraced: it hooked in to (maybe even before?) the window-rendering functions and sent the changes over the network. It’s hard to explain exactly what I mean because everyone is so used to streaming pictures of the screen over the network (if they even use remote access at all), but you could have less than 20ms latency while controlling over the internet while using tiny amounts of data (50kbps? 100? not sure but somewhere around there).
OSX had the opportunity to follow that path before settling on the “render windows, capture the screen, compress the image, send it over the network to be decompressed” VNC-style remote access that’s bog-standard today, and if they had Vision Pro would be set up to be an absolute mind-blowing macOS experience.
Wayland clients don't draw things the way old-school X clients do (neither do modern X clients), so it doesn't make sense at the Wayland level. KDE or GTK could potentially implement something like this though.
isn't that how Xorg remoting used to work as well? the display server and client are separate, so whether the pipe was local or remote didn't matter. In principle, Wayland could do it too, I think, if there were a way to synchronize texture handles (the Wayland protocol is also message-based, but IPCs GPU handles around instead of copying bitmaps.)
I guess one downfall is that that your pipe has to be lossless, and there's no way to recover from a broken pipe (unless you keep a shadow copy of the window state, and have a protocol for resynchronizing from that, and a way to ensure you don't get out of sync.)
> On Windows of all places (95ish to MEish) there was a remote tool called radmin and it had something that I wish companies had embraced: it hooked in to (maybe even before?) the window-rendering functions and sent the changes over the network. It’s hard to explain exactly what I mean because everyone is so used to streaming pictures of the screen over the network (if they even use remote access at all), but you could have less than 20ms latency while controlling over the internet while using tiny amounts of data (50kbps? 100? not sure but somewhere around there).
This has been done many times before (see e.g. X Windows) and has known downsides. Off the top of my head:
- You need the same fonts installed on both sides for native font rendering to work
- Applications that don't use native drawing functions will tend to be very chatty, making the total amount of data larger than VNC/rdesktop/&c. style "send compressed pictures"
- Detaching and re-attaching to an application is hard to get right, so it's either disallowed or buggy.
This can only work until it doesn't, and it won't work in many situations because eg 1. apps aren't going to bother being compatible with it 2. compositing has surprising performance and memory costs, and in this case the destination is more constrained than the source.
IIRC many windows apps at that time were using MFC or otherwise composing a UI out of rects, lines, buttons, etc. Then came Winamp and the fad to draw crazy bitmaps as part of the UI. If everyone does that, shipping draw commands is less useful and shipping pixels makes a lot more sense.
This is how Windows Remote Desktop used to work - it would forward GDI instructions to be rendered remotely.
It falls apart as UIs got richer, browsers in particular: they're entirely composited in-app and not via GDI, because GDI isn't an expressive enough interface. So you end up shipping a lot of bitmaps, and to optimize you need to compress them. You might as well compress the whole screen then.
I wonder how this works exactly. RDP lets me connect to a single monitor Win 11 host and display it on my client's three monitors. Everything is super smooth including browsers (I am connecting via Ethernet). Is the host managing the three screens or does the RDC do it on the client's side?
Note the "used to work" modern RDP will negotiate some form of image (or video) based compression for transferring data[1]. You can even share an X11 desktop over RDP using freerdp-shadow-cli.
Windows does this very well since at least a few years back. When connected via Remote Desktop any native application will get the behavior you describe, so the UI gets updated with almost no latency.
Applications which bypass the native APIs to render their window contents, in particular video players or games, get a compressed streamed video which has very decent performance. The video quality seems to be dynamic as well, so if there's a scene with very few changes you can see the quality progressively improve.
All of this is done per window, so a small VLC window playing a video in a corner gets the video treatment, while everything else still works like native UI.
This only works with two major assumptions, neither of which is true for the VisionPro:
1. The receiving side has to have at least as much rendering power as the original side, since it will be the one actually having to render things on screen. This is always going to be the opposite case with any kind of glasses, where you'll always want to put as little compute as possible for weight and warmth reasons.
2. Each application actually has to send draw instructions instead of displaying photos or directly taking on control of the graphics hardware themselves. No or very few modern applications work like this for any significant part of their UI.
This thing could have been useful on day one for lots of people, from drone operators to cinematographers to programmers. But Apple's sad fear of I/O has crippled yet another product.
People should expect that a $3500 video-display device has a way to get video into it.
I think it will be more like the proliferation of noise-cancelling headphones; it's not about what it brings (music, dialog, etc) but about what it blocks out (the ambient sounds). Yes, you can watch a movie on a plane now, but with a headset you remove the depressing, crowded, and claustrophobic visual environment.
And watching content on a plane's built-in screens is a miserable experience: poor interface, lag, constant forced interruptions for even the most boring announcements (captain, donating foreign coins, meal schedule).
Back in the day I was flying back east and had some movies loaded on my iPod Touch.
And I was casually watching one in my seat, the movie was “Blackhawk Down”. And it occurred to me how a neighbor might not enjoy watching blood, guns, and violence out of the corner of their eye.
So, I can absolutely see value in a headset style movie experience in public places.
I am perhaps slightly selective about what I watch on a plane--depending somewhat on who is sitting beside me. But my observation is that US airlines have gotten quite a bit less editing-heavy in that regard and, at some level, it's not my problem. (Although I'll be reasonable if someone cares enough to ask me to not watch something.)
For international flights they have may have a screen on the back of the seat in front of you. Using a tablet or small laptop works just as well without the discomfort of a headset.
You actually do wobble a bit, and sometimes s lot in turbulence, which can be unnoticeable for you but will throw accelerometers meters at a time. Also routine altitude/speed changes also cause accelerations,
I've seen a few ideas that seem really interesting, just not at the price point Apple is offering. And honestly I'm just going to wait for these headsets to be smaller and lighter too before buying one. I don't think anyone at Apple is under the impression this device is going to be a money maker, but more of a gen-1 device to provide a place to start from for further work.
- Learning instruments in a guitar-hero way (Piano, guitar, drums)
- Cooking with timers and recipes right in front of you (will be even more doable with better internal displays in the future)
- Coding with virtual displays on-demand. This is another thing where we still need more resolution to make it really doable.
- Watching movies. Obviously a solo way of doing this but I could see it being big.
- (once these are much lighter and less intrusive) I could see these being huge for virtual workouts like Yoga, weight lifting, etc.
Also regarding your question, I'm trying to think of the "killer app" is for a currently successful device - iPhone. I mean, camera? Texting? Most people use tiktok a ton but I wouldn't consider that a killer app. I think it's more of the device providing a home for a bunch of different apps.
killer app for the iphone was a truly usable portable web browser. everything else early iphones did had been done many times before, but a web browser that actually worked pretty well on a portable device was new
Cooking sounds like a nice application, but as soon as the headset steams up/get condensation on the glass it will be quite annoying. I can't imagine cooking in ski goggles.
> Learning instruments in a guitar-hero way (Piano, guitar, drums)
What would VR add here? For guitar-hero style instrument learning, there are already Yousician, Simply Piano / Guitar, Gibson and Fender apps, and quite a few others.
For piano it lines up the keys with the notes on your physical keyboard, so you don't need to look at a separate representation of a keyboard, you can literally see the notes fall onto your piano at the right times. Same thing with the other instruments.
The problem is that once you get past the very initial stage, you want to read ahead of what you play at the moment. At that moment Synthesia-style falling blocks notation becomes difficult to read because the blocks don't have any position reference besides the keys and no time values. Even the scrolling sheet music or tab is distracting compared stationary black and white. With guitar there's the additional problem that you can't see both hands at the same time, and usually you won't be looking at your hands at all. And if you ever want to play without help from the app, you need to learn to find the notes anyway.
exactly. Those who think VR headsets are good music teachers probably have never taken any serious music lessons or master any instruments themselves. If you have just taken one month of lesson and have been practicing daily, these things will start to prevent you from making progress, not help you.
(coming from someone who has been taking piano lessons for the past 8 years and have a Quest 3)
I'm imagining an AI VR tutor - heck, with some legal deepfake+AI style evolution, perhaps that tutor could actually be a famous player, talking to you.
The 3D view gives, well, a 3D view, and all the advantages it entails. For example, when you are with a physical real world teacher, you don't have a fixed view. You can observe from any angle, the teacher can observe and correct micro errors. And the senses of scale and proportion are intact, unlike viewing on a 2D screen of arbitrary size at arbitrary distance.
Even the simple POV of VR means that you get to see the techniques in-situ - e.g. seeing the expert's hands on your guitar, and how the technique is supposed to look from POV - rather than the standard teacher-student limitation of e.g. guitar, where the student sits opposite the teacher and sees a reversed image.
But to be clear, my vision of how this would be game changing relies upon a level of interplay between hardware and software that is not yet developed. But I expect it will be, in time, because virtualising real world experiences is arguably the core goal of VR, and (correct me if I'm wrong) but nothing in the music tuition world has yet proven superior to having an extremely skilled one-on-one mentor who can personally guide your every step, "in person" - and this is what VR would seek to achieve here.
May I ask if you play any instruments and taken music lessons for a serious amount of time?
My guess is not, because to me it is obvious you don't understand how this works and are just imagining things.
Speaking as someone who takes private piano lessons and have been practicing daily for the past 8 years:
If this were possible, music teachers would have already been replaced to some extent. But that has not happened yet. AI is not new. VR is not that new. Even without this, just talking about non-personalized, non-VR instructions. Have you seen those websites that "teaches" you piano just by connecting your keyboard to a computer and following instructions? Where are they now? It does not work like that. Maybe for an absolute beginner who has trouble finding a key or fret, but as soon as you get a little bit better it won't work.
Music learning is an extremely personalized experience. People pay 1-1 private lessons because it needs to be done that way. A teacher, especially a good teacher, can tell you exactly what you did wrong and what you should do to get better at it. Plus different teachers often have different opinions of music -- how a piece of music should be interpreted or played. As you get more advanced, you spend even less time on technique itself. Theoretically you could train an AI tutor that does the same thing. But no, it hasn't happen yet and will not happen. Did I mention hallucination? Do you want a teacher that hallucinates?
Plus you need lots of practice -- routines, repetitions, all those scales. AI or VR is just useless. Effective practice strategy is important, but you don't need AI for that.
> My guess is not, because to me it is obvious you don't understand how this works and are just imagining things.
That's great! I'm glad to hear that your 8 years of music practice has given you some firm opinions, and I'm sure you'll continue learning over time. Speaking as a player and composer of 25 years, former music teacher, and maker of the instruments I play - for whatever such things are worth - I believe my ideas have some merit. But only time and technology will show whether-or-not such AR (and AI) musician overlaps eventuate.
Working out with a VR headset on seems like a great way to get the whole thing disgusting. Hope I can put this $3500 face computer through the washing machine.
Unless Apple has truly screwed the pooch, this has been a solved problem. Sometimes, if you are purposely trying to work out with a VR headset, you just wear a headband to catch the sweat. Most headsets also have cheaply replaceable or even machine washable face "gaskets". If the idea of any sweat in your poor device saddens you, just point a fan at yourself while you work out.
Also, since the gaskets just stick on with magnets, I'm sure there will be third-party ones with included fans or whatever available within a couple of weeks.
Or real workouts. I want to be able to have floating text to read on my runs, and real time biometric data directly in my field of view rather than on my watch would be cool too.
I think the killer app will be remote work. The social interaction in VR chat is pretty good, if they can bring your work environment in with lots of monitors, and then share parts of it seamlessly and on demand but also have ample 100% focus time, I can see it being better than an office in some ways
For me, I think the killer app would have to be something that interacts with the real world. Watching movies, FaceTime, or browsing the internet won’t be enough for me to ever buy this. But maybe if there was something like a home repair or car repair app that could in real time identify and inspect the objects, give me visual and audio instructions on how to perform the repair, that could be a killer app. But I’m not sure the AI/ML side is good enough yet to enable that.
Speaking of the MS Hololens: Is it still a thing and is there a community of 3rd-party developers around it? It showed a lot of promise when it was released.
They pivoted to military applications and I haven't seen much news about it. It looks like something out of the futuristic call of duty games. It looks super cool, but apparently soldiers didn't like it.
I think even if it were good enough to place some static instructions over a non-disruptive part of your field of vision, with a pleasant way to display embedded images or video, and a hands free control mechanism, I would find that incredibly useful for DIY projects and stuff.
> According to an exclusive report from The New York Post, NBA Commissioner Adam Silver “said the league is working with Apple to bring a tech-enhanced viewing experience” to its upcoming headset.
When asked about it, he told the outlet: “We’re working very closely with Apple.”
If there is one group that has a track record for laying down piles of money for hardware like giant televisions and expensive streaming services, it's sports aficionados.
Floor seats will be less desirable...or floor seat patrons are wearing Apple Vision/goggles for the AR (joking! but maybe serious)...
I do agree, just watching a sporting event on a projector where the athletes are life-sized is excellent, on-field cameras already provide a better view of the game then any seat.
Now make it more immersive, and the trick of immersion is very cool. Like The Sphere, that immersion is next level.
Add sports-betting to the Vision experience, that is a great side-car app for this. (I am not pro-bet but see the usage).
With standard coverage of sports, they cut between cameras and zoom in/out, pan to follow the action etc. Even people attending the game will be watching big screens showing this content some of the time.
I'm wondering how that translates to VR.. do they just teleport you around the arena? That seems like it would be a bit jarring if not altogether sickening.
It could offer a couple options. One being the main view is like you are there. You see the court in a huge view in your main view. Then, similar, but way better than in person, you have several big views off to the side, for replays, stats, maybe following a view of your favorite player...
I would think they we have a way to make it so your view (switches as the action on the court switches to the other basket). That will have to be a good design but just the idea of something like teleporting from your seat with a great view of this half of the court to now having it on the other half.
And for other sports even better I think. With say soccer/football and football (USA) you can have a big overview (like you are sitting at midfield) but then camera angles for closer view of the action...
I think sports viewing could really be incredible. Figuring out exactly how to do it well will take awhile. But it seems to me the kind of thing Apple could do very well.
I imagine some combination of a single location environment view, floating 3D panels that follow players the same as TV broadcasts, and a miniature arena where you can watch everything at once from a top-down perspective.
For me it’s movies. But the fact that Netflix and Amazon haven’t updated their Quest apps in years — nor has Meta bothered building its own movie streaming solution (e.g. making the Quest Browser compatible with streaming services) leads me to think that the active user base might be slim.
Can’t imagine doing work is the killer app, not while wearing a headset is more cumbersome than opening a laptop.
I'm not sure how Apple can make watching movies in VR that much better than they are already on the Quest series and other existing VR headsets. And as you noted it doesn't seem to be a popular use for those devices. Maybe the higher resolution on Apple's headset will win people over?
> I'm not sure how Apple can make watching movies in VR that much better than they are
Apple is already making it's own 3D content.
For instance watching a concert shot in 3D with spatial audio may be something that people find compelling. Certain artists, for example Taylor Swift, have fans who are well known to be willing to spend a fortune to attend her shows in person with high end tickets running $750.
I tried some of the 3d concerts that Quest recorded a few years ago.. it's a neat trick but Apple will definitely need to do something different to make it compelling
I haven't tried them personally but I think current quest headsets still have visible pixels. The vision pro is Micro OLED with no visible pixels. So it should feel like watching an actual IRL screen.
It’s an incredibly popular use for the quest, Amazon and Netflix just don’t own the market, people are watching content elsewhere like big screen/vrchat/various vr players.
Almost NOBODY uses the Netflix app because it’s terrible, terrible software.
do you really watch that many movies all by yourself? that's the problem for me .... if I'm carving out 2.5 hours to watch a good movie I am wanting to spend that quality time and share the experience with someone else as well.
Quest headsets are already pretty compelling for some use cases, but the tech still needs more advancement (at a reasonable price point) to go fully mainstream.
For a lot of people, Beat Saber and similar games are a killer app for the Quest. It can be good for making exercise fun and accessible at the same time.
Both of these suffer from the fact that a good portion of the target audience is going to want to barf as soon as the camera moves. So games are limited in what kind of gameplay they can offer and real life events are stuck with either a fixed location or "jumping" from camera to camera.
> So games are limited in what kind of gameplay they can offer
I disagree. Games will have limitations. Limitations constrain but foster creativity. Console games were already limited in so many ways yet people found amazing ways to make them fun as hell. When mobile games come I truly believed that nobody would be able to have fun there, yesterday I played the entire game of League of Legends retrofited for mobile and it works! So now VR gaming is great, people will find ways to make it even greater without free movement that makes you sick
Sure they have, it’s funny how often this is repeated. It’s anything social. It absolutely destroys every other technology for interacting with others over distance. There is nothing else like it and when the hardware catches up it will do a lot to shrink the distance you feel between those who live an ocean away from you.
I think we are now at a stage where VR hardware has surpassed software. Between this and the Quest 3, we have powerful, polished and consumer friendly devices, but beyond a few niches (fitness, simulation, gaming to some extent), there is nothing to convince users to put the headsets on.
I am hoping we will see a lot of experimentation in the coming years, and I am excited for what the Apple ecosystem will bring to the table. That said, from what I have seen so far this does not seem to be a revolution compared to the current offerings, but an evolution on various fronts, without addressing the killer app question.
It wouldn't be the first time that hardware gets ahead of software.
In 1988-94, the CPUs available in desktop computers were substantially more advanced than the widely used operating systems. Windows 3 and Mac System 6/7 didn't support pre-emptive multitasking, memory protection, or many other features that define a modern OS.
Maybe we'll look back at today's Quest and Vision Pro as similar transitional devices with one foot stuck in the old paradigm, running old-style software.
The Apple headset is maybe more akin to the expensive workstations of the time which did make fuller use of their CPU facilities but were neither priced nor aimed at consumers. The headset is not nearly as expensive (especially inflation adjusted) and is ostensibly a consumer device but it's current incarnation seems unlikely to have the kind of mass adoption for the analogy to work out.
The fact that the Vision Pro today mostly runs legacy iPad and web software in 2D rectangles kind of makes it feel like Windows/386 which most people just used to run text mode MS-DOS programs inside GUI windows.
> In 1988-94, the CPUs available in desktop computers were substantially more advanced than the widely used operating systems. Windows 3 and Mac System 6/7 didn't support pre-emptive multitasking, memory protection, or many other features that define a modern OS.
That was partly because RAM was over $100/MB (Nominal; ~$230 inflation adjusted) in 1990. Additionally, in the IBM compatible world, many people didn't have a 386 at that point.
Also, minor nitpick on the dates; 1993 saw OS/2 2.1 and NT 3.1, both of which had preemptive multitasking and memory protection.
There's a bunch of VR games/apps that are notable for fitness. Beat Saber is the most well known one that's explicitly a game, Supernatural is maybe the most famous one that's framed as more of a fitness app/service.
But other notable ones include Synth Riders, FitXR, OhShape, Pistol Whip, Thrill of the Fight, and (maybe) Gorilla Tag. And this list is far from exhaustive.
VR is pretty good for fitness just because it can make exercising more interesting, comparable to sports without the need to coordinate with other people (and it's easy to do inside your house, if you have at least a 2m x 2m open space). Major downsides would be having that space available and sweat inside the headset.
Meta (and many third-party manufacturers) offer a wipe-clean silicone facial interface designed for fitness-oriented users. The hardware isn't waterproof, but it's quite well protected and I'd be perfectly confident to work up a sweat. Quest has calorie tracking, can sync with the fitness tracking features on iOS and Android and can pair with heart rate monitors. Fitness is one of the key segments in VR, because fitness apps and fitness-oriented users have vastly above-average engagement and retention rates.
You don't need to worry with most headsets (don't know about AVP). I've been sweating in VR headsets for years and it never did any harm. I saw in the AVP reviews that it has a removable and swappable facial interface, so it should be easy to clean; the same is true of other modern headsets, and before that you had third party face covers or disposable absorbant stickers you could put on the facial interface to keep the sweat away.
That's not really an issue imo, the downside is mostly just getting part of the headset gross (you don't want to use it right after someone else has sweated up a storm in there, believe me).
used a Quest 2 for 2 years, and I sweat a lot when doing intensive exercises. No issue so far. And it is $300, cheaper than the base iPad. What else can you ask for.
I think the discomfort of sweaty eyes is a much bigger problem than damaging the device. It's extremely easy toake the device sweatproof compared to fixing the sweaty eyes issue.
To add to the suggestions by the sibling comments, Eleven Table Tennis and Racket NX are both great racket-based games with multiplayer and a high skill ceilings. Depending on your personality I think applications like this are much more motivating than going to the gym as a workout.
I’m surprised Apple dropped the ball on fitness here given they already have a fitness platform. Imagine rowing in VR and feeling like you’re actually on the water. It would make exercising so much more motivating and interesting.
It turned out the killer Apple Watch feature was fitness, and I don’t see why it couldn’t have been here.
I've used a rowing machine with a VR headset (I don't recall which model, it was a few years ago, but probably some Oculus).
It was fun for a few minutes but not really usable for serious exercise:
- It's heavy and annoying (and this apple product seems even bulkier and heavier). The cable situation is also not great, you need a lightweight cable and ceiling suspension to keep it out of the way, but this is solvable.
- Exercise means you get sweaty. Can't wipe your brow and you have a wet headset on your face.
- You can't see your body and maintain proper form. The VR environment itself is also distracting if you turn your head around to look at stuff moving there.
I much prefer just to have my phone or tablet in fixed place in front of me to watch youtube or some movie.
I really don't think that's the best use-case for VR - if you're doing some kind of virtual rowing, a flat screen is going to offer most of the experience with none of the downsides. VR fitness is really about games that are fun in their own right and happen to be physically active. There's a big cohort of people who hate exercise and would never set foot in a gym, but who will happily spend an hour at their aerobic threshold because they're playing a fun game.
Heavily disagree. Just like running outside is generally more enjoyable than on the treadmil, so is rowing. VR rowing would allow you to row through basically every major city on earth, over Niagara falls, and all sorts of other scenic spots.
No, it would allow you to row in a gym/at home while looking at a pseudo-realistic version of every major city or Niagara falls.
The reason running outside is nicer than on the treadmill is not that you get to look at nicer images. You get to smell fresh air and feel the wind on your body, you get to sometimes interact with people, you get to feel various textures under your feet, you get to move your bodies in more ways as you turn left and right etc.
Rowing outside is even more different, as you get the feel of the water currents and water splashes as well.
VR can mimic none of these things, and it comes with the massive downside of a sweaty face that you can't wipe away.
On the other hand, row fast or you fall down virtual Niagara falls sounds like an entertaining game and might make the rowing interesting... to me anyway.
I mean can’t speak for anyone else, but personally running outside is nicer because I get to look at nicer images. The fresh air is nice too but none of the other things you listed contribute anything for me. Getting wet is actually a downside of rowing outside for me, so that’s another plus of vr rowing.
> I think we are now at a stage where VR hardware has surpassed software.
Totally agree. I'm waiting for a usable Virtual Desktop app to come out. All the ones I have tried which work on my cheap WMR headset fall short of having floating app windows in view.
I guess there is one of those which works on Meta Quest, but not PC headsets. That's really what you need to be effective working in VR. Just like is mentioned the Apple headset supports.
I feel like the killer combination here would be a Virtual Desktop-like app for visionOS where the desktop VR passes through as an environment, but full local app layering is still available in the foreground.
People like to say this but my friend just sent me a recording of “drop dead home invasion” with the Quest 3 and my jaw was on the floor. He says it’s amazing too.
I’ve also heard about players spending a lot of time in counter strike games like pavlov.
At this point it seems like there’s a TON of things to do in VR (and I’m gonna be honest, there were a ton of experiences too on the Quest 1 when I had it).
I’m just waiting for more live shows and concerts that I can attend from the Quest personally.
Oh I agree - I loved my experience with Drop Dead Home Invasion, and there are a lot of amazing experiences overall. But, I do think that most of these have more potential as a "demo", that you do a few times but would not motivate you to use a headset every day, beyond a relatively small group of people.
It is like VR is currently stuck being Kinect in terms of sales and stickiness, while Meta and Apple would both like it to be at least like the Wii, or ideally the iPad.
Personally I have found social experiences to have the best long-term appeal (i.e. Racket NX or Drop Dead with friends), but even there I am not these apps have sufficient mainstream appeal.
> But, I do think that most of these have more potential as a "demo", that you do a few times but would not motivate you to use a headset every day, beyond a relatively small group of people.
This matches everyone I’ve heard talk about it, too. It was fun, they enjoyed a few things and then at some point they realized that their headset had multiple months of dust on it.
I think the big question is when costs come down to the point where it’d be reasonable for a non-rich family to have enough headsets to use together. Most people mention technical limits (resolution, latency) but the thing which everyone mentioned as a dealbreaker was that putting the headset on was shutting out everyone around them.
You’d lose that bet. Console games can be done with friends and family, and they don’t give you headaches or nausea after an hour. The hardware demands push the price up and that makes the social challenge harder, too.
The major problem VR has isn't the games, but all the boring and basic stuff, like using 2D apps in VR or running multiple VR apps at the same time.
The discontinued WMR Portal, essentially the Window's desktop in VR, was so far the only software that tried to be a full workspace in VR. But even that was missing a lot of important features and Microsoft gave up on it years ago and never made it accessible to non-Microsoft headsets. It's currently scheduled for removal from Windows.
VisionPro seems very similar to WMRPortal so far, with a few key improvements like allowing apps do add 3D objects into a shared space.
Underlying a lot of these discussions is the assumption that there is a future where this is an actually useful, mass-market device. I'm still not convinced this is true.
For example, "killer apps/content" never arrived for 3D TVs and they have largely disappeared from the market. Same with various "waggling" input technologies like the Wiimote and Kinect. There were some compelling uses, like Wii Sports, but these were pretty limited and many other uses of these in games was a case of Nintendo shoehorning the technology into the game.
I think the best pessimist argument is the one offered by Folding Ideas in his metaverse video[1]: Text is really, really useful, and a virtual 3D space is not a good environment for either creating or consuming textual content.
I see headsets like this as a way to have multiple, large, monitors that go away as soon as you aren't using them. After having a big dual-monitor setup for years, about a year and a half ago I got rid of them and work with just my laptop's display. I do this so that my desk doesn't have to have monitors on it, and is more conducive to artistic work and mechanical tinkering.
I don't miss multiple monitors so much, but I do often wish for a larger screen. Not enough to put one in my space, though. That's where my interest in the Vision Pro lies - simply a way to project large, high-fidelity, 2d screens.
I’d rather have small monitors than wear a headset while working.
For less than this headset, I can buy an Ergotron arm and some monitors and have multiple large monitors that have zero footprint on my desk.
No, I can’t take my setup with me when I go somewhere, but I don’t want to.
I have a hard time seeing this become a mass market device. It’ll have its adherents and enthusiasts, but personally find it difficult to imagine even wanting one, let alone using it.
But what do I know, I thought the iPad was goofy. Though I never did end up replacing my gen 2 iPad. Realized I didn’t need it.
Yes, my monitors when I had them were on arms - they wouldn't work great in my current setup due to its density, and the things I've put around the desk.
I certainly could make adjustments to accommodate a monitor - I just don't want to, and would love for there to be a way to not have to and still get to use a large display.
I used to think the same, but now I don't see these headsets replacing monitors/TVs any time soon.
I don't have a TV at my small apartment but do have two 27 in monitors. Sometimes I wish I had enough space for a sofa and 55+ in TV for movies and shows. And maybe headsets will solve the problem, maybe I don't even need those monitors any more.
Now? I realize I don't need multiple monitors, 2 is enough. (4k is definitely better than 1080p and 2 is better than 1, but I don't think I can properly utilize a third monitor.) And these are good monitors -- not Pro Display XDR good but still premium 4k monitors -- that have been around for years and I'll keep using them until they die. And about movies -- I have tried using the Quest 2/3 to watch movies. It is usable, and apps even provide "scenes" that make you feel you are in a theater. Do I want to wear them and watch movies on a fake 100" screen? No, for obvious reasons (comfort).
If the day comes when they are light, comfortable to wear, allow very good passthrough and actually do a better job than my 4k screens, I'll take another look.
Wouldn't it be simpler to just wall mount (or edge of desk mount) the monitors on pivoting arms so you can have desk space when you need it, with the monitors floating out of the way? VR seems like such an unnecessarily complicated way to do multi monitors.
Yeah, the Quest seems to be the spiritual successor to Wii sports. It’s the closest thing you can buy to that 2006 experience. The long dream of wielding a lightsaber on the Wii was finally realized on the Quest, to a quite satisfactory level.
And yet, from a gaming perspective, we still have to crack force feedback and natural locomotion before we have a holodeck. Maybe in 20 more years.
The "killer app" here is to have an infinity-sized 'screen' for anything the average user was already going to do with an iPad or Apple TV. The hardware's just not good enough yet, in terms of lightness and comfort, for the average user to put up with as more than a novelty.
The extreme majority of users is perfectly happy with two 24" monitors (actually, the extreme majority is happy with a single 13" monitor). "Infinite monitor space" isn't something that will sell a large enough numbers of headsets to prevent the cancellation of the next iteration.
Do you think it's plausible the hardware will ever be good enough?
People aren't really that happy wearing even vision eyeglasses, and I don't see any realistic chance we will ever be able to have something with the capabilities of the Vision Pro in a form factor as small and light as eyeglasses.
I mean it's the same problem for a lot of stuff, build it and they will come. Apple is super naive to think that they can just jump into this space and yet end up with a polished product (ie Apple's selling point).
I see on scifi all the time where someone flicks/flings a video playing on a device to move it to a larger display surface and it kills me that we actually have the technology to do stuff like this right now...but because every company works in their own interests/don't work together to create standards we don't get to have fun use of tech like that.
Anything goggle-like is a non-starter for me. I'm not even interested in trying it. I don't even like glasses, and I need corrective lenses to be able to see clearly (I wear contacts almost exclusively).
I'm not sure I've ever liked the term "killer app", because I don't think it's particularly useful in describing real user thinking and behaviour. There was a very long journey from VisiCalc and Wordstar to the modern-day ubiquity of office computing. Different user groups have complex, diverse and overlapping sets of needs and wants that can rarely be distilled into a single application. I'm more inclined to think in terms of Bezos's one-way doors - changes in user behaviour that are sufficiently compelling to be largely irreversible.
I agree that progress has been slow in the consumer space and meaningful long-term adoption of VR has been confined to a few niches; that isn't necessarily an indictment of the long-term prospects for VR, because desktop computers spent much longer in that stage than most people remember.
In enterprise, I think things are more advanced and some user groups have decisively gone through the one-way door for some applications. I think the best example is architecture. If you've done a couple of client presentations in VR, you just aren't going back to showing renders on a flat screen, because immersing the client in a physical space is that powerful. It's not just a sales tool, but a communications tool - clients can understand and respond to the environment intuitively and give much better feedback as a result.
Industrial and clinical training is less clearly one-way, but I think we're very close in a lot of areas. AR is still less developed than VR, but I do think we're on the cusp of something significant - a sufficiently comfortable standalone AR headset with sufficiently high-quality passthrough can deliver training experiences that can't practically be replicated through other means.
I think one of the most interesting areas of development is in psychiatry. It's still early days, but we're starting to see real, meaningful benefits in RCTs for VR-based therapy of disorders like phobia and PTSD. Some of the most compelling results have been in the very sickest patients - people with psychosis, who often find it impossible to engage with conventional psychotherapy.
I don't think it's remotely likely that VR will ever replace flat screens, but I do think that VR is slowly growing into a niche but durable HCI platform. Tablets are a reasonable analogy - a lot of people see them as a failure, but they still sell in serious volume and they're often a much better form-factor for specific applications than either a phone or a laptop, especially in industry. Tablets didn't change the world, but nor are they likely to go away.
Everyone is looking for the "killer app" so they have something to anchor the concept. Put simply, the Web fits the bill. Apple has invested in the space because they can't afford not to. Their App Store model starts to show its limitations when you stop staring at the screen and start looking through it.
WebGPU and WebXR are the two big enablers going forward. With WebGPU, developers have a common way to access hardware and that's a big deal across all your devices. A common way to access the hardware that gets you real-time 3D graphics, machine learning, crypto, etc. that works on your phone, tablet, laptop, headset, whatever is a big deal. And it's not just for anyone with Apple gear, but anyone with a compatible browser. Think generative AI/ML streaming Gaussian splats to your retinas via a browser. That's where we're headed.
I personally think MSFS 2020 and Automobilista 2 are killer VR apps. All (wannabe or pro) pilots could learn how to operate any given aircraft in MSFS and relive past glorious racing in A2 on current or historic tracks/cars. Senua and Alyx showed what is possible in gaming as well and why it's so much better than 2D. Elder Scrolls looks great in VR just the controllers make it a joke when fighting (too easy and weird). I still think 4k is too low and 8k will be needed to feel like a 1080p phone.
they are killer apps, but how many people are going to spend $3500, no, just $1500 or $1000 for just for these things? Well, that is almost the entire market of Quest 3 which Apple does not want to be in. Which is why I feel Quest 3 is a product that makes much more sense than Vision Pro.
I think you've got it backwards. There are plenty of reasons you'd want to put on a VR headset if it weighed as much as a pair of glasses, had a 180 degree field of view and like 5x the resolution of current headsets.
The reason the software doesn't exist is because compelling hardware doesn't exist for it to run on, so nobody bothers to write it.
Apple is imagining this device will be used for productivity but it's still painful to actually wear for long periods. We're a long way from being limited by software instead of hardware.
I think we are now at a stage where VR hardware has surpassed software.
How about enabling AIs to create layouts of information on behalf of the user? Like, what if an AI could arrange all of your information for you in a scheme derived from Archy?
> I think we are now at a stage where VR hardware has surpassed software.
It's a 600+ grams headset with a battery on a leash for $3500. I wouldn't say the hardware is mainstream ready or fulfilling it's side of the contract yet.
I think if it was an iphone accessory then it would have convinced more users because it would be $1000 since it’s just the display and sensors where iphone would drive all the compute
But at the same time, the hardware is not where it needs to be for ubiquity. We need ~60PPD, great FOV, and lighter hardware to really break the barrier. Vision Pro is at the limits, but still not quite there, and we all know the 90/10 rule.
I can't wait what the Vision Pro 4S-equivalent would look like and what capabilities it would have.
I already imagine how people from 5 years in the future will be sharing photos of the current Vision Pro asking "Remember when this was the best VR headset hhh??"
Sure, VR headsets existed before Apple's foray into that segment, but also did laptops, smartphones, tables computers, smart watches and bluetooth headsets.
And if one to learn from history, all these products categories were significantly improved after Apple entered their respective markets.
It's not VR though. The AR angle of this device is why it's compelling. VR is completely blind to your surroundings, Apple Pro could in principle make your public park look like Jurassic Park with full size dinosaurs etc. Blending our reality with an overlayed real time rendering is a massive benefit over straight VR.
That could just be an initial limitation of technology, though. The Apple Vision "spatial computing" line could move to glasses eventually.
That said, I think the "you're looking at a digital reconstruction of the world" aspect pretty interesting since it means, in principle, everything can be changed as you'd like. Why not turn day to night, night to day, redecorate, block out ads on billboards, re-paint your house, hang paintings, add an extra window to your wall, etc?
You're looking at reality with half of the colors your eyes can see, a reality with a limited field of view, a reality which looks much worse as the surroundings become darker. Nothing I've seen is impressive. I'm impressed by the lack of things more than what this thing actually offers.
Sure, Apple built the headset so that it mimics AR devices, but it definitely is not one itself.
The technology to build an AR device with the capabilities that Apple boasted is simply not here yet. I'm sure that Apple top-talent is hard at work trying to break the barrier (if it didn't yet), but that'll be for a future version, just not this one.
When you say "AR devices" I think you specifically mean AR devices that do optical compositing vs. AVP's digital compositing, yes?
> I'm sure that Apple top-talent is hard at work trying to break the barrier (if it didn't yet), but that'll be for a future version, just not this one.
To me, it feels like AVP is the final nail in the coffin of the optical compositing evolutionary branch of HMDs. Even if it's someday possible to do well, devices would still need the same realtime subsystem necessary for digital compositing in order to do 3D mapping, object detection, environmental lighting integration, etc.
I understand that in the short-term, camera sensors and displays don't exceed what the human eye can perceive, but those seem straightforward (if not easy) to address.
Same here. If there's some breakthrough they're not going to ignore it, but I would bet anything that current Apple far-forecasting plans for the "Vision 10" amount to current Vision Pro functionality in oversized wraparound shades, relying entirely on the camera passthrough for vision.
When I say AR I mean exactly what it's defined as. Put this in the previous comment, it's the Wikipedia definition:
"AR can be defined as a system that incorporates three basic features: a combination of real and virtual worlds, real-time interaction, and accurate 3D registration of virtual and real objects"
There is nowhere in there that says the output has to have pure optical through. Digital composition seems to fit into the definition.
I don't understand, every definition of AR I can find conforms to what Apple has made. This one from Wikipedia for instance:
"AR can be defined as a system that incorporates three basic features: a combination of real and virtual worlds, real-time interaction, and accurate 3D registration of virtual and real objects"
What types of data exactly are you implying that this device may capture that would be any more extreme than the data which would be captured by any other existing Apple device? I would think the Apple Watch alone has the ability to capture more data about habits and personal information than the Apple Vision Pro possibly could, considering its countless additional sensors on top of its existing ecosystem integration with the iPhone.
I don't care about the data it captures on the wearer. If you're inclined to hand over all that telemetry on yourself to other people, I don't know what to tell you.
I care very much about the high-resolution cameras it has pointed at me. I don't think that's okay.
Unfortunately no I didn't read the article you linked, I still don't see how this device collects any additional data over what is already available for other devices in the Apple ecosystem. Anybody who has used a Mac, iPhone, or iPad has had HD cameras pointed at them.
To clarify, my point with this comment and my first one was never that data collection (and as far as I can tell mass surveillance as well to you?) was okay, rather that I don't see how this device captures any more data on a wearer than the other devices they likely own.
I think GP's point was that it normalizes having cameras pointed at other people at all times while you're wearing it or similar devices.
I see a much bigger issue in this regard with the meta ray ban sunglasses, since those will actually be worn by people outside. We already have cameras pointed all around public spaces, but this device category will increase that coverage to near ubiquity long-term.
When you are in the presence of someone wearing this, you're being captures by 4 or more high-res cameras, in 3d, whether you want it or not.
People don't wear those all day. A scenario where people have cameras on their face all day yields a totally different kind of ability to capture serendipitious moments, etc. That's not to say this value proposition is high enough to warrant it, or that we should expect this technology to actually succeed mass adoption, but it is not the same value proposition as what you're talking about.
The thing I don't get about spacial computing is: why?
I feel like being able to see everything open at once would be incredibly distracting. I like how I can swipe between app screens on my desktop so that I'm only focusing on one app at a time.
Of course I imagine there are some applications where it's useful, but to me it just feels like the Apple Vision Pro is just a very large screen and they haven't quite figured out what to do with it.
It's the same reason (some) people love multiple monitors and don't find it distracting. In fact, I feel extremely limited on just 1 or even 2 monitors. I'm hoping that the 4k virtual monitor in AVP will suffice and with windowing tools I think it will since it can be so much larger.
> I like how I can swipe between app screens on my desktop so that I'm only focusing on one app at a time.
You can shove the app into a corner of your room, out of your view, and it will be there when you walk over to that corner later. That's part of the idea of "spatial computing" - if we can associate computer objects with real locations in space, maybe we can better harness spatial memory and stuff like that when we interact with them.
It is interesting, I do regularly use Immersed in which I have total freedom to place windows any size anywhere I want in 3d space. Do you know what I end up doing mostly? Snapping them into the fixed 3 monitor layout in front of me just like they are physical monitors. It turns out, the way we set up physical monitors is actually how we want the virtual ones too .... what a surprise!
It seems that a lot of basic information is missing in the review:
1. How does it compare to an high-end monitor for text editing/programming, web browsing, watching non-VR video, playing non-VR games? Is it better or not?
2. Is the resolution, latency, FoV and lack of color fringing good enough for it to be indistinguishable from reality in both passthrough and VR modes? If not, how exactly far is it?
3. Can you run VR games on a PC with multiple desktop GPUs and stream to it? How does it compare to current high-end and ultra-high-end VR headsets?
I recommend the video review as well. Seeing the video call between Nilay, Joanna, and MKB shows how much the tech has advanced but also how much it still needs to evolve to be at the level of FaceTime Video.
I’m excited about the Vision Pro but I now see why some reviewers called the FaceTime “Personas” creepy.
They’re like talking paintings in the Haunted Mansion ride with bunch of blur and depth of field. It’s way too weird. Joanna’s looks a bit like she’s been stuck as the replacement person in the Mona Lisa.
A static picture of you or maybe your Memoji (remember those?) would be far preferable.
yeah ... i regularly do Workrooms meetings with avatars and i have to say it looks and feels much less creepy - safely outside the uncanny valley but still conveying all our natural expressions. It's actually quite weird when I think about how much I forget we are avatars and just talk to the other person as if it is really them.
> "Apple sent zeiss lens inserts for reading glasses, but I just used my soft contacts and it was fine."
lol what kind of review says that. It would be like someone who doesn't even use any kind of vision correction saying "I just used it without glasses and it was fine."
(edit: the review is actually quite good. but that line was bizarre)
Apple said that contact lenses might interfere with eye tracking (or maybe with the iris scanning). So it’s a data point if he says he had no problems with soft contacts.
Only some kinds do (mostly cosmetics). If you go through the purchase process there's a questionnaire on if you actually need the inserts.
Also, I don't think people with only near/far vision issues or prism correction know how this interacts with VR. IIRC in general you always use your far prescription and there is no support for prism.
For #3 the question is how you're going to control the games, when the overwhelming majority of existing VR games are built for dedicated VR controllers rather than hand tracking.
Streaming games from a PC to a standalone headset over WiFi has been proven to work with the Quest, but that has proper controllers.
I feel like the simple hacky solution there is to use Joycons connected directly to the PC for the buttons/thumbsticks. Attach the control inputs to whatever virtual tracking points the vOS app is supplying and voila.
Joycons don't have absolute positioning, only relative, they're not good enough for VR. There is one way you could do it, if you have a Valve Lighthouse setup then you could use Index or Vive controllers, but that would require manually calibrating the two independent tracking systems to align with each other and it's not exactly a cheap solution if you don't already have the gear.
It's a very niche setup, but a few people already use it in order to combine Lighthouse-based body trackers with a non-Lighthouse headset.
> Joycons don't have absolute positioning, only relative, they're not good enough for VR.
The point would be that you don't use them for positioning at all, just buttons/thumbsticks. If you've using a Virtual Desktop-like app on the headset then it's already got positioning from the hand tracking.
It does read well as a non-technical review though, where a user would only make note of those things if they were lacking. If the passthrough looked artificial and weird you would imagine that would have come up.
I know it's v1, I know it's expensive, I know it's limited but I cannot wait to get my hands on it this Friday.
Even with just 1 4k floating screen I think it would be a winner for me but I'm also really excited to see what people come up with in VisionOS itself. I think for a while the sweet spot (for me) with be using a virtual monitor and a handful of VisionOS apps as well. Eventually I hope to be able to pull macOS windows out of a fixed box and arrange them wherever I want but I'm fully aware that might not be this year or even next year.
Yeah, I think long-term I'd want a "confluence mode" (a la Parallels) or a large curved screen. Dual/Tri-screens would be fine too but I feel like we can do better with an infinite canvas (though "monitors" might make macOS apps play nicer).
It's a far cry. This thing is a little more that 4k for the entire field of view.
Resolution required to render two 4k monitors at comfy distance is more than an order of magnitude away.
An important other aspect though is how much is visible. With foveated rendering does it matter if I have 50 4k monitors? As long as they aren't all in-focus and in-view at the same time?
I'm not sure where the limitation is right now (if it's purely a software limitation) but my MBP runs 4 monitors right now and I'd put up with a cord (or two) plugged from my MBP to my AVP if I could have all my monitors in VR. I believe they are using AirPlay to give you the virtual desktop but 3-4 AP streams doesn't seem too crazy, but I don't know much about that.
There's foveated rendering, but there isn't foveated physical resolution.
With this cutting edge display resolution you can expect a 1-2 monitors QHD of comfortable angular size and that's it.
You could have more out of view but that's your virtual displays.
Then again, why exactly? Getting used to some heavy steamy gadget on your head at home/at work so you can also do that on a train? Get real.
VR goggles always were a gimmick and will be a gimmick on coming decades. I'd bet we'll have non-visual brain datalink before we have non-gimmick VR.
what i am wondering about these devices is, my eyes get tired and strained from VR. I can only game on VR for limited amounts of time, I am a pretty fit person but i sweat a TON, de-fogging these devices has been a gigantic pain point.. does apple vision pro innovate in these areas ? I just felt the occulus devices as oppressive 30+ minutes of use.
> A: The Vision Pro wasn’t designed to be worn with glasses. Instead you have to order prescription Zeiss optical inserts for $99. The two monocle-looking pieces snap right into place.
It's $99 for readers (non-prescription) and $149 for a prescription. Very odd that she would have gotten that detail wrong since it's so easy to check and it's been repeated so often in coverage of the AVP.
Not to excuse that mistake but FWIW there are third party manufacturers which sell prescription lenses for various headsets for as little as $50, so there will probably be cheaper options once they start making them for the Vision Pro. The cost to actually manufacture the lenses depends on the strength of the prescription, and most VR lens manufacturers reflect that in their pricing, but Apple appears to be charging everyone the worst-case price and pocketing the difference when you order a weaker prescription.
Also, you might be able to take your previous pair of glasses, remove the lenses and grind until they fit. I did something similar to make my own prescription sunglasses (glued in the lenses).
It is interesting how many here are excited about this for productive computer work. It’s also what Apple advertises with.
But what is the account situation like?
For years I’ve been complaining that I can’t easily use my private iPad with my company Mac because they have separate Apple IDs. Things like sidecar for a quick virtual whiteboard are basically impossible.
AirPods have gotten better over the years where today I can freely switch between devices belonging to different Apple IDs with the same AirPods.
But is the Vision Pro like that as well?
It would seem weird to exclude the not-so-small group of people working from home but with company MacBooks
Its wireless operation seems to depend on Hand Off in some capacity. Most companies probably wouldn’t want to grant a personal device access to that on a work laptop, and I bet there are some thorny questions about what to do with incoming Hand Off data from multiple accounts.
The entire screen sharing setup they demo'd in the original Vision Pro demo reels always made me laugh. They've had years to get Sidecar right, and have failed miserably every time. How am I going to believe that they'll get wireless display transmission to work perfectly for this thing?
I haven't used the Vision Pro, so I can't say how well it works in practice... but with macOS 14 this year they redid their screen sharing app to, presumably, use whatever technology is underlying the Vision Pro display-sharing. It's really good. Vast improvement over the previous tech (presumably VNC?).
Assuming the Vision Pro screen sharing works using the same stuff, I have high hopes.
If your company Mac is locked down to the point where you couldn't just create a separate account on it that is tied to your iCloud account, then it is also unlikely that they would allow you to hook up another device to their network and your work computer in order to have this convenience.
(My partner is corpo; I'm startup, but have worked at corpos. No thanks.)
Better to keep it all owned by the company, in my opinion, and have them issue you an iPad for this express purpose.
There’s plenty of reasons why you would want to have a separate Apple ID for a company Mac that have nothing to do with overly restrictive permissions from IT.
The main one being a complete separation of calls, messages, calendar, notes and reminders. For my own sake more than for my employers sake.
And many employees with company phones already have that separation. iPhone and Mac is not that uncommon to provide for employees. But an iPad on top? I think that’s gonna be much harder to find
Sure you can create a separate account on your company Mac. But there's no assurance that whatever work resources needed would be available on that second separate account.
If your work is on the traditional model of perimeter protection and trusted intranet, a non-work device can't join the network as you have correctly pointed out. If your work is on the newer BeyondCorp style model, switching to a second account on your computer is going to invalidate the device trust needed to access work resources.
I thought I was the only one bothered by that! I'd love to use my private iPad with my work Macbook. And at least in my case preventing that definitely won't increase iPad sales: My company won't provide me a work iPad and even if it did it wouldn't work as there are no iCloud accounts attached to our work Macbooks.
Locking you customers into your ecosystem? Fine, whatever. But even within the ecosystem restricting usage in such a way!?
It's been said for years but the iPad could be so much more than a mere media consumption device if it weren't for short-term-profit driven design decisions.
Basically the business and education group is about selling to businesses and schools, so they give them the tools they say they need. This means you wind up having configuration options which sound good to operations, but which break ecosystem support - and on BYOD break personal usage.
Literally the only cloud drive product I know of which doesn't work on my corporate laptop is iCloud Drive, because the EMM gave a checkbox to set a flag. As a result, a huge portion of built-in collaborative features and apps just don't work. I have paid seats in other products only to regain functionality lost by that checkbox.
> For years I’ve been complaining that I can’t easily use my private iPad with my company Mac because they have separate Apple IDs.
I have a similar complaint with my Apple Watch and my corporate issued laptop. When I am using my own computer (mac mini) I love how easy it is to use my watch to login, use it to approve actions, etc. However when it comes to my company laptop I have to type my password in repeatedly. It would be awesome if the watch could be linked to both IDs to make this much more seamless.
No security-conscious corporation is going to allow you to approve any actions with security implications using an Apple Watch secured by a four-digit passcode, rather than an alphanumeric password on a Mac.
MDM is supported by the watch. My organization requires a 10 digit code for watches plus requires biometrics to be resolved on the primary device prior to the watch being able to control that device or interact with apps on that device
It's actually far worse. There's a single user and a "guest mode", but for AR/VR to work with, there's a calibration step, which means that the guest has to go through that step every single time they want to use the device. It might be fine for a real guest using it once, but it would be basically impossible to share the device with someone else. Having to setup the device every single time you use it sounds absolutely terrible.
Of course. They want to sell them locked to a user so that every employee or family member needs their own, and can't use the same one at work and at home.
I don't think the question was about multiple people sharing a VP device, but about the same person using a single VP device with multiple Apple devices on multiple Apple accounts - can you easily switch between viewing your personal Mac's screen and then switching to your work Mac?
> For years I’ve been complaining that I can’t easily use my private iPad with my company Mac because they have separate Apple IDs. Things like sidecar for a quick virtual whiteboard are basically impossible.
This is kinda what Managed Apple IDs are for - the work 'owns' the Apple ID it puts into its management profile and can set policy. Apps write into a separate storage container which the company could remote wipe, without affecting the rest of your personal data. If they want to disable things like sidecar, they can do it.. for the corporate apps/accounts/web domains.
I'd' generally assume the multi-user aspect is worse (because face shields and prescriptive inserts) so generalized multi-account is pretty low on the priority list.
> Things like sidecar for a quick virtual whiteboard are basically impossible.
Zoom has a good Airplay sharing feature that works well in this situation.
But I get what GP means -- I do have a corporate profile, and I made my own @corporation.com Apple ID, but what do I do to use sidecar? Either log out of my personal iCloud on the iPad (gross) or log in to my personal iCloud on my work computer (grosser)
I suspect multi account support is not ‘down the priority list’ but purposefully not implemented at least when it comes to iOS. Why make it easier for customers to share iPads at home when they can buy multiple iPads.
I use my iPad so sporadically that it could easily be the house iPad, but I’m signed in with my email and so on it can’t be.
However, pairing an audio device is an exchange of settings and encryption keys, and Apple will sync that pairing that to your entire account. Hold your AirPods near your Phone and tap the button to create the initial pairing, and they start working with your Mac and Apple TV.
that doesn't explain the previous status quo, which appears to be that previous (and maybe current) incarnations cant Bluetooth pair without an apple ID.
This just is not true, you can pair any model of airpods to any device that supports bluetooth. (Maybe there are exceptions but I'm not aware of them.)
I use my airpods with my Nintendo Switch and Samsung TV all the time. No Apple IDs on there.
I have many fights to pick with Apple. Pairing of accessories is not one of them. Honestly, if I stop using iPhone in the future, it’s likely I’ll stop using my AirPods as well. There’s in either case very little point paying premium for Apple accessories and pairing them over Bluetooth.
Bluetooth is one of the worst protocols to work with. That goes for both implementers and users. It’s one of the key selling points of Apple devices - they just work together.
Troubleshooting which device is hijacking the Bluetooth speaker is not how I want to spend my relax time. And don’t get me started on anything IoT. Solving the pairing issue is a massive selling point.
As for Apple ID specifically, I know that it’s used for tracking lost devices, and connecting with other devices that you may not have paired individually in a n^2 fashion. It could be, for instance, that Bluetooth pairing would conflict with apples own discovery/ownership protocol - say you only use BT (no account) and then an iPhone with temp access to your AirPods could “take over” your AirPods since they didn’t have a logical owner before.
The "freely switch" here is referring to the W-chip multi-device support that will on the fly switch between any number of Apple devices based on what's actively being used at the time, without needing to do any manual connection stuff.
Other non-proprietary Bluetooth devices will generally do 2 devices at most, and getting that to work right with microphone input settings can be kind of a nightmare.
The facial interfaces are just held on with magnets, so it's not unrealistic to think that people might swap them out regularly depending on who's using it. The interface is sized for the user so hygiene aside you'd probably want to swap it for a different one anyway.
Unfortunately Apple is charging $200 per extra facial interface though.
OP is talking about the reverse. Using one Vision attached to two different laptops with two different iCloud accounts, so they can use it with both their work and personal computer.
I maintain that the Vision Pro/its battery pack should have at least one displayport input. It wouldn't support keyboard+mouse sharing, true, but latency would be improved and you'd sidestep all the problems with accounts and locked-down work devices.
Man I've felt this for YEARS and I feel like I'm taking crazy pills with all the youtubers and influencers proclaiming Apple's Connected Ecosystem as such as productivity advantage.
Either you can't sign in with your personal Apple account, or you shouldn't (because MDM). So the only way to access anything associated with iCloud is what is available on the iCloud web portal; which is a horrible experience. You can't do sidecar. You can't do airdrop, copy-paste, continuity camera, nothing.
I've only ever used Macs in a professional environment. I've, also, always had a Mac and iPhone as personal devices. But I've never made the jump toward saying "Ok I'm actually using iCloud Seriously now" for this single reason. The best Google Cloud experience is available in a web browser, which I can be signed-in to on everything. Google Drive is everywhere. The list goes on.
Its such a crystalline example of why Apple's walled garden actually hurts themselves.
I really like Joanna Stern, and how she approaches reviews like this. I’ve watched her review, The Verge’s, and MKBHD’s unboxing video.
However, the best review I’ve found that actually transmits what is possible and what it is like to use is Brian Tong’s 55 minute review video: https://youtu.be/GkPw6ScHyb4
I’m not familiar with him, but unlike other reviews I’ve seen, he spends less time evaluating or summarizing, and more time trying to actually use the device. I didn’t even realize that you can seamlessly use your Mac to control your visionOS apps, for example.
Good review. Most interesting part was at 43:00 discussing the ergonomics and weight, which is the real question for everyone hoping to make this a daily driver.
He said he could wear it 45 mins before needing to take it off, that it was overstimulating so you need to slow down how quickly you use apps and move things on screen, and that gestures also were fatiguing. You could tell he was trying to be fair but positive.
Headsets just haven’t cracked this nut yet, and tho tech may advance somewhat, they may be limitations inherent to the form factor. Even if it gets really light weight, the issues of overstimulation, headaches, and the amount of neck movement implied may keep these products in a niche. (I say this as someone super excited about AVP)
For everyone used to using their computers all day long wanting to do it in a headset, don’t throw your macbooks away just yet.
I've regularly done 2+ hours of light activity (e.g. mini golf, social hangouts) with my Quest 3 without issues, though I will note this is with a third-party head strap specifically designed to be way more ergonomic and comfortable than anything first-party from Meta or Apple [1].
A lot of the physical downsides here are basically self-inflicted by companies trying really hard to hide the "nerd factor" necessary for comfort, to the detriment of the actual user experience.
Yeah industrial design and ergonomics tend not to have the same goals. Personally I was able to use a Quest 2 for ~1hr without too much issue, but it's not something I'd want to do on the regular.
The big product marketing question is what niche do headsets fit in, and thus whats the ideal single session and daily usage goal for a headset?
If it's about replacing laptops or another high usage scenario, that's a pretty high bar, definitely too high for the next 1-2yrs. I imagine some people at Apple wore dummy see-through goggle ergo tester units of varying weights around all day to get at these numbers :) Wonder what they came back with. Even still, that only gets at weight vs the perceptual ergonomics, skin-feel, etc.
The issue I see with headsets is that there may not be a lot of improvement possible without compromising durability or other factors necessary when going to market. E.g. what if they can't get it below ~400g (making AVP ~40% lighter), but to make the headset comfortable for most people for the usage scenario that makes them mass market (e.g. 2h+ sessions daily) requires ~250-300g?
its not really self inflicted when it literally determines how many sales you will have. Plenty of people are put off of VR because of the form factor. Quest looks like a nerdy toy, which does indeed influence who buys and/or uses it
Everyone I know even those who are into VR and even those who WORK in VR have zero interest in working with a headset of any kind on.
And those who work in VR report their coworkers and just about everyone they talk to customer wise feels the same way.
I know there are people who really want to, or think they do, but most would rather just use screens until maybe such a time the form factor becomes a pair of eye glasses.
Is the problem "working in VR" or is it "with a headset on", though?
I think it's completely reasonable to say that this stuff is only going to continue to get lighter and more comfortable every generation. What's that balance of interest going to look like when a headset's eventually got the same weight and form factor as, for example, big ski goggles?
> Is the problem "working in VR" or is it "with a headset on", though?
IMHO both. As for form factor lots of people already opt for contacts over glasses for comfort. I think ski googles are right out for anything that's not recreation or industry/application specific.
As for being in VR people really do like the real word lol. Going in and out of VR is a bit of an ordeal. Anecdotally I like taking small breaks from looking on my screen during the work day. Frequently. I look outside at the trees, at my cats, stuff on my desk, and etc when I'm thinking. If I have to "go in and out of VR" to do that or just not be in VR I'm going to just not be in VR.
For it to catch on mainstream for productivity and day-to-day it's gonna have to be like eye or sunglasses with seamless AR. Via something like retinal projection perhaps. I feel like Google was onto all this hence the Google Glass.
I think the "overstimulation" thing is a bit of a sleeper issue.
At first people think "wow it's so awesome I can be sitting on the moon while I browse the web". But after a bit of time you just get tired, and I think it's precisely because your whole brain is working in overdrive to understand the unnatural environment you are in. None of this manifests explicitly but at the end of it, when people are faced with the choice of putting the headset on or not, they just "feel" like it's a lot of effort.
I say all this as someone who does regularly spend 1-2 hours working in Immersed with multiple giant screens up. And I love it as a break, a way to focus or just relieve the boredom of working in the same space day in day out. But even I feel this effect of it being tiring and not keen to do it for 8 hours a day. And the minute you say that, you lost the use case of this being your "only" computer / replacing your laptop, so it's actually kind of crucial to its central justification as a replacement for a computer or a 'new kind' of computer.
This is headset, app and person dependent. I've done 8 hours in Elite: Dangerous (sat down, can't remember the headset) and well over 4 hours in Fallout 4 VR (stood up and moving around, Valve Index).
Having demoed VR at my old office I can tell you that the range of reactions varies from an immediate "nope" and having to take the headset off to being able to stay in it for a significant amount of time with no discomfort.
I started watching the video, and at 0:40 he asked Siri to "close all my apps". At that point my own iPhone's Siri enthusiastically explained to me how I can close all the apps on my phone.
For me the two takeways from the various reviews are that:
- The Vision Pro is the best VR set that can be done today, with massive investment (rumoured 5e9 USD) and competent staff, and hefty price. It is miles ahead of the competition.
- It’s still not enough for most if any practical use, apart from films maybe. The technical requirements for a really useful VR are still largely out-of-reach, and will be for at least the next 5-10 years.
I was confused as well, so I checked various European languages for what their word for billion is. I suspect they are french, using an abbreviation for milliard, the French word for billion.
This is one of the places where the US has a better naming standard than us in the EU imo. Just feels much more intuitive to count up with Bi, Tri without doubling them up every time.
To set a comparison point, a bike helmet is around 3 times that weight.
Less weight would be better, but there's ways to embrace that and aleviate the issues ("have the product be true to its nature" to channel my inner Jony Ive).
This time Apple chose not to, while Varjo tends to care a lot more about these ergonomic aspects. Looking at the XR-4 strap, it looks like a solid design to me.
I really don't think this is correct. It's a bit better at some things than any of its competition, and it's a little worse in some ways too. Other than the displays I don't think you can say it's substantially ahead at all.
It can't even play games. You can do everything the Vision Pro can do and more on a quest 3 for 1/7th the price. The screen is not as sharp but the FOV is better on the quest (which they mention in the verge article), it has no battery dongle and it has actual tracked controllers if you want to use them (but you can fall back to hand tracking where it makes sense).
"That's the same as guys going, my 1000$ Ubuntu laptop can do everything your Mac does guys"
That's great if that works for you, but most people would agree that Apple just has that finishing touch that 99% of other brands do not achieve. It's all in the details. Apple is luxury tech.
What does this even do better that has any "Apple" finishing touches? There's no games, there's some content but lots missing, what it is supposed to be able to do, it does worse than the competition and it's more than 6x the Quest's price
Apple was never about the games or content or even hard numbers. What is Apple like is: 1. incredible engineering :All very high quality fabrics, machining and optics.
2: The detail oriented software. Look how polished everything looks in Vision OS. It’s just nice to be ‘in’ there.
3: The integration of the software and hardware. Extremely nice handtracking, amazing see-through functionalities, working with a good ecosystem (keyboard, MacBook, trackpads,…)
Those experiences are what makes Apple unique. Not ‘better’ per se. That depends on what you need.
I totally agree, but I'm responding to the idea that it is way ahead of the competition and pointing out that not even the verge article was positive in all aspects (as was suggested in the few parent comments of mine).
I don't think it is ahead of the competition personally and especially when considering the 7x price I fail to see how it could land with consumers. Having said that I do anticipate Apple will finish the tech in the next few years and it will be a much better buy (both in value proposition and with a lower price) a few years from now.
I'd like to understand how this compares to something like XReal with better UX (I'm assuming the software for XReal isn't great, I've never tried it).
1,108 comments
[ 149 ms ] story [ 1700 ms ] threadI don't think that is true. I think 'most people' don't need a computer for more than what an iPad can do, and that your use case is more exceptional than common.
Computers as we think of them, with a somewhat permissive operating system that let's you execute whatever you want are probably going to decrease in relevance to somewhere akin to how they were in the 90s -- incredibly powerful devices useful to some people who need them for particular reasons or just like using them. Everyone else will be fine with whatever the equivalent of the smartphone/tablet OS is.
I've never made my iPad work for me on a business trip except for something very short where I won't really be writing. I also don't draw and, while I could probably do photo editing on it well enough, I wait until I get home and use old-school Lightroom.
It’s true that they’re limited but that doesn’t mean that there isn’t work that’s well-suited for them.
Meta isn't really opening up their headset, since the whole point of that adventure for them was to have their 'iphone' type kingdom.
Our best bet might just be Microsoft but they gave up on mixed reality windows, and Idk if they're really gearing up to jump into this product space.
I'm fascinated in the idea of Vision pro, not sure I'm ready to shell out for what is basically just a really great movie watching experience. I can see the movie pretty well on my 4k tv, and I'm not sure the improvement is worth the cash.
https://aframe.io/
My understanding is that Apple is making excuses about being slow to support WebXR.
As much as the media has benn fixated about the failed Horizon Worlds (which is a very interesting story that I've never seen elaborated from a software dev's perspective) you might not know that the successful model for Quest is the same as publishing on other game consoles. The market for single-player games for MQ is really healthy but multiplayer games are really thin on the ground which makes the "metaverse" idea look pretty dodgy. I mean, there is the multiplayer VRChat but you're going to have to rub shoulders with weaboos and furries which keeps other obnoxious people away.
https://www.roadtovr.com/apple-vision-pro-webxr-support-safa...
While this is true for the desktop and iOS/iPadOS Safari, their VisionOS version of Safari comes with full WebXR support. Which was confirmed explicitly in this Apple’s tech talk[0] and by the fact that it is indeed supported in the VisionOS simulator (which has been available for quite a while already).
0. https://developer.apple.com/videos/play/wwdc2023/10279/ (ctrl+f “webxr”on the transcript, as I don’t know the exact timestamps, but I confirmed by doing ctrl+f myself)
This is something I've been wondering about. The Vision Pro's displays are 4K per eye, for the entire field of view. The monitor I'm sitting in front of is 5K, and takes up quite a bit less than my full field of view.
Surely the virtual Mac screen (and everything, I guess) is gonna be substantially lower resolution than traditional high-DPI screens at normal viewing distances?
Depends on your typical FOV to your monitor.
If the headset manages to be 4000 pixels wide, but that is filling say 110 degree FOV, then if your virtual display is 44 degree FOV, it's only going to be represented by about 1600 pixels wide. 4000/(110/44) = 1600.
I wish we knew what the actual resolution of this headset is and what the FOV is.
They say it's 23 MP total, so if the screens were square that would be 3391x3391.
If the FOV is 100 and our virtual screen was 50, then the screen would be 1695 pixels across.
And finally you lose some effective resolution because of the nature of the virtual screen being not aligned perfectly with the headset's panel. Since your head would be slightly askew in relation to the virtual monitor due to head tracking.
https://kguttag.com/2023/08/05/apple-vision-pro-part-5a-why-...
If you hold your head steady while looking through a window with a plastic screen on it, things outside are obstructed. If you move your head slightly back and forth and focus in the distance, the screen pretty much disappears.
Your brain can do some motion smoothing to determine the "actual" content, even if it's sampled. I'm not sure how you could quantify it, and it only helps a relatively small amount, but it's there.
Another comment mentions that you’re confined to the host computer’s “screen” and can’t break applications away from that rectangle. But you could imagine that being a possibility in the not-too-distant future.
I think it's worried about not being able to apply a 30% tax on third party software.
That's exactly what corporations like Apple want their devices to be.
I think it's meant to be used with a Mac for most productivity use cases. That's how I intend to use mine: VSCode, terminals, compiling all happening on my nearby laptop with the Vision Pro as "just" a 4K monitor, and then extra apps like Slack, Zoom, Safari, Mail, music, etc, floating around as native visionOS windows.
In addition, it can be used as an iPad-like media consumption device, e.g., on an airplane, but I see that as an additional (and for me only supplementary) use case.
That sounds like a way that may actually work (and don't cost too many penny). Just put dozens of programmable LEDS at the edge of your view. And shining correspond colored light when someone actually use a flashlight to point at you. That might even be used as a safety measure without block your view.
"The displays have other limitations: the field of view isn’t huge, and the essential nature of looking at tiny displays through lenses makes that field of view feel even smaller. Apple won’t tell me the exact number, but the Vision Pro’s field of view is certainly smaller than the Quest 3’s 110 horizontal degrees. That means there are fairly large black borders around what you’re seeing, a bit like you’re looking through binoculars."
https://www.theverge.com/24054862/apple-vision-pro-review-vr...
The widest FOV headset you can currently buy is the XTAL 3 at 180 degrees, and it's huge, despite being a PC-tethered design that doesn't need to make space for a SoC or battery.
https://www.xtal.pro/product/xtal-3-mr
We're a few breakthroughs away from having full immersion and a reasonable form factor at the same time.
[0]https://duckduckgo.com/?t=h_&q=nikkor+6mm+f2.8+fisheye&iax=i...
From their own page, the very minimum requirements for the PC are 32GB RAM and an RTX2060 GPU. And that’s a minimum, so I am not sure how well it functions with those specs.
Just the lightest GPU listed in the minimum requirements (RTX2060) is around 950g[0], while Vision Pro battery pack is ~350g. Add to that a motherboard (being able to support that GPU and a i7/i9 CPU and 32GB of RAM), a CPU fan, and a 500W (probably could get away with less, but i decided to be safe) power supply being around 3.9lb(~1.7kg)[1].
Even without accounting for the motherboard, desktop case, and CPU fan weight at all (which can vary significantly and are pretty heavy compared to the rest of the components) and counting them as a zero, you have already crossed into the 3.6kg territory. With those accounted for, you are crossing into the 5kg territory easily. And if you decide to go beyond minimum specs for GPU, I would expect it to be even worse. And the size of that monstrosity would be rather significant too.
I think comparing that to the weight and size of the Vision Pro battery pack is not that good of a comparison at all.
P.S. I am aware that there is a noticeable variation in weight of GPUs depending on the manufacturer and the specific variant for any given model, as well as for motherboard/cpu fan/power supply. However, that would not change the overall conclusion at all, as there is exactly zero way you would be able to get the build below 3kg. And I am being extremely generous here, as my estimate is very likely below even the lowest real-life lower-bound estimate.
0. https://www.msi.com/Graphics-Card/GeForce-RTX-2060-GAMING-Z-...
1. https://www.evga.com/products/product.aspx?pn=100-W1-0500-KR
This is one of those things that Apple never claimed was supported, and yet there's something about that behavior that feels like such a natural intuitive implication to the technology that a lot of people feel alarmed or even cheated when they realize it's not possible (yet). It's been funny to watch the various discussion threads as people pop up talking about their shocked realization and disappointed feelings.
Update: I did realize when watching the WSJ video that the "mirrored" display actually appeared to have greater "resolution" (more pixels in height and width) than what she had on her laptop. So that's something.
If Apple approves it, of course. This is one of my major concerns; there's a lot of potentially useful functionality that could be implemented, but you have to jump through the app store hoops and hope that Apple doesn't decide that it conflicts with their idea of what you should be allowed to do.
So even though you have a sequestered Mac output alongside Vision apps, you can use the same controls for all them simultaneously. This should help in the interim.
With Stage Manager on macOS now, it feels like they have all the primitives in place to "transpose" macOS stage manager windows textures to Vision OS/ the iPad OS foundation.
Though this will be tricky to get right for all apps. Will be interesting to see if it's a macOS App store only feature/ API, opt-in, or some other option
EDIT: -5* doesn't make sense, this is the most polite way you can point out that getting macOS apps windowed on visionOS has ~0 to do with double-buffered windows on iPad OS. n.b. I didn't use half-baked, OP did.
I use iPad Pro as a kind of sidecar daily driver, in the magnetic dock magic keyboard w/ trackpad.
As I type this, the screen shows a traditional MacOS style dock across bottom, four Stage Manager window clusters I can tap with a thumb on the left, and Safari plus Messages taking 2/3 and 1/2 of screen respectively.
There's more app and pixel real estate than most Windows laptops, and bringing screen sets to the foreground or swapping them back to the side is so natural I almost feel like giving up that space on my Mac as well.
The big thing I saw happen from apps over the past two version of iOS is app devs realizing their windows will not always be full screen or 1/2 screen size, but arbitrary size.
By now, most iPad apps of any serious nature are effectively window size independent, making them play well with others in stage manager. It's easy to see how that would make them play well with the headset one day.
They just haven’t done it.
- [Profit on basic innovation] Did they want to wait and see how their customers would adopt VisionOS's native free-floating windows, so as to avoid cognitive overload by commingling with MacOS windows?
- [Benevolence to fellow competitors] Did they not want to takeover the existing market of virtualized VR desktops?
The Vision Pro is effectively using AirPlay to mirror the whole screen. If you used AirPlay to mirror each window as a whole screen, you'd run out of bandwidth pretty quickly.
The windowing system in MacOS, Quartz Compositor, also isn't built to stream window information. Right now it has a big built in assumption that any windows its displaying are on a screen it also controls. It was probably too big a lift across teams to also re-write the graphics stack for MacOS for the launch of the Vision Pro. Hopefully they get it working in the future, but neither of these problems are easy to solve.
In fact, there's already a project for it on GitHub. https://github.com/saagarjha/Ensemble
480 lines total, including comments, headers, whole shebang.
It's all of the edge cases and UX refinements that takes time.
Try it with a full suite of Mac Apps and you’ll find it falls apart because they aren’t all well behaved.
You are making this sound much more complicated than it really is.
The trivial case is simple. You are imagining it's all the trivial case.
What fools Apple engineering management must be, then!
Compositor work isn't tremendously hard when the graphics primitives are already done.
UI state handling and input is a majority of the work from there. I've implemented this work before. A large portion of my background is in compositors and UI primitives.
What's really cool about having window backing layer handles, is you can do all sorts of crazy fun stuff in the office and show off to your coworkers or tech demo that basically will never make it to production because it's totally impractical.
The best example of that which actually did end up in a final product in my opinion was Windows Flip 3D. Totally fun implementation, I'm sure.
Remember the original comment was the claim that Xorg could easily do remote windows years ago. This is only true using the low performance path that modern operating systems have universally rejected.
The first iphone didn’t have copy/paste.
Apple will always prioritize critical scenarios over nice to have. None of these things are technically difficult, it’s just time. I’m willing to believe they released too early, but at some point you have to start learning from real users.
Literally nobody has done it. It's beyond ridiculous that you can't already show or duplicate an application window on any display you want and allow it to be controlled from anywhere it is visible.
Searching for ways to do this lead one into extremely niche software ecosystems. Please is there any collaboration app out there that makes it seamless to toss windows around like everyone actually wants?
Even classic Mac OS, which wasn’t designed to be “rootless” could be run that way could have its windows mixed with OS X windows in the classic days.
As others have pointed out, there is already a PoC app that is doing it, so it seems if Apple wants to, it’s completely in their power. However, does this match their vision? (pun intended) Time will tell.
This would mean the goggles would be basically just a dumb display for the Mac. It would also be weird to try to move an AR app onto your Mac.
But this is sufficient for many use cases (or at least, mine). I pre-ordered one with the idea that my main work will be on the 4K monitor, with most of my superfluous apps floating around as native visionOS apps. That's mail, a web browser, and zoom, which all have apps now, and Slack, which I could just use Safari for but may have a native app in the future.
It's more like 1080p monitor. The virtual monitor only covers a small part of the VisionPro's display. You can compensate a bit for a lack of resolution by making the virtual screen bigger or by leaning in, but none of that gives you a 4k display.
To really take proper advantage of the VR environment you really need the ability to pull out apps into their own windows, as than you can move lesser used apps into your peripheral vision and leave only the important stuff right in front of you. You also miss out on the verticality that VR offers when you are stuck with a virtual 16:9 screen.
Which is the resolution that the majority of PC users are likely using.
“There is a lot of very complicated display scaling going on behind the scenes here, but the easiest way to think about it is that you’re basically getting a 27-inch Retina display, like you’d find on an iMac or Studio Display. Your Mac thinks it’s connected to a 5K display with a resolution of 5120 x 2880, and it runs macOS at a 2:1 logical resolution of 2560 x 1440, just like a 5K display. (You can pick other resolutions, but the device warns you that they’ll be lower quality.) That virtual display is then streamed as a 4K 3560 x 2880 video to the Vision Pro, where you can just make it as big as you want. The upshot of all of this is that 4K content runs at a native 4K resolution — it has all the pixels to do it, just like an iMac — but you have a grand total of 2560 x 1440 to place windows in, regardless of how big you make the Mac display in space, and you’re not seeing a pixel-perfect 5K image.”
We're pretty close to "Write once, (rewrite a bit,) Run everywhere"
even the big UI update recently as completely overwritten how the old app used to look in my memory
the only feature i have tried and not really cared for is the "Canvas" feature
edit: Yes I know you can build apps before they're in the store
Yes I know you can technically do this without a paid dev account, but it's practically useless because it has to be re-done every 7 days.
I think mostly people don't (for high end IDEs) because they expect a keyboard and pointing device that isn't a touchscreen and those are faily rare per ipads sold.
The ipad pro certainly is seeing some more beefy productivity apps it seems.
Windows, BeOS and Commodore 64 apps also don't run natively on the iPad or Vision Pro.
This allows you to restrict full bitrate to a single window with a maximum resolution and keep the user experience high and astonishment low.
It feels like the Vision Pro would definitely be a great replacement for people who (want to) buy multiple expensive monitors, but it doesn't fully reach that potential today, and mostly because of software? Although rendering 3 or 4 virtual workspaces through ad-hoc Wifi at 4K 60fps+ low-latency would certainly be a huge challenge.
OSX had the opportunity to follow that path before settling on the “render windows, capture the screen, compress the image, send it over the network to be decompressed” VNC-style remote access that’s bog-standard today, and if they had Vision Pro would be set up to be an absolute mind-blowing macOS experience.
I guess one downfall is that that your pipe has to be lossless, and there's no way to recover from a broken pipe (unless you keep a shadow copy of the window state, and have a protocol for resynchronizing from that, and a way to ensure you don't get out of sync.)
This has been done many times before (see e.g. X Windows) and has known downsides. Off the top of my head:
- You need the same fonts installed on both sides for native font rendering to work
- Applications that don't use native drawing functions will tend to be very chatty, making the total amount of data larger than VNC/rdesktop/&c. style "send compressed pictures"
- Detaching and re-attaching to an application is hard to get right, so it's either disallowed or buggy.
It falls apart as UIs got richer, browsers in particular: they're entirely composited in-app and not via GDI, because GDI isn't an expressive enough interface. So you end up shipping a lot of bitmaps, and to optimize you need to compress them. You might as well compress the whole screen then.
https://www.anandtech.com/show/3972/nvidia-gtc-2010-wrapup/3
1: e.g. https://learn.microsoft.com/en-us/openspecs/windows_protocol...
Applications which bypass the native APIs to render their window contents, in particular video players or games, get a compressed streamed video which has very decent performance. The video quality seems to be dynamic as well, so if there's a scene with very few changes you can see the quality progressively improve.
All of this is done per window, so a small VLC window playing a video in a corner gets the video treatment, while everything else still works like native UI.
How did you know it sends windows hooks? Was it some sort of binary serialization?
1. The receiving side has to have at least as much rendering power as the original side, since it will be the one actually having to render things on screen. This is always going to be the opposite case with any kind of glasses, where you'll always want to put as little compute as possible for weight and warmth reasons.
2. Each application actually has to send draw instructions instead of displaying photos or directly taking on control of the graphics hardware themselves. No or very few modern applications work like this for any significant part of their UI.
This thing could have been useful on day one for lots of people, from drone operators to cinematographers to programmers. But Apple's sad fear of I/O has crippled yet another product.
People should expect that a $3500 video-display device has a way to get video into it.
I bet Luckey had something like this working in the early days but abandoned it because it wouldn't work for games. What a shame
Movies on airplanes. I’m guessing these will become virtually ubiquitous in the front cabin within a few years.
I couldn't see carrying one of these instead of just an iPad for movies. (But then I'm a very light packer.)
And I was casually watching one in my seat, the movie was “Blackhawk Down”. And it occurred to me how a neighbor might not enjoy watching blood, guns, and violence out of the corner of their eye.
So, I can absolutely see value in a headset style movie experience in public places.
They always have a screen in front of me. And I always have my laptop or iPad. There are still advantages to this, from immersiveness to privacy.
You aren’t unpredictably accelerating most of the time in a plane. It makes sense Facebook would have fucked this up. I’d be surprised if Apple has.
- Learning instruments in a guitar-hero way (Piano, guitar, drums)
- Cooking with timers and recipes right in front of you (will be even more doable with better internal displays in the future)
- Coding with virtual displays on-demand. This is another thing where we still need more resolution to make it really doable.
- Watching movies. Obviously a solo way of doing this but I could see it being big.
- (once these are much lighter and less intrusive) I could see these being huge for virtual workouts like Yoga, weight lifting, etc.
Also regarding your question, I'm trying to think of the "killer app" is for a currently successful device - iPhone. I mean, camera? Texting? Most people use tiktok a ton but I wouldn't consider that a killer app. I think it's more of the device providing a home for a bunch of different apps.
https://www.apple.com/shop/product/MM6F3AM/A/polishing-cloth
What would VR add here? For guitar-hero style instrument learning, there are already Yousician, Simply Piano / Guitar, Gibson and Fender apps, and quite a few others.
This is by the way a great explanation of all the attempts to make music easier to read and how they fail: https://www.youtube.com/watch?v=Eq3bUFgEcb4
(coming from someone who has been taking piano lessons for the past 8 years and have a Quest 3)
I'm imagining an AI VR tutor - heck, with some legal deepfake+AI style evolution, perhaps that tutor could actually be a famous player, talking to you.
The 3D view gives, well, a 3D view, and all the advantages it entails. For example, when you are with a physical real world teacher, you don't have a fixed view. You can observe from any angle, the teacher can observe and correct micro errors. And the senses of scale and proportion are intact, unlike viewing on a 2D screen of arbitrary size at arbitrary distance.
Even the simple POV of VR means that you get to see the techniques in-situ - e.g. seeing the expert's hands on your guitar, and how the technique is supposed to look from POV - rather than the standard teacher-student limitation of e.g. guitar, where the student sits opposite the teacher and sees a reversed image.
But to be clear, my vision of how this would be game changing relies upon a level of interplay between hardware and software that is not yet developed. But I expect it will be, in time, because virtualising real world experiences is arguably the core goal of VR, and (correct me if I'm wrong) but nothing in the music tuition world has yet proven superior to having an extremely skilled one-on-one mentor who can personally guide your every step, "in person" - and this is what VR would seek to achieve here.
My guess is not, because to me it is obvious you don't understand how this works and are just imagining things.
Speaking as someone who takes private piano lessons and have been practicing daily for the past 8 years:
If this were possible, music teachers would have already been replaced to some extent. But that has not happened yet. AI is not new. VR is not that new. Even without this, just talking about non-personalized, non-VR instructions. Have you seen those websites that "teaches" you piano just by connecting your keyboard to a computer and following instructions? Where are they now? It does not work like that. Maybe for an absolute beginner who has trouble finding a key or fret, but as soon as you get a little bit better it won't work.
Music learning is an extremely personalized experience. People pay 1-1 private lessons because it needs to be done that way. A teacher, especially a good teacher, can tell you exactly what you did wrong and what you should do to get better at it. Plus different teachers often have different opinions of music -- how a piece of music should be interpreted or played. As you get more advanced, you spend even less time on technique itself. Theoretically you could train an AI tutor that does the same thing. But no, it hasn't happen yet and will not happen. Did I mention hallucination? Do you want a teacher that hallucinates?
Plus you need lots of practice -- routines, repetitions, all those scales. AI or VR is just useless. Effective practice strategy is important, but you don't need AI for that.
That's great! I'm glad to hear that your 8 years of music practice has given you some firm opinions, and I'm sure you'll continue learning over time. Speaking as a player and composer of 25 years, former music teacher, and maker of the instruments I play - for whatever such things are worth - I believe my ideas have some merit. But only time and technology will show whether-or-not such AR (and AI) musician overlaps eventuate.
¯\_(ツ)_/¯
Or real workouts. I want to be able to have floating text to read on my runs, and real time biometric data directly in my field of view rather than on my watch would be cool too.
Definitely not going to run with the current product lmao.
example: https://www.youtube.com/watch?v=pIsjVaqdNpc
https://en.m.wikipedia.org/wiki/Integrated_Visual_Augmentati...
Apparently they are still making them for military purposes
> According to an exclusive report from The New York Post, NBA Commissioner Adam Silver “said the league is working with Apple to bring a tech-enhanced viewing experience” to its upcoming headset.
When asked about it, he told the outlet: “We’re working very closely with Apple.”
https://www.tomsguide.com/news/nba-games-could-be-apple-visi...
If there is one group that has a track record for laying down piles of money for hardware like giant televisions and expensive streaming services, it's sports aficionados.
I do agree, just watching a sporting event on a projector where the athletes are life-sized is excellent, on-field cameras already provide a better view of the game then any seat.
Now make it more immersive, and the trick of immersion is very cool. Like The Sphere, that immersion is next level.
Add sports-betting to the Vision experience, that is a great side-car app for this. (I am not pro-bet but see the usage).
I'm wondering how that translates to VR.. do they just teleport you around the arena? That seems like it would be a bit jarring if not altogether sickening.
And for other sports even better I think. With say soccer/football and football (USA) you can have a big overview (like you are sitting at midfield) but then camera angles for closer view of the action...
I think sports viewing could really be incredible. Figuring out exactly how to do it well will take awhile. But it seems to me the kind of thing Apple could do very well.
Can’t imagine doing work is the killer app, not while wearing a headset is more cumbersome than opening a laptop.
Apple is already making it's own 3D content.
For instance watching a concert shot in 3D with spatial audio may be something that people find compelling. Certain artists, for example Taylor Swift, have fans who are well known to be willing to spend a fortune to attend her shows in person with high end tickets running $750.
Almost NOBODY uses the Netflix app because it’s terrible, terrible software.
For a lot of people, Beat Saber and similar games are a killer app for the Quest. It can be good for making exercise fun and accessible at the same time.
The other one that could be huge IMO is attending real-life events like sport, concerts, shows, etc.
I disagree. Games will have limitations. Limitations constrain but foster creativity. Console games were already limited in so many ways yet people found amazing ways to make them fun as hell. When mobile games come I truly believed that nobody would be able to have fun there, yesterday I played the entire game of League of Legends retrofited for mobile and it works! So now VR gaming is great, people will find ways to make it even greater without free movement that makes you sick
I am hoping we will see a lot of experimentation in the coming years, and I am excited for what the Apple ecosystem will bring to the table. That said, from what I have seen so far this does not seem to be a revolution compared to the current offerings, but an evolution on various fronts, without addressing the killer app question.
In 1988-94, the CPUs available in desktop computers were substantially more advanced than the widely used operating systems. Windows 3 and Mac System 6/7 didn't support pre-emptive multitasking, memory protection, or many other features that define a modern OS.
Maybe we'll look back at today's Quest and Vision Pro as similar transitional devices with one foot stuck in the old paradigm, running old-style software.
That was partly because RAM was over $100/MB (Nominal; ~$230 inflation adjusted) in 1990. Additionally, in the IBM compatible world, many people didn't have a 386 at that point.
Also, minor nitpick on the dates; 1993 saw OS/2 2.1 and NT 3.1, both of which had preemptive multitasking and memory protection.
But other notable ones include Synth Riders, FitXR, OhShape, Pistol Whip, Thrill of the Fight, and (maybe) Gorilla Tag. And this list is far from exhaustive.
VR is pretty good for fitness just because it can make exercising more interesting, comparable to sports without the need to coordinate with other people (and it's easy to do inside your house, if you have at least a 2m x 2m open space). Major downsides would be having that space available and sweat inside the headset.
https://www.meta.com/us/quest/accessories/quest-3-silicone-f...
https://www.meta.com/en-us/help/quest/articles/in-vr-experie...
It turned out the killer Apple Watch feature was fitness, and I don’t see why it couldn’t have been here.
It was fun for a few minutes but not really usable for serious exercise:
- It's heavy and annoying (and this apple product seems even bulkier and heavier). The cable situation is also not great, you need a lightweight cable and ceiling suspension to keep it out of the way, but this is solvable.
- Exercise means you get sweaty. Can't wipe your brow and you have a wet headset on your face.
- You can't see your body and maintain proper form. The VR environment itself is also distracting if you turn your head around to look at stuff moving there.
I much prefer just to have my phone or tablet in fixed place in front of me to watch youtube or some movie.
I really don't think that's the best use-case for VR - if you're doing some kind of virtual rowing, a flat screen is going to offer most of the experience with none of the downsides. VR fitness is really about games that are fun in their own right and happen to be physically active. There's a big cohort of people who hate exercise and would never set foot in a gym, but who will happily spend an hour at their aerobic threshold because they're playing a fun game.
https://www.youtube.com/watch?v=291LZGxZS5Q
The reason running outside is nicer than on the treadmill is not that you get to look at nicer images. You get to smell fresh air and feel the wind on your body, you get to sometimes interact with people, you get to feel various textures under your feet, you get to move your bodies in more ways as you turn left and right etc.
Rowing outside is even more different, as you get the feel of the water currents and water splashes as well.
VR can mimic none of these things, and it comes with the massive downside of a sweaty face that you can't wipe away.
Totally agree. I'm waiting for a usable Virtual Desktop app to come out. All the ones I have tried which work on my cheap WMR headset fall short of having floating app windows in view.
I guess there is one of those which works on Meta Quest, but not PC headsets. That's really what you need to be effective working in VR. Just like is mentioned the Apple headset supports.
I’ve also heard about players spending a lot of time in counter strike games like pavlov.
At this point it seems like there’s a TON of things to do in VR (and I’m gonna be honest, there were a ton of experiences too on the Quest 1 when I had it).
I’m just waiting for more live shows and concerts that I can attend from the Quest personally.
It is like VR is currently stuck being Kinect in terms of sales and stickiness, while Meta and Apple would both like it to be at least like the Wii, or ideally the iPad.
Personally I have found social experiences to have the best long-term appeal (i.e. Racket NX or Drop Dead with friends), but even there I am not these apps have sufficient mainstream appeal.
This matches everyone I’ve heard talk about it, too. It was fun, they enjoyed a few things and then at some point they realized that their headset had multiple months of dust on it.
I think the big question is when costs come down to the point where it’d be reasonable for a non-rich family to have enough headsets to use together. Most people mention technical limits (resolution, latency) but the thing which everyone mentioned as a dealbreaker was that putting the headset on was shutting out everyone around them.
I bet these people also have consoles that are collecting dust as well
The discontinued WMR Portal, essentially the Window's desktop in VR, was so far the only software that tried to be a full workspace in VR. But even that was missing a lot of important features and Microsoft gave up on it years ago and never made it accessible to non-Microsoft headsets. It's currently scheduled for removal from Windows.
VisionPro seems very similar to WMRPortal so far, with a few key improvements like allowing apps do add 3D objects into a shared space.
For example, "killer apps/content" never arrived for 3D TVs and they have largely disappeared from the market. Same with various "waggling" input technologies like the Wiimote and Kinect. There were some compelling uses, like Wii Sports, but these were pretty limited and many other uses of these in games was a case of Nintendo shoehorning the technology into the game.
I think the best pessimist argument is the one offered by Folding Ideas in his metaverse video[1]: Text is really, really useful, and a virtual 3D space is not a good environment for either creating or consuming textual content.
[1] https://www.youtube.com/watch?v=EiZhdpLXZ8Q
I don't miss multiple monitors so much, but I do often wish for a larger screen. Not enough to put one in my space, though. That's where my interest in the Vision Pro lies - simply a way to project large, high-fidelity, 2d screens.
For less than this headset, I can buy an Ergotron arm and some monitors and have multiple large monitors that have zero footprint on my desk.
No, I can’t take my setup with me when I go somewhere, but I don’t want to.
I have a hard time seeing this become a mass market device. It’ll have its adherents and enthusiasts, but personally find it difficult to imagine even wanting one, let alone using it.
But what do I know, I thought the iPad was goofy. Though I never did end up replacing my gen 2 iPad. Realized I didn’t need it.
I'm on the same boat as you, but ended up just getting a long table so I can move to the empty side when I need it.
I certainly could make adjustments to accommodate a monitor - I just don't want to, and would love for there to be a way to not have to and still get to use a large display.
I don't have a TV at my small apartment but do have two 27 in monitors. Sometimes I wish I had enough space for a sofa and 55+ in TV for movies and shows. And maybe headsets will solve the problem, maybe I don't even need those monitors any more.
Now? I realize I don't need multiple monitors, 2 is enough. (4k is definitely better than 1080p and 2 is better than 1, but I don't think I can properly utilize a third monitor.) And these are good monitors -- not Pro Display XDR good but still premium 4k monitors -- that have been around for years and I'll keep using them until they die. And about movies -- I have tried using the Quest 2/3 to watch movies. It is usable, and apps even provide "scenes" that make you feel you are in a theater. Do I want to wear them and watch movies on a fake 100" screen? No, for obvious reasons (comfort).
If the day comes when they are light, comfortable to wear, allow very good passthrough and actually do a better job than my 4k screens, I'll take another look.
Hasn't VR taken over waggle? I don't think you can say its disappeared when the VR install base is in the 10s of millions.
And yet, from a gaming perspective, we still have to crack force feedback and natural locomotion before we have a holodeck. Maybe in 20 more years.
People aren't really that happy wearing even vision eyeglasses, and I don't see any realistic chance we will ever be able to have something with the capabilities of the Vision Pro in a form factor as small and light as eyeglasses.
I see on scifi all the time where someone flicks/flings a video playing on a device to move it to a larger display surface and it kills me that we actually have the technology to do stuff like this right now...but because every company works in their own interests/don't work together to create standards we don't get to have fun use of tech like that.
I'm still full of myself for postponing getting a 3D TV enough times that the technology died.
> Same with various "waggling" input technologies like the Wiimote and Kinect.
... but I have a PS Move gathering dust somewhere. Which I even preordered.
So the hardware is not good enough yet. It will be good enough when I basically don‘t care, just like I don‘t care with glasses.
I agree that progress has been slow in the consumer space and meaningful long-term adoption of VR has been confined to a few niches; that isn't necessarily an indictment of the long-term prospects for VR, because desktop computers spent much longer in that stage than most people remember.
In enterprise, I think things are more advanced and some user groups have decisively gone through the one-way door for some applications. I think the best example is architecture. If you've done a couple of client presentations in VR, you just aren't going back to showing renders on a flat screen, because immersing the client in a physical space is that powerful. It's not just a sales tool, but a communications tool - clients can understand and respond to the environment intuitively and give much better feedback as a result.
Industrial and clinical training is less clearly one-way, but I think we're very close in a lot of areas. AR is still less developed than VR, but I do think we're on the cusp of something significant - a sufficiently comfortable standalone AR headset with sufficiently high-quality passthrough can deliver training experiences that can't practically be replicated through other means.
I think one of the most interesting areas of development is in psychiatry. It's still early days, but we're starting to see real, meaningful benefits in RCTs for VR-based therapy of disorders like phobia and PTSD. Some of the most compelling results have been in the very sickest patients - people with psychosis, who often find it impossible to engage with conventional psychotherapy.
https://www.psy.ox.ac.uk/research/oxford-cognitive-approache...
I don't think it's remotely likely that VR will ever replace flat screens, but I do think that VR is slowly growing into a niche but durable HCI platform. Tablets are a reasonable analogy - a lot of people see them as a failure, but they still sell in serious volume and they're often a much better form-factor for specific applications than either a phone or a laptop, especially in industry. Tablets didn't change the world, but nor are they likely to go away.
WebGPU and WebXR are the two big enablers going forward. With WebGPU, developers have a common way to access hardware and that's a big deal across all your devices. A common way to access the hardware that gets you real-time 3D graphics, machine learning, crypto, etc. that works on your phone, tablet, laptop, headset, whatever is a big deal. And it's not just for anyone with Apple gear, but anyone with a compatible browser. Think generative AI/ML streaming Gaussian splats to your retinas via a browser. That's where we're headed.
Need an OR to explore a phobia of surgery? Sketchfab has you covered: https://sketchfab.com/3d-models/charite-university-hospital-...
The reason the software doesn't exist is because compelling hardware doesn't exist for it to run on, so nobody bothers to write it.
Apple is imagining this device will be used for productivity but it's still painful to actually wear for long periods. We're a long way from being limited by software instead of hardware.
How about enabling AIs to create layouts of information on behalf of the user? Like, what if an AI could arrange all of your information for you in a scheme derived from Archy?
https://en.wikipedia.org/wiki/Archy_(software)
It's a 600+ grams headset with a battery on a leash for $3500. I wouldn't say the hardware is mainstream ready or fulfilling it's side of the contract yet.
I already imagine how people from 5 years in the future will be sharing photos of the current Vision Pro asking "Remember when this was the best VR headset hhh??"
Sure, VR headsets existed before Apple's foray into that segment, but also did laptops, smartphones, tables computers, smart watches and bluetooth headsets.
And if one to learn from history, all these products categories were significantly improved after Apple entered their respective markets.
That said, I think the "you're looking at a digital reconstruction of the world" aspect pretty interesting since it means, in principle, everything can be changed as you'd like. Why not turn day to night, night to day, redecorate, block out ads on billboards, re-paint your house, hang paintings, add an extra window to your wall, etc?
Without that, you'll be looking at an artificially "crisp" version of the world.
Why not just get a few projectors or more screens if you want digital environments?
Sure, Apple built the headset so that it mimics AR devices, but it definitely is not one itself.
The technology to build an AR device with the capabilities that Apple boasted is simply not here yet. I'm sure that Apple top-talent is hard at work trying to break the barrier (if it didn't yet), but that'll be for a future version, just not this one.
> I'm sure that Apple top-talent is hard at work trying to break the barrier (if it didn't yet), but that'll be for a future version, just not this one.
To me, it feels like AVP is the final nail in the coffin of the optical compositing evolutionary branch of HMDs. Even if it's someday possible to do well, devices would still need the same realtime subsystem necessary for digital compositing in order to do 3D mapping, object detection, environmental lighting integration, etc.
I understand that in the short-term, camera sensors and displays don't exceed what the human eye can perceive, but those seem straightforward (if not easy) to address.
"AR can be defined as a system that incorporates three basic features: a combination of real and virtual worlds, real-time interaction, and accurate 3D registration of virtual and real objects"
There is nowhere in there that says the output has to have pure optical through. Digital composition seems to fit into the definition.
"AR can be defined as a system that incorporates three basic features: a combination of real and virtual worlds, real-time interaction, and accurate 3D registration of virtual and real objects"
What is it missing?
I don't care about the data it captures on the wearer. If you're inclined to hand over all that telemetry on yourself to other people, I don't know what to tell you.
I care very much about the high-resolution cameras it has pointed at me. I don't think that's okay.
To clarify, my point with this comment and my first one was never that data collection (and as far as I can tell mass surveillance as well to you?) was okay, rather that I don't see how this device captures any more data on a wearer than the other devices they likely own.
I see a much bigger issue in this regard with the meta ray ban sunglasses, since those will actually be worn by people outside. We already have cameras pointed all around public spaces, but this device category will increase that coverage to near ubiquity long-term.
When you are in the presence of someone wearing this, you're being captures by 4 or more high-res cameras, in 3d, whether you want it or not.
Does it? What about gopros or similar action cams, being used for over a decade, and for less than a tenth of the price?
I feel like being able to see everything open at once would be incredibly distracting. I like how I can swipe between app screens on my desktop so that I'm only focusing on one app at a time.
Of course I imagine there are some applications where it's useful, but to me it just feels like the Apple Vision Pro is just a very large screen and they haven't quite figured out what to do with it.
You can shove the app into a corner of your room, out of your view, and it will be there when you walk over to that corner later. That's part of the idea of "spatial computing" - if we can associate computer objects with real locations in space, maybe we can better harness spatial memory and stuff like that when we interact with them.
1. How does it compare to an high-end monitor for text editing/programming, web browsing, watching non-VR video, playing non-VR games? Is it better or not?
2. Is the resolution, latency, FoV and lack of color fringing good enough for it to be indistinguishable from reality in both passthrough and VR modes? If not, how exactly far is it?
3. Can you run VR games on a PC with multiple desktop GPUs and stream to it? How does it compare to current high-end and ultra-high-end VR headsets?
Windows even has support for IndirectDisplayDevices - I'm not sure how openXR or SteamVR handles those, however.
I recommend the video review as well. Seeing the video call between Nilay, Joanna, and MKB shows how much the tech has advanced but also how much it still needs to evolve to be at the level of FaceTime Video.
They’re like talking paintings in the Haunted Mansion ride with bunch of blur and depth of field. It’s way too weird. Joanna’s looks a bit like she’s been stuck as the replacement person in the Mona Lisa.
A static picture of you or maybe your Memoji (remember those?) would be far preferable.
I’m surprised Apple is shipping Personas.
Facebook has also shipped meta.
All the money in the world can't buy you an understanding of your market.
This seems alpha. I’m sure it’s a hard problem but I feel like waiting a bit longer would have done them a better service.
They do make it clear that it's in Beta so they know there is work to do.
But if you're having a Zoom meeting with a divorce lawyer or negotiating an M&A a Memoji is the last thing you want.
lol what kind of review says that. It would be like someone who doesn't even use any kind of vision correction saying "I just used it without glasses and it was fine."
(edit: the review is actually quite good. but that line was bizarre)
Also, I don't think people with only near/far vision issues or prism correction know how this interacts with VR. IIRC in general you always use your far prescription and there is no support for prism.
Streaming games from a PC to a standalone headset over WiFi has been proven to work with the Quest, but that has proper controllers.
It's a very niche setup, but a few people already use it in order to combine Lighthouse-based body trackers with a non-Lighthouse headset.
The point would be that you don't use them for positioning at all, just buttons/thumbsticks. If you've using a Virtual Desktop-like app on the headset then it's already got positioning from the hand tracking.
I suppose an ambitious implementation could try to fuse the hand tracking data with the IMU data from the Joycons for better resolution.
Even with just 1 4k floating screen I think it would be a winner for me but I'm also really excited to see what people come up with in VisionOS itself. I think for a while the sweet spot (for me) with be using a virtual monitor and a handful of VisionOS apps as well. Eventually I hope to be able to pull macOS windows out of a fixed box and arrange them wherever I want but I'm fully aware that might not be this year or even next year.
I don't think you'll be able to pull windows out of the mac screen, but apps you might need are in vision OS anyway like safari or messages.
I think my dream would be dual 4k monitors, or maybe a double wide?
I'm not sure where the limitation is right now (if it's purely a software limitation) but my MBP runs 4 monitors right now and I'd put up with a cord (or two) plugged from my MBP to my AVP if I could have all my monitors in VR. I believe they are using AirPlay to give you the virtual desktop but 3-4 AP streams doesn't seem too crazy, but I don't know much about that.
> A: The Vision Pro wasn’t designed to be worn with glasses. Instead you have to order prescription Zeiss optical inserts for $99. The two monocle-looking pieces snap right into place.
It's $99 for readers (non-prescription) and $149 for a prescription. Very odd that she would have gotten that detail wrong since it's so easy to check and it's been repeated so often in coverage of the AVP.
Basically it wouldn't be them to pass up a accessory sell opportunity.
Yeah, uh, they definitely always do that. It "wouldn't be them" to ever not.
(Hell, the monitor even supports True Tone—and it didn't at first, that arrived in an OS update, from Apple)
But what is the account situation like?
For years I’ve been complaining that I can’t easily use my private iPad with my company Mac because they have separate Apple IDs. Things like sidecar for a quick virtual whiteboard are basically impossible.
AirPods have gotten better over the years where today I can freely switch between devices belonging to different Apple IDs with the same AirPods.
But is the Vision Pro like that as well? It would seem weird to exclude the not-so-small group of people working from home but with company MacBooks
Assuming the Vision Pro screen sharing works using the same stuff, I have high hopes.
Ended up creating a new account that was part of my family.
(My partner is corpo; I'm startup, but have worked at corpos. No thanks.)
Better to keep it all owned by the company, in my opinion, and have them issue you an iPad for this express purpose.
The main one being a complete separation of calls, messages, calendar, notes and reminders. For my own sake more than for my employers sake.
And many employees with company phones already have that separation. iPhone and Mac is not that uncommon to provide for employees. But an iPad on top? I think that’s gonna be much harder to find
And edit: a Vision Pro on top…
If your work is on the traditional model of perimeter protection and trusted intranet, a non-work device can't join the network as you have correctly pointed out. If your work is on the newer BeyondCorp style model, switching to a second account on your computer is going to invalidate the device trust needed to access work resources.
Locking you customers into your ecosystem? Fine, whatever. But even within the ecosystem restricting usage in such a way!?
It's been said for years but the iPad could be so much more than a mere media consumption device if it weren't for short-term-profit driven design decisions.
Maybe they do better with the Vision Pro.
Literally the only cloud drive product I know of which doesn't work on my corporate laptop is iCloud Drive, because the EMM gave a checkbox to set a flag. As a result, a huge portion of built-in collaborative features and apps just don't work. I have paid seats in other products only to regain functionality lost by that checkbox.
I have a similar complaint with my Apple Watch and my corporate issued laptop. When I am using my own computer (mac mini) I love how easy it is to use my watch to login, use it to approve actions, etc. However when it comes to my company laptop I have to type my password in repeatedly. It would be awesome if the watch could be linked to both IDs to make this much more seamless.
It's actually far worse. There's a single user and a "guest mode", but for AR/VR to work with, there's a calibration step, which means that the guest has to go through that step every single time they want to use the device. It might be fine for a real guest using it once, but it would be basically impossible to share the device with someone else. Having to setup the device every single time you use it sounds absolutely terrible.
This is kinda what Managed Apple IDs are for - the work 'owns' the Apple ID it puts into its management profile and can set policy. Apps write into a separate storage container which the company could remote wipe, without affecting the rest of your personal data. If they want to disable things like sidecar, they can do it.. for the corporate apps/accounts/web domains.
I'd' generally assume the multi-user aspect is worse (because face shields and prescriptive inserts) so generalized multi-account is pretty low on the priority list.
Zoom has a good Airplay sharing feature that works well in this situation.
But I get what GP means -- I do have a corporate profile, and I made my own @corporation.com Apple ID, but what do I do to use sidecar? Either log out of my personal iCloud on the iPad (gross) or log in to my personal iCloud on my work computer (grosser)
I use my iPad so sporadically that it could easily be the house iPad, but I’m signed in with my email and so on it can’t be.
What the fuck. The fact that an apple ID is even involved is absurd. Should be able to just Bluetooth to any device.
However, pairing an audio device is an exchange of settings and encryption keys, and Apple will sync that pairing that to your entire account. Hold your AirPods near your Phone and tap the button to create the initial pairing, and they start working with your Mac and Apple TV.
I use my airpods with my Nintendo Switch and Samsung TV all the time. No Apple IDs on there.
Bluetooth is one of the worst protocols to work with. That goes for both implementers and users. It’s one of the key selling points of Apple devices - they just work together.
Troubleshooting which device is hijacking the Bluetooth speaker is not how I want to spend my relax time. And don’t get me started on anything IoT. Solving the pairing issue is a massive selling point.
As for Apple ID specifically, I know that it’s used for tracking lost devices, and connecting with other devices that you may not have paired individually in a n^2 fashion. It could be, for instance, that Bluetooth pairing would conflict with apples own discovery/ownership protocol - say you only use BT (no account) and then an iPhone with temp access to your AirPods could “take over” your AirPods since they didn’t have a logical owner before.
The "freely switch" here is referring to the W-chip multi-device support that will on the fly switch between any number of Apple devices based on what's actively being used at the time, without needing to do any manual connection stuff.
Other non-proprietary Bluetooth devices will generally do 2 devices at most, and getting that to work right with microphone input settings can be kind of a nightmare.
These devices are going to have your sweat, makeup, odours etc on them.
So you're really not going to want to share a device with anyone else.
Unfortunately Apple is charging $200 per extra facial interface though.
The moment you use different Apple IDs you lose a lot of nice features of Apple‘s products
Either you can't sign in with your personal Apple account, or you shouldn't (because MDM). So the only way to access anything associated with iCloud is what is available on the iCloud web portal; which is a horrible experience. You can't do sidecar. You can't do airdrop, copy-paste, continuity camera, nothing.
I've only ever used Macs in a professional environment. I've, also, always had a Mac and iPhone as personal devices. But I've never made the jump toward saying "Ok I'm actually using iCloud Seriously now" for this single reason. The best Google Cloud experience is available in a web browser, which I can be signed-in to on everything. Google Drive is everywhere. The list goes on.
Its such a crystalline example of why Apple's walled garden actually hurts themselves.
However, the best review I’ve found that actually transmits what is possible and what it is like to use is Brian Tong’s 55 minute review video: https://youtu.be/GkPw6ScHyb4
I’m not familiar with him, but unlike other reviews I’ve seen, he spends less time evaluating or summarizing, and more time trying to actually use the device. I didn’t even realize that you can seamlessly use your Mac to control your visionOS apps, for example.
He said he could wear it 45 mins before needing to take it off, that it was overstimulating so you need to slow down how quickly you use apps and move things on screen, and that gestures also were fatiguing. You could tell he was trying to be fair but positive.
Headsets just haven’t cracked this nut yet, and tho tech may advance somewhat, they may be limitations inherent to the form factor. Even if it gets really light weight, the issues of overstimulation, headaches, and the amount of neck movement implied may keep these products in a niche. (I say this as someone super excited about AVP)
For everyone used to using their computers all day long wanting to do it in a headset, don’t throw your macbooks away just yet.
[1]: https://www.bobovr.com/products/bobovr-m3-pro
A lot of the physical downsides here are basically self-inflicted by companies trying really hard to hide the "nerd factor" necessary for comfort, to the detriment of the actual user experience.
The big product marketing question is what niche do headsets fit in, and thus whats the ideal single session and daily usage goal for a headset?
If it's about replacing laptops or another high usage scenario, that's a pretty high bar, definitely too high for the next 1-2yrs. I imagine some people at Apple wore dummy see-through goggle ergo tester units of varying weights around all day to get at these numbers :) Wonder what they came back with. Even still, that only gets at weight vs the perceptual ergonomics, skin-feel, etc.
The issue I see with headsets is that there may not be a lot of improvement possible without compromising durability or other factors necessary when going to market. E.g. what if they can't get it below ~400g (making AVP ~40% lighter), but to make the headset comfortable for most people for the usage scenario that makes them mass market (e.g. 2h+ sessions daily) requires ~250-300g?
And those who work in VR report their coworkers and just about everyone they talk to customer wise feels the same way.
I know there are people who really want to, or think they do, but most would rather just use screens until maybe such a time the form factor becomes a pair of eye glasses.
I think it's completely reasonable to say that this stuff is only going to continue to get lighter and more comfortable every generation. What's that balance of interest going to look like when a headset's eventually got the same weight and form factor as, for example, big ski goggles?
IMHO both. As for form factor lots of people already opt for contacts over glasses for comfort. I think ski googles are right out for anything that's not recreation or industry/application specific.
As for being in VR people really do like the real word lol. Going in and out of VR is a bit of an ordeal. Anecdotally I like taking small breaks from looking on my screen during the work day. Frequently. I look outside at the trees, at my cats, stuff on my desk, and etc when I'm thinking. If I have to "go in and out of VR" to do that or just not be in VR I'm going to just not be in VR.
For it to catch on mainstream for productivity and day-to-day it's gonna have to be like eye or sunglasses with seamless AR. Via something like retinal projection perhaps. I feel like Google was onto all this hence the Google Glass.
At first people think "wow it's so awesome I can be sitting on the moon while I browse the web". But after a bit of time you just get tired, and I think it's precisely because your whole brain is working in overdrive to understand the unnatural environment you are in. None of this manifests explicitly but at the end of it, when people are faced with the choice of putting the headset on or not, they just "feel" like it's a lot of effort.
I say all this as someone who does regularly spend 1-2 hours working in Immersed with multiple giant screens up. And I love it as a break, a way to focus or just relieve the boredom of working in the same space day in day out. But even I feel this effect of it being tiring and not keen to do it for 8 hours a day. And the minute you say that, you lost the use case of this being your "only" computer / replacing your laptop, so it's actually kind of crucial to its central justification as a replacement for a computer or a 'new kind' of computer.
Having demoed VR at my old office I can tell you that the range of reactions varies from an immediate "nope" and having to take the headset off to being able to stay in it for a significant amount of time with no discomfort.
- The Vision Pro is the best VR set that can be done today, with massive investment (rumoured 5e9 USD) and competent staff, and hefty price. It is miles ahead of the competition.
- It’s still not enough for most if any practical use, apart from films maybe. The technical requirements for a really useful VR are still largely out-of-reach, and will be for at least the next 5-10 years.
Thank you.
Milion, Miliard, Bilion, Biliard = Million, Billion, Trillion, Quadrillion
(neither are justified because neither start at the beginning, that is at a thousand :) )
Less weight would be better, but there's ways to embrace that and aleviate the issues ("have the product be true to its nature" to channel my inner Jony Ive).
This time Apple chose not to, while Varjo tends to care a lot more about these ergonomic aspects. Looking at the XR-4 strap, it looks like a solid design to me.
A motorbike helmet is heavier (3kg), but the weight is wrapped around your head, not hanging in front of it.
Yes, that's what makes the weight bearable.
Same way the Quest Pro for instance splits the weight around the head to achieve a good balance. Better bands will also help.
At this point we have none of that for the Vision Pro, which is a choice Apple made.
I really don't think this is correct. It's a bit better at some things than any of its competition, and it's a little worse in some ways too. Other than the displays I don't think you can say it's substantially ahead at all.
> the Vision Pro has the best video passthrough I’ve ever seen on the sharpest VR displays any normal person will ever come across
> the eye and hand tracking control system… is light years beyond any other consumer hand or eye tracking systems
It calls out other aspects similarly.
2: The detail oriented software. Look how polished everything looks in Vision OS. It’s just nice to be ‘in’ there.
3: The integration of the software and hardware. Extremely nice handtracking, amazing see-through functionalities, working with a good ecosystem (keyboard, MacBook, trackpads,…)
Those experiences are what makes Apple unique. Not ‘better’ per se. That depends on what you need.
I don't think it is ahead of the competition personally and especially when considering the 7x price I fail to see how it could land with consumers. Having said that I do anticipate Apple will finish the tech in the next few years and it will be a much better buy (both in value proposition and with a lower price) a few years from now.