Mixed reality on the Quest 3 is an exciting novelty, but not its future

Most Meta Quest 3 leaks suggest the headset will improve on the Quest 2 with a faster Snapdragon chip, pancake lenses, more accurate hand tracking, and full-color passthrough. It’s that latter point — new mixed-reality apps and games, just like on the Quest Pro — where the 2023 headset will differentiate itself most visibly.

Yet Meta engineers admit that MR is “still in its infancy” compared to VR, and everything I’ve seen of the tech thus far has been a mixed bag (pun intended).

Meta came to San Francisco’s Game Developers Conference with one clear goal: to get developers invested in mixed reality. At its two official GDC 2023 panels, it explained how its Presence Platform lets you adapt experiences for any physical space, and highlighted the success of indie devs like Cubism’s Thomas Van Bouwel that have brought MR passthrough to their games.

I came away from them looking forward to seeing what other Oculus Publishing devs do with mixed reality. But I’m also more convinced than ever that MR isn’t the killer tool that’ll make the Quest 3 sell like hotcakes; it probably needs another generation to come into its own.

The challenges of mixed reality

Got a chance to show @LaserDanceGame to some folks from @OwlchemyLabs yesterday!Huge fan of their work, and very excited for what they’re building next! pic.twitter.com/cYzSmVHGQLMarch 24, 2023

Cool experiences like Laser Dance (above) show mixed reality’s potential to make your living space a dynamic gaming area without being confined to a small furniture-free circle. But Van Bouwel also outlined how he had to account for the player’s size and mobility, wide vs. narrow rooms, sparse to crowded levels of furniture, and so on when designing it.

Essentially, MR games must either incorporate tons of variables, limit gameplay to a fixed space like a tabletop, or else run into all kinds of unforeseeable issues.

To address that, a mixed-reality game has to incorporate spatial anchors, which create world-locked frames of reference that persist regardless of which direction the headset-wearer faces. Using Meta’s Scene API, the game can capture and model a player’s surroundings to create 2D or 3D “bounding boxes” that the MR content lives in; but it’s up to the developer to take advantage of this tech.

Three people using Wooorld with the Meta Quest Pro's passthrough mode

A mixed-reality experience on the Meta Quest Pro (Image credit: Wooorld Inc.)

You can even share spatial anchors between two headsets, so colocated users can share a local multiplayer experience like playing chess (or something more dynamic) on a real-life dining table. At the moment you’d need two pricey Quest Pros to do that, but in theory, buying two Quest 3s wouldn’t be as onerous.

Still, counting on everyday people buying two consoles just for local multiplayer — or making one person play in black-and-white on their old Quest 2 — isn’t a business model for success, so I’m not certain how many devs will invest time in this tech.

A mixed-reality experience has to account for new factors like depth, too. In Beat Saber, note blocks will appear a dozen virtual meters away; but if you tried a similar experience in MR, a nearby wall could limit the depth that you have to work with, which means the blocks need to be much slower for players to react in time.

Mixed reality is both more and less social than virtual reality, depending on whether you want to play with people physically or digitally nearby.

Meta is aware of the medium’s challenges and created the Presence Platform to help developers address them. But these SDKs aren’t perfect: users have to manually place spatial markers, for example.

When devs at the panel who’ve used them asked about upgrades like automatic anchor recognition, Meta Product Managers Meenal Nalwaya and Aashay Desai mostly responded that they were “considering” or “testing” new features but had no timetable for implementing them.

I asked if mixed-reality games could somehow work across two remote spaces — meaning each player has their own local MR bounding box but manipulates the same content — and Nalwaya said that players can always see their own surroundings via passthrough, but otherwise said she’d have to check (which felt like a “no” to me). So in that sense, MR is more social for staying aware of your surroundings but less social than VR for online multiplayer.

Desai and Nalwaya concluded the panel by saying “we are just now scratching the surface of mixed reality,” which is an important point for Quest 3 would-be buyers to consider. It may feel like a return to the early, experimental days of the Oculus Rift, when VR developers were still experimenting and learning as they went — only for MR games instead.

VR > MR for immersion

Render of the Meta Quest 3

A CAD render of what the Meta Quest 3 could look like, showing its clear front-facing cameras (Image credit: Alvin Suen (via @Lunayian))

If the Quest 3’s full-color passthrough camera is anything like the Quest Pro’s, there will be a clear visual gap between the fidelity of your surroundings and the graphical objects produced by a mobile chipset.

As my colleague Nick Sutrich said in his Quest Pro review, the color passthrough quality is “impressive, but it’s not perfect. There’s still some fringing happening around the edges where lots of contrast exists,” and the colors aren’t always 100% accurate to real life.

In VR, even if everything is low-resolution like in Resident Evil 4 VR, it’s still a consistent aesthetic experience. In mixed reality, the gap between reality and the virtual is more obvious and prone to look artificial, especially along the blurred line between the two.

Plus, Nick explained how the Quest Pro’s open glasses design lets you see your periphery, making these experiences more immersive because you’re always sure of your surroundings.

Mixed reality brings cool experiences into the real world; most gamers would rather leave this world behind entirely.

On the VR-focused Quest 3, you’ll have your front camera view, but limited to its lenses’ field of view, which should fall between 90–100º (or between the Quest 2’s and Quest Pro’s FOVs). You’ll see what’s in front of you, but you’ll still have to be a little more careful of environmental hazards.

There’s a reason why so few mobile games have successfully recreated the Pokemon Go blueprint. Because for the most part, people play games to transport themselves into other worlds, not transport other worlds into theirs.

VR fans wear headsets to avoid looking at their surroundings, which lets them lose hours in their favorite Quest games. I’m sure some won’t mind the option of an MR mode for games like Demeo that were already top-down tabletop experiences. Otherwise, I suspect that many will treat MR as a cool novelty when they first unbox their Quest 3 headsets…and then switch right back to VR.

Where MR makes VR better

VR has plenty of “killer apps” that gamers have latched onto, like rhythm games or exercise games. Augmented reality glasses have further to go, but I had the chance to play Steam Deck and Xbox games on the Nreal Air at GDC and could see why AR has struck a chord with that niche.

Mixed reality, in my mind, is too new for Meta to have found a killer app. The Quest Pro is a prosumer work device that uses mixed reality well, but even with a $500 price drop, it’s still a niche device most people won’t buy. I’m curious if Apple’s $3,000 Reality Pro headset will have more to offer on that front, but all the leaks and scuttlebutt suggest that Apple’s engineers are worried about its chances for success, too.

So why am I excited for the Quest 3’s mixed-reality passthrough, aside from a clearer image when I’m outside of VR games? Because developers like Resolution Games are using passthrough to make games better for people who live in smaller spaces.

In games like Blaston for the Quest 2, you can switch from a virtual arena to a passthrough mode where you can see grayscale outlines of your surroundings, ensuring you don’t accidentally slam a controller into a wall or chair because you never lose sight of them.

For Quest users like me with small apartments and sprinting cats to worry about, passthrough is a really useful tool for a hybrid playspace where I can come right up against furniture without worry and see any person or pet that enters the space.

The only problem is that the Quest 2’s low-res, B&W cameras were never really intended for this use, and the poor graphics reflect that. But I tried Resolution Games’ new Spatial Ops beta at GDC, which uses the Quest Pro’s full-color mixed-reality to great effect. With the Quest 3 and its upgraded camera, we could get many more VR games that subtly incorporate your surroundings into their virtual worlds, without it looking quite as terrible.

By the time mixed-reality tech becomes more ubiquitous, you might be looking at buying the Quest 4 instead of the Quest 3.

Otherwise, well, Meta and its partners really have to sell us on the idea that native MR games on the Quest 3 aren’t just experimental gimmicks. MR might make the headset more accessible to less experienced VR users because there’s no simulation sickness and their surroundings will be more familiar. But it’s also a little less magical than the virtual worlds that people have come to expect from a Quest headset.

Most Oculus headsets last two to three years before they’re replaced by a newer model. Mixed-reality games will continue to come out during that time. But just like the Quest 2 was the breakout VR console after the Quest 1 laid the groundwork, I’m not convinced that MR will have its breakout moment until the Quest Pro 2 or Quest 4 comes along.

Source link

Mixed reality on the Quest 3 is an exciting novelty, but not its future

Which Skincare Product Should You Try Next? Take An "Everything Shower" To Find Out

Amazing Amateur Weather Photos

"The Thing I Hate About Where I Live Is…": People Are Calling Out The Frustrating Home Feature They Didn't Notice Much Later

26 Beauty Products With Before And After Photos That’ll Make Your Jaw Drop