Gear VR: The future is (not quite) here

Posted on December 19, 2015

The line forms to the right, ladies

If you’re a nerd like me, you’ve been hearing buzz about a company called Oculus VR for a few years now. Oculus grew out of faith in what most of us thought was a dead god: virtual reality, the dream of technology that would allow people to immerse themselves in invented worlds.

Founded by Palmer Luckey and based on work he’d done at USC, Oculus claimed to have solved the problem that had killed the first VR boom back in the 1990s — how to create a head-mounted display (HMD) that was powerful enough to update its images as fast as the human eye requires, while being light enough to be worn comfortably. Luckey’s prototype attracted an enthusiastic core of influential fans, including legendary game programmer John Carmack (who would eventually become Oculus’ CTO); a hugely successful Kickstarter followed, with $75 million in venture capital and then acquisition by Facebook for a cool $2 billion not far behind.

It was one of the most meteoric ascents in modern tech history, and it was all driven by one thing: testimony by those who had experienced Oculus’ technology that it was game-changingly, world-changingly good. “Once you experience it, you’ll understand,” they said. But for most people, such experience was elusive, because despite three years having passed since the Kickstarter Oculus has yet to ship a consumer-ready version of its VR technology. All that they’ve shipped are a couple of developer kits and a lot of buzz. So people have been starting to wonder when the revolution Oculus promised is ever actually going to arrive.

If we’re talking about the main consumer-oriented Oculus product, Oculus Rift, the answer is “not anytime soon.” The latest ETA from the company has Rift shipping sometime in the first quarter of 2016 — which, critically for a consumer-tech product, means that they’re going to miss the Christmas 2015 retail season.

But surprisingly, while Oculus won’t have an Oculus-based product for sale for some time, Samsung has one you can buy right now. Called Gear VR, it combines Oculus’ VR tech with the latest generation of Samsung smartphones (the Galaxy S6, S6 Edge, S6 Edge+, and Note 5) to create what they say is a ready-to-go VR environment that turns the Oculus buzz into something ordinary people can experience today. And it does so for a very reasonable price: $99. (Not including the cost of the phone, of course.)

I’ve been wanting to see if the Oculus tech was equal to the Oculus hype for some time now, so I picked up a Gear VR and put it through its paces. Here’s what I found.

The hardware

Gear VR is a head-mounted display that weighs about seven-tenths of a pound. It requires a little assembly on first use, mostly threading the various straps that hold it onto your head into place. Popping off the plastic front plate reveals a slot into which you snap your Galaxy phone with a couple of easy-to-use clips (one of which has a micro-USB connector, which plugs into the phone’s data port) and you’re more or less off to the races.

The first challenge a head-mounted display has to overcome is the simple fact that strapping a bunch of tech to your face is a process that’s difficult to make comfortable. To Samsung and Oculus’ credit, Gear VR mostly overcomes this; it’s light enough that it doesn’t weigh down your head or bite into the bridge of your nose, and all the places where it comes into contact with your skin are sufficiently padded to prevent you getting pinched by plastic. It can fit comfortably over glasses if you wear them, though I found I didn’t need mine — my eye problems are all about distance viewing, and the virtual image Gear VR presents is right up in your proverbial grill.

The device is capable of head-tracking, meaning that moving your head while you wear it changes the point of view you see on the virtual image; swivel your head left and the camera pans left, look up and it pans up. There’s also a directional pad and set of buttons (back, volume up, volume down) on the right side of the headset; they provide an additional means of input for applications that support them. (Not all do; in some apps you select things by looking at them and tapping your finger on the d-pad, while in others just looking suffices.)

Because all the actual processing happens in your mobile device, there’s no power cord, external battery or data cable to some other device required; Gear VR is completely cordless. This is convenient, both because it streamlines the process of getting up and running and because it makes the whole setup portable — just grab the goggles and your phone and you’ve got everything you need to share it with friends at a party.

Gear VR splits your phone’s display into two areas, one for each eye, so the S6’s 1440 x 2560 pixel display gives it an effective resolution of 1440 x 1280 per eye.

The experiences

But you’re not here to wade through mind-numbing spec sheets; you’re here because you want to understand what using this thing is like. So let’s talk about that a bit.

When you plug your device into Gear VR, it undergoes a kind of personality transplant. The old Android interface falls away, replaced by a panoramic Home screen that floats in midair inside what looks like a pretty swank apartment. The Home screen displays buttons to launch your most recently used VR apps, as well as to access your Library (the collection of all VR apps installed on your device) and an online app store operated by Samsung and Oculus from which you can obtain more.

VR is still in early days, which a quick browse through the app store will make painfully obvious. There’s a reasonable number of apps on offer, but most of them are pretty small-scale; bite-sized experiences designed more to demo the hardware than anything else. Thankfully, these are mostly free. There are a few paid apps to be found as well, of varying degrees of polish and content density; Oculus appears to be encouraging developers to follow the pricing model of mobile apps, so these range from $.99 to around $15.

Broadly speaking, the available apps offer a few general categories of experience:

• Panoramic imagery. These apps take a still image, generated either by a special 360° camera or by stitching multiple images from a regular camera together, and let you effectively step into the picture as if it were painted on a dome surrounding you. Moving your head changes the perspective from which you view the image, but since it’s at root a still image you don’t get the sense of things moving around you. The images available include both photographs of real-world locations and completely invented digital landscapes that exist only in some creative person’s head.
• Video. In these apps, you… well, you watch videos. The videos themselves are the same as they would be on YouTube, flat 2D moving images, but by presenting them in a virtual environment Gear VR can make use of some interesting tricks it has up its sleeves. Instead of just seeing the video in a little window, for instance, it can present them as if they were being projected on a giant movie screen, so big it completely fills your field of view. It can also place that virtual screen inside a virtual environment — an empty movie theater, for instance, complete with lights that dim when the video starts and come back up when it’s done. When it’s done right, this effect can be quite striking.
• Panoramic video. These apps present video that’s more tailored for the VR environment, filmed (like the panoramic images above) using 360° cameras so that action all around the camera can be captured at once. Here you’re not looking at a movie screen, you’re placed right in the middle of the shot itself; and as you move your head, your viewpoint moves with you. The best of these combine this immersive perspective with clever use of positional audio, so that (if you’re wearing headphones) you can hear something happening outside your field of view and then turn around and see it. It’s still video, however, so your ability to interact with the experience is generally limited to just turning about within it.
• Games. There are also a small-but-significant number of VR games available. Unlike the apps described above, these generally present you with rendered 3D environments rather than still images or video; because of this, you can interact with these environments in richer, more complicated ways, though the exact type and complexity of these interactions will vary from game to game. Developers are still working out exactly what types of interactions work best in the VR environment, but there are a few types that have already begun to emerge as standards. One is what I’d call the “psychokinetic puzzler,” in which the interaction involves solving puzzles by moving objects around in the VR environment; since Gear VR has no way to capture what you’re doing with your hands, these games typically let you “move the objects with your mind” by focusing on them and then performing some interaction with the d-pad or buttons. (Examples: Land’s End; Esper/Esper 2.) Another is what I’d call the “turret defense” game, in which you are placed inside some kind of gun turret and then tasked with shooting down waves of enemies, typically directing the turret’s fire by moving your head. (Examples: Gunjack, Bandit Six.) But there’s also games which defy categorization, such as Drift, which is played from the perspective of a bullet fired out of a gun; you guide the bullet’s trajectory through space with your head movements, helping it avoid barriers and find its intended target.

The compromises

All of which probably sounds pretty neat. And it can be! But the more you use Gear VR, the more you discover the compromises that have to be made in order to make it possible. And some of those compromises detract significantly from the quality of the experience.

When I first heard about Gear VR, it piqued my interest for a simple reason: it didn’t seem like something that should actually be able to exist. As I said above, Oculus has been working on their own consumer-oriented VR device (the Rift, due sometime in 2016) for several years now. And while the exact shape and capabilities of the Rift have varied with each new developer kit they’ve released, one thing remained constant: the warning that it would only work if you plugged it into a powerful desktop-class computer with a top-of-the-line graphics processing unit (GPU). VR was computationally demanding, they said, so being able to tap into all that raw computing power was absolutely necessary for Oculus to work its magic.

This was a message that made sense, at least to me. But it was also a message that was completely undermined by Samsung’s Gear VR, because here was a device using Oculus tech that was driven entirely by the comparatively meager processing power of a mobile phone. (A top-of-the-line mobile phone, but still.) Both Samsung and Oculus claimed that Gear VR was a real, honest-to-gosh Oculus device, not some dollar-store knockoff. But that seemed impossible, because if you could get a true Oculus experience with just a mobile phone, did the Rift really need that beefy desktop PC? And if the Rift really did need it, how could Oculus and Samsung say with a straight face that Gear VR was a comparable product?

I’ve never had a chance to use a Rift, so I can’t answer those questions with 100% certainty. What I can tell you, though, is that while I no longer doubt that Gear VR can provide a “real Oculus experience,” it does so by cutting corners and pushing your smartphone harder than it’s ever been pushed before.

“Cutting corners” comes across more harshly than what I intend to say here; I don’t think either Samsung or Oculus skimped on this product. What I mean is that, while the initial impression Gear VR makes is amazing, once you’ve used it for a few hours the seams begin to show.

Take those panoramic video experiences I described above, for example. They are definitely novel and (in some cases) immersive. But there is one thing they definitely are not, though: high definition. As noted above, Gear VR’s per-eye resolution is 1440 by 1280 pixels, which sounds impressive considering that over-the-air broadcast HDTV programming is often displayed in the comparatively pixel-meager 720p (1280 × 720) format. But when you actually put Gear VR on you discover that spec sheets aren’t everything, because that 720p image on your HDTV will look crisp and clear while video on your Gear VR is muddy and indistinct.

And that matters, because it distracts from the immersion that Gear VR is straining so hard to provide. It’s fun to be standing in the middle of a virtual crowd crossing a street in Tokyo; it’s less fun to realize that you can’t really make out the expressions on their faces. It’s not enough to ruin the experience, but it definitely detracts from it.

Why the muddiness? That brings us to the other major compromise. It’s kind of amazing that Samsung and Oculus were able to derive a more-or-less convincing VR experience from just the resources in a smartphone. But it quickly becomes clear that Gear VR is using all of those resources, pushing the device as hard as it possibly can, because after you’ve been using it for half an hour to an hour the device becomes physically warm to the touch and your VR experiences start getting interrupted with “Gear VR needs to cool down” warning messages. You can dismiss them and keep using the unit, but once they start popping up they’ll just keep coming at increasingly shorter intervals until you power down and give your phone a breather. (One review I read suggested that this wasn’t really a problem, because it can be mitigated simply by pointing a desk fan at your head. That may be true, I haven’t tried it, but it strikes me as a bit of a reach to expect people to make that a regular part of their VR experience.)

So my suspicion is that the answers to the questions I asked above are simple: you can use a modern smartphone to drive a VR system, but it takes just about every CPU cycle that poor little device has, and even at that there’s only so much fidelity you can expect to squeeze out of it. It just doesn’t have the computational oomph to give any more, the poor thing.

Once you realize this, you start to notice how the smarter developers of Gear VR apps found ways to work around it. The best of the currently available games, for example, is the above-mentioned Land’s End, in which you travel across a set of striking landscapes while solving puzzles along the way.

https://www.youtube.com/watch?v=XwJ9fiH2Ksw

Land’s End is an impressive accomplishment, not least because it creates a sense of immersion more complete than any other game I tried; at one point you’re taken up to the edge of a cliff, and looking down gave me the same alarming sense of vertigo that I’ve experienced when looking down from high places in real life. But you’ll notice from the trailer that while Land’s End‘s production design is artful, it steers hard away from photo-realism; the terrain is chunky and stylized, looking more like a sort of digital painting than any landscape you’d actually encounter on planet Earth.

On one level, their decision to go that route is an artistic choice, in the same way that the stylized look of a movie like 300 is. But I suspect there’s another level in operation as well, namely that Gear VR simply can’t push enough pixels to make a truly photo-realistic landscape possible, at least not while also providing the level of dynamic interaction a game like Land’s End requires. So instead of mounting a futile battle against this limitation, the designers cleverly made the most of it instead by taking their aesthetic in a different, more abstract direction that also happened to be within the bounds of what the hardware could actually handle.

Other titles by less savvy developers call attention to Gear VR’s limitations, instead of obscuring them. One example of this is Action Bowling VR, which is exactly what it says on the tin: a bowling game that plunks you down inside one of several virtual-reality alleys. The alleys themselves are rendered convincingly, and it’s impressive to turn around and take in the environment the game has created around you.

But when the time comes to actually bowl, you immediately run into a fundamental problem: Gear VR has no way of tracking the movements of your hands, so you can’t throw the virtual ball with the same movements you would use to throw a real one. In fact, you can’t really throw it at all. What you have to do instead is swipe a finger across Gear VR’s touch-sensitive d-pad, swiping towards the top to pull the ball left and towards the bottom to pull it right. It works, sort of; but it feels absolutely nothing like actual bowling, and that in turn destroys the immersion. It’s painfully obvious that you’re playing a video game about bowling instead of experiencing bowling itself.

(Oculus is aware of this problem, of course, and are working on a peripheral to let you use your hands to interact with things in the VR environment. But that’s another bit of tech that’s still stuck in their labs instead of available as a real product, and even if/when it ships it will add another layer of gear on top of what you need to hook up to dive into VR, making it even less appealing for casual use than it is already.)

The conclusion

As a technology demonstrator, Gear VR is terrific. It provides an excellent illustration of how thoroughly Oculus has conquered some of the long-standing problems, like head tracking and natural-feeling motion, that kept VR from taking off in the past.

As a product for the general public, though, I’m not sure it’s really there yet. Even putting aside the way its requiring a current-generation Samsung flagship device limits its audience, the limited number of available experiences and variable quality of the ones do exist make it something less than a no-brainer purchase, even if you already have a compatible device and even at its attractive $99 price point.

Spending some time with it also raises, at least in my mind, some more fundamental questions about the appeal of VR technology in general. Namely:

• Will near-future devices (Oculus’ Rift, as well as its also-expected-in-2016 competitors like Valve/HTC’s Vive and Sony’s PlayStation VR) have enough horsepower to provide high-enough quality video and interactive experiences to avoid the immersion-limiting resolution limitations Gear VR suffers from?
• Will the extra oomph made available by tethering the device to an external processing unit like a desktop PC or game console be worth the resulting loss of flexibility and additional complication of setup that will entail compared to just popping your phone in?
• Will someone come up with a “killer app”: a VR experience so compelling that people are willing to buy the hardware specifically to make use of it?

And then there’s the challenge that to me seems like the big one:

• Will anyone have a VR experience so compelling that a world full of people who weren’t willing to put on special glasses just to watch 3D HDTV will be willing to strap hardware to their face (along with hand-tracking hardware and lord knows what else) to get into it?

I’ve enjoyed my time with Gear VR, but so far it hasn’t given me a high degree of confidence that the answer to any of the questions above is yes. It shows a lot of potential, but potential by itself does not a mass-market smash hit make. It has to be fulfilled to get people to open their wallets. And Gear VR, at least as of this writing, is long on potential and short on fulfillment.