How the Oculus Rift Works and Why You Need One
You don’t know why people are so hyped about the Oculus Rift ? That’s understandable.
A lot of systems, from the Kinect to 3D movies , have claimed to be “immersive” over the years, but none of them are all that special — and the last time you tried virtual reality, it made you feel sick and it wasn’t all that fun.
So why should you get excited for the Oculus Rift, a device that shows every sign of being a dumb gimmick? It’s hard to explain if you haven’t actually put one on and tried it for yourself, but I’m going to give it a shot anyway.
Let’s talk a little about what the Rift is, how it does what it does, and why you’re going to love it.
Virtual Reality In a Nutshell
Simply put, the purpose of virtual reality (VR) is to trick your brain into perceiving a virtual world as real.
Note that I say “perceiving” and not “believing”. The technology can’t make you forget that you’re actually wearing a pair of goofy goggles and standing in your living room. What it can do is trick your senses so well that no amount of conscious knowledge can shake the illusion.
Just as roller coasters are scary even though you know they’re safe, VR feels real even though you know it isn’t.
This is fascinating because it’s kind of all-or-nothing. If the brain is fooled well enough, one can experience intensely powerful, close-to-real scenarios. If it isn’t fooled well enough, that’s when you get eye strain and motion sickness.
If VR isn’t done perfectly, it isn’t worth doing at all.
That’s why so many people are confused about the sudden resurgence of VR. The technologies used in the Rift are essentially upgraded versions of the stuff in old VR arcade games.
As such, one would expect a slight improvement in the experience at best — but all combined, these advancements have improved the experience enough to cross that invisible threshold in the brain between “nauseating” and “awesome”.
That’s why people are excited, and that’s why there’s so much money being spent on VR today.
How Virtual Reality Works
So how do the brain and the senses get tricked so perfectly? As it turns out, it’s really hard — so hard that it requires a handful of clever tricks across the board.
Display and Optics
Let’s start with the simplest problem VR developers need to solve: how to actually put an image in front of the user’s eyes.
The Rift uses two display panels and two lenses with each screen covering a different eye, which produces a 3D effect. Note that this is different from 3D movies, which are designed knowing exactly where the users eyes are (and thus knowing the right perspective to emulate).
In a 3D movie, you have to display the same stereo image to everyone no matter where they’re sitting, and that means that it’s never actually perfect. Objects don’t look like they’re the right size and many people get eye strain if they aren’t sitting in the exact center of the theater.
The Rift’s screens are OLEDs — the same technology used in high-end smartphone screens — and sit a few inches from the user’s face. To avoid eye strain, the lenses refocus the light as though it were coming from far away. These lenses also magnify the screen to cover more of your vision, which warps the image quite a bit.
The old approach to solving this problem was to use a second set of lenses to undo the distortion, but this added a lot of weight and cost to the headset. The Oculus Rift solves the issue by reverse-distorting the image on the computer so that it looks normal when viewed through the lenses.
Tracking and Rendering
The Rift also needs to know when the user’s head moves in order to update the image on screen properly — and this information has to be known quickly and accurately. When the user moves or turns, the Rift only has about twenty milliseconds to change the image on screen without the brain noticing the delay.
If the image updates too slowly, the immersion is broken and the user feels sick.
Twenty milliseconds is not a lot of time. If you’re rendering at a modest 30 FPS (frames-per-second), simple math says that each new frame takes 33 milliseconds to render. That’s already too slow, and that’s without considering the time it takes to measure the motion, send the data to the display, and change the pixels.
So VR requires a fast renderer, and that’s why it needs such a beefy PC. The Rift happens to run at 90 FPS, which is faster than most high-end PC games.
At this speed, each frame takes 11 milliseconds to render, leaving 9 milliseconds to handle everything else. The OLED screens can switch pixels in about 1 millisecond, leaving 8 milliseconds to measure the head’s position and get all of the data back and forth across the cable.
As it turns out, no current sensor in existence is fast enough and accurate enough for VR. The fast ones are too inaccurate and the accurate ones are too slow. So the Rift actually uses two sensors: one fast and one accurate.
The fast one is an accelerometer similar to the one in your smartphone, which can detect motion and rotation at an extremely high refresh rate but is notoriously inaccurate. The slow one is a camera that tracks blinking infrared lights on the headset, which can objectively determine position but relies on slow image processing.
The Rift’s software blends these data sources together and uses the slow one to correct the fast one when they diverge too much. Together, they provide high-accuracy, low-latency tracking.
The Rift will ship with an Xbox One controller since most of its launch titles are designed for a gamepad. However, the gamepad is far from the be-all and end-all of VR input.
Ideally, you’ll want a motion controller — and not an inaccurate wavy stick like the Wiimote . You want something that actually makes it feel like your hands are in virtual reality.
Enter the Oculus Touch, an optional add-on for the Rift.
These devices are small controllers that wrap around your hands and are also tracked using the external camera. This allows for the same high-accuracy, low-latency tracking. The Touch has several buttons, a joystick, and a trigger, and can track your thumb, index, and middle fingers, letting you gesture and pick up objects in VR.
So far, a good start… but it isn’t enough.
One problem that most people overlook is so-called “persistence blur”. When you draw a frame to the screen of the Rift, it’s correct only for the first millisecond or two. After that, it’s outdated and needs to be updated — but continues to be dragged around with your head. To the user, this looks like motion blur.
The solution? Strobe the display very quickly.
When the frame is rendered, it’s drawn to the headset where it stays visible for a millisecond or two. Then, the display turns off completely until the next frame is ready. Because this is happening at 90 FPS, we don’t see flicker. In fact, it looks normal even though the screen is actually black most of the time.
This is called “low persistence mode”.
There are also several subtle challenges to designing a VR headset that’s comfortable over long periods of time. It has to fit all of this technology without being too heavy (ideally, less than one pound).
For this reason, the consumer version of the Rift is partially made from a durable fabric to keep its weight down. I haven’t personally use the consumer version, but the most recently available prototype is astonishingly light already.
From Toy to Necessity
Eight years ago, when the iPhone first premiered, it was derided as an expensive toy for nerds with no possible practical value. Today, nobody between the ages of 15 and 85 would be caught dead without a smartphone. That’s just the way technology works.
The Oculus Rift is absolutely going to revolutionize gaming , but that’s not all it’s good for. Not by a long shot.
Viewing the Rift as a gaming device is like looking at the iPhone as an Angry Birds device. You aren’t wrong, but you’re missing the point. The real game-changer here is the Rift’s eventual impact on day-to-day communications.
Right now, Skype is slowly edging out the telephone as the preferred medium of live remote communication. VR can provide an experience that leaves them both in the dust. The consumer Oculus Rift can capture your voice and the motions of your head, hands, and fingers.
That’s already enough to facilitate powerful experiences like the Oculus Toybox demo, which allows two users to share a space and play with toys together.
Looking at the longer term, headsets may eventually be able to track facial expressions and body language, allowing for next-generation VR social experiences that are virtually indistinguishable from reality. That’s the kind of radical functionality that will change lives around the world.
Right now, the Rift is more than rough around the edges. It’s bulky, it’s wired, and you need a powerful PC to drive it. It’s also expensive, new, and everything about it is still in the formative stage. In a lot of ways, it’s the first open beta of the future .
So maybe you aren’t sold on it yet, but if you get the chance to try it, don’t hesitate to do so. I guarantee you’ll be surprised.
Are you excited for the future of VR? Or are you convinced that it’s a passing fad? Let us know in the comments. We’d love to hear what you think!