The science of 3D gaming

Just as 3D films are quickly becoming more prevalent at the box office, the 3D craze is approaching the video-game industry with equal momentum

With the help of leading experts in optical science and 3D technology, we examine the present and future of 3D video games, including how it works and what it ultimately means for gamers.

Hot on the heels of the 3D entertainment revolution hitting the film industry full force-with everything from the next Jackass movie to a Kenny Chesney concert coming to theaters in 3D-it's clear that video games are the next step.

Just as 3D films are quickly becoming more prevalent at the box office, the 3D craze is approaching the video-game industry with equal momentum. First-party companies like Sony and Microsoft, as well as third-party developers like EA, are already embracing the technology for gaming, and Nintendo is even rolling out a 3D gaming handheld, the 3DS.

While few doubt that the role of 3D in gamers' lives is growing, it's not clear what exactly this means for the industry as well as for players. James Cameron's record-shattering 3D science-fiction film Avatar is just shy of 3 hours long, and some moviegoers complained about side effects from watching it in 3D, from minor discomfort to severe headaches. So what happens to your eyes-not to mention your brain-after prolonged exposure during a 6-hour binge of Halo in 3D? In addition to examining how 3D works in games and its imminent revolution, we ask experts about the possible harmful effects of prolonged exposure to playing video games in 3D.


Liquid Crystal Lenses and the Parallax

In a nutshell, 3D is all about sending a different picture into each eye so that the brain can meld the two images together, creating one complete picture. In the real world, we perceive objects to be in 3D because our eyes send a number of different signals to our brain that interpret the object and its place in the surrounding environment.

Light waves tell us the object's size, shape, and contour (this is easily portrayed in 2D) while distance comes from the angle of your eyes as they converge to focus on the target. The closer the object is, the more your eyes angle toward your nose to see it.

"For objects that are relatively near, our brain uses parallax-the images of objects at various distances as seen by the left and right eyes," says Dr. Charles Falco, professor of optical sciences at the University of Arizona. "Another visual clue for 3D information is size. Since most adults are roughly the same height, if one person in a scene appears to be much smaller than the others, our brain interprets that as due to them being far from us."

Since your eyes communicate depth in several ways, a few technologies help create the illusion of 3D. Active Shuttering is the most popular modern technique. This method uses glasses with liquid crystal lenses that can be electrically switched between opaque and transparent and are used to rapidly block one lens at a time. The television and glasses are synced so that the glasses block one eye at precisely the same moment that the television shows the image intended for the other. In just one-sixtieth of a second, each eye is individually shown a slightly different version of an image. This process repeats up to 120 times per second-so fast that you'd never be able to consciously notice the effect-to produce fluid movement in high-definition 3D.

3D dependant on wearing glasses, however, is likely a transitory stage in the technology's evolution. Most experts believe many consumers will hesitate to spring for an expensive 3DTV-which currently sell for as much as several thousand dollars-until glasses-free models are available. While glasses-free 3D, dubbed "autostereoscopic 3D" (see sidebar "Autostereoscopy: The Elimination of 3D Glasses" on how autostereoscopic 3D works), already exists-in Japanese mobile phones and in the upcoming Nintendo 3DS-a large-scale HDTV version of auto-3D is still in its infancy. Every technique so far has flaws, particularly when viewed in high definition. The glasses-free 3D technology currently in development for HDTVs usually involves extremely limited viewing angles and distances in order to work properly, and others can only be done in very small sizes or poor resolutions.

Master Image President James Bower, whose company has already developed functioning autostereoscopic 3D for products like the Hitachi Wooo mobile phone, says it's only a matter a time before we play 3D video games without glasses.

"There are still some things to work out on a TV screen, but I can tell you glasses-free 3DTV will be released; it will be commercializable; and it will have a great impact in the 3D display business and on 3D gaming," Bower says.

3D and the Damage Done

For the last century, the popularity of 3D has come and gone in waves-with the earliest 3D film The Power of Love (no relation to the hit song by Huey Lewis and the News) arriving in theaters in 1922. Rarely does 3D stick around long enough to generate interest in funding extensive long-term research projects that investigate any potential health ramifications. But in recent short-term studies, Dr. Martin Banks, a researcher at the University of California, Berkeley, found that focused viewing for just 45 minutes can cause eye strain and blurry vision.

He explains the way our eyes focus and converge (the angle of our eyes) is hardwired into our brain to work together. But viewing a 3D image separates the eyes in an abnormal fashion. Your eyes may need to angle themselves toward a point 5 feet in front of the screen to see the content, but since light still comes from the television, your eyes will focus on it. The result is a constant fight between your brain and your eyes.

Even in the modern 3D era's relative infancy, evidence exists of some of the potential dangers of prolonged exposure to 3D. Earlier this year, Josh James, an 18-year-old "straight A student" from Plymouth, Massachusetts, flipped over his car on the freeway several times after watching Tim Burton's Alice in Wonderland in 3D. James claims the movie gave him blurry vision, pain in his temples, and eventually caused him to black out and wreck his vehicle.

The great unanswered question about 3D gaming is what will happen when gamers begin regularly putting their eyes through the rigors of 10-hour-long 3D Halo marathons. "We subjected people to 45 minutes of pretty intensive viewing," Banks says, "and that definitely caused them to experience fatigue and discomfort. I suspect that longer viewing would cause a great deal more discomfort."

Beyond that, Dr. Neil Dodgson, a University of Cambridge researcher of 3D display technology, says that long gaming sessions even on normal TVs are already hard on the eyes. "Your eyes are focusing on a single distance the whole time," Dodgson explains. "That is hard on your vision. You are much more used to constantly flicking between focusing near and focusing far."

It's not all bad news, though. Methods exist for mitigating eye strain. Surprisingly, the onus for protecting the eyes of gamers isn't on the 3DTV manufacturer but rather on game developers. Dodgson says that a restrained use of the technology can ease the strain on your eyes, such as keeping the objects in the game from jumping too far out from the screen or diving too far in.

"Game designers need to be careful about pulling things too far out of the screen or pushing things too far back," Dodgson warns. "In the movies, they vary the depth-budget of different scenes. Many scenes are very shallow in depth, which gives the eyes a rest between the big, deep action shots. Game designers are going to have to learn how to use 3D appropriately. Part of that is learning how to ensure they don't harm gamers' vision with excessive depth."

Page Break

A Storytelling Tool

Modern 3D technology has the potential to reinvent the way we play video games, but only if its integration into gaming makes sense to the consumer and doesn't feel like it's being jammed down their throats. For film, the extra dimension is frequently a bonus, but many believe 3DTV could fundamentally enhance video gaming. But will it be enough to convince people to upgrade their televisions and potentially crop their gaming time considerably to avoid eye strain?

Dodgson thinks so, but he's quick to clarify that as of the present, 3D gaming remains just theoretical potential that no one has realized yet. He notes that film directors have already shown how 3D can be more than just eye candy-and even a powerful tool for telling stories.

"In [the film] Up, the 3D was executed well," Dodgson says. "The director used the 3D to help tell the story. The moody, sad scenes had less depth, while the exciting action scenes had more depth. That is one way to use 3D as a storytelling tool."

Of course, 3D films like Journey to the Center of the Earth prove the opposite can be true-sometimes 3D substitutes for a deficient story line.

Dodgson explains that color was once a similar novelty in film. Once filmmakers learned to use color to actually enhance their work, it quickly gained mass appeal. "We know how to use color well, and we are learning how to use 3D well. We need to learn how to use 3D appropriately so that people enjoy watching films or playing video games in 3D while not being constantly reminded of the fact that they're seeing something in 3D."

The Next Stage

As a storytelling device, 3D may not be something consumers absolutely need, but its potential is exciting. Modern 3D entertainment appears to be the precursor of something far greater than what we have today.

Major advances in television technology tend to emerge every couple of years in order to entice consumers to upgrade. And just as 3DTV becomes the norm, TV manufacturers will undoubtedly be working on developing "the next big thing." But what exactly is 3D entertainment leading to?

"The obvious next step is to move to wall-size 3DTV-the movie theater at home," Dodgson says. "Beyond that we move to wrap-around television-a screen on every wall, making the viewer completely and literally immersed. IMAX has already shown that wrapping the screen around the audience can make for a stunning, engrossing experience."

Jannick Rolland, professor of optical engineering at the University of Rochester, takes this one step further. "If the display of images is coupled with haptic [touch] technology as well, you could use a haptic glove to create the feeling of resistance and force to feel the objects, too," Rolland says.

This may all sound a bit far-flung-3DTVs alone have only just recently become available-but consider that the haptic device Rolland describes is little more than a PlayStation Move controller with force feedback, and a fully encompassing 3D room is only one step beyond available modern technology. What we imagine as the be-all and end-all of futuristic, hyperimmersive gaming setups may actually be a lot closer to reality than we previously thought. All of the seeds have already been planted, and once we learn to avoid the inherent pitfalls of this new technology, it's going to be interesting to see what it blossoms into over the next decade.