Producing for 3D - what are the challenges?

“3D adds complexity that goes well beyond what we have seen before,” says ADAM LESH, US Chief Technology Officer of Testronic Labs. He reviews the various technical challenges to overcome in order to ensure consumers will embrace the next frontier in home entertainment.

3D technology was developed over 100 years ago with the advent of stereoscopic photography. Taking two photographs simultaneously, slightly offset, and displaying them using a special viewer was the earliest form of 3D presentation. Of course, it was decades later before the introduction of 3D moving pictures with anaglyph as one of the first technologies. Anaglyph employs the familiar red-blue or red-green glasses.

The biggest problem with anaglyph technology is that it uses colour space to separate the left and right images, so there is a dramatic degradation of color in the picture.

A more recent set of technologies involve encoding both the left and right images into a single standard-definition (SD) or high-definition (HD) frame. There are a variety of methods for implementing this approach.

What should be obvious is that, while this approach is far superior to anaglyph, half of each image is being sacrificed, thus, image resolution is reduced by 50% to each eye. Similar to anaglyph, this approach also uses passive, though in this case, polarized, glasses to separate the images and present the proper “stream” to each eye.

BD 3D offers a new approach which does not sacrifice image resolution, called “Frame Sequential,” utilizing “active shutter” glasses.

Typical HD television signal runs at 60 frames per second. With frame sequential 3D, the signal runs at 120 frames per second. When each frame is displayed, one lens is blocked, so only a viewer’s left or right eye is seeing any one frame. By alternating between left and right frames, in the course of a second, each eye receives a full 60 frame HD stream. The result is full HD resolution to each eye making for a spectacular 3D HD experience.



Evolution of BD authoring & testing

Now we know how we got here. In order to understand the impact 3D has had on Blu-ray, it is necessary to offer some background on the evolution of the Blu-ray authoring and testing process.

When Blu-ray first got off the ground just a few years ago utilizing HDMV mode, it was truly nothing more than a glorified DVD. Increased video quality and resolution, and superior audio fidelity, but not much more. So, (and this is a bit of a simplification), for all intents and purposes, the only discipline necessary to successfully create a Blu-ray disc was DVD authoring.

The first significant evolution in Blu-ray came with the introduction of BD-J (Java) programming in menus and games. The discs changed from a “media disc,” best described as A/V content wrapped in a minimal amount of programming, to a software disc, best described as a software project which accesses A/V content. BD-J introduced two whole new disciplines into the Blu-ray authoring and testing process: software programming and game creation.

Next came BD-Live. BD-Live is a client-server application implemented by having a small disc or player resident application communicate across the Internet with a more robust server resident application. Now, we not only have to worry about the programming on the disc, but also the programming on a distant server and the network that connects them.

While evolutionary in nature as far as the consumer is concerned, these updates, from BD-J to BD-Live, have exponentially increased the complexity of both the BD authoring and the testing process.

Now, along comes 3D, and, as we say on our side of the Pond, “it’s a whole new ballgame.”

3D challenges

There are many challenges when creating 3D content, let alone presenting it in the home. Again, some context is necessary before we get into the difficulties with 3D in the home, so we’ll start by outlining some of the issues filmmakers face when creating 3D theatrical content:

Ghosting – Ghosting occurs when the left and right images are not in perfect alignment. An outline or “ghost” image can be seen next to the primary image.

Temporal sampling flicker and motion judder – This has to do with frame rate. 24 fps (film), 30 fps (SD), 60 fps (BD) and even 120 fps (3D HD) are not fast enough to deliver perfectly smooth motion on film or video. We are so used to some flicker and motion judder (jittery motion) that we hardly notice anymore in a theater or on TV. However, when a projector or video device is attempting to synchronize two simultaneous picture streams this problem can be exaggerated.

Maintain depth within and between scene cuts – Focal issues are very challenging in 3D. One of the reasons people can be uncomfortable watching a 3D presentation is when they are forced to rapidly refocus their eyes. It is important to maintain or gradually change focal points throughout the course of a feature to avoid this issue.

Window violations – When watching 2D content, parts of objects or people are often off screen. For example, you may only see half of a person’s face on screen. In 2D, this is not a problem. A viewer’s brain automatically realises the missing half is there and fills in the gap. However, in 3D, if that person is pushed forward (toward the audience), then a viewer will see half a person floating in mid-air. The brain is not so quick to understand this and the viewer can become uncomfortable. A solution to this problem is called a “floating window.” A frame is put in front of the 3D content, which helps the brain process the information in a comfortable manner.

Maximum focal distance – Studies have shown that there is a maximum distance away from the screen toward the audience that is comfortable for a viewer, but that distance is relative to the size of the screen. In a theatre, that distance is about eight feet, but in the home, that reduces to about 1.5 feet. Therefore, if the 3D is optimised for the theatrical experience, it is not necessarily optimised for the home.

Subtitle placement – There are two issues with subtitle placement in 3D. The first again deals with focal distance. If the subtitle depth is dramatically different from the content depth, it can cause eyestrain as the viewer has to constantly refocus to take in the content and read the subtitles. The second is that subtitles have to remain in front of the content or they can literally collide with an object. Put these two together and you can see how challenging subtitle placement can be.

Left eye/right eye synchronisation – This one is fairly clear. If the two streams are out of synch, the 3D will look completely wrong to the viewer.

The above are mostly content issues. When we take 3D to the home via Blu-ray, we open up a plethora of other challenges:

2D backward compatibility – Content owners can opt to have a 3D disc be playable on 2D players, therefore compatibility with legacy players can be an issue.

Left eye/right eye image inversion – If the images are swapped from their intended eyes, it can reverse the 3D effect (objects supposed to be in front of the screen are behind) or cause eyestrain.

Subtitle placement – Similar to the theatrical issue, subtitles need to be carefully placed, but on a smaller screen and with a different technology driving the effect, there are additional challenges for a good experience.

Decoding artifacts in the dependent view – It is possible to test the base (main) view of a 3D Blu-ray in a 2D player because that is the stream that plays in 2D mode. However, the dependent view (second stream) can only be watched in 3D mode making it more difficult to spot compression artifacts.

Quality Control in a 3D world

As before, it is necessary to understand a bit about the 2D processes before diving into 3D. In testing for 2D Blu-ray linear passes are a full test of the video, audio and/or subtitles for all the A/V content. They are typically done on a variety of players as there are usually several passes for any one disc (the number of different audio languages and subtitles determine the number of passes).

Compatibility is performed on a bank of dozens of players to ensure that the disc functions correctly in most consumer households. Functionality tests the menus and buttons to make sure all the programming on the disc is correct.

3D introduced a whole new level into the testing process. Now, there is technology in the displays to decode 3D. The players and the displays “talk” to each other. In order to cover a majority of consumer setups, each player must be connected to each display to make sure they are talking to each other correctly and therefore the disc is behaving the same regardless of which player and/or monitor a consumer might have. As I said above, a whole new ballgame.

In addition, while training has, of course, always been important, additional challenges for the testers include:

The ability to see 3D – In fact, people can perceive different depths of 3D. Some can see very subtle depths, while others can’t. To test 3D content, testers are required to have a reasonably sensitive perception of 3D.

Eyestrain and fatigue – Studies have shown that there are in fact physiological effects of 3D. Testers cannot test 3D content for an extended period of time so thus must be rotated more frequently during the testing process.

“Flattening” – This one only presented itself recently and we have not had an opportunity to truly investigate it. Several of our testers have reported that after viewing 3D content for a couple of hours, then taking a break, and then returning to the testing, they find that the 3D appears to have more depth. It seems, anecdotally, that the testers become used to 3D over time and actually see less depth. It appears that even a 10-minute break can minimise this effect.

Where do we go from here?

3D for gaming has been around for some time and will continue to grow. Already we are seeing a great deal of excitement in the broadcast world as channel after channel jump on the 3D bandwagon. Announcements have also been made for 3D for mobile and expanded 3D content on the Internet.

One question I am often asked is “is 3D a fad (again)?” Of course, it is impossible to say, but the evidence that 3D is here to say is strong. Theatrically, 3D is the “killer app” behind the adoption of digital cinema and is driving higher box office revenues. In fact, on a per-screen basis, the 3D version of a feature earns more revenue at the boxoffice than its 2D cousin. In addition, filmmakers are beginning to understand how to work 3D into the texture of their movies, rather than treating it as a gimmick. Finally, as more 3D content arrives theatrically, consumers will be compelled to add 3D to their home in order not to diminish the experience of how they saw the movie in the theaters.

With all its challenges, 3D has finally evolved into an exciting technology that enhances movie watching experience and consumers appear to be embracing it as never before in its history.

Biography
ADAM LESH is Testronic Labs’ Chief Technology Officer, North America. He has been involved with process improvement for both DVD and Blu-ray QC, developing comprehensive test plans and procedures for BD-Live, Digital Copy, and, most recently, 3D, for a variety of Studio clients, and expanding Testronic Labs’ services to better ensure a positive consumer experience for the company’s clients' products. Contact: adam.lesh@testroniclabs.com...

Article Comments

comments powered by Disqus