Augmented Reality: where does packaged media fit?

Augmented Reality is still a new technology and could well fall by the wayside. However, it can revolutionise the way the market for physical products is seen by the buying public. ANDREW ELIA, President of Arishi Media Technologies, explains why AR provides a means to embrace a digital partnership where physical media and packaging still has a place.

From research my team conducted on Augmented Reality up to last year (when my colleagues wrote a piece in this magazine), there were certain expectations as to how the Blu-ray format would evolve from a commercial point of view and its mass-market acceptance. It turns out the market hasn't quite moved in that particular direction.

Limits of Blu-ray players

Manufacturers of set-top Blu-ray devices have certainly embraced connectivity, but only as a means for transporting content rather than anything more sophisticated. The new top-of-the-range Blu-ray player I bought just a few weeks ago illustrates this point: I can see Twitter feeds, play videos from YouTube and even push my content over DLNA (although not pull it!). None of this is a particular stretch for the scaling and decoding hardware the standard Blu-ray player would use.

Augmented Reality needs horsepower, and current generations of Blu-ray players still don't cut it. There are two main problems:

The first problem is that standard Blu-ray players don't have much processing power outside their dedicated video decoding tasks. The Blu-ray specification itself doesn't give any guidance in this area, hence manufacturers tend to lean towards the lower-end capabilities because it keeps production costs low and margins in place.

The horsepower issue is actually two-fold. The first is the processing required to detect and track an image - usually a raw processor issue, but more solvable as time goes on because these days even mobile phones have the capability to process such images in real time. The second issue is that throwing 3D graphics around the screen often requires dedicated graphics hardware or a bigger and more powerful main processor. The graphics hardware in standard Blu-ray players is often geared specifically around 2D and not 3D. Note that I use "3D" in the context of a three-dimensional shape represented in a 2D space rather than the now familiar stereoscopic 3D images.

The second, more obvious problem is that most Blu-ray players cannot 'see' anything. They don't have cameras, and for the most part, don't need them. Again, the Blu-ray specification doesn't suggest anything in this area. So, once again, manufacturers are understandably cautious about providing this capability or indeed the capability to connect such a device.

In contrast to last year's article in this magazine, I need to redefine Augmented Reality in the context which it is now more likely to be useful to the Home Entertainment industry.

So, what is Augmented Reality?

It's a common misconception that Augmented Reality is purely related to location-based services and compass-driven applications. That is not actually the case because the 'reality' which is being referenced here is only for fixed objects such as buildings.

The origins of Augmented Reality in a real-world environment go back to its use by the military. Augmented Reality technology was used to recognise objects in, say, a fighter aircraft's field of view, and work out what they are and whether or not they are threats. In essence, Augmented Reality identifies moving targets.

You may also have seen other erroneously-labelled references to Augmented Reality in magazines such as Esquire with Robert Downey Jr (below, right) perched atop a hideous black and white square (known as a 'fiduciary marker' among the developers). Here, you are actually manipulating the reality you are trying to identify - this is not Augmented Reality.



A true Augmented Reality application can be taught to recognise an image, a human face or a real-world object. These are exciting areas, where greater opportunities with Blu-ray, DVD and Home Entertainment in general lie. In essence, if an application can be taught to recognise the front cover of a Blu-ray Disc title, it can unlock premium content or enter someone into a competition because there is a proof of purchase. If an AR application can be taught to recognise a real world object, George Lucas would fall off his chair because all the merchandise he's been pushing over the years can then be used to monetise the assets further still with whatever he can offer online. That is a whole new avenue for revenue generation.

The traditional print-based promotional techniques such as overprinting with unique codes are comparatively expensive and not always that easy to control and track. Using Augmented Reality, everyone gets the same packaging as before. There aren't any inserts, or unique codes. The packaging is the code.

The rules change slightly when Augmented Reality is used through a mobile phone as opposed to the more 'traditional' computer-mounted webcam. Because users can potentially be in their local retailer's store, they can conceivably use the application to access the premium content without having actually purchased the product. If it's a competition, organisers can get around this by asking the winner to send in the cover. If users are watching premium content in the store without having bought the product, is it any different from someone standing in a bookshop reading a book they haven't bought before a surly shop assistant growls "this isn't a library."

Consider the benefits of using Augmented Reality in conjunction with a mobile phone. If a user's phone contains an application to recognise your product to unlock premium content or enter a competition, you can then use that application to offer them other content, and cross sell other products. In summary, a Blu-ray Disc pack combined with an AR application does not just provide a portal to access extra content relating to that specific release, but can further engage the film fans by making sure that they are informed of the next releases.

Real applications

Many of the applications that we have implemented are centred around using the packaging of a product in order to unlock extra content, participate in competitions and generally allow the distributors to say "we know you've bought our product, so welcome to the club." This doesn't wave a magic wand and create the content for you. There's still work to be done there. At the simplest level, it provides a credible route to link online with print.

Very substantial reduction in logistics can derived from this. Promotional campaigns which would normally have to be planned several months in advance because of the production process involving packaging and distribution can disappear and be handled online. No need to overprint unique passwords onto the packaging, the packaging itself becomes the password. It enables measuring the promotional exercise online, with the product packaging as the enabler.

An example of Augmented Reality less geared towards home entertainment, but still covering physical (print-based) media is a project we built for Future Publishing's T3 magazine. The publisher wanted to use the technology to give readers access to additional content (shot by our camera crew). Readers holding the magazine front cover up to a webcam saw the editor bursting through it and giving a brief tour of what to expect in the issue. Moreover, it included competitions, extended 'rants' from some of the editorial team and clips from films reviewing in the magazine.



We implemented another AR application to enhance mobile handset manufacturer Nokia's presence at last year's CES show in Las Vegas. Nokia’s booth was split into four sections, and within each section visitors could attend a briefing about a particular facet of the new product and receive a sticker. Upon collecting all four stickers and placing them on the competition entry card provided, visitors could discover on a screen whether they had won by holding the card up to a camera (below).

The clever part here was recognising that all four pieces were in the right place at the same time. The system also enabled Nokia to gather statistics on all entrants, successful or otherwise, for further analysis. Rather cheekily, they only had a limited number of prizes to give out, so we were asked to adjust the probabilities of winning to match that!



Staying on the promotion front, we collaborated with DVD/BD rental-by-mail provider LOVEFiLM and games publishers Ubisoft. The objective was to give a taste of what their latest Assassin's Creed: Brotherhood title looks like. An Augmented Reality application was created that links the disc envelopes that went out to 330,000 members to the website. Apart from engaging users with the content and giving them a reason to connect, users behaviour measurements could be gathered as well. Just with a magazine, how do you know if someone has looked at an advert, read an article or played a DVD? The AR application essentially brought measurability into a medium that doesn't normally sustain it.



What next?

Augmented Reality is still a new technology and could well fall by the wayside. However, it can also revolutionise the way the market for physical products is seen by the buying public. A wholly digital offering is always going to nip at the heels of that business, so Augmented Reality provides a means to embrace a digital partnership where physical media and packaging still has a place.


ANDREW ELIA is the UK's leading expert in bringing marker-less augmented reality technology to major brands. Prior to running his own company Arishi Media Technology, he co-founded and was CTO of Crossplatform. A member of both BAFTA and Mensa, Andrew is a veteran of the new media and interactive industry, responsible for developing some of the earliest ground-breaking internet-based projects. Contact: www.amediatek.com/

This is one of many editorial features included in the annual DVD and Beyond 2011 magazine. Ask for your free copy....

Article Comments

comments powered by Disqus