Home ›

Augmented Reality with HTML5

Issue 203

From Issue #203
March 2011

Apr 30, 2011  By Rick Rogers
 in
How far can HTML5 go when writing mobile applications?

In a previous Linux Journal article (“Developing Portable Mobile Web Applications”, September 2010, www.linuxjournal.com/article/10789), I looked at HTML5 and how it could be used to write applications for mobile phones. The techniques presented in that article work well for applications that use text, buttons, images, audio and even video, but what about cutting-edge applications that stretch the envelope of what mobile phones can do? In an effort to find out, I decided to implement a rather simple mobile augmented reality application, doing as much as I could in HTML5. This article explores the techniques for extending JavaScript capabilities to write applications that do more than is possible with standard HTML5.

Augmented Reality

Augmented reality (AR) is the name given to a class of applications that combines the unique capabilities of mobile phones to extend users' perceptions of their environments. Layar (www.layar.com) was one of the first AR applications, and it's still one of the more creative. Augmented reality overlays the current camera preview screen with additional information—you can see examples in this YouTube video: (www.youtube.com/watch?v=A6Le50-QN3o&feature=player_embedded). Figure 1 shows what Layar looks like when the “Starbucks” layer is loaded and the camera is pointed at a mall where there is a Starbucks coffee shop.

Figure 1. Layar with the “Starbucks” Layer

This application makes use of a number of mobile phone features:

  • Camera preview.

  • Compass (direction the camera is pointed).

  • Location.

  • 2-D graphics (for the overlay).

  • Database capabilities.

Layar is a very advanced application, with many options to make it easy to use. Again, the essential nature of AR is that the user sees additional information superimposed on a camera preview.

HTML5 Extensions

How would you implement this kind of application using HTML5? For the sake of creating an example application, let's reduce AR to a simple case: show the current camera preview on the user's screen and superimpose the current compass direction on top of the preview. Let's also animate the compass card so it moves as the phone's camera pans around. In principle, the overlay could be anything, but a compass card is a start.

HTML5 has greatly extended the capabilities of HTML applications, but some things still are missing for this application:

  1. HTML5 doesn't include a compass API. You need a way to access the mobile phone's current compass direction and receive periodic updates as the direction changes. You could use the API in one of the Web app toolkits (such as PhoneGap or Titanium) for this, but let's create our own interface and demonstrate how you can access just about any Object from JavaScript.

  2. You need a live camera preview on the screen, and there isn't a camera API in HTML5. Extensions to HTML5, such as WAC (Wholesale Applications Community, public.wholesaleappcommunity.com), are defining APIs for camera preview, but there are no WAC mobile phones yet.

  3. In order to add your own HTML5 extensions to a mobile platform, you have to do some platform-specific code. That means you have to give up some portability, but let's accept the trade-off and focus on one platform, Android. Let's create the needed Dalvik/Java code to implement this simple AR application and take a look at how JavaScript can call Dalvik methods and vice versa.

The ARCompass Application

The application will be a hybrid Dalvik/HTML5 application. The HTML5 part will run in a browser. Android applications create an Internet browser view in one of two ways:

  1. Issue an Intent with the URL to open, and Android will resolve that Intent by opening the browser application and passing it the URL. When you exit the browser, control is returned to the calling application. This approach works fine for regular HTML5 applications, but it doesn't provide a way to add new interfaces to JavaScript.

  2. Inflate a WebView and pass it the URL. There is a lot more flexibility in the WebView compared to the browser application, including a public method, addJavascriptInterface (Object obj, String InterfaceName). This method lets you create your own JavaScript APIs for the scripts run by a WebView. Note that there is a bit of a security hole here—anything you make visible to JavaScript can be accessed by any JavaScript script run by this WebView, whether or not you wrote the script. You want to be sure the user can't navigate to random Web sites that might misuse your interface. In this case, let's include the HTML and JavaScript files in the application and not provide the user any chance to navigate away.

Let's write a Dalvik application that shows the camera preview screen and overlays that with a WebView that will draw and animate the compass card. Of course, you'll also need the compass information passed from Android back to the HTML5 code, so it can animate the card properly.

Assuming you've already loaded the Android SDK (from developer.android.com), you can follow along by downloading the ARCompass.prj project file and the HTML and JavaScript files from ftp.linuxjournal.com/pub/lj/listings/issue203/10920.tgz.

______________________

Rick Rogers has been a professional embedded developer for more than 30 years. Now specializing in mobile application software, when Rick isn't writing software for a living, he's writing books and magazine articles like this one.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Missed the latest developments?

Anonymous's picture
Submitted by Anonymous (not verified) on Fri, 04/29/2011 - 06:31.

It's a shame that you seem to have missed this:

http://my.opera.com/core/blog/2011/03/23/webcam-orientation-preview

HTML5 is awesome!

Mathuseo's picture
Submitted by Mathuseo (not verified) on Fri, 04/29/2011 - 05:43.

Absolutely awesome what HTML5 makes possible. As webworker I'm happy about the features that will follow in next years! Nice posting, I will twitter the post here.

Webcast
More Resources
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers

Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.

Learn More

Sponsored by AMD

White Paper
More Resources
Red Hat White Paper: Using an Open Source Framework to Catch the Bad Guy

Built-in forensics, incident response, and security with Red Hat Enterprise Linux 6

Every security policy provides guidance and requirements for ensuring adequate protection of information and data, as well as high-level technical and administrative security requirements for a system in a given environment. Traditionally, providing security for a system focuses on the confidentiality of the information on it. However, protecting the data integrity and system and data availability is just as important. For example, when processing United States intelligence information, there are three attributes that require protection: confidentiality, integrity, and availability.

Learn more about catching the bad guy in this free white paper.

Learn More

Sponsored by DLT Solutions