By Deb Boyer
Using technology to provide access to the collections held by cultural institutions is nothing new. Using augmented reality, a technology often equated with science fiction movies, to provide that access, however, is much newer. But how does augmented reality actually work? What are the benefits and challenges of developing an AR project? Can this technology be used effectively in cultural institutions?
To address these questions, the City of Philadelphia Department of Records (DOR) applied for and received a Digital Humanities Start-Up Grant from the National Endowment for the Humanities in 2010 to develop the use of augmented reality to display historic photograph collections (see Figure 1). Using images from PhillyHistory.org, a Philadelphia City Archives website that provides access to more than 95,000 photos and maps held by several different institutions (see Figure 2), DOR wanted to create a prototype smartphone application (also known as an app) that would enable users to point their iPhone or Android device at a specific location and see historic photos of that location overlaid on the view visible through their phones’ camera – a technique known as augmented reality (commonly referred to as AR). With this grant, DOR hoped to create a new way for people to view these thousands of images and introduce the photos to new audiences, as well as research the potential uses of augmented reality in cultural institutions.
AR overlays digital data or computer generated imagery on a live view of the physical world, thereby enhancing or “augmenting” what a person can see by providing additional textual or visual information. Research into AR reaches back decades and has resulted in a variety of mobile and desktop-based projects. The recent development of smartphones, however, has dramatically increased the number and type of augmented reality applications available to the general public.
AR technology can be broadly divided into two categories: computer vision AR and sensor AR. In computer vision AR, a user visits a web page designed specifically for the AR experience and holds a marker (a simple image printed on paper) in front of the computer’s webcam. Using computer vision algorithms, the computer recognizes the marker and displays a digital version of the 3D object it has been programmed to associate with that marker. As a user rotates and moves the marker, the 3D object also rotates and moves. The Getty Museum used this type of AR to create an augmented reality version of a seventeenth-century German display cabinet. Users first download a marker from the Getty Museum website and print it on regular printer paper. Then, by holding the printed marker in front of their computer’s webcam, they can view and manipulate a model of the cabinet, thus providing a new and personal way to interact with the museum object.
While computer vision AR allows users to view objects in augmented reality, DOR wanted to use AR technology to enable people to view historic photos of Philadelphia while standing in the places where they had been taken. Sensor AR, the second category of augmented reality applications, is designed for mobile phones and was better suited for this purpose. The Department of Records partnered with Azavea, the Philadelphia based software firm that originally built the PhillyHistory.org database, to research and develop a prototype sensor AR-based app.
Sensor AR uses the global positioning system (or GPS), compass, WiFi, accelerometer, and other sensors available on smartphones to identify both where the phone is located in space and where it is pointing. When a user points the phone at a particular location, the AR app consults its database of available augmented reality points and returns data related to the location visible through the camera view of the phone. For any location with available data in the PhillyHistory AR app database, a number of default images appear as overlays on the screen. The user can also load additional images by selecting various icons. As shown in Figure 3, the user can view a large historic image of the Girard Trust Company building located at Broad and Chestnuts Streets or select the red PH icon to minimize the Girard Trust image and load another photo taken near that same location.
Nearly 87,000 images from PhillyHistory.org are accessible via the AR app, each accompanied by a title, brief description, the date of the photo, and the name of the collection to which it belongs. Five hundred select images are also “pinned” in 3D space. In addition to being associated with a particular location (via latitude and longitude coordinates), these images are also oriented in 3D space using the coordinates of the four corners of the image. Figure 4 depicts a standard image (on the left) and a pinned in 3D space image (on the right). The standard image will always face the user head on while the pinned image appears at an angle more closely aligned with the original angle from which the photo was taken. This more accurate orientation in space can assist users in seeing the relationship between the present and past views. Local historians, including the editors of The Encyclopedia of Greater Philadelphia, a project based at MARCH, have also created descriptive text for twenty images as a way to highlight certain themes and locations in Philadelphia history.
Sensor AR is one of the most common AR techniques, and several commercial providers such as Layar (used in the PhillyHistory app), Wikitude, and Junaio provide basic frameworks to create augmented reality apps. Several cultural institutions have used these frameworks to create AR apps that build on the place-based and mobile digital humanities projects that have become prevalent in the last decade. A non-comprehensive list of sensor AR apps completed by cultural institutions includes:
Hoppala and Superimpose, two augmented reality groups in Germany, developed the Berliner Mauer AR layer, enabling users to stand where the Berlin Wall was formerly located and view a recreation of the wall.
Launched in May 2010, the Museum of London: Streetmuseum app draws upon hundreds of images from the museum’s collection and enables users to see past views of a London location via smartphone.
The Andy Warhol Museum has created a layer in Layar that allows users to view locations in Pittsburgh and New York City connected to Warhol’s work and life.
The Urban Augmented Reality app by the Netherlands Architecture Institute not only shows past images of buildings but also includes design drawings and models for construction that never materialized as well as depictions of projects under development.
These and other newly created humanities-related AR projects indicate an increasing focus on AR as a method for connecting people to the work and collections of cultural institutions. Collectively, these projects have several advantages that can advance the work of digital humanists and public historians:
1. They generate excitement. For many people, AR still sounds like science fiction. Individuals who may not consider themselves “history” or “museum people” suddenly express interest in historic photographs or digital collections if they can access them via AR.
2. They provide access via smartphones. As the use of smartphones grows, these apps provide opportunities for users to access an organization’s information, images, or collections as they have time amidst their daily tasks.
3. They support an interest in place-based history. People often feel strongly about a place that figures in their personal or community history. This connection to place can be enhanced through an AR app by encouraging people to link the past and present of a particular location and think about the events or people associated with that location.
4. They create educational opportunities. AR opens up intriguing possibilities for site interpretation. An AR app can be a new form of a self-guided tour or enhance a docent-led walking tour. Educators can use an AR app to help students understand how a location has both changed over time and remains the same.
Nonetheless, augmented reality development is not without its challenges. It is, first of all, a rapidly changing technology, resulting in changes to the available AR frameworks. During the creation of the PhillyHistory.org app, for example, Layar would occasionally introduce new features that would effectively break the PhillyHistory app and require developers to fix a feature that had previously worked without error. Secondly, AR app development requires a certain level of technical knowledge. AR frameworks such as Layar, Wikitude, and Junaio provide some guidance and serve as a good starting point, but more advanced development may require a challenging level of technical expertise. Sensor AR, while increasingly common, still suffers from usability issues related to the quality of a phone’s hardware, making use of the app often very frustrating. The requisite use of a smartphone to view an AR app also serves as an inherent barrier to access, an issue that can be problematic for many institutions.
One of the larger challenges of using augmented reality is going beyond the initial excitement the app generates to encourage a more thorough examination of the collection upon which it draws. Experimenting with new outreach, engagement, and collections management activities is absolutely essential, but if AR is to be more than just a flashy, one-time experiment, cultural institutions must use it and other new technologies in ways that further their mission and goals. Organizations should consider how they might build a relationship between the institution and the new audiences attracted by the app.
For example, the Museum of London’s Streetmuseum app introduced the museum to many individuals previously unaware of the institution or its collections. Since May 2010, the app has been downloaded more than 200,000 times by individuals around the world. The museum’s mission is “to inspire a passion for London and a passion for learning,” which is done “through increasing public awareness, appreciation and understanding of London’s cultural heritage, its people and its stories.” The public interest and excitement created by the Streetmuseum AR app address several key components in the organization’s mission and demonstrate how this new technology can assist in fulfilling broad institutional goals. In the mid-Atlantic region, Dickinson College created an augmented reality tour of Carlisle, Pennsylvania as part of its House Divided project, which provides teachers, students, and the general public with access to resources on the Civil War era and the wartime experiences of Dickinson students and graduates (see Figure 5). The inclusion of an augmented reality tour of downtown Carlisle as part of this project vividly – and visually – demonstrates the relationships among Civil War era activities, the college, and the local community. If these and other projects currently in development are any indication, augmented reality technology is likely to become increasingly common as a tool for outreach and education within cultural institutions.
To learn more about the creation of the PhillyHistory.org augmented reality application and the use of AR in cultural institutions, consult “Implementing Mobile Augmented Reality Technology for Viewing Historic Images,” a free white paper summarizing the research conducted as part of the Philadelphia Department of Records’ Digital Humanities Start-Up Grant. The paper is available at http://www.azavea.com/augmented-reality.
Deborah Boyer is a project manager at Azavea and also serves as the project manager for PhillyHistory.org.