,Note: This article was originally published on LinkedIn. If you enjoy my article, please click through and consider sharing on social media and connecting with me!
Evolving AR from the tabletop to the great outdoors. Bringing the streets of 1941 Lochee to your street. Emulating a museum experience using just a tablet. These are just some of the challenges I worked on – and I’ve developed a framework to help you do the same and more.
Following this framework allowed us to get close to emulating the kind of interactive storytelling experience we studied, for a fraction of the development time and cost of an actual museum exhibit.
Here’s what you can expect from this article:
I identified 9 major steps that can be implemented in an iterative process to progress from a basic to fully immersive AR game.
I’ll explain my process for arriving at this framework and then introduce the framework itself, along with a detailed breakdown of how I applied it.
I’ll reflect on the suitability of this framework and its application and how it can potentially be improved and adapted for further use.
Plus: I’ll share a design breakdown I produced of a state-of-the-art digital interactive museum exhibit.
Plus: I’ve shared links to great resources to help you do the same using Unity AR Foundation
A New Business Model And A Passion Project
In mid-2020, I started my first official project at InGAME: designing and developing a prototype historical and cultural Augmented Reality experience for the iPad.
Our partner and challenge-holder, TPLD, had come to us with an idea they had for a new business model. They wanted to explore creating mixed-reality experiences for museums and the tourism sector. They wanted us to help them further define and de-risk this idea, so they could turn it into a business model and eventually start a new venture.
They had in mind a perfect way to validate this idea – a little passion-project-prototype, recreating 1941 Tipperary, Dundee, based on one of the challenge holder’s own childhood memories!
Goals
Our objective was to create an immersive and interactive 3D VR / AR application that gives people the sense of what it was like to live, work and socialise in 1938 in an area of Dundee known locally as Tipperary.
We needed to develop a process or framework which could successfully produce a project meeting the following criteria:
1. Recreating 1941 Tipperary, Lochee, Dundee, optimized for iPad 2020
2. Telling fictional stories based on authentic historic material and artefacts
3. Producing a working AR technical demo for the iPad 2020
4. Allowing the player to play the role of a fictional character in the chosen setting
5. Providing an experience close to that of state-of-the-art digital museum experiences
6. Immersing players in this recreation of the chosen setting
7. Exploring outdoors AR
8. Exploring Unity AR Foundation’s available features
Our Process For Designing This Framework
We needed a viable, straightforward, and reliable process to design this experience. They were some unknowns and challenges that helped form this process.
First, the type of experience had to be considered. There have been projects about applying Games and AR technologies for cultural museum experiences, such as experiences that allow users to view, process, and even curate cultural artefacts (Pollalis et al., 2017 - HoloMuse: Enhancing Engagement with Archaeological Artifacts through Gesture-Based Interaction with Holograms.).
However, projects like that are often limited in scale and functionality to single objects on a table or on a shelf and don’t easily provide opportunities for immersive storytelling.
The nature of the experience we wanted to give our users seemed best suited to outdoors AR. This led us to prefer developing our own ad-hoc process for designing the game. A few challenges I considered are:
1. What kind of experience are we trying to design for users?
2. What’s currently possible with AR on our target platform?
3. What kind of data and artefacts do we have about the story we want to tell?
Of course, in addition to this, there are some general questions like project scope, available resources, and timeline.
The answers to these questions helped form a process that naturally led to informing the AR Experience Design Document.
Validating And Understanding The Need - Uncovering Problems With Traditional Museum Experiences
The COVID pandemic has caused major disruptions for museums - most obviously by greatly reducing footfall as well as highlighting a need for updating infrastructure to support current needs.
In museums, often the volume of information can be overwhelming and tiring. Improvements are needed in the manner and medium of delivering this information.
There are three major factors that make digital AR experiences for museums an attractive proposition:
1. Location and accessibility
What we’re exploring here is similar to the concept of “distributed museums”. By not limiting museum experiences to the physical museum space, we make experiences available to online audiences and make them more sustainable across platforms, communities, and time.
(Proctor, 2017 - 'The Museum as Distributed Network, a 21st Century Model').
Historical parks and monuments make for good targets for digital or online museums – due to being physical sites of interest that may be lost over time.
2. Cost of designing and developing a digital museum exhibit
The Smithsonian found the direct costs ranged from $25 to $6,500,000 per exhibit, and that their costs are similar to those at other museums across the United States (Smithsonian Institution Office of Policy and Analysis, 2002 - The cost and funding of exhibitions).
Comparatively, the development of an AR app may cost between a few thousand dollars for a straightforward AR app to $300,000 for a feature-rich one (Lavrentyeva, 2021 - Augmented reality cost: key factors and real-world examples.).
3. Better experiences with increased immersion
Mixed Reality offers a unique tool for people to study the past and get an “immersive” feeling.
Today’s high-end devices are likely to be affordable and mass-produced in as little as a few years’ time, which makes them good targets for R&D work like this.
Step 1: Understanding Existing Experiences We Wanted To Emulate: Design Fundamentals Of Interactive Digital Museum Experiences
I started by looking at state-of-the-art museum experiences, to understand what they were offering. I wanted to break down their design and see if that could be replicated using AR.
Several state-of-the-art digital museum experiences focus on adding a vivid narrative in which visitors assume the role of an interesting character that suits the experience.
The experiences are designed like linear story-based gameplay and are also similar to theme park experiences and rides. Game-like role-playing and storytelling is brought to life using a mix of custom hardware and large screens. There is a strong use of Interactive menus on physical displays, pre-recorded voice and video, and immersive displays that make you feel like you are on-site, living the lives of the people in the story.
These experiences are likely to have multi-million-dollar budgets.
Design Breakdown: Ideum: Penguin Chill Habitat At The Albuquerque BioPark
Source: https://ideum.com/portfolio/penguin-chill
Ideum designed and built exhibitory that complements the experience of watching live penguins by adding a vivid narrative in which visitors assume the role of researchers bound for the Antarctic.
I’d describe it as a high-budget game or experience with multiple hardware/technology working together. The experience flows like linear gameplay or a theme park ride.
After some critical analysis of a video of the experience online, here’s what I could glean:
A clear narrative with role-playing: Ideum chose the narrative for this experience to take the player through a guided role-playing experience. Visitors take the role of researchers on an expedition on the icy shores of Tierra del Fuego.
Setting the context: The experience starts with an interactive menu with videos that introduce you to researchers.
AI NPC interactions: The exhibit lets you “make virtual video calls to real-life contributing scientists to learn more about their research”. Basically, leveraging human-like AI interactions to support storytelling.
Recreating realistic equipment: The science stations are built to look like a shipboard command centre.
