<

Designing a Mixed Reality game

My graduation project at Mirabeau was the last project for my Bachelor's degree Communication and Multimedia Design. For this project, I explored new areas within the field of UX design and tried new prototyping techniques.

Goal

Find out in what kind of way emerging technologies can be used during long haul flights of KLM to create a more comfortable experience for its passengers.

Challenge

The biggest challenge of this project was the limited amount of space that passengers have during flights. Besides that, designing and prototyping for Mixed Reality was a huge challenge. I couldn't find any examples on how to prototype for Mixed Reality, so I thought of my own ways and experimented. You can read more about prototyping for Mixed Reality here.

The research

I started by doing research about the target group and their experiences, the airline KLM, and what other airlines offer their passengers during flights. Simultaneously, I created the first concepts. After the first concepts were clear, I did more research about the possibilities and limitations of XR technologies.

To understand the circumstances on long haul flights, I went to New York to experience it myself and to observe other passengers. The first flight was during the day and the second flight was during the night. This variation on the Service Safari method resulted in a customer journey, which can be seen below (in Dutch).

Photo from the Customer Journey of long haul flights


All the research I did resulted in four guiding principles.

Guiding principles

  1. Reality

    Provide the user with access to another reality to let him forget he's on a long haul flight.

  2. Flight tracking

    Use a flight tracking element in the application.

  3. Comfort

    Reduce the amount of stiffness and cramping of the body of the passenger.

  4. No annoyance

    Make sure the application can be used without disturbing or annoying other passengers in any way.

Concepts

To get a clear overview of the work I had done so far, I listed the insights I got from the research using a form of the Google Design Sprint method. Keeping the insights in mind, I formulated some How Might We questions. By answering these questions, I created the first ideas of concepts. The process from insights to concepts can be seen below.

Insights to concepts, Google Design Sprint

Insights to concepts, Google Design Sprint

Final concept

The final concept is SPECTRE, a mixed reality game that can be played on the Microsoft HoloLens in which the user needs to solve a case by looking for evidence through a 3D model of a crime scene.

The core game mechanic is: Find evidence by using different tools to solve a mysterious and secret case and to complete levels.

The scene that has been further developed is Unidentified Cargo. The story is about a mysterious team who asks the user for help by solving the game. All that the user will know is that an unknown pilot stole a cargo plane. The user needs to investigate the cockpit to find evidence which can help him identify the pilot.

Prototyping

I wrote an article about prototyping for Mixed Reality, which also includes a detailed description about how I prototyped the game during this project. This is why I will only show a small part here. If you are interested in prototyping for Mixed Reality and how I did this during this project, you can click here.

At the start, I knew that I needed 2D and 3D models to make the lo-fi prototype happen. I made the 3D models out of Lego, styrofoam, and real objects such as a notebook. After this, I made sketches of the 2D interfaces I wanted to create. I transferred these designs onto transparent plastic sheets to create a hologram effect. For the hi-fi prototype, I made 2D interfaces in Blender. The different stages of designs can be seen below.

Three different stages of designs. On the left a sketch, in the middle the design for the lo-fi prototype and on the right the design for the hi-fi prototype.

The three stages of design

There is one thing though, I haven't really explained how the guiding principles are part of the concept. You might ask yourself the following questions: Mixed Reality is not a completely different reality, so why will the user forget that he's on a long haul flight? How is it that this Mixed Reality game can work in such a small place? And in what kind of way does the game contribute to the comfort of the user?

Well, the storyline of the game has been written in a way that the user really gets involved and feels important. The user doesn't bother somebody else, because there's a remote that's used to click. The user also needs to look around, which reduces the amount of stiffness of the neck and will loosen up the body.

End product

To finalize my ideas into something visually appealing, I decided to make a video. All of the designs are made in Blender, a 3D modelling environment. The video itself has also been rendered in Blender.

WARNING! Prepare yourself for hearing my voice before watching the video.