Origin Consulting

Experience This Blog

Experience This Blog provides insights and ideas about product, digital, and place-based experience design and explores creating for the human experience.

The 5 S Words of Designing for Mixed Reality: Day 1 at AR in Action (ARIA)

Mixed reality is fast-evolving with new technologies that will vastly change the landscape of industry and people’s everyday experiences. ARIA, hosted at the MIT Media Lab, brought together individuals working with augmented reality across various sectors to discuss new products, emerging trends, and the impact mixed reality could have on our society. The conference takes place Tuesday, January 16 to Wednesday, January 17, 2018 and inspired this article.

From a human experience vantage point, our lives and how we engage are going to change significantly when augmented reality becomes more accessible, enabling widespread consumer adoption. From opening your refrigerator in the morning to notifications of food expiry dates to your bike ride to work paved with AR navigation indicators (check out Solos Smart Glasses) to your job in whatever field you work in (Some examples; AR assisted surgery, diagnostic displays, AR construction site walkthroughs of architectural models, the list continues..); mixed reality is going to transform the way we interact with information and each other.

Many people can’t imagine a world without smartphones because of how much they currently play a part in our lives, but this was a topic discussed by multiple speakers at ARIA’s first day and raised the question of how we design moving into a mixed reality world. MIT Media Lab  academic head and professor Pattie Maes envisions access to information that currently exists online and in our phones becoming more integrated into our environments so we are not as easily pulled from in-person interactions; where you don’t have to look down to recall information or send a message.

So when considering user experience design and building for the people who are going to use these products, we need to reassess the inputs and indicators we use in AR. For a long time, technology has distracted people from experiencing the “real world” because of the interfaces that currently exist to engage with the technology. With mixed reality, many people are eliminating the in-the-box constraints of the smartphone and thinking of new and innovative ways to store, process, and present information.

5 S Words of Designing for Mixed Reality

Here are 5 S words to consider when designing for mixed reality that prioritize the end user. After all, we are building these products for people.. Right? (there’s a whole different post opportunity for that conversation):

1. Sight

Sight is probably the first sense that comes to mind when designing an interface. Visual cues and indicators are key factors to consider when developing applications in augmented reality. Color and vibrancy are also important to take into account when building interfaces that overlay on complex and changing environments. Rui Pereira teaches a class called Magic Windows and Mixed Up Realities at ITP-NYU and is a creative technologist at Google. At ARIA, he spoke about his students work to help you navigate your way through a city guided by Alice and Wonderland’s white rabbit, animating brands by bringing logos to life, and more. Augmented reality opens up an entirely new dimension for visual experiences that excite, aid, and empower.

2. Sound

Neil Gupta, a partner at Indicator Ventures and founder of nonprofit BostonAR, gave a lightning talk at ARIA that discussed the opportunities for AR and AI to consider sound, pattern, and pulse to support indicators in mixed reality interfaces and theorized Apple may be placing a strong focus on this for development of new technologies in the augmented reality space.

3. Sensation

For anyone who has used a smartphone, haptic feedback is a fundamental characteristic of mobile design. The duration and pulse of a vibration down to microseconds can be interpreted by the user to mean different things. So what does sensory feedback look like if there is no more screen? No more device? For the innovative and futurist readers out there, should we include taste as a part of this category? What other sensory indicators could be used as an anchor when designing for mixed reality?

4. Scent

Scent has been shown to enhance and influence people’s memories of past events and have body-altering effects such as reducing anxiety. Pattie Maes referenced “emotional regulation” as an activity AR will play a role in (check out the student project Essence, the first olfactory computational necklace). For readers who watch Black Mirror, she did say that she makes her students watch the show to consider the implications of some of these technologies and how they could have negative effects on society.

5. Sequence

One of the projects mentioned at the MIT Media Lab is called NeverMind. This “memory palace memorization method with augmented reality” helps support long-term memory recall using sequenced items as the user moves through different environments. The factor of time and sequence is a significant consideration in the advancement of mixed reality products.

Moving into the future

As you consider these 5 S words of designing for mixed reality environments, consider how people may hack your product and take time to explore the moral and societal impact of your creation. And, as the AR in Research panel agreed, designing for failure is a great way to build impactful products and help you stay innovative. As always, integrate your end user into the design process as early as you can and prioritize understanding what experience they want and how to deliver that to them.

Reflecting on today, I would like there to be a larger philosophical dialogue around “if we can build it, should we?” Rather, “since it’s going to be built, how do we handle it?” Mixed reality can pave the way for incredible advancements in society; from medicine to infrastructure to education, but it could also expand the doorway to invasion of privacy, lack of control, unequal access, and other factors where the technology could be abused.


Looking to build and launch a product of your own? Read How to Design a Customer-First Product Experience for insight on building impactful products your consumers want.

Joshua Croke is an innovation consultant and creative strategist working at the intersection of design, technology, and society. Engage with Josh on Twitter @JoshuaCroke.

Origin Consulting is an innovation consulting and creative strategy studio. We use human-centered design to build identities, craft experiences, and deliver impactful products and services to consumers. To work with Origin, give us a shout.