Rethinking User Experience in the Era of Mixed Reality
It has been at least two decades since User Experience (UX) became a prominent focus area in technology. It certainly existed even before then, but web design and mobile apps have been crucial in elevating UI/UX to a starring role, as a central component of any product design process. Every interaction between human and machine – every click, page and user flow – is analyzed for efficiency, clarity and consistency. Designers even describe interfaces as being joyful. But despite all the high-mindedness, the basic elements involved have generally been the same – data, screens, mouse-clicks, key presses or touches.
Along the way, users acclimated to new concepts, and grew tired of others. Apple’s Skeuomorphism gave way to Microsoft’s Fluent and Google’s Material Design, putting more focus on efficiency rather than familiarity with older paradigms. Instead of designing for one type of screen, designers now had to provide consistency across web browsers, phones and tablets, incorporating touches, pinches, gestures etc.
Now, the emergence of Virtual Reality – and its descendants, Augmented Reality and Mixed Reality – is redefining both the design and elements involved in how a user interacts with software. First, let’s sort out the vocabulary. Virtual Reality (VR) provides a completely simulated environment with a familiar way of interacting with it. Augmented Reality (AR) incorporates virtual elements into a physical environment, for example a Snapchat filter that superimposes a moustache on your face. AR and VR fit within the culminated spectrum of Extended Reality (XR) – an immersive, three-dimensional, multi-media experience that is touched, felt and experienced rather than simply navigated. Mixed Reality (MR) is the end-product of these technologies; it is a seamless meld of the real, the augmented and the simulated. Pokemon Go is one of the most famous examples of MR; it uses the physical world as a canvas for both augmentation and entirely virtual objects.
Those of us in the software design world pivoted from the term “Interface” to “Experience” many years ago, on the notion that software applications shouldn’t just be a flat two-dimensional world. Back then, it was more aspirational than literal. But now that MR and XR finally deliver on that vaunted goal, what does this mean for the UX designer? Or for the enterprise thinking about a holistic customer experience? Everything, as it turns out. Here are three important considerations:
- XR incorporates a spatial dimension that never existed in UX. Actual physical movement – walking towards or away, leaning in or simple head movement can become essential elements of interaction.
- The XR canvas isn’t a screen, but a three-dimensional world that can be navigated. This broadens the potential for what kinds of complex information can be represented – Information Architecture, in UX lingo.
- New devices and combinations of devices – goggles, gloves, tactile sensors, haptic feedback – challenge and change the very notion of interaction medium. No longer are we in a world of web, desktop, mobile, tablet etc. This is a whole new approach that calls for an entirely new strategy. Just as companies refocused on having a holistic UX or CX strategy, XR strategies will become the new growth area for digital design professionals.
A little more nuanced, but ultimately more significant, XR changes the fundamental role of the user – from a listener, viewer, clicker – to a full-fledged participant. The purpose of an experience is no longer about narrating, guiding or even flowing. It’s one of creating a rich enough environment that empowers the user to easily perceive objects as real and perform common tasks. It requires modern designers to start thinking spatially, and creates harmony between technical and design roles. And lastly, it requires the incorporation of a spatial-focused feedback process involving a wide user base.
These XR trends are evident in many industries. Retailers like Wayfair and Nike are leveraging AR to ease the online shopping experience. Museums are beginning to utilize various MR techniques in their exhibitions to bring history and archaeology to life. And major film festivals now host VR-specific categories. Microsoft has even released a 3D version of their legacy Paint application to foster a new form of creativity. Indeed, the realm of user experience is entering an exciting period.
At 10Pearls Labs, we stay ahead of emerging technologies to distinguish which new tech trends will fizzle and fade, and which will have long-lasting effects in your industry. We study, test, and analyze the impact of artificial intelligence, augmented/virtual reality, machine learning, blockchain, and NLP. More recently, 10Pearls Labs developed separate Microsoft HoloLens apps that feature interactions with navigation services and facial recognition. Contact us to learn more.
