What is XR? [In theory]
From Milgram’s continuum in 1994 to the three-dimensional framework of 2021: what thirty years of research actually change in the way you design an immersive experience.
- XR
- Theory
- UX
Hello to everyone curious about the world of XR! 👋
I have told you what XR is in practice; today, let’s talk about XR in theory! Behind all these fashionable terms sit solid scientific concepts that will help you design better immersive experiences.
Hold on tight, we are diving into theory… but the accessible version! 😁
📚 1994: Milgram and Kishino lay the groundwork
In 1994, two researchers, Paul Milgram and Fumio Kishino, created a model that is still a reference today: the reality-virtuality continuum.
Their idea?
Picture every immersive experience on a straight line, with:
- On the left: the 100% real world (your desk, your morning coffee)
- On the right: the 100% virtual world (a video game, a universe built from scratch)
And in the middle?
All of Mixed Reality! That zone where the real and the virtual meet and interact, and inside it:
- Augmented Reality: your real world with digital elements laid on top
- Augmented Virtuality: a virtual world that takes in elements of the real one
This linear view was revolutionary at the time, but today it is being questioned:
🔄 2021: the shift brought by Smith, Whitton and Skarbez
In 2021, three researchers - Missie Smith, Mary C Whitton and Richard Skarbez - put forward a new approach.
Their 3 main objections:
1. The continuum is not all that continuous
Their observation: "perfect virtuality" does not really exist. Even in virtual reality you keep your physical body, you sometimes hear the real world... So everything is a little mixed, in the end! At least through senses other than sight...
2. Redefining Mixed Reality
For them, MR is no longer limited to "sticking the virtual onto the real". It is subtler than that: it is how the user perceives the blend of real and virtual.
A concrete example: when you are playing in virtual reality and instinctively avoid a virtual wall, your brain is blending real perception (your body) and virtual perception (the wall). That is mixed reality!
3. Simplifying the terms
Their bold proposal: call everything "mixed reality" to put an end to the confusion between terms.
📐 The new approach: a three-dimensional framework
Farewell to the straight line! The three researchers propose a three-dimensional framework for making better sense of XR experiences:
📍 Extent of world knowledge
- What it measures: how well the experience “understands” your real environment.
- Example: an augmented reality application that recognises your furniture, versus one that sees nothing and drops objects anywhere.
🎯 Immersion
- What it measures: how engaged you are in the experience.
- Example: simply watching a scene in XR, interacting with a few objects, or living the experience from the inside as a participant.
✨ Coherence
- What it measures: how much the virtual “makes sense” alongside the real.
- Example: a virtual object that falls naturally to the floor, versus one that floats unconvincingly in your living room.
Why this is so useful for us, as designers
This framework gives us 3 clear levers for improving our experiences:
- World knowledge: how does my app understand the environment?
- Immersion: how do I engage my user?
- Coherence: how do I make the experience believable?
🎯 What it changes for your projects, from a UX point of view
1994 version (Milgram & Kishino)
- ✅ Easy to grasp
- ❌ Too rigid for today's complex experiences
2021 version (Smith, Whitton & Skarbez)
- ✅ More nuanced and more realistic
- ✅ Finer analytical tools
- ✅ Suited to current technologies
My view as a designer: use the practical explanation with your clients, and keep the theory for your own grounding and for your projects.
Project tips:
- ✅ Rate existing apps on the 3 dimensions to spot where they could improve.
- ✅ Each dimension gives you concrete ways to improve your prototype
💡 My view as a designer
What I take from it: these theoretical shifts reflect a simple truth — XR keeps growing more sophisticated, and the tools we use to understand it have to keep up!
How I work with it:
- Explain the practical definition to my clients
- Start from the classic continuum to get my bearings
- Use the 3D framework to sharpen the analysis
- Always keep the user at the centre of the thinking!!
→ a spoiler from the newsletter on design thinking: this analysis belongs in the define phase, built on what came out of your empathy phase
🎓 Ready to put these concepts to work?
Now that the theoretical fundamentals of XR are yours, you are equipped to create genuinely considered experiences!
My next pieces will guide you step by step:
- 5 fundamental principles for starting well in XR
- My favourite tools for prototyping quickly
- The keywords of XR: your guide to starting out with confidence
So, tell me…
- Which framework seems the most useful for your own projects?
- Have you already looked at an XR experience from this angle?
- Any questions about putting these concepts into practice?
Share your thoughts in the comments! And if you would like to dig into these theoretical aspects on a real project, let's talk!