Let's paint ourselves a world, a few decades from now, where smart phones are seen as a relict of the past. Where technology is more immersive than ever before. Where digital elements merge with our factual experiences. Whether we are wearing devices or have them incorporated into us, whether they are altering visuality, enhancing our visual experience or enriching it with information. Whether it is our visual perception or any other form of intake. 

We call those enhancements phygital layers. We call this environment a phygital one. Immersive technology blurs the lines between physical and digital experiences.

Everybody uses these layers — as long as they can afford them. Software providers create packages, that alter peoples perceptions. These packages can be purchased online. Extend and sort of packages used is free to decide by the consumers themselves.

Exploration and Research

The goal was to address a variety of social and emotional conflicts imposed by the phygital technology, while leaving room for interpretation and judgement. Where does the technology come from? Who owns it? Does it have the potential to create a positive image? Does it ultimately lead to a degradation of human society?

We experimented with the concept of multiple points of view. Starting with a written story about two women who meet by chance in a coffee shop and are unable to converse because of their different beliefs towards existing perception-altering technologies.

We moved from sequential to parallel points of view, while at the same time abandoning the black-and-white idea of „one person uses the technology to a high degree ↔ another does not use it at all“. Because we believe that the realistically interesting nuances of interpersonal interactions can be found in relativity rather than extremity.

Storytelling

For the final project video, we created four characters who were together at university and meet again after a few years. They all use the technology, but differ significantly in their degree of immersion and dependency:

Our focus is on the resulting person-to-person (p2p) relationships. But we need to remember that these are heavily shaped by each individual's own hyper-individual reality, shaped by their selection, use and opinions of phygital enhancement layers (p2r). In addition, both personal and societal beliefs and norms are embedded in and shine through the spaces (p2s). The characters therefore pass three spaces within the story: a personal space owned by one of the characters, a public space and a sensitive space:

Result

Let's dive deep into this vision of our future and talk about some keyframes in particular:

Scene 1, Giselle's Flat, Private Space

The group decides to meet at Giselle's flat, who prepares sparkling wine and food until Lotte arrives first, just before Robert.

Keyframe 1 & 2

Giselle blocks reality adaptation layers because she is careful with phygital technology. However, she does use some information layers, e.g. showing a timer or telling her how many layers she is blocking. Even the majority of cautious people rely partially on enhancement layers:

Keyframe 3

Lotte is on the verge of knocking on Giselle's door. She's bought a reality adaptation layer that lets her see sunshine in the greyest of winters:

Keyframe 4 & 5

Upon entering Giselle's flat, Lotte receives a system notification telling her that Giselle is blocking some enhancement layers, including her sunshine layer. But she is not blocking all the layers, as can be seen in a later scene in which an information layer tells Lotte that she has been drinking too much alcohol:

Keyframe 6

Giselle hands Lotte sparkling wine, which she perceives as pink. They don't talk about it because they don't know each other's perception. The importance of subtlety and meaninglessness is highlighted: do they even notice the color of their wine? Do they care about their dissonance in perception? 

Keyframe 7

Robert arrives. The menu bar in his point of view indicates that he has activated a number of information layers. He receives the same system notification, only in a different UI theme, which allows further personification of the embedded software.

Keyframe 8

When Robert and Lotte meet for the first time, they see the same rain of confetti. This raises the question of partially shared digital realities. How exclusive they are is left unanswered:

Scene 2, Street, Public Space

Giselle, Lotte and Robert leave Giselle's flat to catch up with Nick. They walk along a sidewalk and head for a café.

Keyframe 9

The fourth person in the group of friends, Nick, is introduced. His perception differs from the others in a strange, irritating way:

Keyframe 10

For the first time, all four viewpoints are presented side by side. Robert and Lotte share their sunny layer. Robert's information layers are fully activated again, showing names and personal information about others. A helpful yet intrusive feature. Giselle's vision stays close to the factual reality. Nick, with his fully immersive visual enhancements, couldn't enter the apartment after seeing the restriction notification, whether because of his own dependency or an actual phygital barrier stays unclear:

Keyframe 11

As they walk down the street, the extent of Robert's layers unfolds, from information layers ( routes and weather) to productivity layers (phone calls). As the group passes a car playing loud music, Lotte mentions the Beatles. Robert sees a Kendrick Lamar info screen. The other two also listen to different music. But no conversation develops, they just brush it aside and change the subject. Their inability to communicate and their unwillingness to find common ground is manifested for the first time, fragmentation continues:

Keyframe 12

Nick and Robert talk about their layers. Two beliefs clash here: Nick, who modifies (self-codes) his software, bypasses safety rules imposed by the system, and accepts unpolished perception, but gains authorship and autonomy in return. And Robert, who earns his money by developing this software. Who bears the moral responsibility in this proposed future? Their conflict is about to unfold, and the group breaks up. Their physical constellation now represents their emotional position: Robert and Lotte, who share layers and use them similarly, walking at the front. Nick and Giselle in the background. Both emotionally dissonant with the people in front of them. But as fully immersed users and cautious realists, their beliefs towards their phygital technolgies are as far apart as possible. Giselle can't and won't communicate. Nick doesn't even understand this basic and clear social rejection:

Keyframe 13 & 14

In a world where software is embedded in the physical world, so is advertising for that software. So to whom does the public space belong? As a trend junkie, Lotte buys the packages. Another step away from the shared perception with her friends:

Scene 3, Cemetery, Sensitive Space

The group has already lost Nick … or Nick has lost himself. Robert suggests taking a shortcut by passing a cemetery on the way to the café.

Keyframe 15 & 16

Upon entering the cemetery, Lotte, Giselle and Robert receive the same system notification asking them to activate their non-mandatory sensitivity restrictions. Even though Lotte and Robert activate it, Giselle doesn't. Which space-based restrictions are mandatory? Which are not? Who imposed these rules? Society, government, someone or something else?

Keyframe 17

This time Lotte talks about her physical perception, which immediately turns into a pain point and starts a discussion. A constellation change happens. Robert allies himself with Giselle. While there may be people who use and think about the technology in exactly the same way, this shows that social constructs are not rigid, beliefs can be congruent with a person in one aspect but different in another.

In their discussion, Lotte and Giselle confront with what is real. Each has a different understanding of reality that they want to impose on the other. Then, Do their visions really merge due to a technical defect?

Keyframe 18, 19 & 20

Meanwhile, Nick has wandered off. He decides to take a bike, but due to a lack of safety measures and unspecific warnings in his modded visual enhancement layers, he has an accident. This opens another person to a space-related cluster of conflicts that can arise when embedding the digital in the physical. What are the actual physical safety issues and safety measures?

Reflection

In the spirit of speculative design, our video project attempts to condense our vision of a phygital future of embedded sensory enhancement layers, asking questions we stumbled upon and presenting conflicts we discussed. Speculative design methods and workshops helped us to think our way into the process, to engage with the principle of the phygital, and to search for questions rather than answers. We realized that we can imagine a world and ask question after question, but it's the discussions that reveal the really interesting nuances, our own beliefs, fears, desires, and not-to-be-forgotten design opportunities.

It gets even more interesting when one extends this discussion to a legal dimension. We could ask, what is my right as a sender of information, as a recipient of information? Can I somehow ensure that what I say is perceived the way I want it to be perceived? Can I do today? Can we find a moral compass for dealing with this technology that does justice to everyone's beliefs, ethics, and legal concepts? Can we do today? Throwing questions into the future and pulling them back into the present is perhaps one of the most interesting challenges and thought processes we have had while working on this project.