Abstract Deutsch

Das Ziel dieser Masterarbeit ist es, die Rolle und Bedeutung von Empathie in der Informationsvisualisierung und in Augmented Reality zu verstehen. Darüber hinaus konzentriert sich diese Arbeit auf die Frage, warum komplexe soziale Probleme wie der Klimawandel von der Nutzung von Empathie und Augmented Reality profitieren könnten, um abstrakte und weit entfernte Situationen dem Betrachter emotional und physisch näher zu bringen. Darüber hinaus wurde eine Reihe von Richtlinien zusammengestellt, die Ersteller von Informationsvisualisierungen oder jede andere interessierte Person nutzen können, um Empathie in ihren Zielgruppen zu wecken. Die Anwendung dieser Richtlinien wird anhand eines prototypischen Beispiels in Augmented Reality demonstriert, das auf dem komplexen sozialen Problem der Fast Fashion basiert.

Abstract English

The purpose of this master thesis is to understand the role and importance of empathy in information visualisation as well as in Augmented Reality. Moreover, this research concentrates especially on why complex social problems like climate change could benefit from the usage of empathy and Augmented Reality to bring abstract and distant situations emotionally and physically closer to audiences. Furthermore, this thesis gathered a set of guidelines which creators of information visualisations or any interested person can make use of in order to evoke empathy in audiences. The utilisation of these guidelines is demonstrated by presenting a prototypical example based on the complex social problem of fast fashion in Augmented Reality.

Motivation

Seeing globalization as well as individual awareness of global crises caused by or involving humans on the rise, as well as all the information and data that is being communicated and visualised to the public, I started to wonder if our current narrative of fear, information overload and numerical graphs is really informing and communicating the severity and importance of all these different situations to the public. 

Coming from a film background and with my interest in making my audience feel something I started to wonder what this would mean for information visualisation. Especially what does it mean to make audiences empathize with abstract concepts and problems like climate change. One is capable of understanding or feeling what others experience through empathy, but what would this mean in the field of information visualisation and what has been researched so far? 

On the other hand, my interest in immersive and expansive technologies like Virtual and Augmented Reality is prevalent since various big corporations like Facebook or Apple are paving the way into the mainstream development of this technology. Through the affordable prices of the Oculus Quest 2 and technologies like LIDAR being implemented in many of the newest Apple products we will see a “boom” in this technology in the coming years. Because of this, I believe it is necessary for designers to start deciding what and how one can design for these platforms. Moreover, over the summer of 2020 I collaborated with another Master student in the Fachhochschule Potsdam, Sebastian Wloch, in developing a lecture for AR prototyping. The projects that came out of the lecture showed the diversity and potential of re-thinking the interfaces, services and tools that we can produce for Augmented Reality. This made me believe and see even more the possibility for Interface Designers to create new ways of Human Computer Interaction in the XR realm and the same goes for information designers, journalists and storytellers. Augmented Reality gives us the chance to Geo-Locate content, to use pictures as markers where we can place information, to use our surroundings as gaming surfaces…, and I think it can be a very valuable tool for communicating complex social problems. 

Imagine if I didn’t have to go to Syria to understand the gravity of the situation, but I could have an Augmented Reality story in my living room informing me about the situation; or I can scan the package of food that I just got and have the data from were it comes from as well as the information of how much this product polluted. 

There are many ideas which designers can come up with for AR. However, when using this medium – or any for that matter – to communicate subjects like complex social problems, many ethical considerations should be made to not create sensationalism from the pain or the reality of others. This is where I see that the study of empathy is fundamental for human communication and the further development of communication and information design. The moment when we treat information as a tool to elicit compassion and understanding for others is the moment where the creators of the information have to as well empathize and deeply understand what they are trying to communicate to audiences. Overall, Augmented Reality has the potential to be used as an activist as well as an educational tool. It can – as I would like to put it – “bring the museum out to the street”.  

With Augmented Reality, empathy and my quest to understand other novel ways of visualising and communicating information as deep interests in my mind, I wanted to discover how immersive technology like Augmented Reality could create different types of spaces for information to come to life and bring complexity and data closer to audiences in a way in which a screen is not capable of, breaking the division between what is happening in my life and what is happening in the world, since in the end, our lives and world events are a connected ecosystem. But how can designers create a different narrative especially for information which comes from complex problems like climate change or a pandemic outbreak? How can one change the narrative? Could facilitating audiences to empathize with the problem and the consequences be an option for future ways of visual and narrative communication?

Research Question

Complex social problems like for example climate change are difficult to understand and communicate since for many people – especially in developed countries – they seem distant in space and time or a point of contact to make it relatable is missing. My hypothesis is that creating informative visualisations that have an emphasis on one of our human skills – empathy – could bring such topics closer to audiences, helping them understand these topics and letting them come emotionally closer to them. This could also later be used as a mechanism for changing people’s behaviour. Ultimately many complex social problems require a change in policies as well a change in the behaviour of the population. Therefore, new solutions must be developed in the way we transmit and visualise the information regarding these types of problems. 

Augmented Reality has the potential as a technology to bring this information closer to audiences since the digital content is placed in the real world, creating a symbiosis of digital and real. Maybe even creating bridges between what seems like disconnected ideas, concepts or problems. For example, how buying a 10 € pair of jeans in Germany has an unseen consequence in the resources that are used in developing countries like Bangladesh to create such a cheap garment. 

This master thesis presents a discussion on how the creation of an empathic experience from the information about a complex social problem, together with the usage of Augmented Reality could allow people to understand and empathize with the presented subject. 

At the end of this thesis, a set of guidelines will be presented with the intention of facilitating other researchers, designers and creators to create an empathic information visualisation which can be implemented and displayed in AR.

Some Learnings

Terminology:

• Augmented Reality (AR), is an experience which enhances the real world by placing digital content into it to be seen through an enabled device like a smart-device, such as a tablet or a smartphone, or through the usage of a wearable like glasses (Valendu, 2018; Marr 2019).

• Information Visualisation Storytelling refers to the creation of a visual representation of qualitative or quantitative data which then tells a story in order to make it understandable to the public.

• Complex Social Problems is an umbrella term referring to complicated

  situations, which are not easily solved, which occur in our society

  and involve humans. Some of these types of problems can be: climate

  change, drug abuse, gender violence, racism, poverty, world

  hunger; for naming just a few examples. According to DeTombe

  (2005, as cited in DeTombe, 2013, p.1), one can only understand

  complex social problems as systems which affect society on different

  levels and are unstable. It’s unclear how to solve them, due to

  the many different phenomena (situations, opinions, other problems)

  which are intertwined. You need different information from

  different fields in order to solve them and sometimes it can collide

  with the goals of the stakeholders and actors involved. On top of

  that, they can create strong emotional reactions in society.

• “Empathy is the ability to recognize, understand, and share the thoughts and feelings of another person, animal, or fictional character” (Psychology Today, n.d.). Empathy can be divided between cognitive empathy and affective empathy:

• „Cognitive empathy relates to one’s ability to accurately imagine another’s internal states (…)“ (Hoffman, 2001 as cited in Boy et al. 2017, p.3). It does not depend on feeling but rather on analyzing what the other person might be feeling or experiencing. It can be understood as critical thinking or perspective taking, referring to the ability to identify and understand other people's emotions without necessarily feeling them ourselves (Friesem, 2016, pp.26-27; Great Good Magazine, n.d.).

• Affective Empathy on the other side, “(…) refers to the sensations and feelings we get in response to others’ emotions (…)” (Great Good Magazine, n.d.). It is feeling instead of imagining/rationalising what others might be feeling or experiencing. Through affective empathy we stop feeling ourselves in order to feel the others.

Learnings:

Through the master thesis, I researched through different fields: empathy, InfoVis, complex social problems, climate change communication and Extended Reality. The goal was to understand how these different topics are connected as well as the creation of a set of guidelines through which designers could make use when visualizing information in or out of AR which intended to arouse empathy. Overall the thesis has the following key learnings:

• Empathy and emotions haven’t been welcomed and have been understudied in the research community. Therefore there is more potential to keep understanding the role and way through which one can design for and with emotions and/or empathy.
• Empathy is a skill that is not only related to humans or something that appears anthropomorphized (human looking).
• A high percentage of the population – 51% in Germany – doesn't have a high graph and numeric literacy. I therefore suggest to include more emotional and empathetic cues and designs through which audiences can have an easier access to complex social problems and the InfoVis of them.
• When communicating and visualising a complex social problem, like climate change, there are many aspects to take into consideration like: try a different narrative than doom and gloom, be aware that people have mental models – but this can be changed – as well as include a sense of hope and solutions (for more information read the chapter about “Challenge with climate change communication and visualisation”).
• Complex social problems tend to use graphs which a high percentage of the population doesn't understand. Meaning that many people are not empathizing with the problem. Moreover these graphs have to be processed by the analytical part of our brain which has to be trained, but one could trigger rather the experiential processing part of the brain which is in charge of stories and emotions and needs no training.
• Virtual- and Augmented Reality showed that there is potential in empathy arousal through the usage of these platforms as well as in the communication and “confrontation” of audiences with complex social problems.
• Augmented Reality connects the real with the digital world through the overlay of the digital content in the real world. One can create a physical connection with the digital content. Moreover one can make the “unseen” seen.
• The analysis of the guidelines showed that one can target the design for affective empathy or for cognitive empathy.

Tipp: Read the colorful pages of the master thesis, if something really interests you can then read the full chapter

The Guidelines

For more information about the guidelines and how to use them please check the master thesis from pages 136 to 172.

1. Allow viewers to witness the events (Weber, 2020)

2. Have characters (Weber, 2020)

3. Create dialogues (Weber, 2020)

4. Have pictures of the affected people (Genevsky et al. 2013; Wenzel, 2019; Garci-Crespo López, 2021)

5. Create comparisons (Boy et al, 2017; Lee et al. 2020; Garci-Crespo López, 2021)

6. Have physical or DIY objects through which audiences can get closer to the information (Garci-Crespo López, 2021)

7. Illustrated faces/people (Boy et al, 2017)

8. Use animation (Boy et al, 2017)

9. Use visual metaphors (Borgo, 2012)

10. Add sound effects (Boy et al, 2017)

11. Have voice-over narrations (Boy et al, 2017)

12. Use unit visualisation (Boy et al, 2017)

13. Target the experiential processing system of the brain (images, narratives, emotions) (CRED, 2009)

14. Try a different narrative than fear (Wenzel, 2019; Boycoff, 2012; Stoknes, 2017)

15. Include solutions (Leiserowitz, 2018)

16. Aim at creating a sense of hope (Feldman, 2018)

17. Be aware of the finite pool of worry (Linville and Fischer, 1991)

18. Think about how to create emotional engagement in order to keep audiences attention (Linville and Fischer, 1991)* here looking at gamification principles could probably help

19. Be careful which emotions you create to prevent emotional numbing (Linville and Fischer, 1991)

19.1. Decide what risks you want your audience to become more aware of and demonstrate “the connection between those risks” (CRED, 2009)

19.2. Create a balance between emotional response and analytical information (CRED, 2009)

19.3. Be aware that people might have other problems that they are probably worrying about (CRED, 2009)

19.4. Acknowledge the emotional numbing that audiences can have and “encourage them to briefly consider their level of worry and potential numbness to climate change” (CRED, 2009)

20. Try using a “U” shaped storyline. With a start, a fall, a climb and a way out (Arnold, 2018)

21. Be aware that people will probably have confirmation biases, so try to find out which are the audience’s existing mental models and give them the right information to change them

22. Create a frame: What part of the information will you show to your audience (CRED, 2009)

23. Communicate the information in a way in which the gains of the future is more relevant than the losses of the present moment (CRED, 2009)

24. Create 3D visualisations (Sheppard, 2012)

25. Use natural looking characters (Sierra Rativa and van Zaanen, 2020)

26. Create interactions through which audiences can have a direct or indirect effect on the character or object (Sierra Rativa and van Zaanen, 2020; Garci-Crespo López, 2021)

27. Aim at creating a connection to the character or objects represented by allowing users to create the backstory or narration of the characters/objects in which they include events, context thoughts and feelings (Kor et al. 2016)

28. Create visceral engagement through either storytelling or interaction with the environment (Kors et al., 2016)

29. Create moments of reflection (Kors et al., 2016)

30. Create affective appeals through different inputs like smell, movement, etc. (Kors et al., 2016)

31. Design for direct interaction with the data (Ivanov et al. 2019)

32. Have the information geo-located (Efstathiou et al., 2018)

33. Provide a problem-based narrative to support [audiences’] exploration and help them focus. (Efstathiou et al., 2018)

34. Integrate the AR activity with other non-AR activities before and after, that would first prepare the learners for what is to follow, and then give them opportunities to reflect on the activity (Efstathiou et al., 2018)

35. Aim at creating conversation within the audience (Skwarek, 2018)

36. Design visuals and audio with “Kindchenschema” (Batson, 2005; Lishener, 2008; Zickfeld, 2017)

37. Add Music/Ambience (Garci-Crespo López, 2021)

The Prototype

After the research and the creation and gathering of the guidelines, I decided to create a prototype which would make use of some of the guidelines that I had collected. The purpose of this prototype was to experiment how Augmented Reality and the guidelines could tell an informative story of a given topic. To experiment on how to design for empathy.

For the purpose of the prototype, I extensively researched about the consequences of fast fashion and how this industry pollutes and contaminates our environment. The reason for taking information about this complex social problem is because is directly related to climate change and my intention was to understand this connection as well as try to visualise it.

Tipp: To learn more about the information used for the prototype I recommend reading the thesis ranging from page 116 to 133.

The Concept: „Gaia vs Fast Fashion – Hidden costs of the fashion industry“

For the concept of the information gathered, and taking into consideration guideline Nº 32: “have the information geo-located”. I decided to locate the information in one of the most prominent shopping streets in Berlin: Kurfürstendamm also named Ku’damm, an area where all types of brands have gathered, from high-end designers like Coco Channel and Gucci to more publicly available stores like H&M, Zara or Pull&Bear. 

I decided to conceptualise a sort of digital augmented Exhibition which would be divided in different sections:

• Welcoming / Reception Area
• Demand and consumption
• History of fast fashion
• Rapid Production
• Fiber Production
• CO2 Production
• Water Needs
• Waste

Moreover, the concept of a „resting space“ was introduced. Meaning creating sections in the digital exhibition where people can rest emotionally in order to prevent emotional numbing (Guideline Nº 19) as well as offer other sorts of more playful interactions with the characters and thematic of the exhibition.

Below you find the map with the different topics mapped as well as some sketched concepts.

Tipp: To read more about the concept, which guidelines were used as well as the design decisions read from pages 176 to 209.

How the Prototype was done

Story 

In order to create the prototype, I choose some of the information which I gathered from the topic about „Demand and Consumption“. 

The information of the station – each section has different stations –which I decided to prototype was the following:

 1. In the last few years people have started to see shopping as a hobby. 

2. On average (in the USA) people buy a clothing item every 5.5 days 

3. The main drivers for compulsive buying are: 

• Only online ordering and same day delivery 

• Multiple collections that brands like Zara and H&M offer; with Zara offering around 24 new collections each year. 

4. Clothing utilisation before a garment is thrown away has decreased considerably – up to 36 % percent – in the past 15 years

 5. In countries like Germany, consumers buy an average of 60 garments per year but wear them half as long as they did 15 years ago 

6. Some Garments are estimated to be discarded after just seven to ten wears.

The goal of this section is to prime the user to reflect about how much we are consuming and start to empathize with the main victim of this whole topic which is Earth and the Nature that inhabits her. For the purpose of this prototype,

Character

After writing the story and creating the storyboard it became clear that the exhibition could have an identifiable vulnerable victim, namely “Earth”. I decided to create Gaia, getting inspired by the soil and skin color diversity as well as her skin having vitiligo. For her design the “Kindchenschema” (Guideline  Nº 36) was taken into consideration. My wish with this design decision is not only to arouse empathy in audiences but to experiment with the way through which one can perceive planet earth. Instead of the more typical image of “mother earth”, changing it to seeing her as a child, as someone one wants to protect. Also, I decided to anthropomorphize her – contrary to the current finding suggesting that empathy is not increased – and have still tried to give her a natural look through the colors to be more identifiable).

To create the prototype I followed the following steps:

1. Create sketches of all the assets I needed 
2. Create 3D models of these sketches through VR Tilt brush or AdobeDimension
3. Animate Gaia using Mixamo
4. Record sound effects for Gaia and download sound effects as well as atmospheric music
5. Arrange and animate all of these models – and scenes – in AdobeAero
6. Film everything through AdobeAero
7. Edit the Video with PremierPro

Other design tools that were used were Illustrator, InDesign and Audition.

Results

After prototyping the information along Ku’damm it became clear to me that more iterations are necessary to fully understand the potential that AR and empathy offer when visualising information. The guidelines allowed me to approach and visualise the information in a more „playful“ and cinematographic way than I had expected. Although it would have also been interesting to use them to try to visualise graphs and experiment with different ways in which one can create numerical graphs which arouse empathy. Nevertheless, when creating the unit visualisation of the 60 pieces of clothing and the 24 seasons I surprised myself by the represented amount. Here I realized that one could manipulate the feeling that audiences get by scaling the objects, which is why I would recommend to designers to keep the sizing of things as close as possible to their real physical counterpart. 

To fully understand whether empathy is actually evoked by the prototype, it would be necessary to test it with users and gather their feedback. The reason why I couldn't collect this qualitative feedback is due to the prototyping technology currently available to me. With AdobeAero it was not possible to trace the 100 m long area that I needed to prototype everything, so I had to create different scenes. Moreover, my wish to combine 3D Models, animation, narration, music and sound effects had the consequence of creating very heavy scenes that can only load one at the time. That’s also why the video has fade ins and fade outs, I had to create the different information points separately. Moreover, when trying to move around the scenes to film within them, the floor and objects would sometimes “slide” away, which was probably due to the cobble stone floor, which confused the tracking algorithm. To prevent the scenes from sliding a switch to Swift and true geo-location would be useful. This would require a full App development but would also enable proper testing and feedback conditions for users. Besides finding solutions to these technical difficulties I also see the necessity to try to visualise the information in different ways to test how does empathy look visually, meaning I would iterate between different character designs, colors, forms, etc. Furthermore, one could also try to create a design which exclusively targets either affective empathy or cognitive empathy – since the design that I proposed contains a combination of both. This could be achieved by creating a prototype which mostly uses either the affective or cognitive guidelines respectively. Moreover, it would also be recommended to design this information in plain 2D and test that version against the one in AR to compare the effects that the different visualizations have on cognition and empathy in users and see which group is more likely to change their mental models and behaviours.

Another next step to consider, is to test how other designers work with the guidelines, meaning to see how effective they are as a tool and which ones can be further improved or removed. This testing could be done by doing a workshop with fellow designers and asking them to visualise a set of information from any topic – complex or not – and gathering feedback on how useful they found the guidelines.

Video

Conclusion and Outlook

Visualising information in a three dimensional space creates a different feeling for the data. It is no longer embedded in a screen but present in the real physical space. Having the real physical world as background makes it easier to show comparisons and create a symbiosis between the digital and the real. However, one has to keep in mind the technical limitations that Augmented Reality presents. The prototype showed the potential that information visualisation stories could have if using empathy as a method and AR as a tool, though further iterations, comparison to a 2D visualisation as well as feedback and testing are still needed.

There is potential to keep experimenting and designing for empathy and especially in understanding what are the differences between affective empathic design and cognitive empathic design. Moreover, I see potential in researching and understanding what is an empathic image. Meaning what are the visual aspects that arouse empathy in audiences. The “Kindchenschema” (Guideline Nº 36) was one of the guidelines that goes in this direction, but there is potential for further understanding of form, color, expression, movement, etc. Analysing “how does empathy look” would be an excellent complement to the set of guidelines which were created throughout this research. In order to keep discovering and understanding how to design for empathy as well as how an empathic image looks, I recommend researching empathy in aesthetics, meaning how art arouses empathy in audiences.

Another approach to further discover how empathy is designed would be to look into fields like immersive journalism, persuasive/serious gaming, marketing, film and museum exhibitions.

Moreover, I believe that Augmented Reality, especially when visualising information, could profit from gaming principles to create a strong engagement in audiences. Another recommendation for those interested in the further study of information visualisation in Augmented Reality is to look deeper into data physicalization, museum exhibitions and art installations, given the increased spatial awareness that one could gather from researching how to place information in real space as well as how to create a narrative that spans across space.

Overall there is still a long journey for the understanding and development of a design which arouses empathy. Emotions – and therefore empathy as well – have not been welcome in the research community for a very long time, but this might change once we become aware that a large part of the population is not fully understanding the information that is presented in classic numerical graphs. Maybe they could find access to this data if it's made more emotionally available for them. Moreover, with new technologies evolving so rapidly new challenges arise in how one can design and communicate experiences to the audience. With technologies which will have the capacity to merge the real with the physical it is of utmost importance to question the kind of interaction that one wants to give to their audience. What can happen in a world in which digital and physical are merged? Will we always have an „avatar“ like Gaia to accompany us in our daily lives? And with our growing globalisation the question of „How can we communicate with one another?“ it is more prevalent than ever.

In a society that is permanently intertwined with complex social problems understanding how to facilitate audiences to change their perspective and imagine or feel what others might be experiencing could bring research and design in a new positive direction. I therefore invite other designers to keep seeking to understand how to design something that makes us feel and connect ourselves with something outside of our own experience of the world.