Research and tools

Since I have never worked with installation projects, this course gave me the opportunity to experiment and develop new skills in visual graphics and motion sensing input devices. Therefore, the first step was to choose the most suitable software and technical device.

Software: Touchdesigner

https://derivative.ca/

Touchdesigner (Figure 1) is a visual programming language that is developed for real-time visual graphics which suits perfectly for creating an installation.

TouchDesigner has built-in support for Kinect which makes it more practical to start experimenting for visual outputs with human interactions.

Motion Capture: KINECT

https://en.wikipedia.org/wiki/Kinect

Kinect (Figure 2) is a motion device that tracks the human skeleton which was mainly developed for the gaming industry.

Figure 1. Touchdesigner interface combined with the skeleton tracking.

A & B) Geo Instancer: This component takes the data from the CHOP operator and transforms it into with X, Y, and Z values for creating the silhouette of the person. https://docs.derivative.ca/Geometry_COMP

C) CHOPs, short for CHannel OPerators, takes input data from the Kinect sensor. (see Figure 2) https://docs.derivative.ca/CHOP

D) SOP Surface Operators: operators that can generate primitive forms for visual representation. https://docs.derivative.ca/SOP

E) TOP Texture Operators: image operators for real-time visual graphics.

Kinect CHOP

The Kinect CHOP reads positional and skeletal tracking data from the Kinect. https://docs.derivative.ca/Kinect_CHOP

Figure 2. Skeletal tracking data.

Fire aestethics with NVIDIA Flow

NVIDIA Flow is a volumetric fluid-based simulation of a burning gassystem. https://docs.derivative.ca/Nvidia_Flow_TOP

Figure 3. Three examples of fire aesthetics.

Figure 4. Example of voxels visual shape.

Installation

Once the technical aspect was almost arranged, it was time to think further with the installation. After some experiments and consultation, I tried to use a reflective material, such as safety blankets (Figure 5). The result was very surprising as the material is semi-transparent, allowing the projection to go beyond the material but at the same time to reflect the light (Figure 7). The reflected light gave a very interesting aesthetic reminiscent of generative art, as depicted in figure 6.

Figure 5. Safety blanket.

Figure 6. Reflection of the projection through the safety blankets.

Figure 7. Example of the effect of the safety blanket's semi-transparency property.

Video example of the installation

Adjustments

After seeing how this installation might look like with multiple people, it was time to fix some issues. The main problem was that it worked well only with one guest interacting with the installation. As soon as the sensor perceived several people inside the scene, it was no longer possible to distinguish every single interaction. This represented a problem since the installation was based on interactions that had to allow active participation to the public who participate in it.

To solve this problem, I decided to simplify the image of the person projected, in order to have greater clarity and more precision of the silhouette of different people at the same time.

The second part was to give each person who enters the scene a different color. Sounds easy, but it took me several weeks after the course to achieve this result.

After different trials of building new ways in the software Touchdesigner, the most straightforward way to achieve this goal has been to build a separate Geo Instancer (see figure 1, A&B) for each person participating in the installation. Finally, individual Section CHOPs (see figure 1, C) have been linked to the Geo Instancer, in order to pull out the channels for each person separately.

Conclusion and further work

Unfortunately, the year started with a pandemic that surprised all of us. Due to the COVID-19 virus quarantine, it wasn't possible to test my latest improvement of the installation and I couldn't exhibit during the scheduled performance in the semester.

However, I am very happy to have worked on this stimulating project. Since it was my first attempt with an art installation project, I have learned a lot during this journey. Regarding the technical aspect, the discovery of visual-graphics software such as Touchdesigner, has kept my learning curve high during the semester. I can only highly recommend this software for experimenting with interaction design and art projects.

Working with installation projects it's not an easy task. Indeed, in order to achieve the desired result, multiple variables, such as the location, lighting and the number of people interacting with your art piece, have to be taken into consideration.

Unfortunately, the limitation of this work is that there has been no possibility to have visual documentation of the latest improvement, namely the possibility of multiple people to participate in the installation by using the newest and implemented Touchdesigner code.

As soon as there will be again the chance, I will be very happy to show this work to the public, trying to demonstrate the magic that can be created with the combination of analog elements such as safety blankets, and technology.