KONKREET PERFROMER

PERFORMER is the result of a project I realized together with the musicians Gwydion ap Dafydd and Shai Levy and was supervised by Prof.Matthias Krohn. The final app made it to the Apple App Store in early 2011, gained massive positive feedback, was used on big stages all around the world, got featured in an apple commercial and sold around 5.000 times until 2016.

KONKREET PERFORMER uses graphical shapes as a visual representation of the sound, which you can sculpt using multi-touch and gestures, giving a completely new and dynamic method of musical expression.

[Konkreet Performer in the AppStore](https://itunes.apple.com/de/app/konkreet-performer/id417418467?mt=8 „KONKREET PERFOMER im AppStore“)

BACKGROUND

Back in 2009, experiences with popular consumer multi-touch devices (Apple’s iPod Touch, iPhone) had shown huge potential for new interaction strategies, but especially in the field of music, multi-touch’s unique properties were where not used in apps. Everything was more or less an emulation of existing analogue hardware. More precisely: knobs and faders behind a glass surface. (The notable exception was Jazzmutant’s $2000 Lemur controller.)

When we started researching the possibilities of multi-touch, the iPad was not there yet. But the increased speculation regarding affordable larger tablet-format devices motivated us going further with the idea of building a new musical interface based on this technology. When the iPad was released in mid 2010 we already had some working concepts which we then tested on the bigger screen.

It was the opportunity to rethink the traditional paradigms of knobs and faders an go for something different. We wanted an interface that is not only aesthetically appealing, but also gives the musician a new feeling of interaction with sound: something more expressive that, by reinforcing the connection between action and sound, would also lead to a better experience for the audience. A chance to depart from the stereotypical experience of a laptop musician on stage, moving a mouse.

THE CONCEPT & FIRST PROTOTYPES

After short period of developing a basic concept and visual ideas we immediately started building little prototypes to test our assumptions. One thing was clear from the beginning: We did not want to build a music-toy. We wanted to build a music controller that could be used by professional artists in their usual environment. So it had to be a music-controller in a classical way. Only sending control messages to the artists DAW (Digital Audio Workstation), like Ableton-live or Reaktor.

We used the Open Sound Protokoll (OSC) and the open-source toolkit openFrameworks for this as it is quite similar to processing but C++ based and made quick and fast prototyping on i-devices very easy.

From our investigations it quickly became clear that the main interface graphics had to be something solid: an object or a shape. Which then could be manipulated by touches and gestures. Although this was still very abstract — as a sound will never actually have a shape — at a cognitive level it made sense for us that a small shape makes a small sound and a big one something loud; a spiky shape intuitively relates to a more harsh sound, conversely smooth, rounded shapes to calmer sounds, etc. Also, using an object allowed us to use interaction patterns that we knew from the real world: dragging, rotating, stretching and throwing it away. Which can be explored intuitively.

DESIGN PROCESS

The actual-design process had to balance two major constraints:

Aesthetics (Form): To develop aesthetically appealing visuals, which were also not too genre-specific, since they needed to fit a wide range of musical styles
Controllability (Function): Every shape needed to offer obvious points of control that the user can immediately and intuitively reach out and touch without confusion

Again we worked in short, iterative cycles of making small visual sketches, rapidly coding a working prototype, testing it on the multi-touch hardware, collecting feedback on which aspects worked well and which ones not so well, and repeating, either adjusting the code or returning to modify the sketches.

THE FINAL APP

In the end we arrived at an object-vector geometry that consists of a centre and several smaller nodes attached to it. The relative positions of these nodes to the centre as well as their relative position to each other are all sent out as parameter values, via OSC, to control the music software in real-time. Additionally, the overall position of the whole object is transmitted via OSC. A pinch gesture can be used to grow/shrink the object (and hence change the nodes-centre distance) and a rotation gesture would rotate the whole shape (and hence all node positions). These simple rules yield a strong effect. If visually tiny things happen, like a node is moved by a small amount, probably nothing much will change on the audio side. But if, for example, all nodes are «pinched-out» the shape, as well as all the connected sound and effects, will be much bigger.

This strong connection between user action, visual representation, and transmitted messages became the key characteristic of Performer.

It was also a clear conceptual decision not to display the numerical values of the transmitted messages on the device screen. We wanted to move away from analytical sound tweaking, back to more instrument-like exploring of music. Our testing showed that the brain was very quickly able to associate shapes and positions with certain sounds. Similar to the playing of a violin’s fretless fingerboard, there is a feedback loop between the action of the fingers, the resultant notes and constant micro-adjustments back to the fingers. And not only does the musician get a closer relationship to the sound, but because of the visible connection between the shape and sound the audience also could finally see what was happening.

For the graphical representation of the object, the user can choose a combination of up to three «visuals» out of a larger palette. These visuals represent the position of the centre and nodes, discussed earlier. Some visuals are just simple lines and dots connecting these nodes together making quite minimalistic shapes. Others have a stronger shape character which invites bending and twisting of the object. And again others like a star field have rather complex algorithms and are not restricted to the imaginary boundaries of the object, and result in a more ambient and abstract representation of the shape.

INERTIA

A half year after the initial release we gave the app the first big update and introduced INERTIA. It is the option to change the object's friction dynamically. Sounds simple, but together with the snapshots it enables variable morphing from one sound to the other.

MIDI

But the real challenge was to enable real MIDI message output from PERFORMER. Not only was this technically tricky it also made it necessary to come up with an interface to set up and modify all 672 possible parameters. We decided us for a Midi-Mapping mode in which you can select the wanted parameters by touching the object itself. Only showing the options for the selected note.

FEEDBACK AND PRESS

In January 2011 we launched KONKREET PERFORMER via the Apple AppStore. The overall positive feedback was overwhelming.

To name a few:

.

Sweet jesus this is mental! Nick, Liine

.

Performer: A Stunning iPad App Puts Daft Punk to Shame [FastCo Design](https://www.fastcodesign.com/1663191/performer-a-stunning-ipad-app-puts-daft-punk-to-shame-video „FastCo Design“)

.

Zur Steuerung von spezialisierten Instrumenten und Effekten, für Soundcluster und generative Musik ist der Performer perfekt [...] Insgesamt ein ziemlich interessantes Tool, das abseits von altbekannten Standards** neue Wege in Sachen Interfacedesign** betritt und die Möglichkeiten des iPads voll nutzt. Lohnt sich! De:Bug

.

As either Marc or Stephan drag the circles to different points on the iPad screen, the waveform changes, resulting in some remarkably weird and wonderful moments of mid-set improvisation. What’s more, a live feed of the iPad screen can be fed to the projectors in the club, meaning the crowd can literally watch the sounds being manipulated in front of them. iDJ

.

Performer completely changed the way we can manipulate music. It allows me to freely create sound design elements, but most importantly, perform musical ideas that can’t be done any other way.” Sascha Dikiciyan, Sound-Designer

.

Wir sind Apple! Prof. Boris Müller ([After PERFORMER got featured in Apples 30 anniversary commercial](https://incom.org/post/129001 „After PERFORMER got featured in Apples 30 anniversary commercial“))

ON TOUR WITH STEPHAN BODZIN

Stefan Bodzin a known DJ contacted us and took PERFORMER for a premiere in his „Luna“ world tour. For him, we also developed a mirror app, which can be used to project performers visuals in real time.

Performer with it's brilliant function, design and possibility to screen all you're doing in real-time played a leading part in our (audio and visual) concept Stefan Bodzin

What started out as a little side project became a tool that was actually used and loved by musicians and DJs all around the world.

On the way I learned how to develop apps for iOS, building a company brand and keeping customers happy.

I am super thankful for all the great users we have out there. The dozens of youtube videos with fans all over the world demoing PERFORMER really kept us going.

Last but not least I am grateful that I had the chance to work with Shai Levy and Gwydion ap Dafydd on this wonderful project.

[Konkreet Performer in the AppStore](https://itunes.apple.com/de/app/konkreet-performer/id417418467?mt=8 „KONKREET PERFOMER im AppStore“)

Incom ist die Kommunikations-Plattform der Fachhochschule Potsdam

Incom ist die Kommunikations-Plattform der Fachhochschule Potsdam mehr erfahren

KONKREET PERFROMER

BACKGROUND

THE CONCEPT & FIRST PROTOTYPES

DESIGN PROCESS

THE FINAL APP

INERTIA

MIDI

FEEDBACK AND PRESS

ON TOUR WITH STEPHAN BODZIN

Ein Projekt von

Fachgruppe

Art des Projekts

Betreuung

Entstehungszeitraum

Keywords

Links