Visual Languages| Funky Footprints, a visual excarvation.

An excavation of visual forms

Next I looked at the individual shapes in order to organize them into distinct visual categories. The goal was to come up with a simple rule based logic that would help explain how the individual shapes differentiate along a continuum. The overriding organization principle that got applied here is organizing the shapes by continuum: Starting with the simple shapes and adding visual complexity as we move forwared. Have a look at the series of pictures in the section below .

Basic Shapes:

First I singled out plain geometrical shapes starting with rectangles on the left and moving on to distorted rectangular shapes in the middle. On the right are some distorted shapes with simple recessions.

Here are some simple geometrical shapes with more dramatic recessions.

However, there were simple geometrical shapes with protruding elements as well. The picture above organizes these starting with the more dramatic protrusions on the left, moving to less dominant protrusions on the right.

Next I took out the rotations and organized the basic shapes along some other dimensions:

The top row organizes the plain geometrical forms by size. The middle row orgnizes slightly distorted plan geometrical forms by the amount of their distortion. The bottom row also organizes elements by the amount of distortion. This time though we are looking at simple shapes with a recessed area.

This picture reflects the observation that the shapes can be oriented either horizontally or vertically. Hence the forms are grouped by there orientation, starting with horizontally oriented shapes on the left and putting vertically oriented shapes to the right. With regards illustrating the initial catalogue of activities and emotions, I thought it might be possible it render activities that don’t involve much physical action in horizontal shapes and experess physically more demanding one in vertical shapes…

The third dimension along which some of basic shapes were organized was the depth and complexity of their recessed surfaces. The pictures shows a continuum from shallow to deep recessions.

With regards to the catalogue of activities, recessions could symbolize activities that that involve the body to absorb something physical such as eating and drinking or smoking.

Finally the forth way of organizing the basic shapes took the number and complexity of elements protruding from the core shape into consideration. The shapes on the left sport single protrusions and the shapes towards the right side of the graphic feature multiple protrusions. Also note that some protrusions are nested.

When it comes to the catalogue of activities and emotions, protruding elements could translate as interaction with physical artefacts. The size of the protrusion could hint at the size of the artefact relative to the body. And their number could hint at the number of artifacts one interacts with at a given moment...

Inner Shapes

Moving form basic to slightly more complex shapes, I isolated forms with inner cut-outs that resemble rims or stencils. As with the basic shapes I strived to organise them along different dimensions:

Again, I singled out plain geometric shapes first. The illustration above shows most plain rectangular shapes on the left and slightly distorted more complex shapes on the right.

When it comes to the catalogue of activities inner shapes could be used to render cognitive activities or the level of consciousness. E.g. actions that demand a high level of concentration and counciousness could be working on a computer, sewing, doing financial stuff. As opposed to activities that are performed by “autopilot” such as dinking a coffee in the morning or brushing teeth.

Among stencil based shapes found in the map were forms with protrusions into the inner shape. The grapic above roughly organizes these shapes by the amount and complexity of protruding elements.

With regards to the catalogue of activities and emotions, protruding elements that are directed towards the inside of a shape could symbolize activities that invole the body to absorb something that is not physical but rather cognitive, such as reading the newspaper or listening to musig or listen to someone taling etc.

A third category of stencil based shapes found in the map consists of forms with both, an inner shape as well as elements protruding towards the outside. This basically represents a combination of two shape categories that were already discusses above: forms with inner shapes and basic shapes with protruding elements. The combination of both results in some fairly complex visual forms.

With regards to the cataloque, such shapes could represent cognitive activities that involve interactions with physical artefacts. This could be things like working through a research paper on the computer etc…

As with the more basic shapes in the previous section, the next step involved taking out the rotations of the individual shapes and organizing them along some other dimensions subsequently.

The diagram above organizes stencil based shapes by the level of distortion of their core shape. Plain geometrical, rectangular shapes are positioned on the left. Shapes with more complex distortions are displayed toward the right hand side.

Beyond core, we are looking at two groups of elements separately: simple stencil based shapes form the top grouping and stencil based shapes with protrusions directed towards the inner centre of the shape represent the bottom grouping.

Another way to organize the stencil based graphics is by looking at the proportional size of the inner shapes in relation to their overall dimension. Shapes with relatively small inner cut-outs can be found on the left side in the picture above. Accordingly shapes with relatively large inner perforations are positioned on the right hand side. Again we see this organizing principle applied to two distinct groupings: plain shapes in the top row and shapes with inner protrusions in the bottom row.

As already hinted at above, inner shapes could be used to render cognitive activities or actions performed under a high degree of consciousness. Whereby the more cognitive load - the more thinking – is involved the lager the inner shape would be relative to the overall over all dimensions of the symbol. In addition to that, protrusions towards the inner centre of the shape could render activities whereby somebody absorbs something that is not physical but rather of cognitive nature.

In the picture above only the form of the individual inner shapes are compared with each other. They are organized by complexity. Simple geometric shapes without protrusions are located on the left hand side. Moving towards the right, shapes become more complex distored and start to carry inward protrusions.

This comparison chart looks at the forms carrying both, inner shapes as well as outward facing protrusions. The protrusions resemble arms. The shapes above are organizes by the number of the outward protruding elements per shape. The continuum starts on the left hand side with shapes that count only a single protruding arm. Towards the right side the number of protruding arms increases as wells as the complexity of their shapes.

With regards to the catalogue such shapes could represent activities that involve varying degrees of cognitive control – represented by the inner shape – as well as interactions with physical artefacts – represented by the protruding arms.

An other way to organize these symbols is to group them according to the proportional size of their protruding arms relative to the dimension of the core shape that the arms are descending from. In the image above shapes with relatively small arms are located the on the left. Moving towards the right the protruding arms become increasingly larger in relation to the core shape.

In terms of the catalogue of activities and emotions the relative size of protruding elements could be proportional to the size of physical artefacts that someone interacts with in order to perform a specific activity.

Last not least, shapes may be distinguished by the direction of outward facing protrusions. The illustration start on the left with a shape carrying four arms that protrude in two distinct directions. Each protrusion flows along a 90 degree angle, adding extra visual complexity and creation a nested effect. As we move towards the right the number of arms as well as the number of directions they protrude to decreases.

Multiple Inner Shapes.

Within the source image of the map there where also slightly more complex shapes and forms with multiple cut-outs and inner shapes. As in the prevous examples I strived to organise them along different dimensions.

Again, plain geometric shapes were singled out first. In the picture above are the few plain rectangular symbols with multiple inner shapes to be found in the map. On the very right there is an example of a plain but slightly distorted geometric shape.

The graphic here features shapes that have a lot of complexity going in their centre. The elements in the image above are organized by complexity. Starting form the upper left corner, the more visual noise there is going on in the centre of a shape the further right and below it located in the graphic. There are two ways to look at the chart though:

1. one might recognize the symbols as rim-base shapes with multiple complex protrusions towards the inside. The protrusions themselves form a dense mesh of inner cut-out shapes.
2. One might assume that the above shapes carry multiple inner cuts that exist as independent of each other as separate inner shapes.

Regarding the catalogue of activities and emotions I much prefer viewpoint A though – mostly because this rationale would be consistent with the idea that inner shapes may used to render cognitive activities or actions performed under a high degree of consciousness. Multiple protruding elements that form a dense inner mesh could be used to illustrate a whole set of cognitive mechanisms or various lines of thought.

Analogue to the previous examples there is also a class of symbols that feature both multiple inner cut-out shapes as well as protruding elements (or arms) descending towards the outside. Again, this is basically a combination of shape categories that were already discusses above.

As with the examples above, the next step involved taking out the rotations of the individual shapes and organizing them along another dimension.

The illustration above shows all symbols with multiple inner cut-out shapes agganged in a continuum that stretches over two lines. Starting form the upper left the various visual forms are organized by the number of inner shapes they carry. Simpler forms can be found in the upper left, more complex ones in the lower right, respectively.

Open shapes

Finally open shapes, as a third class of symbolic forms in the source image, were explored in isolation and organized along different categories.

As in the previous explorations plain geometric shapes were singled out first. The graphic above displays simple geometric forms and fragments that open up towards one side.

When it comes to the catalogue of activities, open forms could be used to symbolize conversations – both verbal and written – where thoughst flow out of somebodies mind into the outside to be shared with the world.

The next level of complexitiy is represented by open nested shapes that feature depressed and elevated shape sections. The chart above starts on the right hand side with less complex open shapes and turns towards more visually complex forms on the left. The depressions and elevations resemble the in- and outward protrusion of the previous examples.

In terms of the catalogue open shapes with inward facing protrusion could illustrated situations in which somebody is engaged in a conversation with some else: Sharing own thoughts and absorbing the input of the other person all at the same time.

The third and final category of open shapes may be best described as sprawling formations. Here it’s almost impossible to recognize a core elementary shape. The open forms sprawl freetyle into different directions and result in symbols with a random appearance. The diagram of shapes above follows no particular order.

Unlike in the previous explorations I abstained from organizing the open shapes across other categories as well as from dissolving the rotations.

Other shapes

The source image also contained some additonal classes of visual elements that were not considered for a more detailed exploration. However, for the sake of officering a somewhat complete (and exhaustive) archaeology of the visual shapes extracted from the map I added the image below.

The chart above shows the additional triangular and roundish shape classes that were not considered and explored any further.

Visual rationale and shape transformation logic

The following section summarizes some simple rules and logic that were derived form the previous exploration of the visual repertoire.

The rules that are established at this point represent the foundation and shape transformation logic to visually encode the catalogue of activities and emotions. Red bullet points highlight the final principles that were eventually used to visually encode the catalogue.

Basic shape transformation rules

Basic or core shapes may be distinguished by their level of rotation, vertical or horizontal orientation, the level of geometric distortion and the relative size to each other.

With regards to the catalogue of activities only the vertical and horizontal orientation of core shapes were considered to visually encode the various activities. Horizontally oriented shapes represent passive activities – for example lying in bed, or sitting in font of the telly. In turn, vertically oriented shapes represent activities that involve a higher level of physical engagement – e.g. walking to the bus station or jogging through to park.

Additionally basic shapes are determined by recessions and protrusions. Both transformation principals were also used in the visual encoding process. A geometric recession into the core shape represents an activity that involves the body to absorb something physical such as eating and drinking or smoking. In contrast to that, Protrusions descending from the core shape towards the outside represent interactions with physical artefacts. Accordingly, the number of protruding elements would represent the number of artefacts someone interacts with at a time. The proportional size of a protrusion in relation to the core shape could hint at the size of the artefact one interacts with. A simmilar logic could apply for the size and number of shape rescissions.

Inner shape transformation rules

The basic core shape of a symbol can be further differentiated by inner cut-out areas which provide a rim or stencil-like appearance. The cut-out areas are visually defined by their proportional size relative to the overall shape, their position, and protruding elements that descend towards the centre of the shape. The proportional size and the protruding elements where eventually used to visually encode the catalogue of activities and emotions.

Inner cut-out areas are used to render cognitive activities or actions performed under a high degree of consciousness. This could be activities that involve a lot of conscious thinking such as working on a computer, sewing or doing financial statements. The proportional size of the cut-out area relative to the overall shape hints at the cognitive load of a specific activity. Furnishing a complicated drawing, for example, is cognitively more demanding than drinking a coffee after waking up in the morning. Thus, the more cognitive demanding an activity gets the larger the proportional size of the cut out area becomes.

In addition to that, protrusions towards the inner centre of the shape could render activities whereby somebody absorbs something that is not physical but rather of cognitive nature (as opposed to rescissions which encode the act of taking in something physical). Examples for non-physical absorbations include rading a moving book, watching an influential movie or listening to someone talk intersitng stuff. Anything applies here that gets the mind going in some way. Multiple protruding elements that create a dense inner mesh inside the overall shape are used to illustrate activities that may invole whole set of alternate cognitive mechanisms or various lines of thought.

Open shape transformation rules

Last but not least symbols that cary inner cut-out areas can be further differentiates by creating an opening on one side. The shape openings may vary in their size and orientation. In order to visually encode the catalogue of activities open shapes are used to illustrate activities where something gets shared with the outside world, specifically considering thoughts and ideas. Hence open shapes could – amoung other elements – be used to encode conversations with others, writing down ideas/thoughts as well as painting etc. Anything really that involves a line of thought flowing out of the mind to be shared with the outside world.

Simplifing Shapes

At this stage in the process the original shapes of the building footprinst extracted from the map image where simplified and standardized. The following graphics illustrate this endeavour.

Initially the various visual signs need to be placed on a standardized backdrop. The size of the backdrop is 128*128 pixels. The picture above explores the size of the core shape in relation to the backdrop when scaling the core shape into horizontal and vertical direction respectively. These experiments resulted in the decision to use a 30 pixel square as the standard core shape. This is forming the basal form for all the visual transformations required in order to encode the catalogue of activities.

The picture above displays the original basal shape in all its glory and in original size. Base on this shape the most significant original shapes that where extracted form the map got simplified. The poster below contains all simplified shapes together with their original counterparts as found in the source image.

Elements of a single day

The WIP shot below illustrates an attempt to encode a section of the catalogue of activities and emotions based on the rules and visual transformation logic derived from the previous exploration.

The series of symbols represents activities spanning from morning to afternoon of a single weekday.

It starts in the early morning with sleeping, dreaming, and lying awake in bed. All symbols used here are horizontally oriented as they encode actions with little physical activities. Also the inner cut-outs are accordingly small in relation to the overall shape. They become larger as the person engages in more conscious activities. The morning routine consist of activities such as cooking and drinking a cup of coffee, smoking a cigarette, lying in bed and surfing te internet with a computer, brushing teeth, dressing up etc.

Actions that are performed while standing such as smoking in fornt of the window are represented by a square basic shape. All actions that involve less physical activities, such as sitting on a chair or lying in bed are represented by more horizontal base shapes accordingly. In turn, actions involving higher levels of physical activity such as strolling or running are represented by more vertical basal shapes.

Furthermore, all activities that involve interactions with physical artefacts (computer work, showering, smoking, cooking coffee etc.) are visually encoded by shapes with protruding elements that descend towards the out side of the basal shape.

Activities such as eating, drinking, smorking which involve the body to absorb something physical form the outside are represented by recessions cut into the basal shape.

Activities though, that involve absorbing something mentally (rather than physically) such as listening to a talk or reading the newspaper are encoded using rim-based shapes with inward facing protrusions.

And finally, conversations, that is activities where thought and ideas are shared with the environment, are symbolized by open shapes.

Shape transformation logic summary

The illustration below sums up the shape transforming logic in conclusion:

• Horizontal shapes ≙ no physical action
• Proportional size of inner cut-out area ≙ level of cognitive load
• Outward facing protrusions ≙ interaction with physical artefact
• Inward facing protrusions ≙ aborbation of non-physical ideas and thoughts
• Open shpes ≙ sharing ideas and thoughts with the outside world

Five actions over 3.5 days

At this stage of the projective distinct and specific actions were recorded over the course of three and a half days. The data gathered with this exercise was more detailed than the original catalogue of activities that was assembled at the start of the project. This time not only the activity itself was recorded but also the specific time of the day as well as the duration of the activity, where appropriate. The information gathered with this exercise was supposed to form the data foundation on which to build the imaginary software application environment.

The five activities tracked where:

• Smoking cigarettes
• Drinking
• Eating
• Travelling with train, subway or tram respectively
• Sitting in front of the computer.

Data Types

Three data types emerged during the exercise:

1. counter: Here the interest is mostly focused on the total times something is done, eg. Smoked five cigarettes
2. Categorical. Here the interest in focused on the occurance of the same action with different units. E.g. drank coffee
3. Event/duration. Here the point of interest is when and for how long something happens. E.g. worked on computer for six hours

With that in mind, smoking was treated as a simple counter. Drinking was treated as a categorical counter. The categorical units are coffee, tea, and beer. Each of the units was encoded by a disticnt visual symbol. A composite icon of the three was used to visually define the overall activity. The same idea was applied to Eating – also a categorical counter. With Eating the sub units were breakfast lunch and dinner. In contrast to drinking, the duration of eating activities was tracked as well. Train travel activities were treated as an event. The duration of train travel was tracked accordingly. Likewise Computer related activities where treated as a event with activity duration being tracked as well.

The poster below shows the visual representation of the activities tracked over a period of 3+ days. The icons are arranged in reverse chronological order.

Software application environment.

For the final exercise the visual language along with the data points was moulded into a software application. Here the goal was to come up with some kind of time indicator. Since, up to this point, the collection of data as well as the excavation of the visual language from the Berlin map happened in a kind of arbitrary way, I am referring to this app as an imaginary software solution. It hab been put together without any aspiration to real world application or functionality for thet matter.

However, putting together this imaginary software was heavily influenced by Nathan Yau’s http://your.flowingdata.com, a twitter application that lets people collect data about themselves. With that in mind, the imaginary application showcased below may be seen best as visually driven redesign of a largely text based site that already exists.

The following series of images guide through the final design process.

Layout & Grid

Taking http://your.flowingdata.com as a starting point I also decided to have the imaginary Software run in a web browser window. The site layout is based on the Square Grid CSS framework, kindely provided by Avraham Cornfeld at http://thesquaregrid.com Avraham's framework is using 28x28 pixel grid. The picture below reveals the plain grid view in a 1024x768 pixel browser frame. (click to enlarge)

Actions Log

The image belowshows the application in its “Action Log” view. The Action log screen simply lists all recorded activities, together with the accompanying symbol, in reverse chronological order. (click to enlarge)

The symbols where scaled down slightly in order to fit into a 64x64pixel backdrop(the white sqauares). Each backdrop houses a visual sign. Whenever an event duration is tracked a white backdrop square visually extends to a continuous band that frames all other activities that occurred within the event duration.

Below each backdrop is a short text desctibtion of the activity that is encoded by the sign as well as the activity’s date and time. The format of the text is “activity Mmm dd, hh:mm”. Depending on the data type a counter, category or start/stop timestamp may be added to the descriptive text. Clicking on the icon or underlined text brings the user to an action specific screen. The action specific screen (see below) allows to explore a single activity in isolation form the others.

Navigational elements

The main navigation panel is located in the top right corner of the screen. It provides links to the three main views: Overview, Calendar view and the Action log.

A single line of text in the top left provides a headline for the current screen. For the Action Log page the headline indicates the date-range used for filtering the visible elements by date. In the current example all actions entered throughout the last year are displayed. That is a total of 132 activities which is also indicated in the information panel below the main navigation on the right. Below that, the information panel also list the 15 most recent activities. Again clicking on the icon or underlined text brings up the corresponding action specific screen.

Overview screen

The Overview screen (click on the image above to enlarge) can be accessed via the main navigation panel. It features the 17 most recent activates. The very most recent activity is featured most prominently on top.

Located Below the list of recent activities are tree statistical panels. The first of these panels indicates changes of the top three activities compared to the previous 30 days. (However, since I tracked only 3 days there is not much to show here). The second stats panel features a graph that indicates the amount of all activities over the last 30 days. The peak in the flat line in the image above indicates the 3 days during which I tracked my data. The third stats panel shows all activites by the hour of the day they were tracked. There are 24 bars. Each bar represents one hour of a day. The higher the bar rises, the more activities were tracked on average during that hour of the day.

The information panel on the right indicates the amount of activates tracked during the last 30 day. 132 in my case. Below that each activity is featured separately. There are 5 different activities in my case. The corresponding bar diagram indicates how often an activity was tracked. The most tracked activates are displayed on top the list. In my case, smoking was tracked 64 times, drinking 33 times , trainriding 12 times , computer work also 12 times and eating 11 times.

Calendar view

The calendar view can be accessed through the second item of the navigation panel.

On top, the screen features a calendar panel with a colour coded square for each day of the current year. Grey sqares indicate that no activity has been tracked for that particular day. A blue square indicates that there is in fact data available. The darker the color the more activites have been recoreded for that particular day. The four blue squares of February indicate the three and a half days on which I tracked my own data.

Below the calendar panel is a list of activities tracked throughout a single day (24 hours that is). Again, the list is displayed in reverse chronological order. Displayed by default is the last day on which data was entered into the system. Users can cycle through the data of different days by clicking on one of the blue (day)squares in the calendar panel above.

Action Specific view

The action specific screen can be entered by clicking on a specific activity, for example smoking. The action specific view features a statistical panel in the top area of the screen. As shown in the picture above, it provides an overview of how often an activity has been tracked in total, and at what time of the day the activity occurs most. Below the stats panal is the familiar calendar panel to select different days of interest. Similar to the previous view, below the calendar panel is a list of activities that were tracked during the day selected. However, when in action specific mode, only the activity of interest is displayed at full contrast. All other activitiew fade into the background.

Inspiration

As stated previously the imaginary software interface above is basicaly a more visual re-design of a largely text based twitter application that already existed. The app can be accessed at http://your.flowingdata.com

However, the great advantage of your.flowing data is that it is not imaginary at all. Insead it actually works and has a proper backend system attached to it that allows for some better statistical comparisons. Hence, I took the liberty to also enter my data points into this system. The panels below are some screenshots that I took form your.flowing data. They provide a brief summary for the activates I tacked:

Smoking most frequently tracked of all five activites. During the 3.5 days I smoked 70 fags in total. Cigarettes are smoked throughout the whole day exept during bed times. The most cigarrets per hour are consumed around lunch time each day.

Drinking During the 3 days I tracked 38 drinks in total: 21 coffees, 10 bears and 7 teas. Most drinks are consumed in the morning hours. These are mostly several good morning coffees each day. Beers are consumed mostly at night. Tea is consumed mostly at the weekend or during the evenings.

Computer work Computer work was tracked 12 times in total over the 3 day. This translates into 3 start computer-work timestamps and 3 stop computer-work timestamps respectively. On average there are two main computer work sessions each weekday: The first starts in the morning and lasts until the afternoon and the second starts in the evening and usually lasts until late at night.

Traveling by public transport This represents the daily commute to and form Potsdam. 6 start and 6 stop timestamps have been tracked over the 3 days.This represents two journeys a day. Travel time usually starts in the early and lasts for about one hour. Traveling back home happens mostly during late afternoon and early evening.

Eating Only the start and stop timestamp for a single breakfast hast been tracked during the 3 days. The breakfast was on a Sunday. This makes perfect sense since I usually skip breakfast during week days.

Treemap YFD provides a searchable treemap that can be browsed by time. The treemap displays data as a set of nested rectangles. Each rectangle represents one of the five activities. Its area is proportional to how often an activity occurred.

Time Sereies A time series chart allows to explore daily, weekly, or monthly patterns of activities.

Slides

Finally here is the link to the the deck that I showed at the final presentation of the project at FHP.

Funky Footprints slides...