Tracktile

Tracktile was designed and developed in response to a briefing on inclusive design & technologies issued by Microsoft Research. It was selected by Microsoft to be presented at the 2015 Design Expo held from July 27th – 31st in Redmond, Washington – together with the projects developed at seven other design universities from all over the world.

This page provides an in-depth documentation of the design and production process behind Tracktile.

During our digital prototype workshop we came in touch with the Spark Core and the IFTTT service. We chose epilepsy as use-case and built a prototype of a bracelet, which records vibrations during an epileptic fit.

Epileptic seizures can happen anytime and anywhere and the epileptic has no more control and might be in a dangerous position or location. In this situation, it would be useful.. - to call for help - to notify the nurse - to know the intensity and length of the seizure - to save the exact location, date and time - to record all this information directly in the medical file

We connected the Spark Core with IFTTT and created following „recipes“: If spark return 1 (bracelet measures vibrations): → then write date, time, place in google spreadsheet (= medical records) → then get notification on mobile device (= notify nurse) → then write a new tweet on twitter with date and time (= emergency call) → then retweet this message with „help is coming“ and date and time (= answering the emergency call)

We stumbled across a blog of a blind muscian which planted the idea of „sounds“ and „for the blind“ in our heads.

“I challenge all of you, think of this with your eyes closed, now reach out…touch the sound, pick it up, move it around.” — http://hackblindness.tumblr.com/post/44792212213/close-your-eyes-imagine

Soon enough we thought of a topic about preserving memories and sounds. We reached out in the internet and asked, how visually impaired people preserve their memoires.

“I experience a lot of life through hearing. Certain music and also video recordings with sound bring back great memories. I once had a tape of my first intense love playing a guitar and singing. I treasured that tape and carried it with me for many years, even after my boyfriend and I had broken up. It was a wonderful memory much the way photographs are for so many people.” — http://www.quora.com/How-do-blind-and-visually-impaired-people-preserve-memories-of-holidays-and-other-important-events

Why do we save memories? + We forget. + We want to share them. + We want to relive them. + We want to understand time.

How about recording sounds instead of taking pictures?

We started to make our own field recordings in the city or in the nature and actually enjoyed sharing them. „Where did you record this beautiful bell sound?“ - „Just around the corner, you really should visit this church.“

Why is there no tool to share sound recordings as easily as photos? Why can’t I sit in a beautiful Café in Rome, and record the clatter of dishes, the chatter of the people, the jazz music in the background? Why can’t I share this recording with my friends at home? Sounds can be as evocative as pictures, maybe even more! They are immersive, because they require phantasy and because capture the atmosphere of a place.

Our topic should be about travelling with visual impairment. We defined a these questions for our further research and work:

→ How do visually impaired people prepare for travelling to unknown places?

→ How do visually impaired people travel? What problems do they have? What scares them? What do they enjoy? Is their experience different? To what do they pay attention?

→ How do visually impaired people preserve their holiday memories? Do they take photos and post them on Facebook?  Do they record sounds and videos? Do they collect little stones, shells and souvenirs to remind them of their travels?

We had a personal contact to ask our questions, but we needed more insights so we reached out on Facebook and had a little Google questionnaire. ([Questionnaire in English](http://goo.gl/forms/3I7IDupvkz „Questionnaire in English“) / [Questionnaire in German](http://goo.gl/forms/8YqTkWsAkJ „Questionnaire in German“))

Here are our questions with some of the best answers:

1. How do you prepare for travelling? - Along with the general preparation which all other do. Insure the place I am traveling to is accessible: easy to access transport or accessible roads for walking to near-by destinations, if I need someone to guide me then to think about who will do this and insure they can go with me, booking the necessary assistance (airport or train)
- make a list of what to take. Pack more than I need. Be prepared for anything.
- Working out exact routes, looking for the least changes. Booking assisted travel, where available. - I download maps of my destination to my talking gps, I research my destination online, including transit, cab numbers, location of my hotel and what's around my hotel - I make sure that busses into the town bus station are operating before travel, otherwise I have to book a taxi. When planning a journey, I book train or coach reservations online. I usually book assistance at each station I visit where possible. National Express are usually very good with assisting disabled passengers, however many rail networks tend to forget about passengers, which can be a problem.

2. How do you choose holiday destinations? - I usually pick a country where I can speak the language. I also look for good public transport links, such as busses or trains, and book accommodation that is not fair from train station or air port. - Same as sighted people, price range, what kind of vacation I want, whether beach or sightseeing. - Accessability, affordability, location. - making sure that there are nearby locations which are friendly. I.E. not noisy or over-crowded. get a feel from others or research on the locality of the place and how easy it would be for a blind person to travel I.E. no pavements for walking, too much traffic or no traffic lains system et, as well as general attitude of local society. - Must be able to travel there by public transport. Must be possible to walk around or use public transport while there. Find out about destination beforehand including where to get assistance.

3. What effort do you make to find locations that offer interesting non-visual experiences? - I can't think of anywhere that doesn't offer non-visual experiences. - I go for mainstream places and use residual sight and monocular. If mainly visual or 3D I avoid it as I have monocular vision. I don't kniw of any non visual experiences. It would be good to find out. - Ask when I get there, Synge museums allow blind/VI visitors to handle artifacts that are normally do not touch. - I don't really make an effort.

4. How do you preserve memories from holidays and travels? - usually using cameras and taking videos. I also try and buy small souvenirs - Photos for family, and viedos for all of us - audio or video recordings. Making sure that audio is always present. - Pictures and videos. I an see them better than real life, due to zooming - buying postcards and souvenirs

5. What are you looking for when travelling? - Clear audible announcements of location, next stop and final destination. - Reliable up to date public transport, information on arrangements readily available,friendly, approachable people and good landmarks. - To have fun, enjoy and relax. - Cheap,varied, interesting places and good infrastructure, busses, trains, cabs, history. - A nice experience that is comfortable and relatively easy

6. What aspect of travelling scares you? - foreign languages. getting lost. - Poor/no audible announcements resulting in me traveling too far it being lost. Travel staff who say they will make sure you get off at the right place but forget. - Getting lost or stranded when no helpnis available. Especially at night. - Not being able to make connections, not having assistance and being forgotten about.

7. Do you visit “classic” sights (e.g. the Eiffel Tower in Paris) or are these less interesting? - Yes, these are still interesting. - Yes, as above I tend to take photos as I can then see the sight better - I dislike tourist attractions. I prefer to find out about the lical or national culture, 'real life' rather than things constructed ir used entirelyfor the benefit of tourists.

8. What is important for you when travelling? Please see the attached diagrams

9. What do you bring home from your travels? - Usually shells from the beach (if there is one), little items like key rings or fridge magnets, postcards. I usually hold on to tickets and boarding passes and save them for scrapbooking. - post cards, t shirts, locally made goods. - Summering of substance, e.g. a piece of the Berlin wall. I.e. not a keyring. - Photos, tickets, leaflets, items bought locally local handmade items, books, clothes

10. What is your favourite souvenir? Why? - I picked up a beautiful shell from a beach in Majorca. It is very bumpy and smooth and it looks lovely. - bark painting and rainmaker from Australia as it was a ince in a lifetime trip, I travelled alone for over six weeks and learned so much about the history and heritage of Australia - a wooden box made of ceder wood. It brings back memorys of the trip and feels good in my hands.

Research summary

Visually impaired people.. → use the same tools for travel preparation as fully sighted people. → also travel alone. → like to be autonomous. → study infrastructure and public transportation before travelling. → like pictures, sounds and videos. → fear being forgotten, getting lost and poorly audible announcements. → want to feel safe when travelling.

BEFORE - Planning - Explore - Memorize

DURING - Experience - Record - Share (online)

AFTER - Remember - Re-experience - Share (physically)

The service: + order your individual map online + it will be shipped to your home + download the app for your smartphone to record your own sounds

The tactile map: + explore the city + get familiar with the basic topography + the map is enhanced with audible informations, e.g. street names and points of interest + your sound recordings will be applied on your map

The community: + insider tipps and additional informations by other users + ratings of places + sound sharing + your own field recordings

Summary of the benefits: → It’s tactile and audible. → It’s social. → It supports autonomy. → It supports learning and orientating. → It’s fun and explorative. → It works as a souvenir.

Our first prototype is a paper mockup and shows a part of Berlin Mitte. We picked a scale that would show enough area, but still shows enough details. It is build in paper with different thicknesses and surface structures for buildings, parks, streets and water. The width of streets corresponds with their size in reality: main streets are wider, smaller streets are narrower. It has high colour contrasts to fulfill the guidelines of colour accessibility.

First work on the prototype.

[insert time-lapse here]

Some pictures of our prototyping session in the interface lab.

making the prototype „beautiful“ and ready for the presentation.

It is halftime now. We presented our functional prototype in action and our concept at the mid semester presentations with Richard Banks.

We need to work on and define.. → Interaction concept: a key for the surfaces and city landscape → Service: use cases → Data model and hierarchy → Application and website → 3D Printing → Sensor technology → Prototype

We need to work on: - the prototype - the service: app for recording sounds - the service: website (order) [discarded!] - website: description about our project - documentation: the design and work process

We decided to revise the concept and focus on our key feature, to explore the city and fill the map with experiences. So [for the moment] we won't work further on the community aspect (or maybe just less?). Also, we don't want any „featuritis“ ;)

The mobile app is part of the service, as it helps to make one's very own field recordings. Each sound track will be geotagged, so they'll be placed on the right spot on the map again. The app is easy to use, intuitive and reduced on the basic functions: recording, synchronizing and playing recorded tracks. The visually impaired user will memorize the position of the buttons on the home screen. Also, voice-over and other accessibility features by the smartphone are supported.

See the app in action: https://marvelapp.com/8hjgcb

It's not easy to decide wether you should give your prototype detailed informations about streets, kerbs, walls, houses, buildings —or if you should have less details and only work out streets and blocks. So we asked the community again some questions about the orientation in the city:

1. On what do you orientate yourself when you walk in the streets? - I orientate myself on the pavement, my guide dog walks from kerb to kerb so it is all straight lines. - I have enough sight to see pavements, fences and people. - I have a guide dog so orientation isnt an issue, plus the sun gives me navigational aid

2. By doing it, what is important: the street, the kerb, the sidewalk, the walls of the buildings, ..? - I cant see the walls or the kerb so it doesnt make a difference, I can feel when we stop that we have reached the next down kerb. - tube maps. Google maps for detailed directions.

3. what informations are relevant for you, when you plan a city trip? (refering to e.g. orientation or institutions) - If I know the route, if not, do i know the route from an app to get where I want to go

4. Do you check e.g. bus connections before your trip, to be well prepared in interchange situations? - I phone train companies in advance for assistance and memorise the number of stops on bus routes. - Always very well prepared. Usually use stick if travelling alone to new places - I also check x2 all bus connections times and commit them to memory, on trains I use the assistance offered by the train company

I have found the trailing things like walls and grass can actually cause me to get more lost, it limits a persons ability to travel in a straight line through open space, it limits a persons ability to travel when they might be close to a wall or street. That is why I prefer to use traffic sounds, the sun, Cardinal directions, A GPS, and my cities address system.

We met Micha in Berlin, who is fully blind. He gave as a lot of inspiration, insights and addresses for our project. But most important, he was our first test person for our map prototype. - he enjoyed the combination of tactile map and audio - he likes to touch things, this map is a good alternative to GPS smartphone apps - the scale of the map works good, also the sizes of the streets (width) - different surfaces for e.g. parks and buildings would be OK, but it should stay reduced - a symbolic code for sights, museums, etc. would work for him - the whole map should stay minimalistic and should not be overloaded - it should only contain „things, that would help him along.” - „Man soll nicht den Faktor Mensch außer Acht lassen —man kann auch mal fragen.” (you shouldn't set aside the factor human —sometime you can also ask [for the way].)

During a talk with a restorer we got some interesting feedback about recording everyday sounds. She says that some background noises are significant and valuable in terms of cultural history, because like cityscape are subjects to change, their noises also do. + a noise is unique trough its content by its creation + noises belong to our culture like art does. Recording sounds and keeping them on a map stimulates people to collect sounds + this way, a gigantic sound database will be created on a natural way + people would start to „audio scrapboox“, it is not only for the blind + since our environment keeps changing permanently, some everyday noises we don't consider as interesting today, might be extinct tomorrow. + collecting sounds raises the awarness for noises, so it is also a teaching function + very unique and detailled noises will only be recorderd by the natural exploring and less by intentional collecting

It's never too late to have a good read about accessibility ;) so here is an extract of our inspiration:

„A Pocket Guide to Color Accessibility“ by Geri Coady; Published in 2013 by Five Simple Steps
„Erfolgreiches Studium für Blinde und Sehbehinderte“ by Sandra Bergmann, Julia Liebchen; FOM Berlin 2009
ISO zur Standardisierung taktiler Symbole
„Barrierefreiheit. Universelles Design“ - GERMAN UPA Usability Professionals' Association; PDF Fachzeitschrift 2011
„Collaborative Accessibility: How Blind and Sighted Companions Co-Create Accessible Home Spaces“ - Stacy M. Branham, Shaun K. Kane; Korea CHI 2015
„A Blind Person’s Interactions with Technology“ - By Kristen Shinohara and Josh Tenenber; communications of the acm 2009
„RegionSpeak: Quick Comprehensive Spatial Descriptions of Complex Images for Blind Users“ - Yu Zhong, Walter S. Lasecki, Erin Brady, Jeffrey P. Bigham; Korea CHI 2015
http://www.w3.org/standards/webdesign/accessibility
https://developer.apple.com/accessibility/
http://www.google.com/accessibility/

It is possible to download vector maps from Open Street Maps (OSM). Here is the link to our area: http://www.openstreetmap.org/#map=14/52.5145/13.3883 But of course we needed to reduce the layers, because the OSM map contains plenty of visual data we won't need for our project.

Why milling and not 3d-printing? We decided to machine mill the mao, because the results are more precise and have better contrasts. The MDF wood is much more stable than plastic or gypsum and milling needs less editing and rework than 3d-printing.

The width and depth of the streets needed to be defined, also the structure of water and woods/parks. In the end we chose five different heads for the milling machine to sink the wood in different ways. - streets have 4 different widths, depending on their size - water goes deeper and has waves - parks/woods have a bullet pattern and are perforated

The prepared the vector map and gave every sreet-width and pattern new layers (the colors are only a visual support) and used the file as basis for the further work in Visual mill.

Bildschirmfoto 2015-05-29 um 00.59.19.PNG

Bildschirmfoto 2015-05-29 um 00.59.43.PNG

[screen shots visual mill]

Since our first prototype was a simple paper mockup, we decided sink the second map in wood. To keep the high color contrast, we chose a black dyed MDF which was to be primed and painted in matt white.

To check all the tech involved, we made some milling test with only a little part of the final map. It was very important, because we wanted to check if the vector map was fine with the milling machine and we wanted to check the actual with and depth on the streets by feeling them. Also, it is important to check our material: - the milling heads - our MDF - the individual characteristic of the milling machine Working with such powerful tech comes with great responsibility and makes us handle the machinery with great respect. We need to be well prepared and considered the advices of fellow students. A huge THANK YOU to everybody who helped us!

In the next days we made three milling tests: - first with a random piece of painted MDF. We disliked the untidy rims of some streets and changed one of the milling heads - second, we used the black dyed MDF. The results were fine and the rims are sharp, but we still wanted to change the depth and the width of some streets - the third test was a part of our final painted black MDF and made some streets wider and the rims even sharper. We like the contrast and the feeling a lot and can't wait for the final map!

Today we went to the lab to produce the final map. After detailed analysis of the three milling tests, we decided to make a fourth test with some revisions and changements. We added a new pattern for the parks and for the water, which also should sink even deeper.

Then we were prepared for the real thing: we prepared the vectors and Visual mill and then... MILLING IT.

Preparations in the workshop lab @ FHP.

Milling test 4: adding the wave pattern and testing some new details

[video milling test 4]

Ok, we're ready! Let's do this!

The real deal -- milling our FIELD ONE!!! this is so exciting~~~

[time lapse video milling]

<3 <3 <3 we did it!! it is so beautiful, we have tears in the eyes~~~~~

We want to display a use case for our product and service in a short video. It should't be too descriptive, rather it should translate the magic of our idea and model. However, to work out our concept and the model itself, it is very helpful to work with use cases. Here is one of them:

Emma (27) wants to spend her holidays in Berlin. She has almost no more sight left and is not able anymore to read text and signs and is therefore dependent on audible announcements in the public.

She already did some research on Berlins sights and would like to visit the Brandenburg Gate, but also parts of the original Berlin wall, and is interested into Checkpoint Charlie. She choses her hotel after the distance to her main sights, in Berlin Mitte.

To familiarize herself with the city, she decides to order a tactile map. She visits the website www.wearettacktile.co

Emma types in Berlin and the location of her hotel, also places she doesn't want to miss. WE ARE TRACKTILE also gives her ideas for more interesting sights and places. She has the possibility to configure her individual FIELD and can change some details about her preferences.

Emma gets her very own FIELD delivered and now she explores Berlin already out of her home. She can check possible bus connections, so she won't need to fear to get lost. Emma can't wait to visit Berlin.

She starts her trip with a good feeling, because she is able to walk autonom through the streets, she already studied at home. With her smart phone and the TRACKTILE app she records some sounds of the environment. These files sync automatically with the TRACKTILE cloud, including the geo locations of the sounds.

Back home, Emma can't wait to tell her friends and family of her adventures in Berlin. The recorded sounds are already synchronized with her FIELD and she rexplores the places she visited before. Her family and friends are pretty impressed by Emma’s knowledge of the city. Everybody enjoy her stories, which are accompanied by the original sounds and are more exciting and vibrant than any souvenir could ever be.

For our video we need to strip the story down into a minimum, so it won't be too descriptive (and boring ;) ). It took us five (or more) story board revisions and two prototypes of the video, to achieve the magic and feeling we want to transmit.

Our keyframes are: - receiving package - reading the lable in Braille - unpacking the FIELD - exploring the FIELD, ego-inside-the-head-view - exploring the FIELD, ego view - details of FIELD - city scene, recording sounds - synchronizing and re-exploring FIELD - sharing with others

edit: we even decided to have less key frames. Just watch the video ^^

After we finished our prototype, we are ready to show it in action. Since we already did a video prototype of our video prototype ;) we were well prepared and shot the loads of footage.

We met the working group for mobility, environment and traffic of the ABSV (Allgemeiner Blinden- und Sehbehinderten-Verein) in Berlin to show them our prototypes. They recently work on the standardization of tactile symbols for maps and made a lot of studies about e.g. how detailed a tangible map can be.

They gave us qualified feedback to our prototype and loads of inspiration for further work.

There are a lot of interesting projects about maps for the blind, but here are some points why Tracktile distinguishes itself between others:

Talking GPS and audio guides only navigate until the next change of directions. In contrast to these devices, a tactile map provides a better overview of the city and the user can create his own mental map.
Exploring the map with two hands creates a better orientation, because it is possible to e.g. feel the distance between two points.
You don't need any additional installation, gesture tracking, screen reader or devices of any kind to manage the map. The augmented data is in the object.
Information layers can be turned on and off on the map itself.
It is not a bulky installation in a public place, but portable and for your own home.
It is available to everybody. The order runs in the internet, the production is individual, no detours with orders via assistant or health aid or clubs/organizations
It is intuitive and direct: you feel the information, you hear the information.
It is easy to understand and has a good usability. No further explications are needed.
We relinquish to use Braille, so nobody who can't read it will be excluded. Also, without Braille, the map is not labeled and tagged as an object for the blind, but for everybody.
It is modular.
It works as a souvenir and has social aspects.
It is a design object.

During our feedback session with Michael we realized the need of a portable version of the tracktile map. So we chose the technique of printing on swell paper and heating them up in a fuser, to make it tactile. We build our third prototype, which also works with audible announcements like the first and can be folded without breaking the contacts or the paper in the fold.

→ We would like to iterate Prototypes, for which we are in a very good starting position → Finding better technologic solutions or develop our own → Finalizing the App & Service

WE ARE TRACKTILE and we will present our project in action on thursday, 21.05.2015 at 14.00h, room D105 in the Design-Haus @ FHP.

The final presentations of the Microsoft Research Design class took place on thursday with Andreas Koller and Don Coyer. We presented our TRACKTILE project and the three prototypes in only 8 minutes (plus 2,5 minutes interlude with technical difficulties ;) ).

After the group presentations Andreas and Don announced the „winner team“ of our class: it is TRACKTILE!

The three of us are happy, overwhelmed, glad, relieved, amazed and feel just supercalifragilisticexpialidocious about this success. THANK YOU SO MUCH!