Sensory postcards: Using mobile media for digital ethnographies

As a mobile media practitioner interested in mediated everyday experience and urban space, my use of technology for collecting data, sharing impressions and observing cultural practices has shifted from using specialized equipment (high-quality portable recorders, professional cameras and video camcorders) to the smaller, more flexible and already at-hand iPhone (or equivalent Android, Windows, etc.). With a continuous stream of mobile applications and externals, both the design community and the community of ‘prod-users’ and researchers are adopting multimodal tools[i] in their practices. In this piece I want to present ‘sensory postcards’ as a model and method for do-it-yourself digital ethnographies that unite sensory ethnography[ii] and cultural studies[iii] toward questions around urban experience.

Sensory Postcards as method

Screenshots from the author’s sensory postcards using Soundcloud and dbBlox Check, posted on Twitter

So how are sensory postcards a method? Everyday mobile media production deserves study in its own right as a novel form of media literacy, signaled by participation in social media communities such as Instagram, YouTube and reddit (to name just a few). From a research standpoint, sensory postcards are a form of multimodal inquiry that engage sensory ethnography as an access point into urban life, place and human geographies, as well as power relations and models of situated learning. As an inductive approach, generating sensory postcards means sensing first, capturing second, and iterative interpretation as patterns settle into media artefacts. The metaphor of ‘postcard’ here is an attempt to evoke a ‘moment in time’ sensibility while de-emphasizing the visual component. In mobile videocam recordings the narrative of the event or action becomes central; removing that by using a static image and sound recording emphasizes instead the temporality of sound, allowing the listener to engage their imagination in constructing a scene without video filling in the blanks. Clean the palate, re-experience, re-engage. Below is a case study of the use of sensory postcards in one Vancouver neighborhood, starting with sound as a unique entrypoint.

Case Study: Yaletown, Vancouver, Canada

Yaletown is a wealthy area that overlooks the English Bay in the heart of downtown Vancouver. I wanted to explore how different spaces are characterized sonically and visually, and compare recordings I made with my direct experiences. One of the first things that caught my attention was how the landscape and soundscape interacted to form an almost intentionally designed experience. In particular, the careful arrangement of the visual environment tricked my ears into hearing less noise, and ultimately experiencing my surroundings as peaceful and serene (in correspondence with the ‘Sailboat’ postcard below) when the actuality was much more busy and noisy (reflected in the ‘Seawall’ postcard below).

The above are literally two sides of the same street, a few feet away from each other. On one side we have a popular open-patio restaurant, a lot of music and the sound of talking leaking out to the street. Across from it we overlook the marina and the seawall, which is often used by people biking and walking. Curiously, not only is the visual landscape different (and the atmosphere and connotation it carries), but crossing the street shaves off almost 10 decibels from the overall soundscape levels. One reflection here is that the perceptual convergence I experienced in putting together the soundscape with the landscape is less an intentional design (as if city planners actually considered sound in any aesthetic, rather than purely functional sense!), and more a result of habituation to constructed media images, where soundscapes are ‘replaced’ and carefully matched to the mood or atmosphere of each image.

The next set of comparison postcards from around the corner features a community plaza on one hand, and the inside of a supermarket right across from it.

(A compilation of these clips is also available on YouTube.)

The irony of course is that the supermarket is way noisier than the space outside, even though the area is fairly busy with local and distant traffic. This brings attention to the built environment and sound propagation, with sound diffusing outdoors, and being trapped indoors, reflecting off the many hard and shiny (but trendy!) surfaces and glass displays in the supermarket.

The next comparison comes from another block away, a community centre courtyard across from an urban playground. While the green wooded landscape photo made my ears privilege a soundscape of birdsong and soft wind, the similarly natural setting of the playground features elevated children’s voices competing with nearby traffic. On reflection, pre-conceived associations have a great deal to do with not only emplaced experience but also recordings – only later, when re-listening to these did I realize the two soundscapes were nearly identical, despite my ears honing in on different sounds and bringing in different cultural connotations (serene tree-scape versus chaotic urban playground).

Venturing further down the seawall to an alley right next to the water, the relationship between the visual and the audible – as well as the relationship between emplaced experience and cultural associations – came to the fore even more. While looking out to the marina opens a landscape of sky and luxury yachts that connote relaxation and freedom, the gulf is surrounded by modern multistory buildings with French glass windows and aluminum railings: an architectural setting that bounces every bang, clank and whir across the water and back. Yet once again, as I walked and sat along the marina I experienced the pleasant lapping of waves and gentle wind – then, when I listened back to the recordings (right there on the spot) I heard all the other noises.

Doing this case study offers a reflection about the intended and unintended design of urban space, as well as conscious and unconscious approaches to listening in different sociocultural spaces. One thing that stands out is the potential that mobile technology offers for the juxtaposition of different sensory modalities – visual and aural in this case, as well as the juxtaposition of emplaced and mediated experience. These convergences and disjunctions offer, I’d argue, a rich palette of analytical routes to the sensory ethnographer interested in relationships between inhabitation, everyday practice, cultural conditioning, flow and movement.

Multimodal toolbox

Screenshots from the author’s device, left to right: Audioboo, dB value-only, sound level photograph using Blox, Recorder the app, Soundcloud, Picle, SpeakingPhoto.

Throughout my research and ethnographic practice I’ve used mobile media and smart devices as vehicles for re-mediating sensory experience. Using a combination of smartphone apps for audio and visual capture, as well as environmental measurements, I started (back in 2009) to create snippets of sonic experience in a personal exploration of my everyday urban environment, a kind of ‘aural postcards’[i]. While an aural postcard might seem like an oxymoron given sound’s temporal nature, the cultural connotations of a postcard align with the way we create digital archives with mobile technology. Aural or sensory postcards are discrete media artefacts that reference experience, what Barthes calls in film theory a “still in quotation.” A sensory postcard is a kind of story. It’s multimodal: it can contain one or multiple media artefacts, reflection in spoken or written form, a drawing, a map; it can be shared online or kept in a personal digital archive. Over the course of 5+ years, I’ve generated an extensive archive of comparative multimodal vignettes, using the smartphone as a tool for exploring place and everyday media production.

The first application I used was Recorder – a high quality audio recording app. I started recording interesting soundscapes everywhere I went, as easily as pulling out my iPhone and tapping a button. Soon after, I added my first decibel measurement app – Sound Meter Pro by Studio Six Digital (and as of 2014 they have an Android version as well). This decibel meter has A and C scales for more accurate capture, as well as a microphone calibration option (for the audio geeks out there). Interestingly, I ended up taking most of my decibel levels using a more basic app – dB, made by Faber Acoustical. Both companies have entire audio suites for real-time analysis complete with external recording attachments. However, what dB allowed me to do was overlay a photo with the sound level value, adding a visual cue and short note to accompany the measurement (a similar tool I currently use is Blox dB Check).

As my informal ethnographic ‘snippets’ or ‘postcards’ archive grew, I realized I needed multimodal ways of capturing everyday experience in its full richness. My search for visual yet non-ocularcentric options led me to two other apps – SpeakingPhoto and Picle, both of which overlay a short sound recording onto a static photograph. Another aspect that emerged from my ongoing work was the importance of integrating the process of capturing everyday experience into the circuit of social networks. With an extra tap or two I could now post my sound level photographs and “social stories” to a host of micro blogging platforms. In fact, I began using my sound level photographs as a base for my photo-recordings, adding extra information to the sensory artefact.

Podcasting-style social networking platforms such as Soundcloud and Audioboo became a vehicle for me to record and share more extensive live commentaries and reflections from-the-scene of the experience, while short-hand apps such as decibel photographs and time-restricted audio-visual forms (Vine and Instagram being the most prominent social network apps) became ways to collect numerous impressions in ‘postcard’ format and instantly communicate different aspects of urban space and everyday experience.

In conclusion, without proclaiming the smartphone as a tool that democratizes the production of ethnographic knowledge, I want to suggest that participatory media culture and mobile computing have already created the conditions for everyday ethnographies on a wider scale and as a new form of media literacy and digital citizenship; conditions that can offer new models for researching and theorizing about urban design and urban everyday life.

Notes

[i] Pelle Snickars and Patrick Vonderau (eds.), Moving Data: The iPhone and the Future of Media, (New York, NY: Columbia University Press, 2013).

[ii] Sarah Pink, Situating Everyday Life, London, UK: Sage, 2012.

[iii] Henry Jenkins et al., Confronting the Challenges of Participatory Culture: Media Education for the 21st Century, White Paper prepared for the McArthur Foundation, 2006.

[iv] Walter Benjamin, “The Work of Art in the Age of Mechanical Reproduction,” Illuminations, New York: Shocken, 1968: pp. 217-252

SpeakingPhoto gallery: https://speakingphoto.com/galleries/milenad

Soundcloud gallery: https://soundcloud.com/ambient-sonic

Translation

English中文(简体)FrançaisEspañol

by

You can help improve the automatic translation by checking off “Edit Translation” after you select a language. Thanks!