Archive by Author

The tools we use: Supporting Wikipedia analysis

The Ethnomatters team has been wanting to do a review of software tools for a while now but when we got down to writing them, we realized that there are already very comprehensive software reviews in places like the University of Surrey’s website. So we decided to rather compile short posts on the tools that each of us used in our last ethnographic project, highlighting what worked, what didn’t work and what we’re thinking of trying in the future. We’d love to hear from you about your own experiences so please feel free to add yours in the comments below for further reading!

For my latest project (“Understanding sources“), I needed to collect data from a really wide variety of sources. I had interview data, articles and papers from web, and then a multitude of Wikipedia talk pages, edits, history versions, related articles and image and video sources. For interviewing, I use my beautiful and incredibly trustworthy Zoom H2 audio recorder. I do my own transcriptions (as suggested by Jenna in order to get a really close understanding of the data) and for that I use ExpressScribe which seems to work pretty well. I like that you can use “hot keys” to stop and play and that the speed dial is in a good place for slowing down the dictation.Read More… The tools we use: Supporting Wikipedia analysis

Beyond reliability: An ethnographic study of Wikipedia sources

Almost a year ago, I was hired by Ushahidi to work as an ethnographic researcher on a project to understand how Wikipedians managed sources during breaking news events. Ushahidi cares a great deal about this kind of work because of a new project called SwiftRiver that seeks to collect and enable the collaborative curation of streams of data from the real time web about a particular issue or event. If another Haiti earthquake happened, for example, would there be a way for us to filter out the irrelevant, the misinformation and build a stream of relevant, meaningful and accurate content about what was happening for those who needed it? And on Wikipedia’s side, could the same tools be used to help editors curate a stream of relevant sources as a team rather than individuals?

Original designs for voting a source up or down in order to determine “veracity”

When we first started thinking about the problem of filtering the web, we naturally thought of a ranking system which would rank sources according to their reliability or veracity. The algorithm would consider a variety of variables involved in determining accuracy as well as whether sources have been chosen, voted up or down by users in the past, and eventually be able to suggest sources according to the subject at hand. My job would be to determine what those variables are i.e. what were editors looking at when deciding whether to use a source or not?

I started the research by talking to as many people as possible. Originally I was expecting that I would be able to conduct 10-20 interviews as the focus of the research, finding out how those editors went about managing sources individually and collaboratively. The initial interviews enabled me to hone my interview guide. One of my key informants urged me to ask questions about sources not cited as well as those cited, leading me to one of the key findings of the report (that the citation is often not the actual source of information and is often provided in order to appease editors who may complain about sources located outside the accepted Western media sphere). But I soon realized that the editors with whom I spoke came from such a wide variety of experience, work areas and subjects that I needed to restrict my focus to a particular article in order to get a comprehensive picture of how editors were working. I chose the 2011 Egyptian revolution article because I wanted a globally relevant breaking news event that would have editors from different parts of the world working together on an issue with local expertise located in a language other than English.

Read More… Beyond reliability: An ethnographic study of Wikipedia sources

From San Francisco to Cairo and back again: Collaborating across cultures

Annie Lin. Pic by Guillaume Paumier CC BY 3.0

I’ve been trying to talk to Egyptian Wikipedia editors for a project about the experience of Wikipedia editors in the Middle East and am finding it really difficult to connect to relevant people through their Talk pages. And so I went to talk to Annie Lin, Global Education Program Manager at the Wikimedia Foundation about how she engaged with editors in Egypt at the start of a project to get students in local universities to write Wikipedia articles. In this interview, Lin talks about ways for outsiders to gain access by giving up power, encouraging participation and changing communication styles and platforms where the culture demands it. She’s given me some great things to think about as I build a more grounded understanding of editing in the Middle East, and I’m sure there are some gems in here that will help others as they think about doing ethnography starting from online places. 

Annie Lin is excited. The first pilot project that she oversaw in Cairo, Egypt to encourage students in local universities to contribute to Wikipedia has been a success – and although the term has ended, many students are still editing.

May was the last month of classes but a lot of students say they’ll keep editing. It seems that the students are excited about the idea that they’re contributing Arabic topics in the Arab world.

The pilot project, involving 60 students from 7 classes in 2 universities, had students create articles in Arabic Wikipedia either as part of the curriculum or as an extra curricula activity. An initial survey asking students what would motivate them to edit Wikipedia had a sense of contributing information about Egypt or the Arab world as the most common motivation. Lin says that when they show maps of Portuguese Wikipedia compared to Arabic Wikipedia, professors and students are shocked at the low numbers of Arabic articles.Read More… From San Francisco to Cairo and back again: Collaborating across cultures

What does it mean to be a participant observer in a place like Wikipedia?

The vision of an ethnographer physically going to a place, establishing themselves in the activities of that place, talking to people and developing deeper understandings seems so much simpler than the same activities in multifaceted spaces like Wikipedia. Researching how Wikipedians manage and verify information in rapidly evolving news articles in my latest ethnographic assignment, I sometimes wish I could simply go to the article as I would to a place, sit down and have a chat to the people around me.

Wikipedia conversations are asynchronous (sometimes with whole weeks or months between replies among editors) and it has proven extremely complicated to work out who said what when, let alone contact and to have live conversations with the editors. I’m beginning to realise how much physical presence is a part of the trust building exercise. If I want to connect with a particular Wikipedia editor, I can only email them or write a message on their talk page, and I often don’t have a lot to go on when I’m doing these things. I often don’t know where they’re from or where they live or who they really are beyond the clues they give me on their profile pages.Read More… What does it mean to be a participant observer in a place like Wikipedia?

A sociologist’s guide to trust and design

Coye Cheshire at a recent seminar at UC Berkeley’s BID Lab entitled "Trust, Trustworthiness, or Assurance? Considerations for Online Interaction and Technology-Mediated Communication" Pic by Heather Ford licensed under a CC BY SA 3.0 license.

Trust. The word gets bandied about a lot when talking about the Web today. We want people to trust our systems. Companies are supposedly building “trusted computing” and “designing for trust”.

But, as sociologist Coye Cheshire, Professor at the School of Information at UC Berkeley will tell you, trust is a thing that happens between people not things. When we talk about trust in systems, we’re actually often talking about the related concepts of reliability or credibility.

Designing for trustworthiness

Take trustworthiness, for example. Trustworthiness is a characteristic that we infer based on other characteristics. It’s an assessment of a person’s future behaviour and it’s theoretically linked to concepts like perceived competence and motivations. When we think about whom to ask to watch our bags at the airport, for example, we look around and base our decision to trust someone on perceived competence (do they look like they could apprehend someone if someone tried to steal something?) and/or motivation (do they look like they need my bag or the things inside it?)

Although we can’t really design for trust we can design symbols to signal competence or motivation by using things like trust badges or seals that signal what Cheshire calls “trust-warranting” characteristics. We can also expose through design the “symptoms” of trust – by-products of actions that are associated with trust such as high customer satisfaction. But again, by designing trust seals or exposing customer reviews, we’re not actually designing trust into a system. We’re just helping people make decisions about who might behave in their interest in the future.Read More… A sociologist’s guide to trust and design

Online reputation: it’s contextual

This post is the first in a new category for Ethnography Matters called “A day in the life”. In it, I describe a day at a workshop on online reputation that I attended, reporting on presentations and conversations with folks from Reddit and Stack Overflow, highlighting four key features of successful online reputation systems that came out of their talks.

A screenshot from's sub-Redit, "SnackExchange" showing point system

We want to build a reputation system for our new SwiftRiver product at Ushahidi where members can vote on bits of relevant content related to a particular event. This meant that I was really excited about being able to spend the day yesterday at the start of a fascinating workshop on online reputation organised by a new non-profit organisation called Hypothesis. It seems that Hypothesis is attempting to build a layer on top of the Web that enables users, when encountering new information, to be able to immediately find the best thinking about that information. In the words of Hypothesis founder, Dan Whaley, “The idea is to develop a system that let’s us see quality insights and information” in order to “improve how we make decisions.” So, for example, when visiting the workshop web page, you might be able to see that people like me (if I “counted” on the reputation quality scale) have written something about that workshop or about very specific aspects of the workshop and be able to find out what they (and perhaps even I) think about it.

The organisers write that a reputation will be “a way for the user community to collectively calibrate the contributions of its members”. And if work of the new system will be “annotating” content on the web, then the reputation model will be an important part of that system. It turns out that calibrating contributions is not as easy as developing a scale and then marking a measure on a measuring jug. First you have to work out what the measure is. When is comes to peer production projects, the goal might be an vibrant volunteer community that comes together to produce something of public value. Wikipedia, for example, wants to see a growing volunteer community working together to build and improve a free encyclopedia, especially in areas that the encyclopedia is weak. Ushahidi, on the other hand, might want to see volunteers deploying and organizing around content in order to improve decision making and effective action in crisis situations.

When co-founder and general manager of the tremendously successful Stack Overflow and Reddit talked yesterday about how they developed their reputation systems, I was struck by the organic nature of their reputation model building process. Building reputation systems, it turns out, relies on an effective process more than a fancy algorithm. Successful codified reputation systems like those used by Stackoverflow and Reddit have developed their codes the way doctors grow skin on different parts of the body in order to use on other parts. Organically, along with the community, evolving in a process of increasingly shared responsibilities. Just the right amount of adherence to what the community currently values and how they already distribute rewards and attention, with just the right amount favoring or weighting of activities and values that achieved desired communal goals.Read More… Online reputation: it’s contextual

The ethnography of robots

Heather Ford spoke with Stuart Geiger, PhD student at the UC Berkeley School of Information, about his emerging ideas about the ethnography of robots. “Not the ethnography of robotics (e.g. examining the humans who design, build, program, and otherwise interact with robots, which I and others have been doing),” wrote Geiger, “but the ways in which bots themselves relate to the world”. Geiger believes that constructing and relating an emic account of the non-human should be the ultimate challenge for ethnography but that he’s getting an absurd amount of pushback from it.” He explains why in this fascinating account of what it means to study the culture of robots.

Stuart Geiger speaking about bots on Wikipedia at the CPoV conference by Institute of Network Cultures on Flickr

HF: So, what’s new, almost-Professor Geiger?

SG: I just got back from the 4S conference — the annual meeting of the Society for the Social Study of Science — which is pretty much the longstanding home for not just science studies but also Science and
Technology Studies. I was in this really interesting session featuring some really cool qualitative studies of robots, including two ethnographies of robotics. One of the presenters, Zara Mirmalek, was looking at the interactions between humans and robots within a modified framework from intercultural communication and workplace studies.

I really enjoyed how she was examining robots as co-workers from different cultures, but it seems like most people in the room didn’t fully get it, thinking it was some kind of stretched metaphor. People kept giving her the same feedback that I’ve been given — isn’t there an easier way you can study the phenomena that interest you without attributing culture to robots themselves? But I saw where she was going and asked her about doing ethnographic studies of robot culture itself, instead of the culture of people who interact with robots — and it seemed like half the room gave a polite chuckle. Zara, however, told me that she loved the idea and we had a great chat afterwards about this.Read More… The ethnography of robots

Data conversations: Can ethnographers do numbers?

I was grappling for a long time about how to approach numbers, statistics, patterns in my Wikipedia research. I’m an ethnographer, right, we’re supposed to be averse to using numbers. Right?

And then Rachelle sent me this really interesting piece by Ken Anderson and the crew from Intel’s People and Practices Research group called ‘Numbers Have Qualities Too: Experiences with Ethno-Mining’ (Ken Anderson et al, 2009) [PDF].

And I realised that there is no problem with numbers and statistics per se. The problem is when we use numbers divorced from the context in which they are extracted. The problem comes when we use numbers to speak for a community, rather than enabling the community to speak to the numbers.Read More… Data conversations: Can ethnographers do numbers?

New geographies

xkcd’s Updated Map of Online Communities

I arrived in Nairobi last night after an absence of about five years. As I left the plane through the walkway, I took a deep breath and inhaled the familiar southern African smell that I always miss so much living in America. I walked through to customs and baggage claim and to my taxi and hotel and became aware of all the things I was noticing: my slight frustration at the absence of instructions about which line to stand in at the immigration hall; the fact that there was not enough room for my place of birth in the immigration paperwork; the fact that, in stark contrast to the Amsterdam Schiphol Airport that I had come from, this airport seems not to have changed in a decade or so.

I noticed how long we had to wait for our bags to come through, the nationalities of the people coming here, how closely they stood next to one another. And my driver, patiently waiting for me, familiar sign in hand. On the car ride to the hotel, I looked at billboards and noticed what was being advertised and who was being represented, the state of repair of the roads and the roadside flowers and how people drive and the smells of food and industry and bodies.

Read More… New geographies

Why doing ethnography is like walking around in other people’s shoes

by Rachelle Annechino and Heather Ford

Ethnographers must walk in sneakers before they can wear heels by experiencing the everyday in the context that they are studying (pic: CC0)

“Ethnography is the eye of the needle through which the threads of the imagination must pass… Experience and the everyday are the bread and butter of ethnography, but they are also the grounds whereupon and the stake for how grander theories must test and justify themselves. They should not be self-referenced imaginings but grounded imaginings.” (from the Forward to The Ethnographic Imagination, Willis)

So you’re interested in technology research and you’ve heard about this thing called ‘ethnography’ but what is it exactly? What does it mean to be an ethnographer? What makes ethnography special? We take a look at why ethnography is like walking around in other people’s shoes.