February 24, 2012 | | 5 Comments

Online reputation: it’s contextual

This post is the first in a new category for Ethnography Matters called “A day in the life”. In it, I describe a day at a workshop on online reputation that I attended, reporting on presentations and conversations with folks from Reddit and Stack Overflow, highlighting four key features of successful online reputation systems that came out of their talks.

A screenshot from Reddit.com's sub-Redit, "SnackExchange" showing point system

We want to build a reputation system for our new SwiftRiver product at Ushahidi where members can vote on bits of relevant content related to a particular event. This meant that I was really excited about being able to spend the day yesterday at the start of a fascinating workshop on online reputation organised by a new non-profit organisation called Hypothesis. It seems that Hypothesis is attempting to build a layer on top of the Web that enables users, when encountering new information, to be able to immediately find the best thinking about that information. In the words of Hypothesis founder, Dan Whaley, “The idea is to develop a system that let’s us see quality insights and information” in order to “improve how we make decisions.” So, for example, when visiting the workshop web page, you might be able to see that people like me (if I “counted” on the reputation quality scale) have written something about that workshop or about very specific aspects of the workshop and be able to find out what they (and perhaps even I) think about it.

The organisers write that a reputation will be “a way for the user community to collectively calibrate the contributions of its members”. And if work of the new system will be “annotating” content on the web, then the reputation model will be an important part of that system. It turns out that calibrating contributions is not as easy as developing a scale and then marking a measure on a measuring jug. First you have to work out what the measure is. When is comes to peer production projects, the goal might be an vibrant volunteer community that comes together to produce something of public value. Wikipedia, for example, wants to see a growing volunteer community working together to build and improve a free encyclopedia, especially in areas that the encyclopedia is weak. Ushahidi, on the other hand, might want to see volunteers deploying and organizing around content in order to improve decision making and effective action in crisis situations.

When co-founder and general manager of the tremendously successful Stack Overflow and Reddit talked yesterday about how they developed their reputation systems, I was struck by the organic nature of their reputation model building process. Building reputation systems, it turns out, relies on an effective process more than a fancy algorithm. Successful codified reputation systems like those used by Stackoverflow and Reddit have developed their codes the way doctors grow skin on different parts of the body in order to use on other parts. Organically, along with the community, evolving in a process of increasingly shared responsibilities. Just the right amount of adherence to what the community currently values and how they already distribute rewards and attention, with just the right amount favoring or weighting of activities and values that achieved desired communal goals. Read More…

February 21, 2012 | | 0 Comments

DesignInquiry | Design City Berlin | Call for Participation

Reblogged from DESIGNABILITIES:

Click to visit the original post

Design Research Lab Berlin is collaborating with DesignInquiry on another experimental workshop-format that will take place in 2012. Our friends and colleagues Alexander Müller-Rakow and Florian Sametinger have been organizing this promising Berlin event, and DESIGNABILITIES is pretty much looking forward to it!

The Call for Participation is still open… Design City: Berlin In 2006, Berlin, Germany was designated by UNESCO as the first European City Of Design.

Read more… 279 more words

January 26, 2012 | | 5 Comments

Is rapid ethnography possible? A cultural analysis of academic critiques of private-sector ethnography (Part 2 of 3)

Sam Ladner, our guest blogger, started off the new years with a provocative question on Ethnography Matters, “Does Corporate Suck?” In Part 1, she proceeded to dissect this divisive question with a cultural analysis of academics critiques of industry ethnography as second rate or illegitimate forms of ethnography. Her post incited a lot of great discussions and surfaced many tensions that have long been difficult to articulate in both communities. 

In this second post of  her three part installment, Sam extends the cultural analysis from her first piece and offers methods that are more fitting for the shorter cycles of industry ethnography. In her final post, Sam will discuss how to maintain reflexivity in the both the private and academic settings.

Sam points out that research output can be compromised regardless if the ethnography is working in corporate or academic settings. What methods do you use to avoids compromising research in private-sector ethnography or academic setting ethnography? Please share in the comments!

A cultural analysis of academic critiques of private-sector ethnography

“The fact that there is no such thing as a perfect anti-sepsis does not mean that one might as well do brain surgery in a sewer.”

– Robert Solow

Robert Solow was an economist, but he could tell anthropologists a thing or two about how to deal with real constraints on the research process. Solow became famous for the “Solow Residual,” or contribution to productivity growth that remains “unexplained” even after careful, empirical analysis. Solow asserted that this unexplained residual was due to technological change.

Is it possible that Solow was wrong? Certainly. Economic growth during that period was accompanied by several other significant shifts, including but not limited to a rise in homeownership, more women entering the workforce, and the elimination of “separate but equal” education systems. Solow could have been wrong in so many ways, but the relevant question is not whether he was right, but whether he contributed insight to an empirically observed phenomenon.

This anecdote is a roundabout way of addressing the question: is rapid ethnography possible? Of course it’s possible. Will it provide us with unequivocal evidence of a given social phenomenon? Will it provide as deep insight as traditional ethnography? Will it be “perfect”?  No, no and definitely no. But, again, the relevant question here is whether it will give us meaningful insight into an empirically observed phenomenon. Read More…

January 16, 2012 | | 0 Comments

Cheering up the chatbot

The speech to text tool on my phone is convinced that “ethnography” = “not greasy.” (At least “not greasy” tends to be a postive thing?) Generally STT and voice commands work great on it though. You have to talk to it the right way: Enunciate; dramatic pauses between each word; don’t feed it too many words at once. The popular speech recognition application Dragon NaturallySpeaking emphasizes that users train the system to recognize their voices, but there’s always an element of the system training its users how to talk.

For entertainment purposes, it’s best to avoid the careful pauses and smush things together, producing text message gems like “Send me the faxable baby.”  It’s the mismatches between human intention and machine representation that can make using natural language interaction tools like STT, chatbots and speech prediction both frustrating and hilarious. When it’s bad, it’s really really good.

I’ve been playing with the game Cheer up the Chatbot the last couple days (from RRRR, “Where the games play you”).

Chatbot has an unusual way of interacting with people, as so many chatbots do.

Screen explaining Chatbot's mental disorders

Understandably, Chatbot is sad.

Chatbot with sad face

The goal is to get Chatbot to smile.

Chatbot with smile face

Open-ended questions make robots happy.

The game is a mix of bot and human-to-human chat, where you switch between talking to the game’s bot and to different players who are presented as the “Chatbot” speaker to each other.  When you hit a moment where there are enough players with different agendas online — including some who don’t know how the game works, some presenting as Chatbot, and some presenting as people — it can get weird.

Read More…

January 16, 2012 | | 0 Comments

Challenges of Urban Fieldwork: A Scavenger Hunt Approach

street scene, Ring Road, Accra, Ghana

My favorite and most longstanding site for fieldwork is a city, Accra, Ghana. There are some peculiar difficulties of urban fieldwork. The size and scope of such a “site” makes it difficult to know what to do, where to go, what to observe, etc. It can also be difficult given the element of anonymity and social distancing in cities. Cities contain diverse populations. You can never really arrive at the sense that you’ve mastered such a place, that you understand it comprehensively. Cities are culturally layered and contain much that is transitory and impermanent, they reflect the promiscuous intermingling of influences.

Beyond the particular topic you might be studying, how can you come to get a sense of the flavor or style of a city? I’ve been compiling for some time a list of questions that might be useful to ethnographers who are trying to figure out what to do, where to go, and what to ask in urban settings. Especially for short-term research stints, seeking out the answers to these questions (scavenger hunt style) could be a way to ramp up research quickly and get richer context for more specific research questions. As this is an evolving list, I invite additional suggestions.

On a related note, over at Kiwanja.net Ken Banks offers a list of 15 suggestions for travel habits when traveling in Africa that are usefully oriented to the demands of research. In particular, his suggestion to buy local newspapers and consume other local media I totally agree with. Having a TV where I’m staying I find indispensable, just to see what issues are presented on the local news and especially how they are presented as well as the kinds of local and foreign media content that are available to people living there. I tend to bring home piles and piles of newspapers from my fieldwork excursions with interesting articles marked with post-it notes. Read More…

January 15, 2012 | | 8 Comments

The ethnography of robots

Heather Ford spoke with Stuart Geiger, PhD student at the UC Berkeley School of Information, about his emerging ideas about the ethnography of robots. “Not the ethnography of robotics (e.g. examining the humans who design, build, program, and otherwise interact with robots, which I and others have been doing),” wrote Geiger, “but the ways in which bots themselves relate to the world”. Geiger believes that constructing and relating an emic account of the non-human should be the ultimate challenge for ethnography but that he’s getting an absurd amount of pushback from it.” He explains why in this fascinating account of what it means to study the culture of robots.

Stuart Geiger speaking about bots on Wikipedia at the CPoV conference by Institute of Network Cultures on Flickr

HF: So, what’s new, almost-Professor Geiger?

SG: I just got back from the 4S conference — the annual meeting of the Society for the Social Study of Science — which is pretty much the longstanding home for not just science studies but also Science and
Technology Studies. I was in this really interesting session featuring some really cool qualitative studies of robots, including two ethnographies of robotics. One of the presenters, Zara Mirmalek, was looking at the interactions between humans and robots within a modified framework from intercultural communication and workplace studies.

I really enjoyed how she was examining robots as co-workers from different cultures, but it seems like most people in the room didn’t fully get it, thinking it was some kind of stretched metaphor. People kept giving her the same feedback that I’ve been given — isn’t there an easier way you can study the phenomena that interest you without attributing culture to robots themselves? But I saw where she was going and asked her about doing ethnographic studies of robot culture itself, instead of the culture of people who interact with robots — and it seemed like half the room gave a polite chuckle. Zara, however, told me that she loved the idea and we had a great chat afterwards about this. Read More…

January 13, 2012 | | 35 Comments

Does corporate ethnography suck? A cultural analysis of academic critiques of private-sector ethnography (Part 1 of 2)

  Ethnography Matters is happy to start the new year with a series of posts from guest writer, Sam Ladner. In this piece, Sam examines the different temporal conceptions of ethnographic fieldwork in industry and academia. 

Stay tuned for Part 2 of Sam’s discussion where she discusses how corporate ethnographers can avoid compromising research.   

Sam is a sociologist specializing in the social aspects of technological change. She mixes private-sector consulting work with academic research and teaching. Primarily an ethnographer, Sam is founder and principal with Copernicus Consulting, a social research company that consults on digital and industrial product design, organizational change, and consumer culture. She is also a Postdoctoral Fellow at the Ted Rogers School of Information Technology Management at Ryerson University in Toronto. She  has published in peer-reviewed journals such as Time & Society and The Canadian Journal of Communication. She is currently managing the Mobile Work Life project, which is investigating smartphones and work/life balance.

Part 1: A cultural analysis of academic critiques of private-sector ethnography

Corporate ethnography’s emergence ignited criticism that its quality and rigour was not as good as the ethnography practiced by academics. Academically trained social scientists have argued that private-sector practitioners are often not trained in anthropology or sociology, much less in the actual method of ethnography. Academics have argued that using ethnography for marketing and advertising is just more evidence of underhanded marketers attempting to dupe people into consumerism (Caron & Caronia, 2007).

And they are right.

Much of private-sector ethnography is as banal as it is ironic. In its bland quest to “understand the consumer,” it reduces culture to mere consumerism and thereby fails to achieve its own stated goal of understanding. This cynical veneer of cultural research disregards the truly transformative effect of “going native,” which is the first step to deriving both deep insight and innovation. Private-sector “ethnographers” are frequently ignorant to what ethnography actually is. The real essence of ethnography is the study of culture or as Geertz would say, the “webs of significance” or the meaning individual social actors ascribe to objects, events, or people. “Ethno” derives from the Greek word “ethnos” meaning folk or culture, while “graphy” derives from “grapho” or “to write.” Most corporate ethnographers neither study culture nor write about it. Instead, ethnography is simply as “on-site research,” such as an in-home interview, and “written up” as a series of meaningless video clips or as the truly stupefying Power Point presentation.

But these critical academics are also wrong. Read More…

November 25, 2011 | | 2 Comments

Qualitative research is not research at all?

Image of building with torn sign reading "Rant"

Rant this way ~ Photo by Nesster, CC BY-SA

Heather pointed out these comments by Bob Garfield from a recent broadcast of On the Media (“Sentiment Analysis Reveals How the World is Feeling“):

I’ve been arguing for years that qualitative research, focus groups and the like, are not research at all. They don’t generate data. It’s statistically insignificant, easily manipulated, and from my perspective just as likely to be exactly wrong as exactly right.

Garfield then adds:

But it seems to me that what you’re dealing with is something that deals with all of my objections, because you’ve got the world’s largest focus group.

Sigh. This is wrong on so many levels, and anyone who is interested in ethnography already knows why, but just to touch on some of the problems:

  • Qualitative research can generate data. The tweets used in Johann Bollen‘s [1] sentiment analysis (the subject of this OTM episode), interview transcripts, field notes, photos, audiorecordings, visual recordings: all data. Some research within the qualitative tradition also generates numeric data [2] by, for example, calculating measures of intercoder reliability, or in the analysis of card sorting tasks.
    -
  • There is a lot more to statistical testing than statistical significance (and some controversy among statisticians about overuse of significance testing). There is also more to quantitative analysis than statistical testing. Bayesian inference, for example, could be thought of as quantitative analysis that is not necessarily statistical testing.
    -
  • Similarly, qualitative research cannot be reduced to “focus groups and the like”. The purposes, strengths and weaknesses of focus groups are very different from those of other qualitative methods such as [participant-]observation and one-on-one interviews [3].
    -
  • Using statistical testing as a marker for what is or is not research omits work that has formed the backbone of the sciences such as classical experimentation, disconfirmation by example, comparative methods for creating typologies and analyzing artifacts, etc.
    -
  • “Easily manipulated”? Yup, research findings in general can be manipulated. Statistical testing is really easy to manipulate.

Garfield’s statement also suggests either ignorance or dismissal of mixed methods research, which, I would argue, is increasingly becoming a gold standard for research in some fields, such as public health.

There’s a hint at why mixed methods have become so important in public health research in Garfield’s comment about “the world’s largest focus group.” Bollen’s use of a large collection of corpora is well-suited to his purposes, but other purposes can require different or additional kinds of work.

Let’s say I do a giant public health survey. If a minority in my sample doesn’t interpret a word or phrase in the same way that the majority interprets it, if some questions make no sense at all from their perspective, if people writing the survey have no idea what minority members’ concerns or experiences even are much less how they’re relevant to health, then the survey results will be meaningless for that social group.

There is no such thing as a survey that is not culturally informed. Without ethnographic work and awareness, surveys, public health information and campaigns, etc., will likely be culturally informed by those who are most powerful and/or in the majority. Qualitative research is indispensable for addressing structural health inequities affecting the less powerful. Should ethnographic work focused on these inequities be patted on the head and assured that it’s nice, but it’s not-really-research? Fortunately, the NIH does not think so.

Sometimes I wonder if people miss how widespread and useful qualitative work is because it can be invisible (see Tricia‘s related post about the ‘Invisibility of Ethnography‘). A couple recent episodes of On the Media may clarify the kind of research that Garfield is dismissing here, while at the same time (perhaps unknowingly?) depending on it.

On Nov. 4th, Garfield spoke with social media researcher danah boyd about “Parents helping kids lie online.” The paper [4] behind this interview presents quantitative summaries of survey data — “real” research, perhaps, to Garfield.  But hmm, how and why was this survey designed?

Read More…

November 23, 2011 | | 2 Comments

Data conversations: Can ethnographers do numbers?

I was grappling for a long time about how to approach numbers, statistics, patterns in my Wikipedia research. I’m an ethnographer, right, we’re supposed to be averse to using numbers. Right?

And then Rachelle sent me this really interesting piece by Ken Anderson and the crew from Intel’s People and Practices Research group called ‘Numbers Have Qualities Too: Experiences with Ethno-Mining’ (Ken Anderson et al, 2009) [PDF].

And I realised that there is no problem with numbers and statistics per se. The problem is when we use numbers divorced from the context in which they are extracted. The problem comes when we use numbers to speak for a community, rather than enabling the community to speak to the numbers. Read More…

November 22, 2011 | | 0 Comments

Technology Unfolds Over Time

Ghana Internet Cafe pic by Rachel Strohm CC BY ND 2.0

Many ethnographers stick to one place or region throughout their careers. Perhaps the memory of the trials and travails of entering the field in the first place, of early incomprehension and discomforts, the exhaustion of language learning, makes them shudder to imagine starting that all over again. Social ties to the field can be maintained in new ways (such as through Facebook). For example. Over time these relationships become deeper, richer. It becomes easier to ask more sensitive and private questions. One develops a growing capacity for insight into a culture, for the non-public side of society, for a better understanding of social performances vs. personal idiosyncrasies, the cleavage points beyond a society’s well-ordered face.

After 7 years of traveling to Ghana, I’ve started to see this sense of time and of change emerge in my own work as well. In part experiencing this more private side of life, but also observing firsthand the changes made sharper and more apparent by my absences. The Internet café scene in Ghana is not what it was when I started fieldwork in 2004. It was around 2008 that I started to see reports from the news media and people in Ghana about ‘sakawa’ a vernacular term that referred to Internet fraud. This was a term only whispered about during my fieldwork but had emerged around 2008 as part of a very public moral debate and was incorporated into the narratives of Ghana’s popular culture – in music and local video-films.

To formalize this sense of passing time I re-interviewed 12 individuals from my 2004 fieldwork. I believe this makes my study the very first longitudinal examination of the Internet in Africa. I was especially interested in whether, with time, the Internet yielded benefits to this group, delivered on their initial enthusiasm and conviction in the way the Internet worked (which in 2004 was bolstered by astonishing second-hand stories/rumors of big gains but very little successful direct experience among users). Read More…

← Older posts
Newer posts →
Follow

Get every new post delivered to your Inbox.

Join 39 other followers