In this final post for The Person in the (Big) Data edition of Ethnography Matters, we provide a collection of five mixed methods used by researchers to shine a light on the people behind the massive streams of data that are being produced as a result of online behavior. These research methods use a variety of digital and traditional methods but they share one thing in common: they are aimed at discovering stories. As Tricia Wang wrote on EM back in 2013, ‘Big Data needs Thick Data’. While ‘Big Data delivers numbers; thick data delivers stories. Big data relies on machine learning; thick data relies on human learning.’ In the methods outlined below, researchers outline how they have made the most of digital data using innovative methods that uncover the meaning, the context, the stories behind the data. In the end, this is still the critical piece for researchers trying to understand the moment in which we are living. Or, put differently, the ways in which we may want to live but are often prevented from by a system that sometimes reduces the human experience rather than to enable its flourishing. – HF, Ed.
1. Real-time Audience Feedback: The Democratic Reflection Method
Democratic Reflection is a new methodological tool for researching the real-time responses of citizens to televised media content, which was developed by a team of researchers from the University of Leeds and the Open University in the UK as part of a larger research project on TV election debates. The research for the project began by developing an inductive understanding of what people need from TV election debates in order to perform their role as democratic citizens. Drawing on focus groups with a diverse range of participants, the research identified five key demands — or ‘democratic entitlements’ — that participants felt debates and the political actors involved in them should meet. Participants felt entitled to be: (1) addressed as rational and independent decision makers, (2) given the information needed to make considered political judgements, (3) included in and engaged by the debates, (4) recognised and represented by the political leaders, and (5) provided with meaningful choices that allow them to make a difference politically. In the next phase of the research, the research team developed a new web-based app (accessible via mobile phones, tablets, and laptops), which allows viewers to respond to televised debates in real time and evaluate them using a range of twenty statements based on the five democratic entitlements. An experiment using the Democratic Reflection app was conducted with a panel of 242 participants during the first debate of the 2015 UK General Election, generating a dataset of over 50,000 responses. Analysis of the data provides a valuable new way to understand how viewers respond to election debates: we can explore general patterns of responses, compare different individuals and groups, track changes over time, and examine how specific moments and performances during the debates may relate to particular responses. – Giles Moss
More details:
Giles Moss’s post in this edition: ‘Democratic Reflection: Evaluating Real-Time Citizen Responses to Media Content‘
Coleman, Stephen, Simon Buckingham Shum, Anna de Liddo, Giles Moss, Brian Plüss and Paul Wilson. 2014. ‘A Novel Method for Capturing Instant, Nuanced Audience Feedback to Televised Election Debates. Election Debate Visualisation. Available at: http://edv-project.net/wp-content/uploads/2014/03/EDV-Briefing2014.04.pdf.
Research projects employing this method:
Coleman, Stephen and Giles Moss. 2016. ‘Rethinking Election Debates: What Citizens Are Entitled to Expect’, The International Journal of Press/Politics, 21 (1) 3-24.
2. Selfie workshops
The Rethinking media and sexuality education workshops sought to explore the institutional and individual barriers to a ‘practice-based’ or ‘strength-based’ approach to young people’s digital cultures (and digital literacies) within sex education and/or sexual health promotion settings. The workshops involved a short presentation of three relevant theories of media
communications, followed by practical activities adapted from The Selfie Course developed by Kath Albury, Terri Senft and colleagues. These activities invited participants to take (and discuss) their own selfies, as a means of eliciting critical reflection on their understandings of young people’s digital practices, and the ways these practices could be linked to other modes of communication, including expressions of sexuality and gender. Participants also reflected the ways young people’s digital practices were framed by educators and health professional, and the ways they were supported (or suppressed) by formal and informal workplace cultures and policies. Follow-up surveys assessed the extent to which participants found the workshop’s content approach useful, relevant, and applicable to their work. The project generated productive insights into the ways digital practices, platforms and technologies can be employed within qualitative media and cultural studies research, and the role that digital ethnography can play in social engagement. – Kath Albury
More details:
Kath Albury’s post in this series: ‘Thinking with Selfies’
Albury, K. and Byron, P. ‘Rethinking media and sexuality education report’ [open access version]
3. Trace ethnography
In my ethnography of Wikipedia, I found it increasingly difficult to explain what I was seeing to other academics without pulling up revision histories and pointing to codes like “rvv” or “{{db-a7}}” in metadata fields. Trace ethnography emerged out of a realization that people in mediated communities and organizations increasingly rely on these kinds of techniques to render their own activities and intentions legible to each other. There are jargons, conventions, and grammars learned as a condition of membership in any group, and people learn how to interact with others by learning these techniques. However, the affordances of mediated platforms are increasingly being used by participants themselves to manage collaboration and context at massive scales and asynchronous latencies. Trace ethnography is based in the realization that these practices around metadata are learned literacies and constitute an essential part of what it means to participate in many communities and organizations. In this understanding, reading through log data can be seen as a form of participation, not just observation — if this is how members themselves spend their time. However, it is crucial that this approach is distinguished from more passive forms of ethnography (“lurker ethnography”), as trace ethnography involves an ethnographer’s socialization into a group prior to the ability to decode and interpret trace data. – R. Stuart Geiger
More details:
“Trace ethnography” was first used in “The Work of Sustaining Order in Wikipedia: The Banning of a Vandal,” which I co-authored with my then-advisor David Ribes in the proceedings of the CSCW 2010 conference. We then wrote a followup paper in the proceedings of HICSS 2011 to give a more general introduction to this method, in which we ‘inverted’ the CSCW 2011 paper, explaining more of the methods we used. We also held a workshop at the 2015 iConference with Amelia Acker and Matt Burton — the details of that workshop (and the collaborative notes) can be found at http://trace-ethnography.github.io.
Research projects employing this method:
Ford, H. and Geiger, R.S. “Writing up rather than writing down: Becoming Wikipedia literate.” Proceedings of the Eighth Annual International Symposium on Wikis and Open Collaboration. ACM, 2012. http://www.stuartgeiger.com/writing-up-wikisym.pdf
Ribes, D., Jackson, S., Geiger, R.S., Burton, M., & Finholt, T. (2013). Artifacts that organize: Delegation in the distributed organization. Information and Organization, 23(1), 1-14. http://www.stuartgeiger.com/artifacts-that-organize.pdf
Mugar, G., Østerlund, C., Hassman, K. D., Crowston, K., & Jackson, C. B. (2014). Planet hunters and seafloor explorers: legitimate peripheral participation through practice proxies in online citizen science. InProceedings of the 17th ACM conference on Computer supported cooperative work & social computing (pp. 109-119). ACM. http://dl.acm.org/citation.cfm?id=2531721
Howison, J., & Crowston, K. (2014). Collaboration Through Open Superposition: A Theory of the Open Source Way. Mis Quarterly, 38(1), 29-50. http://aisel.aisnet.org/cgi/viewcontent.cgi?article=3156&context=misq
Related methods/projects:
- Documentary ethnography
- Participant-generated ethnography
- Cultural probes
4. Trace interviewing
Trace interviewing is a mixed method approach involving three steps: the visualisation of a user’s traces on multiple platforms, the (re)presentation of those visualisations during the interview process and the analysis of user behavior and interaction after the interview. The researcher and participant work together during the interview to interpret the data in visualizations. Users are encouraged to add missing contextual information, to reflect on the activities that led to the datapoints being created, and to alert researchers to the existence of inaccuracies and/or missing data and information. Trace interviews are useful for enhancing recall, validating trace data-generated results, addressing data joining problems and responding to ethical concerns regarding consent. Potential limitations or challenges include variable levels of visual literacy among interviewees and the need to recognise that, although interviewees are encouraged to co-create the analysis, data being presented to the user is not free of assumptions, and the researcher makes a variety of analytical and editorial decisions when generating useful visualizations. If the challenges of the method are successfully navigated, trace interviewing could allow researchers to respond creatively to new questions about the current, complex political communication environment. – Elizabeth Dubois and Heather Ford
More details:
Elizabeth Dubois’ post in this series:
Dubois, E., & Ford, H. (2015). Qualitative Political Communication | Trace Interviews: An Actor-Centered Approach. International Journal Of Communication, 9, 25. Retrieved from http://ijoc.org/index.php/ijoc/article/view/3378
Research projects employing this method:
Ford, H. (2015). Fact Factories: Wikipedia and the Power to Represent. (Doctoral dissertation). University of Oxford.
5. The Walkthrough Method for Mobile Apps
The walkthrough method systematically examines a particular app. It can generate knowledge about its environment of expected use – such as the vision of users put forward by developers, its operating model and its forms of governance. The app itself is also examined exploring registration processes, everyday use and ways of account suspension and closure. Unexpected appropriation may be explored, referring to ways of use not anticipated, and sometimes unsanctioned, by developers. This method draws from science and technology studies and cultural studies while acknowledging user testing practices from human-computer interaction and vernacular walkthroughs, such as those associated with video games. The method can be enacted as an imagined user, where a researcher engages with the app under certain conditions (e.g. by creating a particular kind of user profile). It is also possible to engage the method by sitting alongside people as they engage with an app and its environment of use. – Jean Burgess, Stefanie Duguay, Ben Light
More details:
Burgess, J., Light, B., & Duguay, S. (2015). Studying HookUp Apps: A comparative platform analysis of Tinder, Mixxxer, Squirt and Dattch. Panel presentation at ICA 65th Annual Conference: Communication Across the Life Span. 21-25 May, San Juan, Puerto Rico. Presentation slides: http://www.slideshare.net/doggyb/ica-presentation-final-copy
Research projects employing this method:
Working paper: “Is being #instagay different from an #lgbttakeover? A cross-platform investigation of sexual and gender identity performances” [open access version]
Related methods/projects:
McVeigh-Schultz, J. & Baym, N.K. (2015). Thinking of you: Venacular affordance in the context of the microsocial relationship app, Couple. Social Media +Society, 1(2), 1-13.
[open access version]
Light, B., & McGrath, K. (2010). Ethics and social networking sites: A disclosive analysis of Facebook. Information Technology & People, 23(4), 290-311. [open access version]
