Rachel Shadoan
Editor’s note: This post for the April ‘Ethnomining‘ edition comes from Rachel Shadoan and Alicia Dudek. Following on the past posts about hybrid methods, this one features another interesting case study involving an on-line role-playing game. Their work correspond to a different approach, based on visualizations, than what we saw in the two previous posts.
Rachel Shadoan @RachelShadoan likes to find answers to interesting questions, and build interesting things using those answers. Currently she is answering interesting questions in the Intel Labs using a combination of data visualization, data mining, and ethnographic techniques.
Alicia Dudek
Alicia Dudek @aliciadudek is a design ethnographer and user experience consultant. Her passion is finding unusual solutions to the usual problems. Currently, she is finding unusual solutions for Deloitte Digital, where she specializes in engaging stakeholders in research insights through participatory design workshops.
____________________________________________________________________________
This is one of the visualizations we created for the research. The horizontal axis is date; the vertical axis is time of day as recorded by the Plant Wars server. Each mark on the chart represents a single instance of in-game training. In Plant Wars, a player can train their plant in one of three areas–speed, defense, or attack. The different colors of the marks corresponds to the characteristic being trained, while the radius of the mark encodes the number of stat points received from the training. The time shift can be seen clearly in this visualization: prior to June, this player starts playing in earnest around 16:00 server time. By October, the player has somewhat settled on starting play around 10:00 am.
A few weeks into our study of Plant Wars, an online text-based fighting RPG developed by Jon Evans of Artful Dodger Software, we encountered a mystery. We had visualized the server log data that records the players’ in-game activities, and discovered a pattern as obvious as it was inexplicable: in June 2009, the top Plant Wars players began slowly shifting the time of day in which they were playing. Over a period of six months, the time that the top players started playing each day shifted by nearly six hours. We poured over the server log data, checking the processing code for errors, for time zone issues, for any possible explanation of this shift in play pattern. Using only the server log data, we came up empty-handed. What was going on?
This mystery appeared during our final project for an MSc in Design Ethnography at the University of Dundee, in which we were exploring research techniques that make use of the strengths of both numbers and stories. During our time at Dundee, we had encountered the work of ken anderson, Dawn Nafus, Tye Rattenbury, and Ryan Aippersprach of Intel [1], in which participant tracking data was visualized and incorporated into ethnographic interviews as a scaffolding for participants to hang their stories on. This approach appealed to us, as it leveraged Rachel’s background in computer science and Alicia’s passion for visual communication, so we began looking for an opportunity to explore its possibilities.
Time, however, was limited—we knew we would be unable to build tracking software, track participants, analyze the data, and conduct an ethnography in the span of the few months we had to complete the project. Thus, we needed an opportunity with existing data stores. The Plant Wars developer, whom Alicia knew from her undergraduate days, was kind enough to open his system logs to us for visualization and analysis, and his player community to us for interviews and co-design.
Thus, we embarked on a study to understand both how the Plant Wars players played and why they played. Visualizing the data generated by the player’s in-game actions provided the map, answering the how and what questions. Interviewing the participants and participating in the game ourselves provided the key to that map, answering the why questions.
Before embarking on the data analysis, we announced the research to the player community and offered the opportunity for players to opt out. One of the players declined to have his data included: Jon stripped his information from the data before transmitting it to us for analysis. The data, spanning from January 2009 to June 2010, was a time stamped record of each action a player took, from training their plant for battle to participating in the in-game marketplace. The most frequent players could have 50,000 instances of training or more. Once we had the data, we processed it using Java. Largely, this consisted of reading in the text files exported from the server, splitting each user’s data into separate files, and translating the timestamp from 12-hr time to 24-hr time. Once that was completed, we began experimenting with visualization. Because the Plant Wars community was spread across the world, most of the interviews we would be conducting remotely. For this reason, we looked at creating web-based interactive visualizations. We cycled through a variety of tools, finding prefuse flare to be too slow to handle as many data points as we had and processing a bit to involved to learn within our time constraints. In the end, we put interactivity aside, settling on the unlikely Microsoft Excel. Excel gets short shrift as a visualization tool, but we found it to be both powerful (it could handle half a million data points, with a tolerable amount of whirring and laptop-gear grinding), and flexible. Instead of aggregating and abstracting the data, we opted to keep the visualizations as close to the player activity as possible. Using date as the horizontal axis and time of day as the vertical axis, we plotted each instance of player activity, watching the players’ daily patterns evolve over time. We constructed two varieties of visualizations. One variety used data from the whole community, the other, only an of individual player’s data. The cross-player visualizations we found to be interesting but not revelatory, especially considering the difficulty in dealing with visualizing the large number of data points. The visualizations of individual player’s patterns soon found their way into interviews.
Meanwhile, on the ethnographic side of the fence, we were immersing ourselves in the Plant Wars experience: playing Plant Wars, participating in the forums, and building relationships with other players. As we developed a feel for the community, we sought out particular players for interviews. Once a player had agreed to an interview, we created a series of visualizations of that player’s data, showing their training and battling behavior over time. The interviews, most conducted by Skype, were divided into halves: we began with open-ended discussion of Plant Wars and gaming in relation to the player’s life. Then, utilizing screen-sharing, we introduced the the data visualizations, asking the participant to explain the patterns therein. We used those stories to annotate the visualizations, creating artifacts that present both the how and the why of the patterns present.
This is the participant that solved the mystery for us. The shift in time, present not just in his playing patterns, but also in the patterns of the other top players, turned out to be a result of this player graduating from high school and adopting an erratic sleep schedule.
Which brings us back to our mystery. After much angsting over the possible cause of the strange time-shift in the patterns of the top Plant Wars players, we posed the question in the interviews. One of the top players, a mathematician whose careful training ratios are beautifully explicit in his visualized data, agreed with us that it was likely some bizarre bug in our processing code, or perhaps a server migration. It wasn’t until interviewing another top player, who had graduated from high school in June 2009, that we identified the true cause. After graduating, he told us, his sleep schedule had become much more erratic. Because the top players have few other challenging sparring partners in the game, their sleep schedules had shifted along with the recent graduate’s, so they would be able to battle each other. In the end, the relationships the players had with each other was the driving force behind that–and many other–patterns in their play.
In the end, the visualizations helped us access a world of player strategy and interaction that we would otherwise never have known existed. They provided the launching point for questions we didn’t even know we wanted to ask. But without the accompanying ethnography–the stories to hang on the visualization’s patterns–they’re just pretty pictures, providing little special insight of their own.
Check out past posts from guest contributors! Join our email groups for ongoing conversations. Follow us on twitter and facebook.
Reblogged this on Thinking about the future & ethnography and commented:
Hey c’mon everybody! And READ! Rachel wrote this about our badass project from back in the Masters day! Hooray!