Anthologize & User Studies
Friday, August 13th, 2010oy! Sometimes Twitter is not the right medium to have a discussion. Since I have more than 140 characters to say on the Anthologize user study and user experience discussion making the rounds.
Earlier this week Claire Warwick decided to Rain on Anthologize’s Parade; kicking off an active discussion about place of user studies in this kind of digital humanities tool building.
In a tweet earlier today, I suggested not throwing the “user study” out with the bathwater. I’m realizing now that part of the confusion in this conversation is that “user study” is a somewhat vague term that can mean several things. In Claire’s post she expressed concern:
I am really pleased that Anthologise had a UX team. But I can’t quite see what serious UX work you can actually do in a week. And this is a problem because if you are going to design something that users really want you need to study them first. You need to talk to them, interview them, observe them, feed back the results of this into the design process, try it again, retest, get more feedback etc.
While there have been several helpful responses from the UX team to Claire’s post (see Kathie’s comment on clarifying that interviews (albeit rapid) were conducted and the expertise of those involved), the comments I’m responding to are the ones that have felt more defensive – along the lines of “but look at all the downloads for @zotero, @omeka, @anthologize!” (sorry Tom) or @clioweb “Most academics write books that other academics haven’t thought of or asked for. No user surveys asking for what they want.”
Maybe it’s unfair to get my feathers ruffled about those (given the work that was done), but there has been less articulation of what sounds to me like a participatory design process. Rather than gathering requirements, they gathered people who are representative of users who would want Anthologize. While that’s different than the survey/interview/analysis process, it is a legitimate approach in and of itself. As is the rapid/agile kind of development I’ve seen come out of CHNM shops. I think proudly owning these approaches and articulating them more clearly would be a better response than pointing to the number of downloads or devaluing different approaches. (and as I’ve been writing, it seems like that’s been happening on twitter). This isn’t an argument about involving users, but rather what methods should we use?
While scholars may not conduct a survey to decide what the topic of their next book is (sorry Jeremy), the book (and it still is a book at this point, no?) that they do write is bounded by currently accepted practice – or it makes a case for a new set of practices. Perhaps this conversation can open up a new round of discussion about what UX means for the digital humanities, particularly an exploration of what kinds of UX methods most benefit the kinds of projects we are undertaking. Talking about “users studies” isn’t very helpful if you don’t know who they are (or who you’d like them to be). If your developer is just a developer, scratching their itches might not be the best approach. But what if your developer is also a recognized humanities scholar who is blending their expertise to create new tools? How are the UX methods that have emerged from corporate development environments compatible with humanities scholarship and tools? UX methods are often grounded in different kinds of social science – quantitative surveys, qualitative ethnography, structured interviews, etc. Is there a humanist UX approach that has been overlooked? (thinking about Johanna Drucker’s critiques of visualization at the MIT Hyperstudio conference). How do you match the effort at UX with the overall budget of the project (just how much did One Week cost anyway?). What are the right forums for sharing the scholarship of studying humanities tools? Kathie says that there will be ongoing work in this direction, I would be interested in hearing more about what that research will look like.
As a final word on this, there are worse ways of going about this. I’ve worked on projects that did user studies and then ignored much of what was learned to produce inferior (IMHO) products. My own user-centered work is often hampered because I don’t have the development chops I need to give people what they want. As much as Anthologize is about a product, it’s also about building a community that has the ability to support the tool as it grows and evolves. Already on-board are people who can commit code changes. What’s the vision for including people conducting systematic UX evaluation and assessment? (as opposed to the anecdotal stream of bug reports and feature requests that will come in.)
