Friday, February 22, 2013

What one application designer thinks of user research

I found this image by soldierant while browsing the Flickr Commons for photos tagged with "ethnography". Cute :)

It takes two: How researchers and clients can help facilitate successful ethnographic fieldwork


Ethnographic research has become an increasingly popular approach for understanding consumers and users of products and services. Corporate research managers, product designers, marketing departments, executives and other business decision-makers (also known as research clients) are becoming more intrigued with the benefits of ethnography as a tool for discovering and exploring new market opportunities, unmet needs, and insights that can be used to improve the customer or user experience.

In the business world, clients often like to be a part of the research projects they commission by attending research sessions or field visits as observers. This can be beneficial for many reasons and is a great way for them to get more bang for their buck. After all, they're the ones who will be involved in making important business decisions based on the results.

When clients attend research as observers, they are able to:

  • Become true stakeholders and get in on the "action" of research through direct exposure
  • Listen to what actual consumers/users have to say about a product, service, or concept
  • Make note of the insights they personally find most important and contribute their own ideas (rather than passively read about the findings in a report)
  • Understand how different types of research are actually conducted (which potentially facilitates better understanding of findings later on)
  • Corroborate (or counter) the findings of the research team based on what they saw or heard
Bringing clients into the process can also provide the research team with a knowledge base of business processes, systems, products/services and internal dynamics/factors to consider and incorporate during collaborative discussions of iterative research design, data analysis, the implication of research results, and formulating recommendations.   

---
 
For facility-based qualitative research methods like focus groups, one-on-one interviews, and usability testing, clients normally do not interact with research participants. They play a very laissez faire role and spend their time in separate quarters, observing research sessions through a one-way mirror and participating in post-research debriefs with the moderator. Sometimes during a session they might send in a note to the moderator if they have an urgent question or need something to be clarified. Other than that, their presence is purposefully minimized.

Attending ethnographic or observational research is quite different from learning about people in a facility. The benefits of having clients join in are the same in that they are exposed to the research process, can learn from consumers/users firsthand, and can share their perspectives throughout. Perhaps the biggest difference is that rather than hanging out behind a glass wall all day, they're coming along for the ride out into the real world, encountering firsthand the real lives of real people in their natural environments.

In consumer ethnography, you can't plan exactly how your research is going to go. It just isn't the same old controlled, contrived atmosphere you find in a research facility, so the dynamics that influence researcher and observer behavior are very different. Overall, there is more to consider in terms of doing good research, collecting good data, and building and maintaining rapport with participants. 

Based on my experiences as both a researcher and a client of research, I've put together a list of things that researchers and clients can do when preparing for and conducting ethnographic research to facilitate a successful experience for everyone. 

Originally, this post started out as a list of suggestions for clients who attend research, especially those who have never been exposed to ethnographic fieldwork. But the more I thought about my experiences, I was reminded that it's also up to the researcher to ensure as smooth a ride as possible. In my mind, the researcher has an even bigger responsibility in this regard because she's the one leading the fieldwork.

The following is by no means a complete list. I would love to know about the experiences of others, so please feel free to share your thoughts on additions or revisions to the list in the comments section below.

For researchers hosting clients in the field:
  • Establish a dialogue with your clients prior to fieldwork and maintain an open line of communication throughout the project. If necessary, educate inexperienced clients about the benefits and limitations of qualitative methods and what to expect when conducting fieldwork (first find out about their backgrounds and experience with research before making any assumptions - more on this below).  Provide ample opportunities for the client to ask questions. All of this helps to set expectations for the long haul.
  • Come up with a logistics plan and make sure everyone is on the same page about the fieldwork schedule and individual roles.
  • Take a moment to describe your personal approach to fieldwork. Emphasize that every researcher is different in his or her approach, but they usually share some commonalities.
  • Make sure to let the client know if you have any specific "fieldwork guidelines" of your own that you would like to have them follow. For example, some people prefer that clients hold off on asking questions of participants for at least one hour into the visit.
  • Try to get an understanding of each client's relationship to the project. What are their titles and roles at the company? Are they key stakeholders/research commissioners or do they play secondary roles? What do they plan to do with the findings? What are their perspectives on the project? Do they have personal or team agendas?
  • Never assume that your client doesn't know anything about research (and don't assume that they do, either). As a client, I once attended a series of in-home interviews being conducted by a major design research firm. The researcher I went with into the field made some incorrect assumptions about my background simply because I was a corporate representative. Another researcher at a different firm sold our company on some in-home ethnographies that ended up being mere in-home interviews. When I was asked to attend the research and told her that I was a trained ethnographer, she was a bit embarrassed, to say the least.
  • See if there are any relevant topics or questions the client would like to have addressed during fieldwork.
  • On the day of fieldwork, brief the client once more on what to expect out in the field. It helps if you can drive to the field site together, or meet for coffee or breakfast before the day starts to review the plan and wrap up last-minute concerns.
  • Before fieldwork, come up with a role for your client(s) to play and stick to during visits. For example, they can be a note taker, a videographer, or simply an observer. 
  • After fieldwork, take time to debrief with your clients by asking them what they found most interesting, what was most surprising, and what their key takeaways were. Reciprocate with some of your own thoughts. If necessary, remind them about the importance of avoiding sweeping generalizations based on just one or two participants.
  • Be open to making iterative changes to the research and field guide. Have discussions to address areas that need more or less attention. Likewise, be firm about staying focused on the key research questions at hand.
  • Have empathy. It's easy to get frustrated at a client who seems to forget the conversations you had about what to do and what not to do in the field. Just remember that he or she may be new to fieldwork and to research in general. Clients are likely not out to sabotage things.
  • Remember that clients have good ideas, too. You are the one being hired for your research expertise, but consider allowing the client to occasionally chime in with a question or thought during fieldwork. It creates a better partnership and makes things more enjoyable. In fact, clients may notice things you don't, and ask questions that yield useful information.  

For clients accompanying researchers into the field:
  • Make sure you're getting what you're paying for! This is probably one of the most important items on your checklist. Lots of research vendors claim to do ethnography when really they're just doing what could also be done in a research facility. If you're sitting in someone's living room while the researcher is mechanically going through an interview questionnaire, that is not ethnography. Familiarity with the approach will help you know what to expect.
  • Set expectations and clarify project roles early in the process. Share as much useful information with the research team as possible to help inform and guide each phase of the project.
  • Let the researcher do her thing. This is crucial to ensuring successful research. She is there to do the legwork, and you are there to listen and observe unless otherwise noted. Many researchers are very particular about their approach, which is usually based on extensive experience in the field. For example, I have a "mental checklist" of the topics, themes and issues I want to cover with participants during fieldwork. I appreciate it when clients have faith in my abilities and know that there is a rhyme and reason to what I'm doing. I once witnessed a coworker of mine (on the client side) literally take over the researcher's job and start asking the participant a series of poorly formed, guided questions. It was frustrating, totally out of line, and didn't help the research at all. It also put the researcher in a really tough position.
  • Let the participant talk. Sometimes the researcher will ask a question for which the answer is completely obvious, but it's important to hear the participant talk about it in his or her own words. You might already know why people do a certain thing, but you may be pleasantly surprised by how someone else thinks of and describes their own behavior.
  • Although the researcher is skilled at being able to cover a lot of ground in a little bit of time, sometimes she'll end up spending more time on a particular topic than originally planned if there's a good reason for it.
  • No two participants are alike, so expect to encounter people from all along the spectrum of humanity, including people who have very different values, lifestyles and worldviews than your own.
  • Tell the researcher if you feel something important was left out of a session, as long as it is relevant to the study's major research questions. This way she can be sure to cover it next time.
  • Respect any preferences the researcher has around interacting with participants.
  • Take notes. Yes, you'll receive a research report when all is said and done, but you may notice things that the researcher or those analyzing the data (footage, transcripts, fieldnotes, etc.) do not.
  • Take a pulse on any assumptions, biases, or preconceived notions you might be carrying with you into the field. It's impossible to let them go completely, but make sure to keep them in mind when trying to understand how a participant lives his life or how he makes decisions. 
  • If the researcher doesn't mind, feel free to ask the occasional question. If you do, try to formulate them using neutral/objective and open-ended language. For example, instead of asking someone "Do you prefer X brand soda or Y brand soda?" you can ask "What brand of soda do you prefer to drink?" This will let the participant decide which direction to take the conversation in and come up with her own answer, rather than choosing from limited response options.
  • Try not to agree or disagree with anything that the participant says, or to give advice on  anything about which you may be an expert (e.g., home appliances, insurance policies, web design, engineering, etc.)
  • Much of what we learn through fieldwork isn't readily obvious during research, but comes out of the woodwork once we sit down to do our analysis.
  • If possible, please save any overly technical or specific questions for the end of the session. I personally do not mind when clients ask questions that are not related to the research after I've gotten through everything I need to cover. Just make sure you find out the researcher's preferences prior to fieldwork.
  • I've said it once, and I'll say it again. Fieldwork never, ever goes according to plan. It's best just to have a general idea of how you would like things to happen, and go in with an open mind and a flexible attitude. People's lives are crazy and in constant flux; they don't follow a script, which means that real-world research cannot follow a script. Be prepared for anything!