Thursday, May 04, 2006
talking usability in the meeting room
"We need to get a decision about this -- I'll book a meeting so we can get resolution on it."
In business, meetings are where action happens, where information is disclosed and debated and decisions are made. Meetings allow many parties to be involved in discussion and decisions more efficiently than one-on-one discussions, or (in many cases) group emails. Business culture gripes about the time meetings take (often simply a sign that people are too busy generally), adding to the pressure to make meetings productive. A meeting is not successful unless issues are closed out, and next steps agreed that are substantively different from what had been discussed at the current meeting. Employees are drilled in this culture, taking courses on running meetings, and being effective participants in them. Meeting attendees who don't follow the code of conduct are admonished in front of their colleagues.
The culture of efficient business meetings doesn't transplant well to issues involving design and user needs. Because meetings are so deeply ingrained into the corporate mindset, organizations often view meetings as the best way to make decisions about the usability of designs. Although meetings have their place in a user centered design process, they can equally hinder a user centered solution.
A standard organizational behavior is to call a meeting to get "resolution" about an issue. If we aren't sure how to design something, let's call a meeting to get resolution on it. The assumption is that if you get a cross sample of stakeholders in a room together, they can have a rational discussion and get a decision then and there. What also is assumed, sometimes without full consideration, is that everyone involved in the discussion has adequate knowledge and information to make a good decision.
The more finely-tuned the meeting process, the less obvious that meeting participants may not have sufficient information to decide usability issues. A carefully considered invitation list, a well structured agenda, and PowerPoint slides can give the appearance that decisions can be made. For many business issues this is in fact possible -- participants have top-of-the-head knowledge necessary to respond to topics that arise in the meeting, so decisions can be made then. But for usability-related issues, top-of-the-head knowledge of general business participants is generally inadequate (this would not necessarily be an issue if we are talking about a more specialized group meeting, such as an in-house design team.) The stakeholders at decision-related meetings are generally not the same people who will be using a system, and are not the right people to comment on user needs. And users aren't the right people to make final design decisions. Research and decision making are separate activities.
Some consultants have tried to adapt usability techniques to fit corporate meeting rituals, but I question the quality of information such techniques develop. The temptation is to borrow techniques familiar to corporate employees taken from decision making and training workshops, such as voting or role playing, and use them as the basis for making decisions about designs. One evaluation technique I have seen discussed involves a set of rules for different participants to pretend they were different kinds of users. The participants have to fill out worksheets, follow participation rules, then make decisions based on what they saw in the role play. I can imagine that participants, committed to doing a good job and spending effort to follow the process faithfully, would believe they are accomplishing something valuable at such a meeting. But we can't expect Hal in marketing to know how a customer who lives or works in very different circumstances will behave. People struggle enough trying to recall how they do their own work on computers when they aren't sitting down in front of one. Getting a surrogate to imagine how a mythical user does a task strains credibility even more.
So when are meetings useful in user centered design? In general, meetings are useful in the beginning, at the end, but not in the middle of design research. In the beginning, meetings can be useful to explore issues early. At this point, we aren't making design decisions, we are just trying to get a picture of what issues might need detailed consideration, so top-of-the-head comments are useful. We'll verify this information later through contextual or behavioral research, or user testing. Research needs to be done outside of a meeting setting. Group workshops are fine, since unlike meetings their purpose is simply to elicit user data, not to make decisions. At the end of of a design research phase, we have some results to report at a meeting, which can provide a basis for decisions about next steps.
The pressure will always be on for instant answers to allow quick decisions. The seduction of meetings is that they collapse the answer and decision making processes, which need to be separate activities in usability. Business people are accustomed, once showing up at a meeting, to be rewarded with a decision at the conclusion. It is therefore vital to emphasize the difference between a workshop and a meeting, and not to agree to meetings unless there is sufficient user data to make design decisions.
But in addition to knowing what is not appropriate about meetings in usability, we need to be appreciate why meetings are attractive to business people, and address the underlying needs that make them attractive. By combining discussion (the offering and analysis of information) with decision making, meetings can be fast, and they make clear the connections between the data with decisions. This suggests we need to work to make data gathering and analysis as quick and clear as possible, to make make data responsive to decision making (gathering data to address the agenda of a scheduled meeting) and transparent (decisions reflect data that was widely understood by decision makers.) These aren't new goals, but there's still plenty of room left for improvement.
In business, meetings are where action happens, where information is disclosed and debated and decisions are made. Meetings allow many parties to be involved in discussion and decisions more efficiently than one-on-one discussions, or (in many cases) group emails. Business culture gripes about the time meetings take (often simply a sign that people are too busy generally), adding to the pressure to make meetings productive. A meeting is not successful unless issues are closed out, and next steps agreed that are substantively different from what had been discussed at the current meeting. Employees are drilled in this culture, taking courses on running meetings, and being effective participants in them. Meeting attendees who don't follow the code of conduct are admonished in front of their colleagues.
The culture of efficient business meetings doesn't transplant well to issues involving design and user needs. Because meetings are so deeply ingrained into the corporate mindset, organizations often view meetings as the best way to make decisions about the usability of designs. Although meetings have their place in a user centered design process, they can equally hinder a user centered solution.
A standard organizational behavior is to call a meeting to get "resolution" about an issue. If we aren't sure how to design something, let's call a meeting to get resolution on it. The assumption is that if you get a cross sample of stakeholders in a room together, they can have a rational discussion and get a decision then and there. What also is assumed, sometimes without full consideration, is that everyone involved in the discussion has adequate knowledge and information to make a good decision.
The more finely-tuned the meeting process, the less obvious that meeting participants may not have sufficient information to decide usability issues. A carefully considered invitation list, a well structured agenda, and PowerPoint slides can give the appearance that decisions can be made. For many business issues this is in fact possible -- participants have top-of-the-head knowledge necessary to respond to topics that arise in the meeting, so decisions can be made then. But for usability-related issues, top-of-the-head knowledge of general business participants is generally inadequate (this would not necessarily be an issue if we are talking about a more specialized group meeting, such as an in-house design team.) The stakeholders at decision-related meetings are generally not the same people who will be using a system, and are not the right people to comment on user needs. And users aren't the right people to make final design decisions. Research and decision making are separate activities.
Some consultants have tried to adapt usability techniques to fit corporate meeting rituals, but I question the quality of information such techniques develop. The temptation is to borrow techniques familiar to corporate employees taken from decision making and training workshops, such as voting or role playing, and use them as the basis for making decisions about designs. One evaluation technique I have seen discussed involves a set of rules for different participants to pretend they were different kinds of users. The participants have to fill out worksheets, follow participation rules, then make decisions based on what they saw in the role play. I can imagine that participants, committed to doing a good job and spending effort to follow the process faithfully, would believe they are accomplishing something valuable at such a meeting. But we can't expect Hal in marketing to know how a customer who lives or works in very different circumstances will behave. People struggle enough trying to recall how they do their own work on computers when they aren't sitting down in front of one. Getting a surrogate to imagine how a mythical user does a task strains credibility even more.
So when are meetings useful in user centered design? In general, meetings are useful in the beginning, at the end, but not in the middle of design research. In the beginning, meetings can be useful to explore issues early. At this point, we aren't making design decisions, we are just trying to get a picture of what issues might need detailed consideration, so top-of-the-head comments are useful. We'll verify this information later through contextual or behavioral research, or user testing. Research needs to be done outside of a meeting setting. Group workshops are fine, since unlike meetings their purpose is simply to elicit user data, not to make decisions. At the end of of a design research phase, we have some results to report at a meeting, which can provide a basis for decisions about next steps.
The pressure will always be on for instant answers to allow quick decisions. The seduction of meetings is that they collapse the answer and decision making processes, which need to be separate activities in usability. Business people are accustomed, once showing up at a meeting, to be rewarded with a decision at the conclusion. It is therefore vital to emphasize the difference between a workshop and a meeting, and not to agree to meetings unless there is sufficient user data to make design decisions.
But in addition to knowing what is not appropriate about meetings in usability, we need to be appreciate why meetings are attractive to business people, and address the underlying needs that make them attractive. By combining discussion (the offering and analysis of information) with decision making, meetings can be fast, and they make clear the connections between the data with decisions. This suggests we need to work to make data gathering and analysis as quick and clear as possible, to make make data responsive to decision making (gathering data to address the agenda of a scheduled meeting) and transparent (decisions reflect data that was widely understood by decision makers.) These aren't new goals, but there's still plenty of room left for improvement.