Your team has just been given a new project to work on.
You think a minute about what to do first, and you come to the conclusion that you want to understand whom you are trying to help (i.e. your stakeholders), and what their needs are. Your first inclination is to go talk to those stakeholders.
But what if I told you they are going to lie to you?
You now think I must have had some pretty traumatic experiences to have such a cynical outlook on life, but hear me out. (That’s not to say that I haven’t been on few doozies in my life time).
Your stakeholders are lying to you, but they generally don’t mean to do it, and don’t realize they are doing it. They are being impacted by one or more cognitive biases that impact all humans. A cognitive bias is a pattern of deviation in judgment that occurs in particular situations and when it plays a part in elicitation activities, it often has the effect of your stakeholders lying to you, or perhaps even more disturbing, you lying to yourself.
Of the over 100 cognitive biases, a few have an especially potent impact on elicitation activities, and can be experience both by the stakeholder and the analyst.
Those experienced by the stakeholder:
- Response Bias – the tendency for stakeholders to answer a question based on what they think the analyst wants to hear, rather than their true belief.
- Herd Instinct – a form of group think where stakeholders adopt the opinions of the majority to feel safer and avoid conflict. This one is can be especially apparent when talking to a group of stakeholders.
- Bandwagon Effect – related to herd instinct, this is the tendency to believe things because many other people believe the same thing.
- Curse of Knowledge – when knowledge of a topic diminishes a stakeholder’s ability to think about it from a less informed and more neutral perspective (i.e. that of the analyst). This is especially prevalent when talking to very knowledgeable subject matter experts.
Those experienced by the analyst:
- Confirmation Bias – the tendency of an analyst to search for, interpret and remember information in a way that confirms their own preconceptions.
- Framing Effect – drawing different conclusions from the same information depending on how or by whom that information is presented.
- Mirror-Imaging – the analyst assumes that stakeholders think like they do.
Ok, there’s lots of opportunity for deception, either our stakeholders deceive us, or we deceive ourselves. Is all hopeless? Should we just give up and call it day because we can never trust anyone again, including ourselves?
No, of course not. There are some things you can do to address deception by stakeholders.
1) Don’t interview them.
There are plenty of ways to elicit information above and beyond interviewing your stakeholders, and because of the cognitive biases I described above, what they say they will do in certain circumstances will not always jive with what they actually will do. To find out what really goes on you are better off observing the process in action rather than just talking to people about it. At an insurance company where I used to work, the process improvement team would “staple themselves to a claim” (not literally of course) to understand and identify improvements in the claim adjudication process.
Another way to understand what really happens is to use data available to you about people’s use of a system or website. If you are working on a project to improve a system, find out what log information is available and analyze that information to determine actual patterns of use. This will help you make more informed decisions about how people actually use the system without falling into the trap of getting told what people think you want to hear.
For more insights into how to find out about your stakeholders without talking to them, see this article about Brandon Carlson’s session at Agile2012 Stop Listening to Your Customers.
2) If you must interview stakeholders, ask the right questions
Of course you aren’t going to be able to get away from not doing interviews at all, so when you do interview your stakeholders, make sure you structure the discussion so that you don’t increase the chances of cognitive biases showing up. They way to do that is to formulate questions in such a way as to get the data you need without those nasty biases showing up.
1. Talk about their life instead of your idea
2. Ask about specifics in the past instead of generics or opinions about the future
3. Talk less and listen more
Even though the Mom Test is intended for testing new business ideas with potential customers, there is a great deal of applicability for interviewing stakeholders as well. Rob wrote up the Mom Test and other ideas in his book titled The Mom Test. It’s a quick read, and has a lot of great ideas on how to structure good interview questions.
How do you avoid deceiving yourself?
This one’s a little tricky, but fortunately you have it within your control. The most important thing is to realize that these cognitive biases exist and the part they play in elicitation, analysis, and decision-making. Once you are aware of them and the impact that you have, you can stop occasionally and ask yourself questions such as “ok, am I falling into a confirmation bias here, and should I be looking for conflicting information?” or “am I putting different emphasis on some information compared to others because of who I heard it from?” Keeping those things in mind will prevent you from immediately falling into the trap that cognitive biases can often present.
If you find you need some help understanding cognitive biases and how to identify them, a couple of good books on the subject are Thinking, Fast and Slow by Daniel Kahneman and Predictably Irrational: The Hidden Forces that Shape our Decisions by Dan Ariely.