(Note: This is the second installment of a multipart series on conducting customer interviews and discovering true migraine problems. And check out the first article in the series.)
Ok, you’re on the bandwagon. You understand that you need to a get to a new level when it comes to understanding your customers and finding their migraine problems. You need to understand them better than they know themselves.
Here’s the thing: When you go out to interview customers, they are probably going to LIE to you.
I’ve lost count of how many companies I’ve seen fail while insisting that everyone they’ve talked to loved their idea. They think they’ve conducted the customer interviews they needed, but they didn’t understand one of the most important rules to customer interviews:
People’s natural inclination is to lie during interviews
Think about the last time a friend told you about their “brilliant” new idea that you thought was really dumb. Were you 100 percent honest with your friend? I bet you weren’t. You, and most people, would probably lie a little bit to get out of the awkward situation. It’s a lot like that for your customers. Their natural inclination is not to be completely honest with you during an interview. They want to figure out the answer you’re looking for and give you that answer so they can leave the conversation as soon as possible. Customers don’t lie to be malicious; it’s just the opposite.
They can clearly see how passionate you are about an idea, and they don’t want to be the one person to hurt your feelings.
When I was pregnant with my son and trying to decide on names, I conducted a fun experiment. I wanted to see what people really thought of the names my husband and I were considering, so when someone asked me what I was naming my son, I would give one of two responses. To one group, I said I wasn’t really sure, but I liked names like Branch, Major and True. To the other group I would say I was naming my son True, and that I was really excited about it. I quickly found that the two groups reacted very differently to my responses. When I told people I was still deciding, more than 90 percent would tell me they didn’t like any of the names I had picked. But when I acted excited about one name, over 90 percent of people told me they loved the name True and would go on and on about how unique it was. Obviously I wasn’t hearing the truth from everyone.
With my son’s name, I was just conducting an experiment. I didn’t really care what people thought. But, when you’re talking about starting a company, hearing the truth from customers is vital to your success. The problem is, if they can sense the answer you are hoping for, they will jump to give it to you.
The key to understanding your customers real pain and current ways of solving existing problems is to get them talking. You want to ask them open-ended questions that will elicit a lot more valuable details than the response to the specific question. (More on open ended questions coming soon – make sure you are subscribed to get the article)
Now, think about the last time you got a call from a telemarketer trying to solicit a donation or get you to respond to a survey. I bet with every question they asked, you went through a mental Rolodex of responses that might make the call end as soon as possible. You weren’t thinking of the real answers to their questions, let alone additional information they might find useful. You were feeling uncomfortable and willing to say anything just to get it to end.
Most customer interviews are the same. If you don’t do a good job of getting them comfortable and engaged before starting to ask them questions, they will just be looking for the fastest way to exit the conversation, and usually that means saying something like “That sounds awesome, I would probably buy that.” They know that if they disagree with anything you said that you’ll want to argue with them, so they know that by agreeing with you and telling you how great your idea is, you’ll have no option but to end the conversation and go finish building your new product.
Many times when you present your idea, you’ll outline all the different ways it can benefit their lives, putting your potential customer in a situation where they look dumb if they disagree with you. For instance, you can tell them all of the money they are wasting each year from poor insulation in their home and all the benefits your product could provide to solve it for them. What could they say? “I don’t really care about all that money I’m wasting.” Or “I don’t really care about the environment.” Instead, they would likely try to save face by telling you how interesting it sounds and how they will think about buying your product. They aren’t going to. They just don’t want to look dumb, uncaring, or uninformed.
Once you know that your customers are inclined to lie to you, your whole strategy should change. Your objective should be to trick them into telling you the truth, and to make it hard for them to lie by keeping them in the dark about what you’re really after. It’s the difference between asking someone’s opinion on a few ideas you’re considering and asking their opinion on one idea in which you obviously have a vested interest.
Interviewing customers is one of the most difficult steps in the entrepreneurial process. It’s not as exciting as launching a website or raising money. The truth can hurt too, especially if customers don’t end up liking your big idea. But, the only thing worse than not talking to customers is talking to customers who you don’t realize are lying to you.
The data you choose to measure to test the success of a project is very important. If you measure vanity metrics instead of measuring real value based on the strategic goals of the organization, you could end up with a warehouse full of cupcake supplies.
The metrics you choose to measure the success of any project are very important. If you measure the wrong information, you might end up with a false positive. You might think a project is going well only to figure out later that it was a waste of time, money and effort.
But, if you measure the right metrics and approach Value Metrics like a scientist, you'll be able to weed out wasted effort much faster and more efficiently.
When Quantitative Strategist Brandon Ritzo began working for a home loan company in 2012, each customer was receiving a custom cupcake to remind them to fill out their loan paperwork. For $25 a pop, the cupcakes were made and shipped overnight to their destinations. Brandon was tasked with figuring out if the cupcakes scheme benefitted the company's bottom line.
You have to choose Value Metrics very carefully because if you end up using the wrong ones, you're going to see value where there isn't any. For example, in the cupcake experiment, people responded very positively to the cupcakes on social media by posting pictures and sending virtual thank you notes. Although interacting with customers on social media is a form of engagement, it's also a vanity metric. “Likes” and sales are two very different things.
The real Value Metrics of the cupcake scheme, Brandon thought, should be whether or not the cupcakes resulted in more customers closing loans. After all, closed loans directly added value to his employer's bottom line.
To know if an initiative is working, you have to have a control group. When the cupcake creators thought up their idea, they decided to buy the supplies in bulk and rent a warehouse to store the excess. Brandon put some of the extra cupcake supplies to good use by differentiating the cupcake group from the non-cupcake group. Without a control group, you can't test the results of your efforts.
When Brandon's team started to compare the control group to the cupcake group, they discovered that the cupcakes were resulting in more customers turning in their paperwork! To a less scientific person, this might indicate it's time to throw an office party and rent another cupcake warehouse.
But even though signed paperwork was the Value Metric the group originally set—the cupcakes came with a note reminding people to send in their paperwork—it turned out to be a Vanity Metric, a piece of data that makes you feel really good, but is actually concealing bad results.
Brandon's team found that the cupcakes had no effect on the real Value Metric: closing loans.
Not only were the cupcakes, shipping and warehousing costing the company money, but processing the increased amount of paperwork resulting from the cupcakes was costing the company money without any benefit to the bottom line.
“If we get extra paperwork, but we aren't getting to the final stage, everyone is worse off. We've wasted everyone's time and created more work,” Brandon said. “When you apply that logic, you're not only spending money on cupcakes but you're also costing the company money in overhead.”
After discovering the real cost of the cupcake experiment, Brandon's employer stopped shipping cupcakes. Even if it's hard to admit that an idea may be a bad one, the only other option is to trudge through bad ideas, accept sub-par results, and ultimately have to admit an even bigger failure down the road.
After shutting down the cupcake scheme, Brandon began looking for other projects that weren't going well. The project managers knew something was broken, so Brandon's team would come in, break the process apart entirely and see where they could deliver more value.
For example, the company also provides free credit counseling for people who don't qualify for a home loan. “We were drowning in clients for this service,” Brandon said. People would sign up, but then many would almost immediately stop participating. For this example, the Value Metric was whether or not the customer was able to improve their credit score to the level required by most banks to get a mortgage.
The conventional wisdom was to work with wealthier customers who would have the resources to dig themselves out of the hole, or to work with customers closest to achieving their goals because there was less distance to cover. But Brandon's team was able to isolate a much more influential factor: how well these customers lived below their means. So, they developed some methods to identify these customers and encourage counselors to focus on these customers first.
From there, Brandon and his team streamlined ways to test ideas as quickly as possible. From the initial results, they could slowly scale up successful projects before putting the full weight of the company behind something that might not add value to the organization.
Even if Brandon's team has improved an initiative, they never stop measuring Value Metrics. For example, after the credit counseling experiment, 2 percent of all call volume continued to follow the previous standards. Even improved initiatives aren't perfect. You should still measure the right Value Metrics, have a control group, and keep experimenting and refining to avoid perpetuating errors from the past.
