Strategy Planning Analogy #558: Counting People

TWO STORIES

1. High School Survey

The US government used to do an annual survey of high school students. The objective of the survey was to track things like the levels of drug use and sexual activity in that age group. Because the writers of the survey were afraid that high school students would lie about their own personal activities, the government asked the students to estimate what percentage of students in their high school they felt were doing each of these types of things.

At first, the government would average all of the responses for each question to get an estimate of how prevalent various activities were. But then, someone dug deeper into the results. What they found was that although the average percentage number was somewhere in the middle, very few of the individual students ever answered with a number in the middle. Instead, there were two clusters of answers on each question—one cluster of students which gave a very high percentage answer and one cluster of students which gave a very low percentage answer. This caused the government to take a second look at the results.

As it turns out, the researchers found out that most students do not hang out with a large percentage of their fellow students. As a result, the students had no idea of the drug and sexual activity of the greater student body. Instead, they only had the reference point of their small band of close friends. 

And these close bands of friends tended to behave similarly to the others in the same band. So, if a student participated in these activities, most of their friends did also, so they concluded that most of the people in the school must also do these things. Similarly, if a student did not participate in these activities, most of their friends also did not, so the student would assume that most of the students in the entire school also did not.

Based on these insights, the government shifted its emphasis from tracking the change in the averages from year to year to tracking the relative sizes of the high and low clusters from year to year.

2. Amazon Music Reviews
Before buying music, I like to read the reviews in Amazon. Over time, I realized that the vast majority of all the music on Amazon has an average ranking of 4.5 out of 5. With nearly everything rated equally, it became impossible to use these average ratings to decide which music to purchase.

But then, I started thinking. The problem with Amazon music ratings is that they are voluntary. This is not a random sampling. People only turn in a rating when the mood hits them. And typically, the mood only hits them if something they hear is especially good or especially bad. And it in the vast majority of cases, the review came from people who thought the music was especially good.

Once I figured that out, I stopped looking at the average rankings of the review and instead started looking at the number of reviews in total for a particular piece of music. My logic was that if only the ones who loved the music send in a review, then the more reviews sent in, the more people loved that music.

This has turned out to be a far more effective way to use the Amazon review process.

THE ANALOGY
Companies like to base their strategies on facts. Sometimes, they try to get their facts directly from the consumer. This tends to happen most often at three phases of strategic planning:

1.     At the beginning, when trying to understand the market place.

2.     In the middle, when testing concepts

3.     At the end, when assessing whether the strategy is working.

The good news is that in today’s interconnected world, there are lots of ways to get consumer input.

The problem is that these sources can often have flaws like the ones mentioned in the stories. Complainers and the people who rate companies online are not a random sample. They are biased towards people who like to rate or towards people with extreme views (like the Amazon music ratings). If you just look at the average ratings and comments, you will most likely come to the wrong conclusion. It may be better to count the reviews, rather than average them.

And even well designed surveys can with random sampling can have flaws. After spending decades in consumer research, I discovered that people will try to honestly answer all of your questions well, but they often just don’t know the answer, so they guess—and often very wrongly (like the high school survey). I have found this to be particularly true when asking people to predict their future behavior in areas where they have little experience (like how they would react to a new strategic scenario).

Therefore, we need to be careful in how we interpret this data.

THE PRINCIPLE
The principle here is that one cannot run a strategic planning process based solely on research, especially if you only look at averages. Part of this is due to some of the research flaws mentioned earlier. Another part is due to the nature of strategic planning itself.

Strategic planning is looking for ways to build a new and better future. It can be about finding new white spaces which have never been exploited. It can be about inventing solutions which never before existed. It can be about building business models that break all the old rules. It can be about finding uncharted “Blue Oceans” of opportunity. In other words, strategy is a lot about trying to get ahead of the curve and be an early adopter of the next big thing.

Of course, if you are trying to lead the way into the future, you may be several steps ahead of the general population. Questioning the general population may not be very useful at such an early stage. 

However, if you wait to move until the consumers can speak as experts, it is too late to be at the front end of the strategic revolution. That’s one reason why Steve Jobs didn’t believe in consumer research. He knew that consumers can’t speak meaningfully about a future not yet envisioned.

But that doesn’t mean that research is useless. As seen in the stories above, there are creative ways to look at data to get insights. But may mean you cannot take the initial results at face value. For example, we saw that instead of averaging out what people say, it may be better to just count how many say something. Therefore, be careful when looking at your results. Don’t necessarily take it at face value. Search deeper for the true implications—especially as the questioning looks into the future or is not randomly sampled.

Other things to keep in mind:

1. Although consumers may be unable to articulate how they will act in an inexperienced future, they can articulate what irritates them in the current state. Knowing the irritations of today can help you when designing the newer future.
2. Some people may be living closer to the leading edge than others. Focusing research on leading edge people may give better results.
3. Even though solutions my change over time, attitudes/concerns/desires regarding the problem may be more stable. If you focus on researching the more stable problem issues, it may give insights into how to develop better, innovative, new solutions.

But probably the most important thing to understand is that strategy is not pure math or pure science. It also has an element of artistic creativity. Creating the future is, by definition, creative. Eliminate the creative and you will never find what you are looking for.

SUMMARY
Although we may want a data-based approach to strategy, relying only on data—or taking the data at face value—will probably lead you in the wrong direction. The future is not precise, so you cannot take just a precise approach to get there. The consumer is not always very helpful or knowledgeable in looking beyond incremental change. Therefore, one will need to also rely on artistic creativity to get to the future. In fact, the creative part is likely be more important than the scientific part.

FINAL THOUGHTS
You cannot find the future if you are only looking backwards. And looking backwards is where the consumers are. Sometimes you have to look forward, to the places where the customers have not yet arrived. Creative insight, rather than research, may be more useful.