It’s Better to Consistent Than To Be Accurate

THE STORY
Awhile back I was working on a project with one of the world’s largest investment banking firms. My task was to create a model to predict the detailed monthly financials of a retail chain going out about 5 years into the future.

My team created a very complex model which taxed the extremes of the capability of an excel spreadsheet. This model was so complex, and involved the interplay between so many different inputs, that the output in any given month fluctuated around a bit.

These little fluctuations bothered the people at the investment bank. They wanted nice, smooth progressions in the numbers over time. Therefore, they scrapped the huge, complex (but fairly accurate) model and built a simplistic little ditty based on extrapolating a couple of metrics. It didn’t exactly tie to all of our detailed assumptions, but all the numbers moved in smooth progressions over time and it roughly matched the macro trends of the complex model.

What these investment bankers told us was that it was better to be consistent than to be accurate. So the simple little model won out over the complex one.

THE ANALOGY
Businesses are made up of hundreds of thousands of little tasks which come together to create the financial outcomes. In addition, there are probably just as many activities in the external environment which also impact one’s financials. Depending on how all of these activities come together, one will get different results.

Trying to model all of these events and accurately guess how they will play out in the future is an extremely difficult task. It is easy to get lost in the minutiae and lose site of the big picture.

The task of strategy is not to accurately predict all of the events of the future in great detail. The task of strategy is to provide enough insight into the future so that whatever decisions you have to make “today” will have enough futuristic context so that you can properly choose a path that will improve your condition over time. Additional details do not always cause additional insight. Sometimes, they just cloud the issue and make it more difficult to see the big picture (for more on this topic, see the blog “Too Many Clocks”).

Therefore, instead of using up valuable time in building extremely complex (but potentially more accurate) models, that time could be better spent in understanding the strategic implications of the big picture (which can be derived with a simpler model) and developing the right strategic alternatives.

THE PRINCIPLE
Strategists are in the business of selling—selling visions, selling ideas and selling alternatives. For example, if you cannot sell a vision of the future, then you cannot get consensus on what to do to improve yourself in the future. Facts are a key element in the selling process, but it is not the only element. Beyond a certain point, additional facts do not increase your persuasive abilities. Other issues also come into play. That is why it can be more important to be consistent than to be accurate. In particular, there are three principles which cause this statement to be true.

1) The Principle of Focus
2) The Principle of Credibility
3) The Principle of Large Numbers

These are discussed in more detail below.

1) The Principle of Focus
It is difficult enough trying to reach strategic decisions when people are focused on the right issues. It is virtually impossible if people are focused on the wrong thing. In selling a vision of the future, what you want is to have people focused on the key assumptions which would cause you to reach a different conclusion, depending on how you think the assumption would turn out.

For example, if you were trying to create a strategy in the health care industry, you might come to a different conclusion depending upon your assumptions around how active the government will be in managing health care in the future. Therefore, discussions around expectations of government involvement in health care management would be very important in developing your strategy.

If you produce data which looked like the data that came from the complex model I referred to earlier, your added accuracy would cause your numbers to have little wiggles in them over time. Your audience could get fixated on the wiggles and start asking questions about all the nuances in you model which caused the wiggles. Then your conversation would be side-tracked into all sorts of minutiae. The big issues, like how much the government will get involved in health care, could get squeezed out of the discussion.

Models are only representations of assumptions. Their goal is to help roughly quantify the direction and magnitude of the impact of an assumption on your business model. That way, you can see the impact of the assumption on your business and then make decisions which optimize under that assumption’s scenario. If your model is so complex that it clouds the impact of the assumptions, then the model is no longer useful in helping you make decisions. Rather than focusing your audience on the key assumptions, it gets them focused on “wiggles.”

In general, most key assumptions revolve around how you think an issue will trend—for example, will it get stronger, weaker, or stay the same over time. Since we tend to think of these assumptions in terms of smooth trends, then the model is more effective in helping us understand these assumptions if it also reflects consistently smooth trends. Again, the goal is not accuracy, but usefulness in making decisions. Consistently smooth trends help keep us focused on the assumptions and their general impact. That is typically enough information to make the right decision for today.

2) The Principle of Credibility
One’s ability to be effective at selling is directly related to one’s level of credibility. If your audience believes you have credibility, then you can be more effective in selling your visions, ideas and alternatives. Conversely, if you have no credibility, it doesn’t matter what you say or do, because nobody will take you seriously.

We are conditioned to believe that life tends to move rather consistently through time. If we think of our assumptions in consistent terms (e.g., things getting gradually better or gradually worse over time), we would also expect their impact to be consistent over time on the model. If your model does not have this type of consistency, it makes people question the accuracy of the model. Complex models with wiggles in them are difficult for people to understand. If the model is so complex that they cannot assess its accuracy themselves, they are less at ease and have to trust even more in your credibility. But if the wiggles cause them to think that there must be something wrong with the model, because “it doesn’t look right,” then you have lost your credibility.

It is better for your credibility to be a little less accurate in your modeling and create models with smooth consistency over time, so that the model appears more “believable” to your audience. As long as the simpler model does not distort the facts enough to come to the wrong conclusion, the simpler model will be a more effective selling tool.

3) The Principle of Large Numbers
The law of large numbers says that it is often easier to forecast an aggregate outcome of many factors than to forecast the all of the individual outcomes of every factor. This is true because there is often more variability in the outcomes of the individual components than in the outcome of total integrated unit. When you aggregate many parts together, the variabilities of the individual parts tend to offset one another. Because the offset, they reduce the variability of the whole.

For example, if you were a retailer trying to forecast your gross margin, it may be easier to forecast the aggregate gross margin for the entire company than to forecast the gross margin on every single item you might sell and then add all of the items up. This is because the variability on the gross margin for each individual item sold is much higher than the variability over time in the aggregate for the entire company.

Therefore, building a simple model that creates smooth consistent trends on a few key aggregate outcomes might actually end up being more accurate than a model which tries to forecast all of the individual components. Hence, by concentrating on consistency over accuracy, you might end up with better consistency AND better accuracy.

SUMMARY
Effective strategy building requires effective salesmanship. Better salesmanship (and hence better strategy) usually comes from simple models that are easy to comprehend and follow smooth trends. That is why it is more important to be consistent than to be accurate.

FINAL THOUGHTS
One of the problems we can often run into is trying to make too many decisions too soon. Frequently, some of the more tactical issues are better handled when delayed until closer to the time when the tactic must be implemented. By keeping strategic models relatively simple, it keeps discussions on the strategic level (where more lead time is needed) rather than getting into the tactics too soon.