Does data-driven decision-making make you boring?

With all the focus on data and AI, it was simply a matter of time before the countermovement started. Reflecting on several discussions around this topic that I’ve had over the last year, the key theme seems to be that data and AI are predicting the future based on the past and as long as the future is like the past, this works fine. However, the world is in constant flux and these technologies cause stagnation as we can’t predict fundamental shifts and disruptive innovations. Even worse, we don’t even look for them as we look at data in a short-sighted fashion.

Although I most certainly believe that there’s a very important place for human creativity and insight, I also think that not exploiting the advantages data and AI offer is simply akin to shooting yourself in the foot or tying one arm behind your back. There are several reasons for this.

First, for all the criticism on machine learning for predicting the future, the fact is that in most cases, humans are even worse at it. Even for highly variable data, ML algorithms often manage to exploit patterns that humans fail to detect. For large retailers, predicting the amount of product to order and then allocating it to each individual shop used to be a human task, but it’s clear that ML algorithms, given sufficient data, do a better job. A counterargument used frequently recently is that these algorithms didn’t predict the Covid-19 disruption, but of course, humans didn’t predict it either, leaving many stores with a significant surplus of goods.

Second, I still meet people that continue to express beliefs about the world, their industry, their customers or their own performance that simply aren’t true. Although some, like Steve Jobs, were known for their “reality distortion field,” for virtually all of us, just wishing for something to be true doesn’t make it so. As William Edwards Deming famously said: in God we trust; all others must bring data.

Third, data-driven practices don’t remove human creativity but instead focus it on the formulation of hypotheses. In traditional organizations, one can build a career on making strong statements that are hard to verify and being vocal about them. Often, these are based on individual instances and storytelling, to which we as humans are very sensitive. When adopting data-driven practices, the focus should be on formulating testable hypotheses and being less concerned with being proven wrong. Even hypotheses that are creative and novel but don’t pan out provide ample opportunity for learning.

Fourth, when using data-driven practices, you need to know what you’re optimizing for. In virtually all companies that I work with, features are prioritized and developed based on the beliefs of some product manager. The effect of the prioritized feature on the customer or system behavior and the way it generates value is often described in qualitative and vague terms. The worst argument here is that it’s a “strategic investment.” Rather than prioritizing a feature to be developed based on the beliefs of a product manager, it’s much better to treat the feature as a hypothesis, define its expected, quantitative effect and then measure its impact as you iteratively develop the feature slice by slice.

Working in a data-driven fashion doesn’t make you boring. Instead, it instills a higher level of discipline in the organization, uses technology where it fits best and focuses creative energy on the areas where humans provide the most value. It helps organizations to shed so-called “shadow beliefs” (beliefs that everyone in the organization considers to be true but that are not) and, through that, remove hypotheses that don’t hold from the pool of ideas. Neither humans nor machines can predict the future. However, although history never repeats itself, it often rhymes. And machine learning is better at detecting the rhymes than you.