The TV show Rookie has been following me everywhere lately. There’s an episode where Nolan, who’s no longer a rookie himself, onboards a rookie named Celina, who believes in mysticism and prophetic dreams. At some point, they have a dialogue where Nolan agrees that her visions and dreams really did help, but he’s convinced that this is the result of all her life experience: attention to detail, searching for inconsistencies in stories, and so on. It’s not just mystical intuition; it’s colossal experience that has transformed into habits, character traits, and general perceptiveness.
Why is this important? I always wonder what basis our company uses to make various decisions. Is it data driven or gut feeling? Spoiler: everyone usually says data driven, as if admitting it’s gut feeling would drastically lower their authority in others’ eyes. Actually, there’s no single right solution, and I’ve gathered some of the loudest examples of both failures and successes of the data and intuition approach, and after them I’ll share what I think about it.
Successful Data Driven. Let’s start with successful scenarios. Data Driven. Actually, the scenario is controversial from an ethical standpoint, but nevertheless, it achieved its goal quite successfully. Target created an algorithm that analyzed purchases and predicted customer pregnancies with high accuracy. So high that it learned about a minor daughter’s pregnancy before her father did. This story shook the internet at the time. I think there’s no need to talk about contextual advertising, which has already leaked everywhere.
Successful Gut Feeling. A successful Gut Feeling example, of course, wouldn’t be complete without Steve Jobs, who generally said that “People don’t know what they want until you show it to them.” This was in 2007 when he presented the iPhone. Yes, the iPhone wasn’t the first in the smartphone category with a touchscreen, but how Jobs pulled this off remains legendary. He rejected all marketing research and focus group data and conquered the smartphone market.
Failed Data Driven. And here’s one of the unsuccessful Data Driven approaches: Amazon once tested AI for hiring. It turned out that the AI discriminated against women. This happened because the models this AI worked on contained historical data where, as it happened, men dominated. And for the AI, this became the norm.
Failed Gut Feeling. And here’s an example of unsuccessful Gut Feeling. Kodak refused to transition to digital because management thought it wasn’t their path. Ironically, the first digital photograph appeared thanks to Kodak engineers.
I don’t make decisions at the level of Kodak directors or Steve Jobs (yet), but I use both approaches. I think you do too, by the way. For example, when you’re asked to estimate a big new feature (usually this needs to be done right now), you, having context of the product and your team, can give this estimate in certain categories (quarters in my case). And you don’t need to conduct long research, cross-reference with data, you just give an estimate. Yes, with margins for uncertainty, risks and all that, but tell me, when you give such estimates, do you pull up data from previous releases?
Or another example from my life, related to team dynamics and health. If there’s a desire to implement some process, tool, or reconsider team composition, then it would be interesting to look at what was before the change and what came after, rather than just saying it got better or worse. After all, when we use comparative adjectives, we’re basing them on some data we’re comparing with, right?
In general, using one approach or another should be dictated by your position and conditions. Steve Jobs could afford any approach; I can’t (yet). My words should be backed by numbers and data that don’t lose the context of what’s happening. I can afford to say that I sense that this and that won’t go according to plan, or that, conversely, a new technology will definitely take off, trust me, but decision-making at my level should mostly be driven by data.
Trusting intuition is fast. One moment and the decision is there! Data collection and processing can take a lot of time. Sometimes there’s no such privilege, and decisions need to be made quickly. The main thing is, when I reach Steve Jobs’ level (someday!), not to forget about intuition that was trained for years by precisely this data.
