Metrics mania: Are Americans too reliant on numbers?
Stop me if you’ve heard this one. A guy strikes up a conversation with another guy on a long plane flight to South America. They are over the Amazon.
Guy 1: “Did you know that the Amazon is 1,000,003 years old?”
Guy 2: “Really? How can you be so precise?”
Guy 1: “I was on this same flight three years ago, and a geologist told me the Amazon was a million years old.”
In the physical sciences, numbers rule. You can’t predict an orbit if you haven’t recorded a planet’s path. It’s best not to wing it when building a nuclear reactor or mixing jet fuel. Even in the squishier arena of human behavior, probability and statistics are essential. The more successful caveman returned to his happy hunting grounds because the probability was higher that he’d end up with something for the barbecue than if he just wandered around the nearby tundra.
In recent years, businesses have embraced metrics as never before, citing management expert Tom DeMarco’s famous line: “You can’t control what you can’t measure.” To stay within a budget, you need to track spending. If you measure defects on an assembly line, you can zero in on a fix. The US military uses metrics to assess progress in Iraq and Afghanistan. In sports, Bill James used sophisticated baseball data to help the Boston Red Sox end their 86-year World Series drought.
Wall Street boffins have been masters at data crunching, using powerful computers to exploit small advantages and rack up big gains – until they didn’t. In the early years of this battered decade, massive financial bets were based on flawed assumptions. Market data were pouring in, but as Nassim Nicholas Taleb argued in his “black swan theory,” no one expects the unexpected – the black swan – which in this case was the massive financial freeze-up brought on by the collateralized debt fiasco.
Wall Street has learned many lessons. You can be certain that one of them won’t be to shun data. The black swan factor will simply be added to the algorithm. For despite its flaws, metrics-based management is usually more scientific than the alternative, where big decisions can be made based on a hunch, ego, or fear.
And so we measure. Take this article. Monitor weekly readers read it first. While I hope you enjoy it, I’ll only know that if I get an e-mail or letter, and even then the missives – for which, thank you – are anecdotal. Does one cross e-mail mean a lot of people were angry or only one? When this article goes online, a tiny “cookie” of code will let us measure the number of people who click on it. That seems scientific. But many online readers block cookies. And clicking on a page doesn’t mean you read it. We learn something – but not everything – with online metrics.
Mr. DeMarco reconsidered his metrics mantra a few years ago, warning that “at its worst, it can do actual harm.” Harm occurs when metrics are unquestioned, manipulated, misunderstood, or stifle creativity. The wandering caveman might discover a great new mastodon takeaway a few glaciers over. Serendipity is real. Inspiration, too. Metrics work best as a benchmark, not a definitive measurement.
Think about what’s really going on when we measure. We lay a yardstick on the deep blue ocean. Below is unfathomable vastness. Can we really measure that?
So here’s my formula: Metrics + Grain of Salt = Somewhat Useful Information.
Still, even if we can’t trust data absolutely, we can extract meaning. We may not know how old the Amazon really is, but we know one thing for certain: It is three years older than when Guy 1 first flew over it.
John Yemma is editor of the Christian Science Monitor. You can follow him on Twitter.
