The limits of quantitative forecasting
ZeroHedge points out that JP Morgan has now taken out a $3 billion reserve to hedge against the potentially faulty judgments of their quantitative analysts.
For those of us in the world of largely qualitative analysis, this is a fairly unprecedented move, one that cuts across the grain of most schools of modern managerial thought. As my colleagues in intelligence often say, fake numbers will trump real insight almost every time. For example, consider that 87% of all statistics are made up on the spot to support faulty arguments. (Ahem.)
We’re not against hard numbers; collect as many of them as possible in every analysis. Still, you should be able to analyze the assumptions behind those numbers. Speaking of which, the ZeroHedge article pulls a shocking statistic out of the history of the subprime debacle. Check out what the quantitative model predicted subprime losses to be, as opposed to the actual losses, factors of 100 greater. Holy cats…
I might add, if you prefer real forecasting to fake numbers, we’re teaching a course on the subject around lunch time, March 4. Why not join the class?
What real forecasting looks like
Competitive Futures has a theme for all of our products and services in 2010: real forecasting.
When we look back at the good and bad decisions of the past twenty years, we ask, “Were you looking at real forecasts or fake forecasts?” Did you listen to forecasts that told you, comfortingly that the positive scenario was 5% growth, the negative scenario 1% growth, or the “middle” scenario dead on at 3% growth? Or did you ask which assumptions were behind that view of the future? Did you ask complex questions and get mature answers?
More on this – much more – in the month of January.
In the meantime, to illustrate the difference, check out Chris Nelder at GetREAList in his blistering takedown of the EIA energy forecasts, entitled What if the Annual Energy Outlook Were Written by an Honest Person: Why the EIA Should be Statutorially Barred from Making Predictions:
Suppose you worked at the Energy Information Administration (EIA), the agency within the U.S. Department of Energy charged with keeping data and making projections on energy, and you had to produce an annual report with a scenario for the next 25 years.
Being an intelligent and informed investor, you might grapple with the $147 to $33 range in oil prices over the last year and try to imagine how such volatility might happen in the future.
You might be tempted to model a few economic factors such as GDP growth rates and credit availability, and how they affect investment in energy supply.
You might consider the price at which producing a barrel of oil or a thousand cubic feet of natural gas becomes profitable, and the price at which it becomes too expensive and destroys demand.
You might take peak oil, peak gas, and peak coal into account, since the best available models on those subjects all suggest peaks within the time frame of your scenario.
Wow, great analysis from Mr. Nelder, and an appropriately systemic approach to energy forecasting. That’s what the EIA forecasts look like, right?
But then, you’re not working for the EIA.
If you were, you’d do something like this…
You’d get out your crayons and your graph paper, and starting with your most recent data, you’d plot a nice, steady 1.5% global growth rate for energy demand over the next 25 years.
You’d do something similar for supply so that it matches demand at prices that also climb at a nice steady rate. For oil prices, call it, oh, how about 0.4% per year? That sounds pretty good.
You’d draw basically flat lines into the future for all the fuels dominant today, since you know they have serious challenges ahead, and then draw sharply rising lines for the latest and greatest technology, projecting enormous growth rates for things like shale gas and enhanced oil recovery.
You’d be sure to count all possible supply from new sources — like a new gas pipeline from Alaska — even if those projects don’t yet exist. Hey, it could happen!
You would not, however, factor in any CO2 reduction, because policies to control it don’t exist.
Naturally, you’d assume that the next 25 years would show gradual economic growth, so there wouldn’t be any troublesome issues like credit availability or depressed consumer demand to worry about.
Ouch.
If you are making forecasts about where to build a factory, planning to ship your finished goods to the markets of the world, and you are calculating energy costs per unit, whose forecasts are you going to use? The EIA is more of an authority, but then again – who’s got the more authoritative forecasts?
Evaluating forecasts in the 2010 – 2020 economy
Round numbered years are boom times for forecasters. There is something intellectually symmetrical about round numbers that makes people hungry for the future. Back in 2000, it was really easy to get people excited about forecasts for 2005, 2010, 2020, 2050 – all divisible by ten, mathematically elegant! There is something more confusing about analyzing trends from 2007 – 2016 than there is from looking at 2000 – 2020. The brain is a funny thing.
As we begin a new decade, you will see quite a bit of retrospective about the last ten years, and copious forecasts around the year 2020. Now is a good time for executives to brush up on their skills in evaluating forecasts, so they can critically assess what is really coming down the road. Given the high level of complexity facing our economies, this is really quite important.
Assessing forecasts is more important than the act of collecting the data itself. When you see numbers about the future in particular, we are instantly attracted to their comforting certainty. Rarely do we ask, “which assumption are packed into this trend line?” For example, take a look at this projection of potential unemployment in the United States from Q4 2009 through Q4 2020. I got this forecast from the indispensable Mike Shedlock, and he got these projections from a mix of Bureau of Labor Statistics reports and assumptions.
There is also a spreadsheet on offer “detailing” the assumptions, getting into the specifics of how many jobs might be created and when they might enter the economy.
Here’s how my analyst’s brain immediately sees this in terms of the future: I want to know what will really be HAPPENING in order to create this abstract curve. Curves like this are pure PowerPoint candy, giving us beautiful visual aids to support our discussions of the future, but the lack of explicit discussion of the future leaves us an intellectual vacuum. To really evaluate the validity of such a scenario about unemployment in the United States, we can’t leave this trend line alone – we must complicate it with all the factors that will actually go into economics in the next ten years
- Aren’t the Baby Boomers retiring during this time, or at least leaving active work?
- The Boomer “retirement” will leave a talent crisis in its wake. Will unemployment mean the same thing?
- Is there a tipping point of high employment at which American society will change and become less stable? 12%? 15%? 20%?
- What about peak oil? What if economic activity is squeezed from high energy costs?
- Which industries are assumed to create the jobs? Why will they provide more value in the next ten years?
- Does this curve assume any backlash from the Ponzi scheme meltdown of the banks? What if we crash again in 2010 or 2011?
- Will US unemployment rates roughly follow the world economy, or is this trend indicative of a shift in economic power to Asia Pacific, Europe, or Latin America?
These questions are not meant to knock down the validity of such forecasts – I’d rather have an unexamined forecast that gets us talking rather than fixate on more blather from the day’s stock market trades. More forecasts lead to more discussions and better decisions.
Hopefully 2010 will be a banner year for forecasting and strategy. If you want to more, we can recommend the book Future Inc, which dives deeper into this methodology. If you want to know even more, get in touch with us to schedule training to make your analysis more valuable to your organization.
AT&T ads from 1993 describe services we use today
Further to our series of “Forecasting Works” blog posts, dig these ads for AT&T from 16 years ago, 1993. They forecast, based on their own knowledge of technology and some educated guesses:
- E-books
- Telepresence
- EZ-Pass digital toll collection
- Online concert ticket sales
- In-car GPS navigation
Were they completely accurate in these visions? Not entirely, but you’ll have to admit that it is all frighteningly close.
They engaged in a rational process of thinking about the impact of current trends, and it helped light the way.
You can do likewise.
Forecasting works: Functional foods 1999 – 2009
Today, the airwaves are filled with advertisements for consumer foods that aren’t simply nourishing but portrayed as practically medicine. A slew of softdrinks are marketed as hangover cures, energy, memory enhancers, cognitive enhancers, help with clairvoyance, and fuel for flight. Fish isn’t just fish, it’s OMEGA-3 FATTY ACIDS. And somewhere along the way, trans-fats replaced “Ebola virus” as the world’s deadliest substance. Is this random or could you see it coming?
Food as medicine was a theme we predicted for 2010 way back in 1999 when studying the future of food and health for a group of global consumer product manufacturers. The world seemed to be at a turning point at that moment, with a number of trends appearing to collide in the decade to come:
- Super-size and family value packs had reached their apex, due to increasing penetration of fast food and big-box retail throughout the world
- Obesity epidemic reaching a pitch, not only in America but also in unexpected places like France, Greece, China
- Litigious American culture had finally apexed with its war on cigarette liability, and a new target was likely to be next
- Biotechnology was promising new technological abilities for all plant life (this was the era of the Human Genome Project and techno-positive rhetoric was off the chart)
- Boomers were aging, and increasingly interested in immortality on the cheap
- Sustainability was increasing as a concern, and farming would be one of the most effected industries
- The “Slow Food Movement” was beginning to point back to heirloom breeds of livestock and produce and encourage local diversity in favor of industrial solutions
You are not crazy, and forecasting works
If you watch the TeeVee Box, the world and its institutions seem inherently irrational. It’s a world of crazy risk, cataclysmic downfalls, nonsensical solutions from people who ought to know better.
One of America’s high priests, Ben Bernanke, has just been taken on for a second term at the head of the powerful and enigmatic Federal Reserve bank. See my post from yesterday to understand why this surprised me. For a moment I had the all-too-common though:
In a world this nuts, why even forecast? I mean, why study housing prices, water tables, healthcare expenditures, and all the rest if the world comes down to the actions of a select, semi-rational few.
Then I thought it over. The last year has unfolded in a strictly rational way. The trick to understanding the future (and the method I teach) is to analyze a combination of three things:
- structural trends
- actor decisions
- wildcards
Understand what’s happening, the options available to actors in a system, and the crazy stuff that can happen when you’re not looking.
Look at the economics of 2008 – 2009 through that lens and it all makes sense.
This is the subject of today’s podcast, so KEEP THINKING.
About Eric Garland
Eric is one of the world's pre-eminent thoughtleaders in forecasting and strategy. He helps leaders of business and government make critical decisions about their futures.New project: Manufacturing 2020!
Competitive Futures is
proud to announce a new research project on the future of manufacturing in North America through 2020. Download a prospectus today and learn where to invest for the next decade.Get Eric's blog delivered!
Connect with us
Read Future Inc.
Learn
a time-tested, profitable method of discovering business opportunities in the future with Eric Garland's groundbreaking Future Inc: How Businesses Can Anticipate and Profit from What's Next, available at Amazon and fine independent booksellers.
