Small Business Labs

Data is the New Plastics

Those old enough to remember the movie The Graduate will no doubt remember the "plastics" scene.  It takes place at a college graduation party for the lead character  - Benjamin Braddock, played by Dustin Hoffman. 

Braddock, who is totally lost about what to do next with his life, is getting career advice from the friends of his parents. 

At one point a man walks up to Braddock and says "Plastics .... enough said", and walks away.

The intent of the reference is that Braddock should enter the plastics industry, a rapidly growing field with lots of opportunities. 

Instead of getting a job in plastics, Braddock has an affair with Mrs. Robinson, the much older wife of his father's business partner.  He also drives around northern California while Simon and Garfunkel music plays in the background. 

Eventually he decides he wants to marry Mrs. Robinson's daughter, which leads to a far-fetched ending.  But I digress.

Last week we held a workshop on data and analytics with our research partners at Intuit.  We had a mix of experts on data and analytics from both large and small organizations. 

There was a very clear consensus that there is a shortage of people with data and analytical skills  - and this shortage is going to get worse and last a long time. 

So my advice to those in school?  Data ... enough said.  Oh, and stay away from my wife. 

Lessons from a Failed Forecast

Workforce Week's (tiered subscription) Fast Forward: 25 Trends That Will Change the Way You do Business appears to have been written in 2003.  This is great.  We don't often get a chance to see how trend forecasts turn out. 

Overall I'd give them a B to B- on forecast accuracy.  I'm most intrigued with their 2003 jobs forecast for 2010.  The qualitative analysis looks OK:

"... a futurist specializing in workplace issues, says pressure on baby boomers wanting to retire will be so great that they will be pulled back into the labor market. Even so, he says, older workers won’t show up in large enough numbers to fill the millions of jobs available. Herman says the problem will be aggravated by the shortage of skilled, educated workers already occurring in manufacturing, health care, and various technical fields."

But their quantitative assessment of what this means is way off.  They forecast there wouldn't be enough people to fill the jobs available in 2010.  The forecast is a shortage of 10 million workers by 2010.  Obviously a big miss.

From the article:

The convergence of several trends--declining births, retiring baby boomers, and expected business growth--will create more jobs than there will be workers to fill them by 2010, experts predict. The math is relatively simple. The civilian labor force will increase by 17 million, reaching 158 million in 2010, reports the Bureau of Labor Statistics. But by then, the BLS says, the number of jobs will reach 168 million.

So why did they miss by so much?

1.  They missed the Great Recession:  Almost all longer term forecasts assume steady economic growth at a "normal" pace.  The underlying assumption behind this forecast was that the economy would grow 3%-4% a year between 2003 and 2010.  Few forecasts consider the impact of low probability, game changing events such as a major recession.  While I don't grade them down for this, users of forecasts should know these type of black swan events generally aren't included in forecasts. 

2.  They missed the social shift towards people working past traditional retirement age:  They assumed boomers would retire just as the prior generation did.  This was wrong, although to be fair the Great Recession has played a part in this.  But the bigger reason boomers are delaying retirement is they want to keep working to stay active and engaged. 

3.  They got their math wrong:  Because most futurists are qualitatively oriented, quantitatively challenged forecasts happen a lot more than you would think.  In this case, a quantitative review of demographic data would have shown neither declining birth rates nor retiring boomers were going to have a major impact on the labor force during this forecast time frame. 

4.  They missed the productivity impact of automation, globalization and outsourcing on the job market:   Simply put, companies need fewer U.S. workers per unit of output than they did in 2003. 

So what can we learn from this?

1.  Understand the underlying assumptions driving a forecast, test them and consider the impact of low probability events.  We use scenario planning to do this at Smallbizlabs (click here to learn more about scenario planning).

2.  Scan for social shifts.  Many forecasts assume a constant social environment instead of incorporating shifts in the way we live.  It was pretty clear to those watching baby boomers in 2003 that they weren't going to retire in the same manner as their parents.  Being aware of this would have helped this forecast. 

We use a variety of scanning methods including following sites that cover a wide range of consumer and social trends.   

3.  Do the math:  The longer the forecast time frame, the greater the impact small changes in forecast growth rates have.    Also, check the math in the forecast assumptions. 

4.  Always consider feedback loop impacts.  Whenever I see a forecast that suggests a large shortage of something or a major dislocation, my first reaction is to check on potential feedback loops. These are reactions to an event that either enhance or buffer the event's impact.  Most forecasts, including this one, ignore them. 

There was no way, even with a good economy, the U.S. was going to be short 10 million jobs today because of the feedback loops caused by technology, automation and outsourcing (for more on feedback loops, see this article). 

BTW, I think lack of thinking about feedback loops is the single greatest reason so many forecasts are wrong. 

No human can predict the future, which is why forecasts are generally wrong.  But we can better plan for the future by learning from our mistakes and using sound forecasting tools and methods. 

New York Times on Analytical Software

The New Times has a good article today covering data mining and analytical software.   The title is "Reaping Results: Data Mining Goes Mainstream" and the article describes how police departments and large corporations and using data mining techniques.  The article mentions a number of large corporations including WalMart, Kohl's, Harrah's, Whirlpool and Capital One.  A good summary quote from the article:

"Faster and cheaper computing and ample sources of information in digital form — plucked from enterprise resource planning systems, point-of-sale devices and Web sites — mean that most companies now have the tools to do the kind of competitive analytics that only a relative handful of elite companies could do in the past."

Just as more large corporations can now take advantage of sophisticated data mining techniques, so can small businesses.  In the next installment of the of the Intuit/IFTF Future of Small Business forecast report (due out in June), we talk about how analytic software and data mining will be increasingly important to small businesses.