Why you need an excellent data scientist

The data science hiring space in a nutshell:

Remember, the driver is as important as the car. If you want to make the best use of your BI application, your organization needs the right people to exploit it. BI is not just about reporting and visualization anymore. It involves intensive and creative analysis, along with data management, to create value for an organization. - Got BI? Now You Need to Hire a Data Geek. Here’s What to Look For.

Hal Varian, Google’s Chief Economist, was interviewed a few months ago, and said the following in the McKinsey Quarterly: “The sexy job in the next ten years will be statisticians… The ability to take data—to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it—that’s going to be a hugely important skill.”  - The Three Sexy Skills of Data Geeks 

Data geeks are a hot commodity. Why?

Data is piling up around the industry's ears. We humans are suddenly generating a mountainous drift of accumulating data, growing exponentially, that nobody anticipated having. That mountain is filled with profitable, scientific gems that we are just beginning to learn how to mine out.

The market is unprepared for the demand. Even with the rush to train data miners, the market isn't coming close to keeping up with the pace of the data mountain's growth. Folks like me are hounded by recruiters; folks with +5 years data mining experience/ education are actively stalked.

CNN coverage: Companies that want to make sense of all their bits and bytes are hiring so-called data scientists - if they can find any. [...] A recent report from the McKinsey Global Institute says that by 2018 the U.S. could face a shortage of up to 190,000 workers with analytical skills.

Data science salaries are growing. The supply/demand disparity is driving up salaries. According to one survey, in 2010, the average data miner salary in the US was $103k; in 2011, it was $113k.

So if professional data miners are so hard to get, why do you need one of us?

[Your Company Here] needs an excellent data scientist. If humans use your digital product, your company is already generating an enormous quantity of ultra-rich data. Based on that fact alone, I can make the following safe bets:

Your data is buggy. No matter how good your testing is, there will be bugs. The bigger the data, the badder the bugs. You are going to need a data analyst who can identify dirty data, scope the damage, and prescribe a solution. Skip that, and risk spending months acting on an interesting data trend that, in the end, describes nothing but a broken javascript call. I've seen it happen over and over.

Skill #2: Data Munging (Suffering). The second critical skill mentioned above is “data munging.” Among data geek circles, this refers to the painful process of cleaning, parsing, and proofing one’s data before it’s suitable for analysis. Real world data is messy. - The Three Sexy Skills of Data Geeks 

Your data has a fluid architecture. As time moves on, your product evolves. Add a new option? Remove a feature? Need to view user behavior through a whole new lens? Like it or not, you have to change your data architecture while it is live. Every time that happens, you add more complexity. You need an analyst who can keep up with that.

Your data has, or should have, journeyman-level richness. If you send an apprentice-level data-miner into that trove, you're going to come out with a handful of iron ore. You can hire an apprentice analyst to run the queries you specify and graph them. You can't hire a apprentice to ask big, hot, actionable, counter-intuitive questions. Those questions grow out of an elbows-deep daily dialogue with your data set, composed of statistics and good old-fashioned nerdly zeal. If you want the gems out of that mine, you need a real industry-level data miner.

With big data comes big headaches; also big, big opportunity for a reputation for genius product development. If your analyst can't handle the big hairy real-world data mess, or doesn't know statistical relevance from a hole in the wall, you get bunk analysis. If your analyst just plain isn't that into it, you will get shallow, token inquiry. If your analyst is a passionate data/social science geek, you get game-changing analysis, and stand to score media-worthy customer relationship coups.