Data scientists are fast becoming the rock stars of the 21st century. Thanks in part to Nate Silver's eerily accurate election predictions and Paul DePodesta's baseball-revolutionizing Moneyball techniques, math nerds have become celebrities. It's debatable how much their work differs from what statisticians have done for years, but it's a growing field, and many companies are desperate to hire their own data scientists.
The irony is that many of these math nerds aren't as math nerdy as you might expect.
Some of the best minds in the field lack the sort of heavy math or science training you might expect. Silver and Paul DePodesta have bachelor's degrees in economics, but neither has a PhD. Former Facebook data scientist and Cloudera co-founder Jeff Hammerbacher -- who helped define the field as it's practiced today -- only has a bachelor's in mathematics. The top ranked competitor at Kaggle -- which runs regular contest for data scientists -- doesn't have a PhD, and many of the site's other elite competitors don't either.
"In fact, I argue that often Ph.D.s in computer science in statistics spend too much time thinking about what algorithm to apply and not enough thinking about common sense issues like which set of variables (or features) are most likely to be important," says Kaggle CEO Anthony Goldbloom.
Data scientist John Candido agrees. "An understanding of math is important," he says, "but equally important is understanding the research. Understanding why you are using a particular type of math is more important than understanding the math itself."
Candido has a master's degree in psychology, but not a PhD in math or physics. Still, he's done quite well for himself in data science game. After graduating, Candido started predicting the outcomes of Mixed Martial Arts matches based on fighters' past performance on the site Fight Metric. That landed him a column for ESPN. Now he does data science for ZestFinance, a company founded by former Google chief information officer Douglas Merrill.
Candido says that although his master's program gave him a good background in statistics, nothing prepares you for data science like actually doing it. He recommends participating in the data mining competitions hosted by Kaggle.
"If you have a PhD, you'll come to a problem with more background, but you'll still need to get your hands dirty to solve it," Candido says. "I don't want to downplay the value, but don't think it's an absolute necessity."
Merrill -- Candido's boss at ZestFinances -- agrees. "We hire data scientists from all walks of life with backgrounds in several different areas, and we do have people on our team without graduate degrees," he says. "That's because math is only half the problem when it comes to data science -- it's also an art as well. The artistry comes in the form of people who have intuition and who creatively approach a problem."
Some software vendors have used the perception that data science requires rare and expensive talent to pitch business intelligence applications that can be used by less technical employees to mine data. But while data analysts and business intelligence professionals tend to know what data sets to analyze and what to look for, data scientists are more experimental. They have to find data sets, figure out what to mine from them and how. Off the shelf software may simplify the mathematics, but there's more to data science than crunching numbers.
No matter what degree they earn, Candido says, data scientists are never done learning. "Staying up on what's new in the field is extremely important, if you don't you will get left behind very quickly," he says. "Participating in data mining contests is one way to stay up. Keep an eye on the people who are better than you."
