Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Nate Silver knows a thing or two about the value of Big Data. He famously predicted who would win 49 out of 50 states in the 2008 Presidential election and the following year was named by Time one of The World’s 100 Most Influential People. Silver’s Five Thirty Eight site focuses on “data journalism.”
But Silver warned in a recent keynote at the Rich Data Summit that Big Data can be misunderstood and oversold. A popular notion, Silver said, is that “you get your data, you press a button and all of a sudden you have extremely valuable output. This idea is very wrong and dangerous.”
In fact, the work data scientists do is far more complex. “Data scientists aren’t interested in data for data’s sake, we’re interested in relationships,” he said.
Jenny Dearborn, author of the book Data Driven: How Performance Analytics Delivers Extraordinary Sales Results, says we are a tipping point in our ability to collect, manipulate, analyze and act on big data.
“We finally have the ability to manipulate all this data we’ve been collecting and understand what to do with it,” says Dearborn, Chief Learning Officer at SAP. “We certainly had the information before, but it was hard to access and compile and get a big picture view of it; it was very much nose to the tree. Now we can see the forest and patterns and trends and ‘what does all this mean?’
“With all this information we can for the first time really answer some very big strategic questions: ‘What is the business problem we’re trying to solve?’ ‘What are the big trends here’?’” says Dearborn.
But she agrees with Silver there is no magic button to realizing the benefits of Big Data.
“It’s challenging, because it’s not just having the data, but knowing what to do with it,” says Dearborn. “Knowing what questions to ask, what business problems you are trying to solve and how do you apply analytics to the data you have to answer those big questions. There’s a lot of big strategic thinking that needs to happen in front of looking at your stacks of data or all your reports, and sometimes companies don’t take the time to ask those big questions. “
Discovering New Flavors
Some Big Data insights are relatively straightforward. Coca Cola has leveraged results from its network of Freestyle drink dispensers to create a new flavor. Freestyle, a kind of drink factory in a touchscreen box, lets customer mix and match over 170 brands of beverages at fast-food outlets, movie theaters and elsewhere. The soft drink giant is able to collect and analyze all those choices. When it saw a pattern of customers mixing Cherry and Vanilla Coke flavors, voila, it created a new, instantly popular flavor, Coke Cherry Vanilla.
Analyst Doug Henschen at Constellation Research points to manufacturing companies like GE and John Deere who are using Big Data to anticipate when parts are going to need to be fixed, resulting in savings on inventory and maintenance costs.
Henschen’s advice for companies considering investment in Big Data is that it’s okay to be innovative but “start with the business challenges you have today, don’t try and boil the ocean.”
Analyst Bob O’Donnell agrees and adds that even with the right structure and investment Big Data may fall short of expectations.
“There is a sense with some Big Data projects that it’s going to lead to constantly ongoing insights, but some of these are one and done,” says O’Donnell, Founder and Chief Analyst at Technalysis Research. “You do discover some insight and then you can track that one thing and maybe it leads to an 80 percent improvement in some process or product line initially, but going forward, there’s a diminishing return.”
O’Donnell also says some companies aren’t prepared to leverage Big Data results. A Big Data analysis may help Company X find, for example, that its product doesn’t appeal to single women over 40, but there may be no support internally at the company to change the product or strategy to address that market.
Big Data Rules of the Road
Andreas Weigend, the former Chief Scientist at Amazon.com who now runs Social Data Lab, shared some rules of the road at Rich Data Summit when it comes to starting a Big Data project.
1) Start with the problem, not with the data. If you start with data it grows exponentially and it will be a hose you can’t clean fast enough. Be clear about what question or problem you are trying to solve.
2) Be wary of consultants who say ‘Give us all your data and we’ll give you insights.’ Focus on decisions and actions you can take yourself.
3) Use metrics that matter to your customers. If you’re in a business that ships products to consumers, it may seem great to find out they’re arriving a day ahead of schedule. But actually that’s a hassle for the customer who planned to be home a day later to receive the package and finds an ’undeliverable’ note on their door.
4) Let people do what people are good at, and computers do what computers are good at.
5) Don’t blame technology for problems that you have in your institution. Weigand uses the example of not being allowed to use third party software when he was teaching at Stanford. “I got a note for using LinkedIn in one of my courses,” he recalled. “You wonder what planet this person is living on.”
Big Data and the Cloud
Analyst Charles King at Pund-IT says the growth of open source frameworks for handling Big Data sets like Hadoop and Apache Spark have led to more companies experimenting with and embracing Big Data.
“You can put together a Hadoop system relatively cheaper, though there’s a lot of assembly and technical expertise required,” says King. “Or you can have a third party like HortonWorks do it.”
He notes that operating a Big Data platform typically requires trained data scientists, who are in relatively short supply. King also expects to see more cloud-based Big Data projects that require a minimum of on-premise infrastructure. “Certain types of one-off projects could run only 1-6 months,” he said. “As Big Data matures, I think in the short term we’re going to see a growing number of companies offer Big Data as a service with the cloud as the backend.”
Big Data is also entering new areas such as physical store locations. A company called RetailNext helps big retailers do Big Data analysis in part by analyzing video feeds of how customers act in retail locations, e.g. what displays they gravitate to or ignore.
“If you look at Amazon.com, Macys.com or any ecommerce site, they have so much data and they use analytics to constantly improve the way they run their websites,” Alexei Agratchev, CEO of RetailNext, recently told The San Jose Mercury News. “Then you walk through Nordstrom or Victoria’s Secret and nobody has any idea what happens.”
Weigand says whatever the Big Data project you embark on, keep an eye on how it’s going.
“Does your product or service get better or worse with a Big Data project over time? I think we all know examples from both.”
Photo courtesy of Shutterstock.
