As technology to break down data siloes and analyze data improves, business can be transformed in all sorts of ways. The advances in analyzing Big Data allow researchers to decode human DNA in minutes, which makes businesses like 23andme feasible.
Researchers are able to predict where terrorist plan to attack, which gene is mostly likely to be responsible for certain diseases and, of course, which ads you are most likely to respond to on Facebook.
In fact, a recent study published in PNAS found that the things you "like" on Facebook reveal all sorts of probable traits about you, such as your intelligence, your gender, your sexual preference, your political leanings and more.
The businesses cases for leveraging Big Data are more compelling than your addiction to curly fries. For instance, Netflix mined its subscriber data to put the essential ingredients together for its recent hit House of Cards, and subscriber data also prompted them to bring Arrested Development back from the dead.
Another example comes from one of the biggest mobile carriers in the world. France's Orange launched its Data for Development project by releasing subscriber data for customers in the Ivory Coast. The 2.5 billion records, which were made anonymous, included details on calls and text messages exchanged between 5 million users.
A number of researchers accessed this dataset and sent Orange proposals about how this data could serve as the foundation for development projects. Proposed projects included one that showed how to improve public safety by tracking cell phone data to map where people went after emergencies; another showed how to use cellular data for disease containment, which Twitter actually already helped do during a cholera outbreak in Haiti.
Finally, the NSA's Prism program relies on Big Data analytics as its justification for even existing. The program vacuums in metadata from cell phone calls, email exchanges, IM chats, social media and who knows what else?
As government officials defend the program, they often fall back on Big Data analysis as the key defense. If someone is a suspected terrorist, then that person's phone records could unearth other terrorists or even help Homeland Security officials pinpoint the likely target of an upcoming attack.
The larger Big Data market is a confusing one, comprised of everything from the underlying databases to storage to infrastructure platforms on up to the application layer, where we try to make sense of the data.
Today, the Big Data analytics market is still in its infancy. Big incumbents, such as Software AG, Oracle, IBM, Microsoft, SAP, EMC, and HP, compete with scrappy startups like Datameer, Alpine Data Labs, SiSense and Cloudmeter.
While the incumbents have collectively spent billions acquiring software firms in the data management and analytics space (Apema, Jacada, More IT Resources, Vertica and Vivisimo, to name only a few), the startups are sitting on impressive stacks of VC funding.
To further complicate things, companies that are a bit too old to be labeled startups have a foothold too. These include Pentaho, Splunk and Jaspersoft.
Finally, many Big Data analytics startups are targeting specific niches, such as social marketing (DataSift), programmatic advertising buying (Rocket Fuel), application performance (Cloudmeter) and even job searches and recruiting (Bright.com).
According to Wikibon, the total Big Data market reached $11.4 billion in 2012, ahead of Wikibon’s 2011 forecast. Wikibon projects that the Big Data market will reach $18.1 billion in 2013, an annual growth of 61 percent. This puts it on pace to exceed $47 billion by 2017, which translates to a 31 percent compound annual growth rate over the five year period 2012-2017.
Clearly, there is plenty of room for a number of vendors, as this market sector is still a land grab, but expect more consolidation in the near future.
This is where Big Data analytics comes in. What we're after isn't just raw data. We want the knowledge that comes from analyzing that data.