10 Big Data Predictions for 2017 and BeyondLook for technologies like artificial intelligence, prescriptive analytics and real-time streaming to play a bigger role in big data solutions.
1. More Data
If there's one big data prediction for 2017 that seems almost guaranteed to come true, it's that enterprises will be dealing with more data than ever before. According to the most recent Digital Universe study conducted by IDC on behalf of Dell EMC, the world's storage systems held 4.4 trillion gigabytes at the end of 2013. Given that the volume of data is roughly doubling every two years, that means storage systems will contain roughly 17.6 trillion gigabytes of data by the end of 2017. Much of that data growth is resulting from other important IT trends, like the Internet of Things (IoT), social media and the proliferation of mobile devices.
Source: Dell EMC
2. More Spending
In order to store and analyze all that data, enterprises are going to have to increase their spending. Analysts at IDC predict, “Worldwide revenues for big data and business analytics will grow from nearly $122 billion in 2015 to more than $187 billion in 2019.” That's a 50 percent increase over five years.
More than half of that spending will go to services providers, while companies will spend about $55 billion on big data software and $28 billion on hardware in 2019. Sectors like utilities, resource industries, healthcare and banking are likely to see their big data spending rise the fastest, while manufacturing will remain the biggest big data spender overall.
3. The Cloud
Analysts believe much of that new spending will go towards cloud-based big data analytics (BDA) solutions. In fact, IDC forecasts, “Through 2020, spending on cloud-based BDA technology will grow 4.5x faster than spending for on-premises solutions.”
IDC adds that many of those cloud-based solutions will be based on open source big data solutions. Technologies like Hadoop, Spark, Storm and others have come to dominate big data analytics, and their prevalence looks likely to continue for the foreseeable future. All of the leading cloud computing vendors, like Amazon Web Services, Microsoft Azure, IBM and Google Cloud, offer big data analytics products, and many smaller firms offer cloud-based analytics products as well.
One of the ways that big data analytics vendors may look to distinguish themselves from one another is through their artificial intelligence (AI), deep learning and machine learning technologies. Over the past year, these types of solutions have begun to hit the market in a major way, and developers are integrating AI capabilities into many types of applications, including big data analytics applications. Next year could be an even bigger year for AI in big data. Forrester predicts that in 2017, “Investment in artificial intelligence (AI) will triple as firms look to tap into complex systems, advanced analytics, and machine learning technology.”
5. Predictive and Prescriptive Analytics
Those AI and machine learning capabilities will give big data analytics solutions better predictive and prescriptive capabilities. In the past, enterprises relied on their analytics primarily to help them make sense of historical data, but the next generation of BDA tools will also include predictive capabilities that forecast what is likely to happen in the future and prescriptive capabilities that tell organizations what they should do in order to take advantage of those forecasts. IDC says, “By 2020, 50 percent of all business analytics software will incorporate prescriptive analytics built on cognitive computing functionality.”
Organizations that are farther along in the big data analytics maturity chain will likely have an advantage over the competition.
Big data analytics tools like Hadoop are very good at analyzing batch data, but enterprises have begun looking for ways to narrow the window between when data is received and when it is analyzed and acted upon. Ideally, many would like to be able to analyze data in real-time, and as a result, they have begun investing in solutions like Kafka and Spark that have streaming analytics capabilities. According to Markets & Markets, “The streaming analytics market is estimated to grow from USD 3.08 Billion in 2016 to USD 13.70 Billion by 2021, at a Compound Annual Growth Rate (CAGR) of 34.8 percent.”
Source: Markets & Markets
7. High Salaries
With so many organizations looking to hire staff with big data skills, data scientists and database professionals can expect salary increases for 2017. According to Robert Half Technology, big data engineers will see an average 5 percent pay increase for salaries ranging from $123,000 to $158,000. And data scientists will likely bring home salaries between $99,500 and $132,000, a 3.3 percent increase from last year. Other experts say developers working on big data projects are also enjoying high pay, and the trend is likely to go on as long as the demand for talent continues to outstrip the available supply of big data experts.
Source: Robert Half Technology
With data scientists and other big data experts in short supply, many organizations are looking for analytics tools that allow business professionals to self-service their own needs. And the vendors have quickly begun responding to that need with next-generation big data analytics tools that feature drag-and-drop interfaces which simplify the creation of reports. IDC forecasts, “Through 2020, spending on self-service visual discovery and data preparation market will grow 2.5x faster than traditional IT-controlled tools for similar functionality.” These tools could help enterprises become more agile and increase the competitive advantage provided by their big data projects.
9. Ethical Questions
Of course, big data analytics doesn't only afford organizations with new opportunities, it also offers new challenges, particularly in relationship to privacy and security. As businesses collect, store and analyze more information about their customers, the risk grows that someone within the company will use that data improperly.
Gartner forecasts, “By 2018, half of business ethics violations will occur through improper use of big data analytics.” With that warning in mind, market analysts recommend that businesses take a cautious approach toward analytics and make slow, incremental progress towards their goals. In some cases, complementary technologies like machine learning may make it possible to achieve the desired outcomes without storing as much personal data.
10. Greater Productivity
Those organizations that successfully overcome the challenges inherent in big data could see a big payoff in terms of greater productivity. IDC predicts, “By 2020, organizations able to analyze all relevant data and deliver actionable information will achieve an extra $430 billion in productivity benefits over their less analytically oriented peers.” The words relevant and actionable are the keys here. Organizations understand that much of their big data might not be of any real value to their executives and managers. More and more the focus will be on identifying which data is important and speeding up the process of delivering actionable insights to key personnel.
Big data analytics is no longer a new technology. Today, most businesses recognize that they need to be actively mining their data stores for insights if they want to remain competitive in the constantly evolving marketplace. They've seen the benefit of big data solutions — and now they want more.
For 2017, the primary trends in big data will revolve around refining enterprises' core big data capabilities. They are looking for ways to analyze more data, more quickly. Having seen the payoff from their initial investments in big data technology, they are looking to expand their big data projects to achieve even greater financial results.
However, the big data talent shortage continues to be a problem. The lack of qualified job candidates is keeping salaries high even as organizations look for ways to do more analysis with fewer, less qualified staff. Leading vendors are making big data tools easier to use in hopes of attracting more users.
Organizations also face some potential challenges surrounding the privacy and security of their big data stores. But for those who successfully navigate the obstacles, the rewards could be substantial.
This slideshow highlights analyst predictions about current trends in the big data market. These are the issues that experts believe will be important in the coming year and into the future.
Image Source: Shutterstock