dcsimg

10 Big Data Predictions for 2018

  • 10 Big Data Predictions for 2018

    10 Big Data Predictions for 2018
    Real-time analytics, AI, cloud computing, and deep learning will play key roles in the big data market this year.
  • 1. Revenue Growth

    Revenue Growth

    For years, sales of big data solutions have been on a steady upward trend. And that is unlikely to change in 2018. In its Worldwide Semiannual Big Data and Analytics Spending Guide, IDC predicted, "Commercial purchases of BDA-related hardware, software, and services are expected to maintain a compound annual growth rate (CAGR) of 11.9 percent through 2020, when revenues will be more than $210 billion."

    According to IDC's Dan Vesset, part of the reason for that continued growth is that big data has become a standard part of doing business. "After years of traversing the adoption S-curve, big data and business analytics [BDA] solutions have finally hit mainstream," he stated. "BDA as an enabler of decision support and decision automation is now firmly on the radar of top executives. This category of solutions is also one of the key pillars of enabling digital transformation efforts across industries and business processes globally."

  • 2. Real-Time Analytics

    Real-Time Analytics

    A lot of this year's increased big data spending will go towards real-time analytics solutions. In the past, enterprises put a lot of focus on batch reports that they generated daily, weekly, monthly, quarterly or annually. Now, they want to be able to run reports constantly throughout the day and receive instantaneous alerts about important data trends. As a result, they are investing heavily in real-time analytics.

    According to Gartner, "Between 2016 and 2019, spending on real-time analytics will grow three times faster than spending on non-real-time analytics." The analyst firm says this trend results from both customer demand for fast response and executive desire for digital transformation and data-driven culture.

  • 3. Data as a service

    Data as a service

    The early days of big data analytics, enterprises focused on mining their data for their own personal use. However, analysts say that enterprises are increasingly opening their data troves to third parties. Organizations recognize that big data is a financial asset, and so they are looking for ways to extract value from that asset.

    In its IDC FutureScape: Worldwide IT Industry 2018 Predictions, IDC predicted, "By 2020, 90 percent of large enterprises will generate revenue from data as a service — from the sale of raw data, derived metrics, insights, and recommendations — up from nearly 50 percent in 2017." It added, "Everyone will be a data provider."

    However, the analyst firm made that prediction before the recent Cambridge Analytica scandal at Facebook. It remains to be seen whether enterprises will scale back their data sharing in response to concerns from customers and regulators.

  • 4. Big data in the cloud

    Big data in the cloud

    Another key trend in big data is greater use of cloud analytics. In a blog post, Forrester vice president and principal analyst Brian Hopkins wrote, "Global spending on big data solutions via cloud subscriptions will grow almost 7.5 times faster than on-premise subscriptions. Furthermore, public cloud was the number one technology priority for big data according to our 2016 and 2017 surveys of data analytics professionals."

    Hopkins said he believes that cloud computing, specifically insight platform as a service offerings, offer so many cost advantages as well as access to innovative technology, that on-premise big data solutions simply won't be able to compete. He stated, "In five years you’ll be using Insight PaaS for big data in the public cloud. On-premise won’t be an option."

  • 5. Open source, proprietary and cloud

    5. Open source, proprietary and cloud

    Today, nearly every commercial big data solution supports open source tools like Hadoop and Spark. In fact, the entire big data ecosystem, including both cloud and on-premise solutions, has achieved an astoundingly high degree of interoperability.

    The 2018 Analytics Predictions and Priorities report from the International Institute for Analytics lists as one of its forecasts for the year, "Open source, proprietary analytical software, and the cloud are fully intertwined." It added, "Analytics professionals can now choose the best mix of software and the best mix of on premise and cloud for their needs." It recommended that organizations use a combination of proprietary, open source, cloud and on-premise software and that they steer away from becoming too reliant on any single technology.

  • 6. Deep learning

    Deep learning

    Artificial intelligence (AI) is also becoming intertwined with big data analytics, and deep learning is a key part of that trend. In fact, Gartner predicted, "By 2018, deep learning (deep neural networks [DNNs]) will be a standard component in 80 percent of data scientists’ tool boxes."

    Not too long ago, deep neural networks were so cutting-edge and required such advanced hardware that only the very largest enterprises could afford to experiment in this area. But the rise of cloud-based tools has put deep learning within reach for almost every organization. And while it is primarily data scientists using deep learning today, over time organizations can expect more vendors to add deep learning capabilities to tools used by ordinary business analysts and other professionals without advanced data science training.

  • 7. AI value

    AI value

    Speaking of AI, the amount that this category of technology is adding to the economy is staggering. In an April 2018 press release, Gartner predicted, “Global business value derived from artificial intelligence (AI) is projected to total $1.2 trillion in 2018, an increase of 70 percent from 2017." It added, "AI-derived business value is forecast to reach $3.9 trillion in 2022."

    And AI-based products that perform big data analytics are likely to be key contributors to that business value. John-David Lovelock, research vice president at Gartner, said, "AI promises to be the most disruptive class of technologies during the next 10 years due to advances in computational power, volume, velocity and variety of data, as well as advances in deep neural networks (DNNs)."

  • 8. AI failure

    AI failure

    But while AI is going to be huge (and already is), not all of those AI projects will be successful. Forrester has gone so far as to predict that "75 percent of early AI projects will underwhelm due to operational oversights."

    The firm's Predictions 2018: A Year of Reckoning report noted that last year's AI projects largely focused on narrow use cases as a way to prove the value of AI technology. While those efforts were largely successful, the benefits were "too narrow and short-lived." This year, enterprises are broadening the scope of their AI projects, but Forrester sees many organizations failing to model operational considerations with negative consequences. However, the firm said it believes this could ultimately be beneficial because it will drive "business leaders to reset the scope of AI investments — and place their firms on a path to realizing the expected benefits."

  • 9. People challenges

    People challenges

    Other key challenges related to big data center around people. Enterprises can never seem to find enough skilled big data employees to meet their needs, and the people they do have are dragging their feet on embracing a data-driven culture.

    In its Big Data Executive Survey 2018 report, NewVantage Partners predicted, "People challenges loom greatest as firms strive to create a data-culture." It noted that 98.6 percent of the executives surveyed are striving to create a data-driven culture, but less than a third (32.4 percent) said they have achieved that goal. That's actually fewer than said the same thing in 2017, which doesn't bode well for cultural transformation efforts.

  • 10. Data scientists

    Data scientists

    The other ongoing big data people issue — staff shortages — is leading to a watering-down of the data scientist title. One of the International Institute for Analytics' predictions for 2018 is that "Everybody claims to be a data scientist."

    Part of the reason for this trend is that data scientists make great money. The Robert Half Technology 2018 Salary Guide listed data scientist as one of the most in-demand professions and noted that median pay for the job title in the United States is $119,000, with top earners brining in $168,000 or more. That kind of money is attracting a lot of job seekers who may not have the full set of skills typically associated with data science.

    The report recommends that employers carefully vet job seekers to make sure that they have the skills enterprises need in order to be successful with their big data projects.

  • 1 of

10 Big Data Predictions for 2018

  • 1 of
  • 10 Big Data Predictions for 2018

    10 Big Data Predictions for 2018

    Real-time analytics, AI, cloud computing, and deep learning will play key roles in the big data market this year.
  • Revenue Growth

    1. Revenue Growth

    For years, sales of big data solutions have been on a steady upward trend. And that is unlikely to change in 2018. In its Worldwide Semiannual Big Data and Analytics Spending Guide, IDC predicted, "Commercial purchases of BDA-related hardware, software, and services are expected to maintain a compound annual growth rate (CAGR) of 11.9 percent through 2020, when revenues will be more than $210 billion."

    According to IDC's Dan Vesset, part of the reason for that continued growth is that big data has become a standard part of doing business. "After years of traversing the adoption S-curve, big data and business analytics [BDA] solutions have finally hit mainstream," he stated. "BDA as an enabler of decision support and decision automation is now firmly on the radar of top executives. This category of solutions is also one of the key pillars of enabling digital transformation efforts across industries and business processes globally."

  • Real-Time Analytics

    2. Real-Time Analytics

    A lot of this year's increased big data spending will go towards real-time analytics solutions. In the past, enterprises put a lot of focus on batch reports that they generated daily, weekly, monthly, quarterly or annually. Now, they want to be able to run reports constantly throughout the day and receive instantaneous alerts about important data trends. As a result, they are investing heavily in real-time analytics.

    According to Gartner, "Between 2016 and 2019, spending on real-time analytics will grow three times faster than spending on non-real-time analytics." The analyst firm says this trend results from both customer demand for fast response and executive desire for digital transformation and data-driven culture.

  • Data as a service

    3. Data as a service

    The early days of big data analytics, enterprises focused on mining their data for their own personal use. However, analysts say that enterprises are increasingly opening their data troves to third parties. Organizations recognize that big data is a financial asset, and so they are looking for ways to extract value from that asset.

    In its IDC FutureScape: Worldwide IT Industry 2018 Predictions, IDC predicted, "By 2020, 90 percent of large enterprises will generate revenue from data as a service — from the sale of raw data, derived metrics, insights, and recommendations — up from nearly 50 percent in 2017." It added, "Everyone will be a data provider."

    However, the analyst firm made that prediction before the recent Cambridge Analytica scandal at Facebook. It remains to be seen whether enterprises will scale back their data sharing in response to concerns from customers and regulators.

  • Big data in the cloud

    4. Big data in the cloud

    Another key trend in big data is greater use of cloud analytics. In a blog post, Forrester vice president and principal analyst Brian Hopkins wrote, "Global spending on big data solutions via cloud subscriptions will grow almost 7.5 times faster than on-premise subscriptions. Furthermore, public cloud was the number one technology priority for big data according to our 2016 and 2017 surveys of data analytics professionals."

    Hopkins said he believes that cloud computing, specifically insight platform as a service offerings, offer so many cost advantages as well as access to innovative technology, that on-premise big data solutions simply won't be able to compete. He stated, "In five years you’ll be using Insight PaaS for big data in the public cloud. On-premise won’t be an option."

  • 5. Open source, proprietary and cloud

    5. Open source, proprietary and cloud

    Today, nearly every commercial big data solution supports open source tools like Hadoop and Spark. In fact, the entire big data ecosystem, including both cloud and on-premise solutions, has achieved an astoundingly high degree of interoperability.

    The 2018 Analytics Predictions and Priorities report from the International Institute for Analytics lists as one of its forecasts for the year, "Open source, proprietary analytical software, and the cloud are fully intertwined." It added, "Analytics professionals can now choose the best mix of software and the best mix of on premise and cloud for their needs." It recommended that organizations use a combination of proprietary, open source, cloud and on-premise software and that they steer away from becoming too reliant on any single technology.

  • Deep learning

    6. Deep learning

    Artificial intelligence (AI) is also becoming intertwined with big data analytics, and deep learning is a key part of that trend. In fact, Gartner predicted, "By 2018, deep learning (deep neural networks [DNNs]) will be a standard component in 80 percent of data scientists’ tool boxes."

    Not too long ago, deep neural networks were so cutting-edge and required such advanced hardware that only the very largest enterprises could afford to experiment in this area. But the rise of cloud-based tools has put deep learning within reach for almost every organization. And while it is primarily data scientists using deep learning today, over time organizations can expect more vendors to add deep learning capabilities to tools used by ordinary business analysts and other professionals without advanced data science training.

  • AI value

    7. AI value

    Speaking of AI, the amount that this category of technology is adding to the economy is staggering. In an April 2018 press release, Gartner predicted, “Global business value derived from artificial intelligence (AI) is projected to total $1.2 trillion in 2018, an increase of 70 percent from 2017." It added, "AI-derived business value is forecast to reach $3.9 trillion in 2022."

    And AI-based products that perform big data analytics are likely to be key contributors to that business value. John-David Lovelock, research vice president at Gartner, said, "AI promises to be the most disruptive class of technologies during the next 10 years due to advances in computational power, volume, velocity and variety of data, as well as advances in deep neural networks (DNNs)."

  • AI failure

    8. AI failure

    But while AI is going to be huge (and already is), not all of those AI projects will be successful. Forrester has gone so far as to predict that "75 percent of early AI projects will underwhelm due to operational oversights."

    The firm's Predictions 2018: A Year of Reckoning report noted that last year's AI projects largely focused on narrow use cases as a way to prove the value of AI technology. While those efforts were largely successful, the benefits were "too narrow and short-lived." This year, enterprises are broadening the scope of their AI projects, but Forrester sees many organizations failing to model operational considerations with negative consequences. However, the firm said it believes this could ultimately be beneficial because it will drive "business leaders to reset the scope of AI investments — and place their firms on a path to realizing the expected benefits."

  • People challenges

    9. People challenges

    Other key challenges related to big data center around people. Enterprises can never seem to find enough skilled big data employees to meet their needs, and the people they do have are dragging their feet on embracing a data-driven culture.

    In its Big Data Executive Survey 2018 report, NewVantage Partners predicted, "People challenges loom greatest as firms strive to create a data-culture." It noted that 98.6 percent of the executives surveyed are striving to create a data-driven culture, but less than a third (32.4 percent) said they have achieved that goal. That's actually fewer than said the same thing in 2017, which doesn't bode well for cultural transformation efforts.

  • Data scientists

    10. Data scientists

    The other ongoing big data people issue — staff shortages — is leading to a watering-down of the data scientist title. One of the International Institute for Analytics' predictions for 2018 is that "Everybody claims to be a data scientist."

    Part of the reason for this trend is that data scientists make great money. The Robert Half Technology 2018 Salary Guide listed data scientist as one of the most in-demand professions and noted that median pay for the job title in the United States is $119,000, with top earners brining in $168,000 or more. That kind of money is attracting a lot of job seekers who may not have the full set of skills typically associated with data science.

    The report recommends that employers carefully vet job seekers to make sure that they have the skills enterprises need in order to be successful with their big data projects.

Someday, artificial intelligence (AI) will advance to the point where it can analyze all the data about big data and come up with its own predictions about the market. However, until that day comes (and it may be sooner than you expect), human research analysts probably offer the best forecasts about the future of the market.

The following slides detail analyst's big data predictions for 2018. The market is clearly growing rapidly, and enterprises are investing in some different types of technologies than they have in the past. Integrating some of those new technologies into business processes is presenting challenges for organizations, and many are likely to experience some failures. In addition, people and organizational challenges continue to hamper organizational efforts to become more data-driven.

And while AI hasn't yet taken over the role of market predictions, it is increasingly taking over big data. If there's a theme to the 2018 big data predictions, it's that AI and big data analytics are becoming indistinguishable. As vendors update their solutions, look for them to incorporate more and more AI technologies.

Check out the following slides and see if you agree with analysts' predictions for where big data and AI are heading for 2018.

Submit a Comment

Loading Comments...