Maybe five or six years ago I first heard business intelligence vendors try to explain that not everyone needed real-time data and insights. "What they usually really need is right-time data—the right information to the right person at the right time," they would say.
A subset of data to a subset of users delivered overnight would solve the problem at reasonable cost, they insisted.
No one wanted to deal with the cost and complexity of revamping billions of dollars of online transaction processing (OLTP) systems and converting them to online analytical processing (OLAP). The vendors insisted that the traditional approach – bolting a data warehouse onto the OLTP environment to provide users with reports and an occasional OLAP engine for real time info – was an effective and economically efficient solution.
Forgetaboutit, as they say in New York. A combination of new technologies, user expectations based on the consumerization of IT, and accelerated business conditions has made real time the new right time for a large swath of business users, as well as for IT itself.
A new series of surveys – plus numerous conversations with IT industry officials – makes it clear that IT professionals must rethink their traditional response when a manager demands real time access to BI or analytics systems.
Forty percent of respondents to a recent survey I managed said their organizations would have real-time access to analytics and business intelligence information instead of batch reports within two years. (Survey included more than 300 corporate executives).
Furthermore, real-time reporting was the number one investment priority among 10 different technology projects including analytics, consolidating data centers, Big Data and mobile access (see table for a sample of their answers):
Note that another 38% were already getting reports in real time rather than batch and 39% were already providing mobile device access to financial data and systems.
Recent surveys by Ventana Research, a pre-eminent BI/analytics benchmark research and advisory services firm, found even stronger interest in real time BI and analytics. Almost six out of ten respondents (mostly business users) to its Operational Intelligence Benchmark Research project said managing performance, detecting fraud, or compliance demands drove their demand for real time (see table):
Source: Ventana Research Operational Intelligence Benchmark Research
Ventana's recent Business Technology Innovations Benchmark Research, mostly of IT officials, also indicated strong interest in real time BI/analytics for security, network and systems management (see table):
Source: Ventana Research Business Technology Innovations Benchmark Research
Tony Cosentino, VP & Research Director of Ventana Research, noted that the survey found even stronger interest in implementing real time BI/analytics in the next two years via complex event processing. "We are currently past the early adopter market for event processing, with 18% already having the capability,” he said. “Another 29% of survey respondents said they would have it within the next 12 months and an additional 49% said they would have it within the next 24 months.”
Furthermore, he added, "2013 will see a much broader discussion about real time BI and analytics.”
Indeed, much of the discussion will be to clarify terms. Real time BI/analytics means different things to different people. The needs vary by department, industry, and national and corporate culture.
"There is still a lot of confusion over what right time and real time really are," noted Cosentino. "When we talk about real-time transaction processing, with data streaming into mobile devices right from transaction systems, rules-based analytics trigger actions. This complex event process is like when a car crashes and the air bags deploy. These rule-based systems are different than the complex algorithms that may be at work behind the scenes."
In addition to the immediate reaction to an event, Cosentino and others warn IT that everybody expects everything right away. "If someone waits more than two seconds for something to appear on their screen, they will jump to another screen," Cosentino added.
For more information about the Ventana Research data, here's a video of a Cosentino presentation to clients.
The impatience to get data and analytics fast is one of the byproducts of the consumerization of IT. Michael Driscoll, CEO of cloud analytics provider Metamarkets, summarizes customer demands this way: "We're entering the Age of Enlightenment for enterprise IT. Executives are starting to ask ‘Why does it take us days to get an internal report, when Google tells me instantly what is out there? Why are our systems slow but others are fast?'"
IT professionals who balk at delivering real time BI/analytics, citing standard explanations for the delays and standard caveats about moving to a real time environment, are doomed. Senior business execs will reject warnings that the existing IT infrastructure was built for batch mode and that the costs and complexities to shift to real time are prohibitive.