Storage, in many respects, is a mess. On the storage technology side, organizations typically include a mix of Direct Attached Storage (DAS), Storage Area Networks (SAN) and Network Attached Systems (NAS). At the same time, they have to deal with multiple hardware systems each with built in proprietary software to manage RAID, tape drives and other storage management functions.
These systems, in turn, run on the gamut of operating systems -- various versions of Windows, UNIX, Linux, NetWare and Mainframe OS's, not to mention the fact that many have their own unique file systems.
Other sources of storage chaos include widely separated sites, out of control disk space requirements, and staffing levels that have not kept up with growth. Add to that tightening IT budgets, and you are faced with a significant challenge.
"With eBusiness related storage requirements doubling annually, organizations are faced with three main challenges: Building storage environments that can scale to meet demand, effectively managing these increasingly complex environments and coping with ever-shrinking back up windows," said Bill North, research director for storage software at International Data Corp. (IDC).
One possible solution to the chaotic nature of enterprise storage is the storage portal. In this article we take a look at the growth of storage, its repercussions in the enterprise, and how storage portals may simplify some of the issues facing the industry.
According to IDC, storage needs grow on average at 50% to 100% annually. Other analysts put growth levels even higher. The usual answer has been to ensure enough capacity through over-provisioning. i.e. throw more servers at the problem. Result: storage resources tend to average a utilization rate of about 30%.
This situation is compounded by a growing management/personnel headache. An SRC (Strategic Research Corp.) study found that over the next three years enterprise storage demand would increase seven-fold while the management resources needed would remain flat.
"Complex storage systems with out-of-control management costs have created an intense need for better Enterprise Storage Management (ESM) tools," said Michael Peterson, president of SRC.
The blame for this situation, though, can't be conveniently attributed to storage hardware vendors. Products such as Compaq Insight Manager (CIM), Dell OpenManage and Hitachi High Command do a great job of managing vendor specific systems, and in some cases, in interoperating with some types of generic storage hardware.
What ends up happening, though, is that storage administrators end up having to understand dozens of tools: they want to see how RAID is running on their Dell servers -- they open up the Dell OpenManage screen; they want to view their Compaq RAID arrays -- time to open up CIM; they want to check utilization, performance and bandwidth levels on various systems and they end up going to the screens of various proprietary tools offered by BMC, EMC and a host of others.
So to get anything done, never mind understand the overall storage environment, administrators must hop from console to console to configure, allocate, reallocate and care for the various storage assets at their disposal. This often means going from building to building and system to system in order to back up, restore, mirror or troubleshoot.
A seemingly basic upgrade in one department from DAS to NAS, for instance, can turn into a nightmare: Changes must be made to all backup, recovery and business continuation/disaster recovery procedures. For backup alone, this means going to a different screen for every backup product in use in the enterprise -- in a surprisingly large number of companies, this can run from two to a half-dozen.
"Storage management is often an ad-hoc collection of disparate tools and predominantly independent device management utilities," said Nancy Marrone, senior analyst at Enterprise Management Group.
Marrone is a strong advocate of Enterprise Storage Automation (ESA) as a means of streamlining storage management. One such ESA approach that may solve many of the above issues is the storage portal.
"What has been missing is a storage management layer that sits above existing applications and systems," said Phil Triede, a storage management executive at Islandia, NY-based Computer Associates (CA). "A well designed storage portal can integrate and access disparate systems and allow control of all of them automatically from a single console."
While CA's BrightStor Portal is probably the most advanced storage portal currently available, other vendors aren't too far behind. HP offer the Compaq Storage Management Portal, and AlphaNet recently announced their Managed Enterprise Storage Portal (which also utilizes BrightStor technology).
These tools, in greater or lesser degree provide the following elements:
Such portals, though, are not yet the finished article. There is still a way to go before we achieve end-to-end ESA -- where administrators monitor and manage the entire scope of the storage environment including laptops, workstations, servers, switches and arrays from one screen, and all routine tasks are automated.
"Instead of carrying out as many as 15 manual steps to add new storage to a SAN environment, policy-driven intelligence would automatically discover new storage capacity, configure it appropriately and move data accordingly," said Triede.
The key elements missing, then (though coming very soon according to CA and other vendors), are:
Dynamical provisioning: the ability to automatically discover new storage capacity, configure it and takes care of moving the data. Thus storage management applications can conduct error-free, timely and automatic processing of storage needs without the administrator having to involve himself in the myriad details such as where the capacity is physically located or its low-level configuration.
Automated storage best practices: Defining storage best practices allows you to map out the key processes and procedures and incorporate them into a workflow engine that consolidates sequences of tasks such as backup and restore so that their many steps can be automated. Administrators can then focus on true storage management and troubleshooting.
Topology-guided management: Constant change is the order of the day in the land of storage. But that makes it difficult for storage operators to keep current on all the latest updated needed to applications, processes, systems and components that correspond to each change. Topology-guided management automatically modifies all aspects of the environment. This separates out the business of storage from the underlying technology. Thus backup and recovery operations will be able to determine the best path, route and procedures to accomplish the tasks based on current up-to-date storage topologies.
Such additions to the storage administrator's arsenal are less than a year away according to storage vendors. When that day comes, new storage devices, platforms and systems will be automatically discovered, allocated against enterprise-wide policies, provisioned based on best practices and deployed where most needed. It will soon be possible to balance data across storage volumes, perform backup/recovery tasks and optimize storage paths according to bandwidth availability and storage loads without administrator involvement.
ESA's top-down approach to storage management is no pipe dream either. While it represents an evolution of storage management to a higher level, it is really only catching up to what is now regarded as the norm in systems and network management -- where achieving a unified view across the enterprise, once a newsworthy subject, is now regarded as the norm.