Storage Management via Web Services

Depending on the analyst you read, data storage technology consumes between 33 and 70 cents of every dollar spent on IT hardware.  And that is only a fraction of the total cost of ownership of disk storage.  The management - or lack thereof - of disk infrastructure contributes significantly to its associated TCO.

Unless you have only one array -- or in some cases if you buy your disk arrays only from one vendor -- the tools provided with the storage equipment you buy are generally insufficient to manage storage efficiently.  The result is a cost spiral:  the more unmanaged storage you add, the more folks you need to hire to manage petabytes of capacity.

While many vendors in the storage industry have sought to address this problem, most have focused on adding functionality to the array controller -- essentially a computer itself -- that automates the processes for allocating capacity to applications and to optimize capacity utilization in non-granular ways (e.g., without knowing much about the actual business value of the data itself).  One outcome of the addition of value-add software to the array controller is that the storage array tends to become an isolated island of data, a stovepipe, that cannot be managed effectively in conjunction with other arrays from other vendors or in some cases from the same vendor.

For this reason, so-called "smart storage" often translates into costly storage -- to buy and to own.

What is desperately needed to drive cost out of storage infrastructure is a unified management approach that enables products from different vendors (called "heterogeneous storage infrastructure") to be managed conveniently by fewer numbers of staff.  Three approaches exist to realizing this goal.

1. Virtualize the storage.  This approach uses a server or switch equipped with virtualization software functionality to aggregate all of the disks in your storage infrastructure.  Thus pooled, software tools are provided that enable storage administrators to broker capacity out to applications and end users that need it.  Some products include sophisticated tools for applying policies to data and to storage to facilitate the movement of data over infrastructure over time (so that faster, more expensive disk is not used to host files that are infrequently accessed), or to ensure that specific services are availed to data that need them (such as archive, replication, etc.).
2. Manage via Storage Resource Management software.  In this approach, a software vendor sells a storage management software product called SRM that is installed and connected to storage rigs via a combination of standards-based hooks and proprietary hardware vendor-based application programming interfaces (APIs).  SRM tends to be challenging to deploy and requires training, plus SRM vendors are susceptible to the vicissitudes of hardware vendor politics.  APIs that were once shared by a hardware vendor may be withheld by that vendor, or their licensing or validation schemes may be changed to raise hurdles for SRM vendors.
3. Manage via Open Standards.  Similar to SRM, this approach leverages open standards to collect information about storage and to facilitate configurations changes.  It works when vendors cooperate in conforming their array management designs to the standards themselves, something that they have shown a general inclination not to do.  One standards-based management approach, the Storage Management Intiative - Specification from the Storage Networking Industry Association (SNIA), produced a method nominally embraced by most of the vendor community, but implemented on only a fraction of the products available in the market today. 

For the most part, vendors have preferred to provide storage management "by-the-box" in the form of proprietary array controller functionality and box-specific element management.  It could be argued that the vendors use this approach to make it difficult for a consumer to deploy a competitor's gear.  The CEO of a leading storage vendor said, in 2001, that there is no difference between the component parts in all vendor arrays, so "to maintain and grow our profit margins, we need to join proprietary software to the hip of a proprietary controller that locks in the consumer and locks out the competition."

Breaking with this formula is Xiotech Corporation, an Eden Prairie, Minnesota-based array vendor that acquired in 2008 the Advanced Storage Technology Group of Seagate Corporation and with it a technology called Intelligent Storage Element or ISE.  ISE proffers some interesting design advantages over other storage rigs today, but a key value feature is that hardware products based on the technology are instrumented for common management via open standards W3C Web Standards. 

Prior to acquiring the business unit of Seagate, Xiotech was already pursuing Web Services-based management of its storage arrays.  They found other management paradigms, including APIs and SMI-S, to be too laborious and time-consuming to design and deploy on their rigs and, more importantly, considered the environment in which storage operates.  Applications already talk Web Services (SOAP, XML, REST, etc.)  In effect, the application extends its hand into the infrastructure and asks for available resources, including storage, only nothing responds back to the request.

Xiotech saw an opportunity to leverage the development work already done with application, hypervisor and operating systems vendors to enable their products for unified management and operational control via Web Services and decided to explore implementing Web Services based management on their own storage arrays.  That decision produced a remarkable result.

Today, using a smartphone or a convenient terminal device like an Apple iPad, a storage manager can access petabytes of distributed storage simply and conveniently to determine status, to perform configuration changes, or to make the myriad other changes that are required for day to day operation.  In the video below, Eric Lomascolo, Xiotech's Director of Product Management, demonstrates Web Services managent of ISE arrays. 

• Interview With Eric Lomascolo, Xiotech, Regarding ISE and Web Services-based Management

Lomascolo knocked our socks off with a demonstration of Xiotech Emprise array management using, first, an iPAD, then an Apple iPhone.  He noted that Xiotech sees W3C standards not only as a way to manage their storage, but to manage the arrays of any vendor who elects to instrument their platforms for management via open standard Web Services.  He said that his company's design choice not only minimizes the hassles and cost of storage management, but also provides a glue that he hopes will be leveraged by third party developers to create an ecosystem of hardware and software products that avail themselves of common management.

IT-SENSE believes that Xiotech is the de facto thought leader in this space.

We applaud them not only for their technical accomplishment, but also for opening up the technology for third party use at their web forum:  cortexdeveloper.com.

Note:  Eric Lomascolo's full interview, as well as additional information about ISE and Web Services technology, is provided on our sister site, c4project.org.