Storage is getting very cheap. Even a home user can get a 1TB LaCie external desktop drive for less than $US1000. The bad news is that because storage prices have fallen so sharply, capacity is growing 60 to 100 percent annually, so the amount of capacity each storage administrator needs to manage is going through the roof. The complexity of storage architectures is also increasing.
George Rodriguez, lead systems programmer at ABC Distributing oversees the catalogue and online retailer's IBM Enterprise Storage Server 2105 Model F20 storage array, which services a z800 mainframe as well as Unix servers running Oracle Financials. "The amount of storage available to the system is 4.3TB," he says. "Without an automation tool, managing this amount of storage would be an impossible task."
Without the tools to gain visibility into and control over storage, the company was running out of space, and that was causing delays in batch processing. A year ago, Rodriguez installed BrightStor CA-Vantage, a storage resource management (SRM) tool from Computer Associates, to provide a common interface for both the z800 and Unix storage. On the z800 side, he uses the software to manage the storage groups defined in the system using the Web publishing scripts that come with CA-Vantage. He also uses it to extract data generated by CA's BrightStor ARCserve backup utility to produce reports validating the backup results. Rodriguez says he set up the CA-Vantage graphical user interface on his own in less than a day, without any special training on the product.
"Setting up my own views took a little longer but was well worth the effort," he says. "Once you start using the facilities of the product, you can set up the same look and feel on both the mainframe and open-systems sides."
SRM can improve the efficiency of storage use and reduce the management load by bringing a number of functions into a single interface. Some SRM tools are stand-alone products, but SRM features are also found in some management applications. Functions vary by product, but they can include data collection, backup and recovery, user authentication, provisioning and performance monitoring.
"SRM is very useful as a capacity management tool, since it is the only tool that can do discovery of data characteristics for information life-cycle management, capacity management or, in some cases, change management," says Michael Peterson, president of Strategic Research Corp. "By themselves, they don't reduce complexity, but they do offer a view into an area that is hard to get your hands around, especially for unstructured data."
In fact, while SRM tools are designed to help reduce complexity, the complexity of the tools themselves hampered early adoption.
"In the past, SRM was trying to bite off too big a chunk for most to swallow," says Steve Duplessie, an analyst at Enterprise Strategy Group. "It was too expensive and did so many things that no one could really use it."
That's no longer the case. According to figures issued by market research company IDC last month, the worldwide storage software market has experienced double-digit growth, hitting $2.1 billion in the second quarter of this year, an 11.8 percent increase over the previous year. SRM sales represented about one-third of that overall figure. Part of the growth is a result of SRM following the path taken earlier by ERP and framework packages: products are being broken down into smaller modules.
"Some 45 percent of larger enterprises have already adopted some SRM somewhere in their world, and 20 percent more will do so this year," Duplessie says. "SRM will take off now that it is cheaper and simpler and geared to where the midmarket can afford and implement it."
Chris Meredith, manager of technical services at $1 billion company Lincare Holdings that provides oxygen and respiratory services for in-home patients across the US, says he, too, found it easy to set up an SRM tool. It took Meredith three to four hours to install the Northern Storage Suite from Stockholm-based Northern Parklife. He uses it to manage 4.5TB of storage at Lincare's headquarters and another 30TB at its primary pharmacy facility.
"With Sarbanes-Oxley [other regulations], future retention of certain types of information became more relevant, so we decided to take a proactive approach," Meredith says. "We wanted to start limiting users on how much storage they could use and start implementing retention policies before it became a problem."
He's currently using the Northern software to limit storage shares, setting a 150MB cap on users' home directories. When users get close to that limit, the SRM system sends them an e-mail telling them where to go to view their files and delete anything they no longer need.
"An added benefit we have seen is that people no longer hoard information in their home directories," Meredith says. "When we started putting in hard caps on their home directories, they started moving that information into a place such as a departmental share where more people can access that information."
Lincare has saved money with SRM by cutting down on the amount of storage capacity needed. Meredith has been able to block employees from saving MP3 data, for example, and as they move files to department shares, there are no longer multiple copies of the same document stashed in different home directories. He has also found it useful for capacity planning to have accurate information on what's being stored. "If I hear another manager saying that he will need additional capacity, I can go into a meeting and say that I ran a report which shows that 60 percent of the data is stale," Meredith says. "Rather than having to buy [storage-area network] space, we can just archive the data."
While Lincare is now using the Northern suite just on users' home directories, Meredith says he plans to use it for Exchange and database files as the company continues to develop its retention policies.
"Overall, the software gives me a better snapshot of how we are using storage from an enterprise level," Meredith says.
Lincare uses its SRM tool to manage a single storage tier. Credit reporting firm Experian Information Solutions, on the other hand, has more than 115TB of three-tier storage at its data warehouse.
Tier 1 is EMC DMX-type disks holding primary databases. Tier 2 consists of EMC Clariion CX700 disks and 146GB drives for Exchange and file servers and other processes that are less I/O-intensive than the databases. Tier 3 is network-attached storage (NAS) disks or slower ATA disks used for flat files or files that are being transferred from mainframes to distributed computing.
"It was tough to manage and keep track of how it was all allocated in order to keep the costs in check," says architecture expert Ernie Demers. "It was a burden on our operations people who had to spend time manually creating reports by going to each server to see which file system it had and what percentage was being used."
To cut the management load, two years ago Demers had EMC install its ControlCenter multivendor SRM suite on a Dell server. And Experian recently upgraded to Version 5.2 on its own. Like Meredith, he says the SRM has more features than his staff can use. But as they learn more, they'll use more. For example, Demers is using the reporting functions to allocate space more efficiently and to put together the business case for buying more storage. But he isn't using it to migrate files from tier to tier.
"It wouldn't be bad to get to that point in the future, where we would have a truly well-oiled machine as regards tiering," Demers says. "But we have a little work to do before we get to that point."
Nevertheless, the software has already greatly cut down on the management workload. Experian used to have three to four people helping out. "Now we have more than 110TB managed by one storage admin," he says. "That is pretty phenomenal considering the type of data we have and the different types of data storage."
Defining the storage path
The main problem with managing storage resources is a lack of smooth interoperability among various hardware and software products. To address this issue, the Storage Networking Industry Association (SNIA) has created the Storage Management Initiative Specification (SMI-S), which defines standards for the storage data path (applications, file systems, volume managers, operating systems, host bus adapters, SAN switches and fabrics, and storage devices) and the storage management path.
"Because of a lack of data-path interoperability, a simple patch upgrade on an OS -- say, for security -- can unravel the stability of the data path," says Wayne N. Adams, chairman of the SNIA board of directors.
"Lack of management path interoperability results in the need to have several SRM tools to manage portions of the storage configuration, which raises the cost of operations for training, tools and tool maintenance and requires a high skill set to oversee the environment," says Adams.
The initial objective for SMI-S was to provide a common interface for storage devices to address management path interoperability. SMI-S Version 1.03 addresses storage resource managers, storage arrays and SAN switches. Vendors have incorporated that version into more than 200 products to date. Version 1.1, released in September, adds support for tape libraries, NAS and iSCSI and add data movement in the form of copy services for snapshots and mirrors.
"I've been able to create a custom report of the ARCserve backup showing detailed information, including the [tape volume serial number] -- the system used to place the data on tape," says Adams."