More data? Get used to it

IDC projects that by 2010, global data creation and storage will reach 988 exabytes

As part of the IT team at the Virginia State Police department in the U.S., Lt. Pete Fagan's job is to ensure that criminal investigators, police officers in the field and other authorities get the most accurate, timely and detailed crime-related information possible. Crime never stops, so the data volumes are huge, dynamic and getting bigger every day.

Criminal records often stretch back decades and include source material from countless systems in myriad formats. Multimedia content is a growing necessity. Metadata, a new necessity, is gobbling up precious disk space. Storage capacity, as you might imagine, is an ongoing problem for the department.

But that's not the half of it.

You see, Fagan's goal is to record every request for information on the police department's storage-area network. Not just a log notation of the request, but the entire transaction as it happened. According to the lieutenant, the system will store everything the requestor sees at the moment the request is fulfilled. If the requestor gets to view dozens of JPEGs, MP3 files, videos and other capacity-hogging content, Fagan wants all that information stored separately from the original data store.

Fagan isn't a data pack rat. He's simply doing his job with the tools at hand. For him, information in the moment, knowing exactly what was known and when, could prove crucial to the criminal justice process. Just having the most recent files won't do, because some criminal cases can hinge on when something was known and by whom. Because those criminal files are always in flux, Fagan needs a storage system that can function as a time machine to retrieve not just a facsimile of the past, but the actual files and data from another time. And he needs to store those exact past experiences for more than 300 million requests per year.

Fagan's time machine is emblematic of an impending storage crisis facing IT. More broadly, will it be a harbinger of loftier roles for CIOs? Or will it -- and projects like it -- ultimately condemn IT executives to being perceived as technologists with tactical, not strategic, value to the business?

Content explosion

The Virginia State Police department is in the midst of updating its 1TB storage infrastructure with new gear from Fujitsu to handle the estimated 11TB of data capacity that will be needed in the near future. While 11TB pales in comparison with the petabyte of data stored on average by Fortune 1,000 companies, the department's growth rate is much faster than larger organizations'. That puts it in the middle of the inexorable march to the global zettabyte storage requirement.

According to an IDC study, 161 exabytes of digital information were created and stored worldwide last year. (An exabyte, you'll recall, is 1 billion gigabytes.) IDC projects that by 2010, global data creation and storage will reach 988 exabytes, a mere 12 billion gigabytes shy of a zettabyte.

"The data explosion means the role of IT managers will expand considerably," said the report. The IDC report happened to be funded by EMC, and when I chatted about it with EMC executive Chuck Hollis, he speculated that a savvy CIO can take this growth as an opportunity to become "less of a technologist and more of an informationist." He argued that IT leaders can emerge as the primary arbiters of information standards and processes for companies -- much like CFOs raised themselves from lowly accountants to powerful executives sitting at the right hand of the CEO.

Wish it were true.

Even if IDC is right about storage demands skyrocketing in the next three years, you won't see CIOs casting a strategic eye on their companies' information policies or dictating data-retention policies to their business units. Instead, they'll be running around with their hair on fire trying to keep up with the ever-increasing amounts of information pouring into their shrinking corporate SANs. Oh sure, the CIO can outlaw certain file types or act as a trusted adviser who shows business leaders how to approach information life-cycle management. But a storage capacity crisis won't enhance IT's reputation. It will undermine it.

The only way CIOs will improve their standing during a global storage shortage is to ensure that their businesses don't suffer from it. I don't care how clever a CIO thinks a new information management scheme might be or how powerful he thinks it will make him. Such an approach will be meaningless to people like Lt. Fagan, who need data to do their jobs. And I wouldn't want to be the IT executive who tries to force a new storage policy on Fagan. After all, the lieutenant does carry a gun.

Join the newsletter!

Error: Please check your email address.
Rocket to Success - Your 10 Tips for Smarter ERP System Selection
Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Mark Hall

Show Comments

Cool Tech

SanDisk MicroSDXC™ for Nintendo® Switch™

Learn more >

Breitling Superocean Heritage Chronographe 44

Learn more >

Toys for Boys

Family Friendly

Panasonic 4K UHD Blu-Ray Player and Full HD Recorder with Netflix - UBT1GL-K

Learn more >

Stocking Stuffer

Razer DeathAdder Expert Ergonomic Gaming Mouse

Learn more >

Christmas Gift Guide

Click for more ›

Most Popular Reviews

Latest Articles


PCW Evaluation Team

Edwina Hargreaves

WD My Cloud Home

I would recommend this device for families and small businesses who want one safe place to store all their important digital content and a way to easily share it with friends, family, business partners, or customers.

Walid Mikhael

Brother QL-820NWB Professional Label Printer

It’s easy to set up, it’s compact and quiet when printing and to top if off, the print quality is excellent. This is hands down the best printer I’ve used for printing labels.

Ben Ramsden

Sharp PN-40TC1 Huddle Board

Brainstorming, innovation, problem solving, and negotiation have all become much more productive and valuable if people can easily collaborate in real time with minimal friction.

Sarah Ieroianni

Brother QL-820NWB Professional Label Printer

The print quality also does not disappoint, it’s clear, bold, doesn’t smudge and the text is perfectly sized.

Ratchada Dunn

Sharp PN-40TC1 Huddle Board

The Huddle Board’s built in program; Sharp Touch Viewing software allows us to easily manipulate and edit our documents (jpegs and PDFs) all at the same time on the dashboard.

George Khoury

Sharp PN-40TC1 Huddle Board

The biggest perks for me would be that it comes with easy to use and comprehensive programs that make the collaboration process a whole lot more intuitive and organic

Featured Content

Product Launch Showcase

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?