How big data is changing the game for backup and recovery

In today's distributed databases, getting a reliable snapshot of all those petabytes isn't easy

It's a well-known fact in the IT world: Change one part of the software stack, and there's a good chance you'll have to change another. For a shining example, look no further than big data.

First, big data shook up the database arena, ushering in a new class of "scale out" technologies. That's the model exemplified by products like Hadoop, MongoDB, and Cassandra, where data is distributed across multiple commodity servers rather than packed into one massive one. The beauty there, of course, is the flexibility: To accommodate more petabytes, you just add another inexpensive machine or two rather than "scaling up" and paying big bucks for a bigger mammoth.

That's all been great, but now there's a new sticking point: backup and recovery.

"Traditional backup products have challenges with very large amounts of data," said Dave Russell, a vice president with Gartner. "The scale-out nature of the architecture can also be difficult for traditional backup applications to handle."

Today's horizontally scalable databases do include some capabilities for availability and recovery, but typically they're not as robust as those IT users have become accustomed to, Russell added.

It's a problem that can leave large enterprises vulnerable when outages strike. But it's also where a new class of data-protection products is beginning to enter the picture.

Datos IO's RecoverX is one of those.

"If you have a traditional database like Oracle or MySQL, it's scale-up, and there's always the notion of a durable log," said Tarun Thakur, Datos IO's co-founder and CEO.

In such scenarios, a copy of that log is what constitutes a backup when problems arise.

In the world of today's next-generation databases -- where data is distributed across small machines -- it's not quite so simple.

"There is no concept of a durable log because there is no master -- each node is working on its own stuff," Thakur explained. "Different nodes could get different rights, and every node has a different view of an operation."

That's in part because of a trade-off that's been required to accommodate what's commonly referred to as the "three V's" of big data -- volume, velocity, and variety. Specifically, to offer scalability while accommodating the crazy amounts of diverse data flying at us at ever-more-alarming speeds, today's distributed databases have departed from the "ACID" criteria generally promised by traditional relational databases. Instead, they've adopted what are known as "BASE" principles.  

It's a critical distinction. Most pertinent is that where traditional databases promise strong consistency throughout -- that's the "C" in ACID -- distributed ones strive instead for what's called "eventual consistency." Updates will be reflected in all nodes of the database sooner or later, but there's a time lag.

"If you need scalability, you need to give up consistency -- you have to give up one or the other," Thakur said.

That makes it tough to get a reliable snapshot of the big picture for point-in-time recovery. Not only is it more difficult to track which data might have moved where in a distributed database at any given moment, but the resiliency features that often come "baked" into newer distributed databases -- replication, for example -- won't protect you if data gets corrupted, said Simon Robinson, a research vice president with 451 Research.

"You just replicate that corrupted data," he said.

Earlier this month, Datos IO launched RecoverX to address those concerns through features including what it calls scalable versioning and semantic deduplication. The result is cluster-consistent backups that are both space-efficient and available in native formats, the company says.

Souvik Das, who until recently was CTO and managing vice president of engineering with CapitalOne Auto Finance, has felt the backup crunch first-hand.

After years of using traditional databases, CapitalOne underwent a "massive transformation" a few years back that included rolling out new distributed technologies such as Cassandra, said Das, who is now senior vice president of engineering at healthcare-focused startup Grand Rounds.

That meant looking for a new strategy for backup and recovery.

"Most of the backup vendors and software are typically tuned to the type of systems that they're backing up," he explained.

Using an older-style backup product with a newer distributed database could spell trouble, he said.

"Either that software would completely fail because it has no idea how to back up the new data stores, or it would work in a very suboptimal way," Das said. "We knew going in that we would have to have different backup solutions."

CapitalOne has been evaluating Datos IO as well as Talena, another major player in the space, Das said.

Vendors of more traditional backup products are gradually adjusting their own technologies for big data as well.

"It usually takes the incumbent backup vendors some time to support the newer technologies," 451 Research's Robinson said.

"Rewind 10 years and it was very difficult initially to easily do backups for VMware virtual machines," he added. "This opened the door for players like Veeam to enter and steal the VM backup market from under the noses of the incumbents."

Join the PC World newsletter!

Error: Please check your email address.

Our Back to Business guide highlights the best products for you to boost your productivity at home, on the road, at the office, or in the classroom.

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Katherine Noyes

IDG News Service
Show Comments

Father’s Day Gift Guide

Most Popular Reviews

Latest News Articles

Resources

GGG Evaluation Team

Kathy Cassidy

STYLISTIC Q702

First impression on unpacking the Q702 test unit was the solid feel and clean, minimalist styling.

Anthony Grifoni

STYLISTIC Q572

For work use, Microsoft Word and Excel programs pre-installed on the device are adequate for preparing short documents.

Steph Mundell

LIFEBOOK UH574

The Fujitsu LifeBook UH574 allowed for great mobility without being obnoxiously heavy or clunky. Its twelve hours of battery life did not disappoint.

Andrew Mitsi

STYLISTIC Q702

The screen was particularly good. It is bright and visible from most angles, however heat is an issue, particularly around the Windows button on the front, and on the back where the battery housing is located.

Simon Harriott

STYLISTIC Q702

My first impression after unboxing the Q702 is that it is a nice looking unit. Styling is somewhat minimalist but very effective. The tablet part, once detached, has a nice weight, and no buttons or switches are located in awkward or intrusive positions.

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?