How big data is changing the game for backup and recovery

In today's distributed databases, getting a reliable snapshot of all those petabytes isn't easy

It's a well-known fact in the IT world: Change one part of the software stack, and there's a good chance you'll have to change another. For a shining example, look no further than big data.

First, big data shook up the database arena, ushering in a new class of "scale out" technologies. That's the model exemplified by products like Hadoop, MongoDB, and Cassandra, where data is distributed across multiple commodity servers rather than packed into one massive one. The beauty there, of course, is the flexibility: To accommodate more petabytes, you just add another inexpensive machine or two rather than "scaling up" and paying big bucks for a bigger mammoth.

That's all been great, but now there's a new sticking point: backup and recovery.

"Traditional backup products have challenges with very large amounts of data," said Dave Russell, a vice president with Gartner. "The scale-out nature of the architecture can also be difficult for traditional backup applications to handle."

Today's horizontally scalable databases do include some capabilities for availability and recovery, but typically they're not as robust as those IT users have become accustomed to, Russell added.

It's a problem that can leave large enterprises vulnerable when outages strike. But it's also where a new class of data-protection products is beginning to enter the picture.

Datos IO's RecoverX is one of those.

"If you have a traditional database like Oracle or MySQL, it's scale-up, and there's always the notion of a durable log," said Tarun Thakur, Datos IO's co-founder and CEO.

In such scenarios, a copy of that log is what constitutes a backup when problems arise.

In the world of today's next-generation databases -- where data is distributed across small machines -- it's not quite so simple.

"There is no concept of a durable log because there is no master -- each node is working on its own stuff," Thakur explained. "Different nodes could get different rights, and every node has a different view of an operation."

That's in part because of a trade-off that's been required to accommodate what's commonly referred to as the "three V's" of big data -- volume, velocity, and variety. Specifically, to offer scalability while accommodating the crazy amounts of diverse data flying at us at ever-more-alarming speeds, today's distributed databases have departed from the "ACID" criteria generally promised by traditional relational databases. Instead, they've adopted what are known as "BASE" principles.  

It's a critical distinction. Most pertinent is that where traditional databases promise strong consistency throughout -- that's the "C" in ACID -- distributed ones strive instead for what's called "eventual consistency." Updates will be reflected in all nodes of the database sooner or later, but there's a time lag.

"If you need scalability, you need to give up consistency -- you have to give up one or the other," Thakur said.

That makes it tough to get a reliable snapshot of the big picture for point-in-time recovery. Not only is it more difficult to track which data might have moved where in a distributed database at any given moment, but the resiliency features that often come "baked" into newer distributed databases -- replication, for example -- won't protect you if data gets corrupted, said Simon Robinson, a research vice president with 451 Research.

"You just replicate that corrupted data," he said.

Earlier this month, Datos IO launched RecoverX to address those concerns through features including what it calls scalable versioning and semantic deduplication. The result is cluster-consistent backups that are both space-efficient and available in native formats, the company says.

Souvik Das, who until recently was CTO and managing vice president of engineering with CapitalOne Auto Finance, has felt the backup crunch first-hand.

After years of using traditional databases, CapitalOne underwent a "massive transformation" a few years back that included rolling out new distributed technologies such as Cassandra, said Das, who is now senior vice president of engineering at healthcare-focused startup Grand Rounds.

That meant looking for a new strategy for backup and recovery.

"Most of the backup vendors and software are typically tuned to the type of systems that they're backing up," he explained.

Using an older-style backup product with a newer distributed database could spell trouble, he said.

"Either that software would completely fail because it has no idea how to back up the new data stores, or it would work in a very suboptimal way," Das said. "We knew going in that we would have to have different backup solutions."

CapitalOne has been evaluating Datos IO as well as Talena, another major player in the space, Das said.

Vendors of more traditional backup products are gradually adjusting their own technologies for big data as well.

"It usually takes the incumbent backup vendors some time to support the newer technologies," 451 Research's Robinson said.

"Rewind 10 years and it was very difficult initially to easily do backups for VMware virtual machines," he added. "This opened the door for players like Veeam to enter and steal the VM backup market from under the noses of the incumbents."

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.
Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Katherine Noyes

IDG News Service
Show Comments

Cool Tech

Toys for Boys

Family Friendly

Stocking Stuffer

SmartLens - Clip on Phone Camera Lens Set of 3

Learn more >

Christmas Gift Guide

Click for more ›

Brand Post

Most Popular Reviews

Latest Articles

Resources

PCW Evaluation Team

Aysha Strobbe

Microsoft Office 365/HP Spectre x360

Microsoft Office continues to make a student’s life that little bit easier by offering reliable, easy to use, time-saving functionality, while continuing to develop new features that further enhance what is already a formidable collection of applications

Michael Hargreaves

Microsoft Office 365/Dell XPS 15 2-in-1

I’d recommend a Dell XPS 15 2-in-1 and the new Windows 10 to anyone who needs to get serious work done (before you kick back on your couch with your favourite Netflix show.)

Maryellen Rose George

Brother PT-P750W

It’s useful for office tasks as well as pragmatic labelling of equipment and storage – just don’t get too excited and label everything in sight!

Cathy Giles

Brother MFC-L8900CDW

The Brother MFC-L8900CDW is an absolute stand out. I struggle to fault it.

Luke Hill

MSI GT75 TITAN

I need power and lots of it. As a Front End Web developer anything less just won’t cut it which is why the MSI GT75 is an outstanding laptop for me. It’s a sleek and futuristic looking, high quality, beast that has a touch of sci-fi flare about it.

Emily Tyson

MSI GE63 Raider

If you’re looking to invest in your next work horse laptop for work or home use, you can’t go wrong with the MSI GE63.

Featured Content

Product Launch Showcase

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?