Facebook sees need for Terabit Ethernet

Data center applications already demand the coming 100-Gigabit version

Facebook's data centers already need 100-Gigabit Ethernet and ideally could use 1-Terabit Ethernet, according to a senior network engineer at the company.

The popular social-networking service's growing bandwidth needs reflect the explosion in overall network traffic that enterprises and technology companies are trying to address, according to speakers at the Ethernet Alliance's Technology Exploration Forum, held Tuesday in Santa Clara.

Facebook builds its own data centers out of many identical low-cost servers, linked via standard Ethernet, and pools their processing power to run its core applications. The network fabric that links those systems is a critical piece of the infrastructure, Facebook Engineer Donn Lee told the Ethernet gathering.

"Already, there is a need for 100-Gigabit Ethernet, and where we're going for our upgrades, there is already a need for 1 terabit," Lee said. Facebook has so many servers, and those servers can process data so fast, that they could fill 64 Terabit Ethernet pipes in the backbone of one data center, Lee said.

Ethernet is moving toward 100 Gbps (bits per second), with the option of 40G bps, but those long-awaited specifications aren't expected to be complete until later this year.

Though carrier backbone networks get a lot of attention, the escalating speeds of Ethernet may be needed first in data centers. The servers in Facebook's centers use a huge amount of network capacity as they collectively solve problems such as backing up databases, according to Lee.

"Just one of our tasks could eclipse the bandwidth of all of our users combined on any given day," Lee said.

Facebook is different from many enterprises in that it throws many servers at a single application rather than dividing up each server into multiple virtual machines. That means it faces a special challenge of knitting the many servers together. But its bandwidth challenge is rooted in fundamental advances in technology. All server motherboards come with Gigabit Ethernet built in, and today's multicore processors can easily fill those pipes, Lee said. Meanwhile, 10-Gigabit Ethernet is the fattest standard pipe Facebook can use to tie the whole data center together, and the company can't find switches that pack in enough 10-Gigabit ports to run the network at optimal speed, he said.

Ultimately, network bottlenecks slow innovation at Facebook, Lee said. Faster network connections would mean faster development, as new internal or customer-facing applications were quickly run through their paces and refined. Ideally, each server could send a full 1G bps to any other server in the data center, but that would require the 64 Terabit Ethernet pipes Facebook can't buy today -- or 6,400 of today's 10-Gigabit Ethernet connections, which isn't really feasible, he said. A fabric of that size would require 160 of the largest switches available, Lee said.

"Just imagine trying to manage these," Lee said. "There are a lot of things that are missing from this data center fabric. All of these things are essential to running an Ethernet fabric, yet none of them exist at this scale." Lee said his job is to piece together the elements that are available.

Join the PC World newsletter!

Error: Please check your email address.

Tags Networkingterabit ethernetData CentreFacebook

Our Back to Business guide highlights the best products for you to boost your productivity at home, on the road, at the office, or in the classroom.

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Stephen Lawson

IDG News Service
Show Comments

Essentials

Lexar® JumpDrive® S57 USB 3.0 flash drive

Learn more >

Microsoft L5V-00027 Sculpt Ergonomic Keyboard Desktop

Learn more >

Mobile

Lexar® JumpDrive® S45 USB 3.0 flash drive 

Learn more >

Exec

Lexar® Professional 1800x microSDHC™/microSDXC™ UHS-II cards 

Learn more >

HD Pan/Tilt Wi-Fi Camera with Night Vision NC450

Learn more >

Audio-Technica ATH-ANC70 Noise Cancelling Headphones

Learn more >

Lexar® JumpDrive® C20c USB Type-C flash drive 

Learn more >

Budget

Back To Business Guide

Click for more ›

Most Popular Reviews

Latest News Articles

Resources

PCW Evaluation Team

Azadeh Williams

HP OfficeJet Pro 8730

A smarter way to print for busy small business owners, combining speedy printing with scanning and copying, making it easier to produce high quality documents and images at a touch of a button.

Andrew Grant

HP OfficeJet Pro 8730

I've had a multifunction printer in the office going on 10 years now. It was a neat bit of kit back in the day -- print, copy, scan, fax -- when printing over WiFi felt a bit like magic. It’s seen better days though and an upgrade’s well overdue. This HP OfficeJet Pro 8730 looks like it ticks all the same boxes: print, copy, scan, and fax. (Really? Does anyone fax anything any more? I guess it's good to know the facility’s there, just in case.) Printing over WiFi is more-or- less standard these days.

Ed Dawson

HP OfficeJet Pro 8730

As a freelance writer who is always on the go, I like my technology to be both efficient and effective so I can do my job well. The HP OfficeJet Pro 8730 Inkjet Printer ticks all the boxes in terms of form factor, performance and user interface.

Michael Hargreaves

Windows 10 for Business / Dell XPS 13

I’d happily recommend this touchscreen laptop and Windows 10 as a great way to get serious work done at a desk or on the road.

Aysha Strobbe

Windows 10 / HP Spectre x360

Ultimately, I think the Windows 10 environment is excellent for me as it caters for so many different uses. The inclusion of the Xbox app is also great for when you need some downtime too!

Mark Escubio

Windows 10 / Lenovo Yoga 910

For me, the Xbox Play Anywhere is a great new feature as it allows you to play your current Xbox games with higher resolutions and better graphics without forking out extra cash for another copy. Although available titles are still scarce, but I’m sure it will grow in time.

Featured Content

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?