Facebook engineers identify Graph Search's big data challenges

"There's still a lot of work we have to do," engineers said

Facebooks engineers have many challenges ahead of them as they work to scale up Graph Search, the sites new social search tool. One stumbling block: an over-abundance of data to sift through.

Take the example of searching for Japanese restaurants in New York City liked by people from Japan. A search that would seem to generate hundreds if not thousands of results only spits back two measly businesses.

The search engine, in its current beta form, simply does not have the processing power to sift through the millions of connections among Japanese people on the site to perform the search, Facebook engineers said Thursday during a small media briefing at the companys headquarters in Menlo Park, California.

Theres still a lot of work we have to do, said software engineer Michael Curtiss. A query like this is very difficult computationally, to start with the 100 million in Japan, and then in a fraction of a second to sort through all the pages liked by people in Japan, he said.

This is virtually intractable in the limited amount of time that we have, said the engineer, who helped to design the sites Unicorn search engine that provides Graph Searchs infrastructure. What we end up having to do is cut out possibly good results.

Facebook is taking a variety of approaches to solve this and other big data problems associated with Graph Search.

One strategy involves a concept in computer databases known as query optimization, to improve the speed and efficiency of certain types of searches.

In the case of the Japanese restaurant search, the technique could be applied to start first with the restaurants that are liked instead of starting with Japan, and then filtering down the likes by people, Facebook engineers said.

The company is also addressing the challenges at the hardware level, by adding additional flash memory and other new features to the servers it uses at data centers, to accommodate the increase in search traffic caused by Graph Search.

We need to do extra work in data centers, buying new hardware platforms, [with] new types of servers being put up to support the computational needs of Unicorn, said Soren Lassen, who led the search infrastructure team behind Graph Search.

Facebook began rolling out Graph Search last month to a limited number of users in the U.S. The search tool is designed to let people comb through the social networks 1 trillion connections among users to search for people, places, photos and interests using phrases in plain English.

In principle, nothing can stop users from typing in a query that is unusually long, such as Employers of friends of my friends who live in New York and who like Downton Abbey, engineers said, since Graph Search uses cues such as Likes and check-ins to more easily rank the results.

Eventually Graph Search will incorporate other metrics such as user comments and status updates to compile and rank results, but thats further down the line, the company said.

Zach Miners covers social networking, search and general technology news for IDG News Service. Follow Zach on Twitter at @zachminers. Zach's e-mail address is zach_miners@idg.com

Join the PC World newsletter!

Error: Please check your email address.
Rocket to Success - Your 10 Tips for Smarter ERP System Selection

Tags Internet-based applications and servicessocial networkingsocial mediainternetsearch enginesFacebook

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Zach Miners

IDG News Service
Show Comments

Most Popular Reviews

Latest Articles

Resources

PCW Evaluation Team

Matthew Stivala

HP OfficeJet 250 Mobile Printer

The HP OfficeJet 250 Mobile Printer is a great device that fits perfectly into my fast paced and mobile lifestyle. My first impression of the printer itself was how incredibly compact and sleek the device was.

Armand Abogado

HP OfficeJet 250 Mobile Printer

Wireless printing from my iPhone was also a handy feature, the whole experience was quick and seamless with no setup requirements - accessed through the default iOS printing menu options.

Azadeh Williams

HP OfficeJet Pro 8730

A smarter way to print for busy small business owners, combining speedy printing with scanning and copying, making it easier to produce high quality documents and images at a touch of a button.

Andrew Grant

HP OfficeJet Pro 8730

I've had a multifunction printer in the office going on 10 years now. It was a neat bit of kit back in the day -- print, copy, scan, fax -- when printing over WiFi felt a bit like magic. It’s seen better days though and an upgrade’s well overdue. This HP OfficeJet Pro 8730 looks like it ticks all the same boxes: print, copy, scan, and fax. (Really? Does anyone fax anything any more? I guess it's good to know the facility’s there, just in case.) Printing over WiFi is more-or- less standard these days.

Ed Dawson

HP OfficeJet Pro 8730

As a freelance writer who is always on the go, I like my technology to be both efficient and effective so I can do my job well. The HP OfficeJet Pro 8730 Inkjet Printer ticks all the boxes in terms of form factor, performance and user interface.

Michael Hargreaves

Windows 10 for Business / Dell XPS 13

I’d happily recommend this touchscreen laptop and Windows 10 as a great way to get serious work done at a desk or on the road.

Featured Content

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?