Redfin CTO says machine learning needs human help

Workers have valuable insights that should be added to the mix, she says

Machine learning can automate the handling of huge troves of data to help companies make and save money. However, they’re not without pitfalls, as the real estate tech company Redfin learned.

As Redfin began building its own machine-learning capabilities, it ran into a problem: Employees weren't using them. Bridget Frey, the firm’s CTO, said in an interview that there was a key reason for that: At first, Redfin didn't leave room in these systems for the real estate agents who were supposed to use them to make modifications.

For example, a Listings Matchmaker feature generated a list of personalized recommendations for homebuyers, based on their interests. In its initial iteration, agents weren’t able to add recommendations they thought would be useful.

“And the agents were saying ‘I’m an expert in this neighborhood, I know that this house is perfect, why are you not letting me add it in?’ And we just realized that our engineers ... were relying too much on the machine learning,” Frey said. “We needed to take what was special about our agents and use that to enhance the algorithm and enhance the experience for our customer.”

Any company that expects its employees to stand behind the product of an automated system needs to let them manually edit the results, Frey said.

What’s not clear is which set of insights -- the machine's or the agent's -- actually led to better outcomes. In the case of Listings Matchmaker, Frey said users didn’t seem to click on one set of recommendations over another, but she didn’t know if the company had data about which recommendations resulted in a purchase.

The more important difference is that agents used the feature more once they were able to recommend particular homes, she said.

Frey’s comments were echoed by Alphabet Executive Chairman and former Google CEO Eric Schmidt during an on-stage interview at the RSA security conference in San Francisco last week. He said humans must remain in the loop of machine learning systems.

“It’s very important with these systems to understand that they are advisory,” Schmidt said. “They help you understand something, but ultimately you want humans to be in charge of these things.”

Another concern is algorithmic bias, or the tendency of automated systems to return results that are skewed because of factors like the data that goes into them and the way they're designed.

Frey said Redfin has been trying to build a diverse team to work on its technology, in part to identify and correct for those biases.

“It requires constant vigilance to fight against the biases that can creep into software. Ultimately, human beings are the ones who write the algorithms and choose the data sets,” Frey said in an email. “So we’ve decided, for example, not to collect and process certain demographic data that could introduce bias into our algorithms. But if the humans powering these algorithms have one inherent perspective, it is likely the algorithm will adopt a similar disposition.”

Join the newsletter!

Error: Please check your email address.
Rocket to Success - Your 10 Tips for Smarter ERP System Selection
Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Blair Hanley Frank

IDG News Service
Show Comments

Most Popular Reviews

Latest Articles

Resources

PCW Evaluation Team

Matthew Stivala

HP OfficeJet 250 Mobile Printer

The HP OfficeJet 250 Mobile Printer is a great device that fits perfectly into my fast paced and mobile lifestyle. My first impression of the printer itself was how incredibly compact and sleek the device was.

Armand Abogado

HP OfficeJet 250 Mobile Printer

Wireless printing from my iPhone was also a handy feature, the whole experience was quick and seamless with no setup requirements - accessed through the default iOS printing menu options.

Azadeh Williams

HP OfficeJet Pro 8730

A smarter way to print for busy small business owners, combining speedy printing with scanning and copying, making it easier to produce high quality documents and images at a touch of a button.

Andrew Grant

HP OfficeJet Pro 8730

I've had a multifunction printer in the office going on 10 years now. It was a neat bit of kit back in the day -- print, copy, scan, fax -- when printing over WiFi felt a bit like magic. It’s seen better days though and an upgrade’s well overdue. This HP OfficeJet Pro 8730 looks like it ticks all the same boxes: print, copy, scan, and fax. (Really? Does anyone fax anything any more? I guess it's good to know the facility’s there, just in case.) Printing over WiFi is more-or- less standard these days.

Ed Dawson

HP OfficeJet Pro 8730

As a freelance writer who is always on the go, I like my technology to be both efficient and effective so I can do my job well. The HP OfficeJet Pro 8730 Inkjet Printer ticks all the boxes in terms of form factor, performance and user interface.

Michael Hargreaves

Windows 10 for Business / Dell XPS 13

I’d happily recommend this touchscreen laptop and Windows 10 as a great way to get serious work done at a desk or on the road.

Featured Content

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?