Zensors app lets you crowdsource live camera monitoring

Workers in the cloud can train machine learning tools to take over monitoring tasks

Zensors is a mobile app that can use crowdsourcing and machine learning to automatically monitor an area of interest and send alerts to users. The graph at right shows that that the input question at left has been answered affirmatively.

Zensors is a mobile app that can use crowdsourcing and machine learning to automatically monitor an area of interest and send alerts to users. The graph at right shows that that the input question at left has been answered affirmatively.

If you feel like you need eyes in the back of your head, there's a crowdsourcing app for that.

Zensors is a smartphone application that can monitor an area of interest by using a camera, crowdsourced workers and artificial intelligence.

Developed by researchers from Carnegie Mellon University and University of Rochester, the idea behind Zensors is to use any camera in a fixed location to detect changes in what's being monitored -- for instance whether a pet's food bowl is empty -- and automatically notify users.

The developers say it's a cheap, accessible way to add sensors to the environment, part of the move toward building smart homes and smart cities.

The project, presented at the 2015 Computer-Human Interaction Conference (CHI) in Seoul this week, is based on simple user questions written in everyday language about the area being monitored.

For example, a question could be: is there a car in the parking space? The presence of a car would trigger a positive response in the alert to the user, which could be sent via email or text message.

The camera could be the image sensor in any mobile device, provided it has been set up to monitor something, or a webcam, security camera or any other connected camera. It will capture images at an interval set by the user.

Users first select a region of interest in the camera's view by circling it with a finger on a touchscreen -- that's intended to limit the surveillance and protect the privacy of people who might walk into part of the frame.

Next, a question is input in the Zensors app, and the job of monitoring the images is farmed out to the Internet. Redundant images in which nothing has changed are automatically ignored.

The people who do the initial monitoring could be staff at a call center or an outsourcing service such as Amazon's Mechanical Turk, which was used in the CMU study. When the monitors decide that the question has an affirmative answer, a graph in the app soon changes; it could also issue alerts to users.

Zensors gets interesting, however, when the process becomes automatic. After a certain period of human monitoring, machine learning algorithms in the software can learn when a certain condition has been met. For instance, they could learn to recognize that a pet's food bowl is empty.

To ensure the algorithms' accuracy, the system would be periodically checked by workers, which could take a more hands-on role if the area monitored has an unexpected change.

Computer vision tools can also be added to the data processing, allowing the system to perform tasks such as counting cars or people in a certain area.

In a demonstration, a smartphone running Zensors was placed face-up on a table. A question was keyed in: "Is there a hand?" After holding a hand over the phone's camera, the app's graph changed, showing that Mechanical Turk workers had answered from afar. The researchers blamed network latency for the fact that the answer took about 30 seconds.

With better responsiveness, Zensors could be used in a variety of business and home applications. A restaurant manager could use it to learn when customers' glasses need to be refilled, and security companies could use it for automatic monitoring.

"We are the first ones, as far as I know, to fuse the crowd with machine learning training and actually doing it," said Gierad Laput, a PhD student at Carnegie Mellon's Human-Computer Interaction Institute, who also showed off new smartphone interfaces at CHI.

The cost of human monitoring is 2 cents per image, according to the researchers. It costs about US$15 worth of human-vetted data to train the algorithms so they can take over.

By contrast, having a programmer write computer-vision software for a sensor that answers a basic yes or no question could take over a month and cost thousands of dollars.

"Natural-language processing, machine learning and computer vision are three of the hardest problems in computer science," said Chris Harrison, an assistant professor of human-computer interaction at CMU. "The crowd lets us basically bypass a lot of that. But we just let the crowd do the bootstrapping work and we still get the benefits of machine learning."

The researchers plan to keep improving the Zensors app, now in beta, and then release it to the public.

Tim Hornyak covers Japan and emerging technologies for The IDG News Service. Follow Tim on Twitter at @robotopia.

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

Tags smartphonesinternetperipheralsconsumer electronicswebcamsamazon.comCarnegie Mellon UniversityUniversity of RochesterInternet of ThingsCHI

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Tim Hornyak

IDG News Service
Show Comments

Brand Post

Most Popular Reviews

Latest Articles

Resources

PCW Evaluation Team

Luke Hill

MSI GT75 TITAN

I need power and lots of it. As a Front End Web developer anything less just won’t cut it which is why the MSI GT75 is an outstanding laptop for me. It’s a sleek and futuristic looking, high quality, beast that has a touch of sci-fi flare about it.

Emily Tyson

MSI GE63 Raider

If you’re looking to invest in your next work horse laptop for work or home use, you can’t go wrong with the MSI GE63.

Laura Johnston

MSI GS65 Stealth Thin

If you can afford the price tag, it is well worth the money. It out performs any other laptop I have tried for gaming, and the transportable design and incredible display also make it ideal for work.

Andrew Teoh

Brother MFC-L9570CDW Multifunction Printer

Touch screen visibility and operation was great and easy to navigate. Each menu and sub-menu was in an understandable order and category

Louise Coady

Brother MFC-L9570CDW Multifunction Printer

The printer was convenient, produced clear and vibrant images and was very easy to use

Edwina Hargreaves

WD My Cloud Home

I would recommend this device for families and small businesses who want one safe place to store all their important digital content and a way to easily share it with friends, family, business partners, or customers.

Featured Content

Product Launch Showcase

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?