Microsoft Research aims to curb Web spam

Microsoft has released a new report and tool to prevent the exploitation of search engines to drive traffic to spam sites.

Researchers at Microsoft have released a new report and tool aimed at preventing Web spammers from exploiting Internet search engines to drive traffic to spam URLs.

The tool, called the Strider Search Defender, identifies spam URLs (uniform resource locators) that are being distributed through social networking, forum and blog-hosting Web sites, and can prevent those URLs from being indexed by search engines, said Yi-Min Wang, group manager of the Cybersecurity and Systems Management Research Group in Microsoft Research.

Instead of commenting on user pages of popular forums and blog sites -- such as Google BlogSpot or MySpace -- spammers will send URLs that link to spam Web sites to as many Internet forum pages as they can, he said. Since these URLs appear so frequently on valid Web sites, search engines such as Google, Yahoo and Microsoft's own MSN will index them and they will begin appearing in search results, Wang said.

"They create a URL they want people to click and they put that into every possible open forum and guest book they can," he said. "Some search engines will see that this URL is everywhere on the Web so [they think] it should be popular. But it doesn't have the kind of relevance to be in the top search-engine results."

The tool uses elements of technology previously developed in Microsoft Research in projects called Strider, HoneyMonkey and Typo Patrol to search forums that have been spammed and to identify spam URLs in the hope of removing them before they are indexed by search engines. It also has an element that can distinguish between legitimate URLs on Web forums and spam URLs, Wang said.

In the cases when a spammer uses what is called a "doorway domain" to set up a spam site, the tool can identify the domain that is being exploited and notify its administrators, he said. A doorway domain is a legitimate URL, such as, that spammers use to set up a spam site so it looks like a valid Web site, and thus will fool users and search engines.

"If they put [what looks like a] blog URL into your forum and everyone else's, they will fool the search engine," Wang said.

In addition to specifications for the tool, Microsoft Research also published information in its report to encourage owners of free Web-hosting sites, search engines and publicly accessible Web forums to do what they can to prevent Web spammers from exploiting search engines.

Wang said free Web-hosting sites such as MySpace and Google BlogSpot can use Microsoft's methodology to identify spammers that might be using their sites as doorway domains. He said he hopes that search-engine companies will use the specifications for the tool described in the report to optimize their search engines to ferret out spam URLs.

Additionally, users who have blogs or forums on Web-hosting sites can help alleviate the problem of Web spamming by shutting down sites that are still active online but that they no longer visit or use, Wang said.

The Microsoft Research report on Strider Search Defender can be found here:

Join the newsletter!

Error: Please check your email address.
Rocket to Success - Your 10 Tips for Smarter ERP System Selection
Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Elizabeth Montalbano

IDG News Service
Show Comments



Sansai 6-Outlet Power Board + 4-Port USB Charging Station

Learn more >



Back To Business Guide

Click for more ›

Brand Post

Most Popular Reviews

Latest Articles


PCW Evaluation Team

Louise Coady

Brother MFC-L9570CDW Multifunction Printer

The printer was convenient, produced clear and vibrant images and was very easy to use

Edwina Hargreaves

WD My Cloud Home

I would recommend this device for families and small businesses who want one safe place to store all their important digital content and a way to easily share it with friends, family, business partners, or customers.

Walid Mikhael

Brother QL-820NWB Professional Label Printer

It’s easy to set up, it’s compact and quiet when printing and to top if off, the print quality is excellent. This is hands down the best printer I’ve used for printing labels.

Ben Ramsden

Sharp PN-40TC1 Huddle Board

Brainstorming, innovation, problem solving, and negotiation have all become much more productive and valuable if people can easily collaborate in real time with minimal friction.

Sarah Ieroianni

Brother QL-820NWB Professional Label Printer

The print quality also does not disappoint, it’s clear, bold, doesn’t smudge and the text is perfectly sized.

Ratchada Dunn

Sharp PN-40TC1 Huddle Board

The Huddle Board’s built in program; Sharp Touch Viewing software allows us to easily manipulate and edit our documents (jpegs and PDFs) all at the same time on the dashboard.

Featured Content

Product Launch Showcase

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?