Microsoft Research aims to curb Web spam

Microsoft has released a new report and tool to prevent the exploitation of search engines to drive traffic to spam sites.

Researchers at Microsoft have released a new report and tool aimed at preventing Web spammers from exploiting Internet search engines to drive traffic to spam URLs.

The tool, called the Strider Search Defender, identifies spam URLs (uniform resource locators) that are being distributed through social networking, forum and blog-hosting Web sites, and can prevent those URLs from being indexed by search engines, said Yi-Min Wang, group manager of the Cybersecurity and Systems Management Research Group in Microsoft Research.

Instead of commenting on user pages of popular forums and blog sites -- such as Google BlogSpot or MySpace -- spammers will send URLs that link to spam Web sites to as many Internet forum pages as they can, he said. Since these URLs appear so frequently on valid Web sites, search engines such as Google, Yahoo and Microsoft's own MSN will index them and they will begin appearing in search results, Wang said.

"They create a URL they want people to click and they put that into every possible open forum and guest book they can," he said. "Some search engines will see that this URL is everywhere on the Web so [they think] it should be popular. But it doesn't have the kind of relevance to be in the top search-engine results."

The tool uses elements of technology previously developed in Microsoft Research in projects called Strider, HoneyMonkey and Typo Patrol to search forums that have been spammed and to identify spam URLs in the hope of removing them before they are indexed by search engines. It also has an element that can distinguish between legitimate URLs on Web forums and spam URLs, Wang said.

In the cases when a spammer uses what is called a "doorway domain" to set up a spam site, the tool can identify the domain that is being exploited and notify its administrators, he said. A doorway domain is a legitimate URL, such as, that spammers use to set up a spam site so it looks like a valid Web site, and thus will fool users and search engines.

"If they put [what looks like a] blog URL into your forum and everyone else's, they will fool the search engine," Wang said.

In addition to specifications for the tool, Microsoft Research also published information in its report to encourage owners of free Web-hosting sites, search engines and publicly accessible Web forums to do what they can to prevent Web spammers from exploiting search engines.

Wang said free Web-hosting sites such as MySpace and Google BlogSpot can use Microsoft's methodology to identify spammers that might be using their sites as doorway domains. He said he hopes that search-engine companies will use the specifications for the tool described in the report to optimize their search engines to ferret out spam URLs.

Additionally, users who have blogs or forums on Web-hosting sites can help alleviate the problem of Web spamming by shutting down sites that are still active online but that they no longer visit or use, Wang said.

The Microsoft Research report on Strider Search Defender can be found here:

Join the newsletter!


Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.
Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Elizabeth Montalbano

IDG News Service
Show Comments

Brand Post

Most Popular Reviews

Latest Articles


PCW Evaluation Team

Luke Hill


I need power and lots of it. As a Front End Web developer anything less just won’t cut it which is why the MSI GT75 is an outstanding laptop for me. It’s a sleek and futuristic looking, high quality, beast that has a touch of sci-fi flare about it.

Emily Tyson

MSI GE63 Raider

If you’re looking to invest in your next work horse laptop for work or home use, you can’t go wrong with the MSI GE63.

Laura Johnston

MSI GS65 Stealth Thin

If you can afford the price tag, it is well worth the money. It out performs any other laptop I have tried for gaming, and the transportable design and incredible display also make it ideal for work.

Andrew Teoh

Brother MFC-L9570CDW Multifunction Printer

Touch screen visibility and operation was great and easy to navigate. Each menu and sub-menu was in an understandable order and category

Louise Coady

Brother MFC-L9570CDW Multifunction Printer

The printer was convenient, produced clear and vibrant images and was very easy to use

Edwina Hargreaves

WD My Cloud Home

I would recommend this device for families and small businesses who want one safe place to store all their important digital content and a way to easily share it with friends, family, business partners, or customers.

Featured Content

Product Launch Showcase

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?