Microsoft correctly predicts reliable exploits just 27% of the time

Exploitability index ratings useless, experts argue; Microsoft disagrees

Microsoft's monthly predictions about whether hackers will create reliable exploit code for its bugs were right only about a quarter of the time in the first half of 2009, the company acknowledged Monday.

"That's not as good as a coin toss," said Andrew Storms, director of security operations at nCircle Network Security. "So what's the point?"

In October 2008, Microsoft added an "Exploitability Index" to the security bulletins it issues each month. The index rates bugs on a scale from 1 to 3, with 1 indicating that consistently-successful exploit code was likely in the next 30 days, and 3 meaning that working exploit code was unlikely during that same period.

The idea was to give customers more information to decide which vulnerabilities should be patched first. Before the introduction of the index, Microsoft only offered impact ratings -- "critical," "important," "moderate" and "low" -- as an aid for users puzzled by which flaws should be fixed immediately and which could be set aside for the moment.

But in the first half of this year, Microsoft correctly predicted exploits just slightly more than one out of every four times.

"Forty-one vulnerabilities were assigned an Exploitability Index rating of 1, meaning that they were considered the most likely to be exploited within 30 days of the associated security bulletin's release," Microsoft stated in its bi-annual security intelligence report , which it published Monday. "Of these, 11 were, in fact, exploited within 30 days."

That means Microsoft got it right about 27% of the time.

Microsoft also tallied its predictions by security bulletins -- in many cases a single bulletin included patches for multiple vulnerabilities -- to come up with a better batting average. "Sixteen bulletins received a severity rating of Critical," it said in its report. "Of these, 11 were assigned an Exploitability Index rating of 1. Five of these 11 bulletins addressed vulnerabilities that were publicly exploited within 30 days, for an aggregate false positive rate of 55%."

The company defended its poor showing -- even on a bulletin-by-bulletin level it accurately predicted exploitability only 45% of the time -- by saying it was playing it safe. "The higher false positive rate for Critical security bulletins can be attributed to the conservative approach used during the assessment process to ensure the highest degree of customer protection for the most severe class of issues," said Microsoft.

"There's some validity to that," agreed Storms. "They're going to err on the side of caution, if only to prevent people saying 'I told you so' if an exploit appears later."

Join the newsletter!

Error: Please check your email address.
Rocket to Success - Your 10 Tips for Smarter ERP System Selection

Tags securityMicrosoft

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Gregg Keizer

Computerworld (US)
Show Comments

Cool Tech

SanDisk MicroSDXC™ for Nintendo® Switch™

Learn more >

Breitling Superocean Heritage Chronographe 44

Learn more >

Toys for Boys

Family Friendly

Panasonic 4K UHD Blu-Ray Player and Full HD Recorder with Netflix - UBT1GL-K

Learn more >

Stocking Stuffer

Razer DeathAdder Expert Ergonomic Gaming Mouse

Learn more >

Christmas Gift Guide

Click for more ›

Most Popular Reviews

Latest Articles

Resources

PCW Evaluation Team

Edwina Hargreaves

WD My Cloud Home

I would recommend this device for families and small businesses who want one safe place to store all their important digital content and a way to easily share it with friends, family, business partners, or customers.

Walid Mikhael

Brother QL-820NWB Professional Label Printer

It’s easy to set up, it’s compact and quiet when printing and to top if off, the print quality is excellent. This is hands down the best printer I’ve used for printing labels.

Ben Ramsden

Sharp PN-40TC1 Huddle Board

Brainstorming, innovation, problem solving, and negotiation have all become much more productive and valuable if people can easily collaborate in real time with minimal friction.

Sarah Ieroianni

Brother QL-820NWB Professional Label Printer

The print quality also does not disappoint, it’s clear, bold, doesn’t smudge and the text is perfectly sized.

Ratchada Dunn

Sharp PN-40TC1 Huddle Board

The Huddle Board’s built in program; Sharp Touch Viewing software allows us to easily manipulate and edit our documents (jpegs and PDFs) all at the same time on the dashboard.

George Khoury

Sharp PN-40TC1 Huddle Board

The biggest perks for me would be that it comes with easy to use and comprehensive programs that make the collaboration process a whole lot more intuitive and organic

Featured Content

Product Launch Showcase

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?