Microsoft doesn't report all security vulnerabilities that it fixes in its software. Bug comparisons between vendors therefore paint an incorrect picture.
"We don't document every issue found," Mike Reavey, director of the Microsoft Security Response Center (MSRC), said at a meeting with reporters at the company's corporate headquarters in Redmond, Washington.
Microsoft will issue a Common Vulnerabilities and Exposures (CVE) number to a vulnerability for flaws that share the same severity, have an attack vector and a workaround. If several flaws share all the same properties, they will not be reported separately, Reavey said.
The nondisclosure of fixes was brought to light early this month by a company called Core Security Technologies. After studying the Microsoft patches MS10-024 and MS10-028, it noticed three silent fixes. Security bulletin MS10-028 addressed a flaw that would expose a user of Microsoft Visio to a buffer overflow attack, which would allow an attacker to take over control of the system.
Microsoft didn't report the additional flaws that it patched in the Visio case because: "The attack vector was exactly the same, the severity was exactly the same. From a customer's perspective, the same workaround -- not opening Visio documents from untrusted sources -- applied," Reavey told Webwereld, an IDG affiliate, in an interview after his presentation.
Adobe too is keeping quiet about internal vulnerability fixes. During a presentation at the Microsoft event, Adobe's director of product security and privacy, Brad Arkin, admitted that it won't assign CVE numbers to bugs that the firm found itself. Adobe considers these updates "code improvements," Arkin said. CVE numbers are used only for bugs that are actively exploited or that were reported by external researchers.
Keeping quiet on security updates isn't without consequence. Both vendors and security researchers have used CVE counts to assess the security of different operating systems and applications. At the press event, Microsoft showed a slide comparing the number of security updates that issued for UbuntuLTS, Red Hat Enterprise Linux 4, OS X 10.4 as well as Windows Vista and Windows XP. In a different slide the company compared security updates for SQL Server 2000, SQL Server 2005 and an unnamed competing database.
Reavey admits that these vulnerability count comparisons are flawed, but argues that they still make sense as a basic comparison tool. "There is a lot of different ways that you can measure security. Vulnerability counting is one way, and it is not perfect." Comparing the number of vulnerabilities per line of software code provides another metric. Reavey argues that vulnerability counts are in fact a useful way to compare between products from one vendor.
The Microsoft director downplayed the vulnerability count discussion as "accounting" that mostly distracts researchers from fixing flaws. When one flaw is reported, the firm would prefer to optimize a research tool that surfaces 200 related bugs through fuzzing. Technically, the code may contain 200 vulnerabilities, but "you change one line of code and it blocks all 200 potential fuzzing issues. Is that one vulnerability? Is that 200 vulnerabilities? I don't really know."
"If we spend time to get the accounting correct, that is time that we take away from getting a solution out to protect customers," he said.