Microsoft last week confirmed that it inadvertently released a pre-release version of an Exchange Server 2007 update that could push servers into an endless series of crashes.
The Update Rollup 4 for Exchange Server 2007 released to users via Microsoft Update and Windows Server Update Services (WSUS), Microsoft's two most popular update mechanisms, was a preliminary version, the company acknowledged.
"For a brief period of time on 9/9, a pre-release version of Update Rollup 4 for Exchange Server 2007 Service Pack 1 was inadvertently made available to Microsoft Update, the Microsoft Update Catalog and WSUS servers for download," an unidentified Microsoft employee said in a post to the official Exchange blog.
Once Microsoft discovered its error, it pulled Update Rollup 4 -- a collection of previously-disclosed bug fixes -- from the update services, but warned those who had already installed the rollup that it could cause problems. "An issue exists with this pre-release version of the Rollup 4 with regard to the Exchange Web Service (EWS) that creates the potential for a continuous crashing cycle," the blog post continued.
Some users reported they were unable to backup their Exchange server after installing the rollup during its window of availability, while others had more dire stories to tell. "It bricked one of my 'just about to go live' servers (services wouldn't start, can't uninstall). Spent today building another one," said Alex Britton in a message posted to the Exchange support forum. "Serves me right for not disabling the recommended auto-update auto install."
Microsoft recommended that users who had installed Update Rollup 4 uninstall it, then install the previous incarnation, Update Rollup 3.
That wasn't always easy, however, as a Microsoft Exchange engineer admitted in a comment added to the blog post. "I want to point out that there is an uninstall case that we just uncovered that people may hit," said Scott Roberts of the Exchange team. After uninstalling Update Rollup 4, Roberts said, EWS is unable to read a configuration file; administrators must open the configuration file and manually edit it to replace instances of an incorrect path name.
Later in the comment thread, however, Roberts told a user to simply install Update Rollup 3 atop the faulty version 4.
Microsoft did not provide a timetable for issuing a working edition of Update Rollup 4, although Roberts said that from this point forward, such updates would be released on the second and fourth Tuesdays of the month. Next Tuesday, September 23, is the fourth Tuesday of this month.
The whole episode left a bad taste in some users' mouths. "I am sorry but this is absolutely unacceptable," said a user identified as "Andy" in the first comment added to the Exchange blog post. "If a pre-release patch can get into Microsoft Update, you leave me no choice but to disable Automatic Updates."
This isn't the first snafu in Microsoft's update services. In June and July, the company had to fix two bugs in other patching mechanisms, including WSUS and the higher-end System Center Configuration Manager 2007, that had kept administrators from pushing patches to end users' PCs.
"We apologize for any inconvenience and are working to make sure this does not happen again," Microsoft said in the Exchange blog.