Laptop battery life benchmarks are out of juice

Vendors AMD and Lenovo say laptop buyers are being misled

In an industry that gets off on throwing obscure benchmarks at buyers ( "pixel fill rate," anyone?), laptop battery life is one of the easiest to understand.

It's also long been one of the least useful, critics charge, due to the industry's deceptive use of the dominant standard, the MobileMark benchmark created by the Business Applications Performance Corp. (BAPCo), an industry consortium whose members include Intel Corp., Dell Inc., Hewlett-Packard Co., Advanced Micro Devices Inc. and others.

"Everyone in the industry knows this benchmark is wildly optimistic and that the actual battery life you'll get is often less than half what MobileMark suggests," wrote analyst Rob Enderle last month. "This is because MobileMark measures battery life much like you might measure gas mileage if you started the car, put it in neutral, and coasted down a long hill."

The latest MobileMark 2007 report measures laptop battery life under three scenarios: reading a document, watching a DVD movie, and doing a "representative" mix of productivity tasks, such as reading and composing documents, editing photos and encoding Flash videos (see page 13 of white paper).

But rather than using an average time based on all three measures, BAPCo designates its third scenario as the way most people use their laptop.

There are several problems with this, according to critics. First, doubters such as AMD say that the productivity test assumes that the notebook is idle 90% or more of the time.

Second, MobileMark's "productivity" scenario assumes that users, when active, are using only software such as Adobe Photoshop or Microsoft Office. They don't test usage of music or video applications such as iTunes or Windows Media Player, games or Web browsing. The test also assumes that Wi-Fi is turned off.

That seems particularly unjustified today, since netbooks are touted as on-the-go, Web-centric devices, or high-definition video-capable machines.

Finally, MobileMark 2007 allows PC vendors to set their laptop screen brightness at the lowest possible setting, provided it is no lower than 60 nits (a nit is a measure of brightness. The problem, again pointed out by AMD, is that 60 nits is quite dim, being only about one-fifth of most notebook PCs' maximum screen brightness.

According to an informal reader poll at Neowin.net, a Windows community site, fewer than 15% of respondents run their notebooks that dim.

BAPCo defended its MobileMark benchmarks. "The content of BAPCo benchmarks are vigorously debated and cooperatively developed by BAPCo members according to a long and rigorous process," the company said in an e-mailed statement. "As is the case with all BAPCo benchmarks, MobileMark 2007 was approved by BAPCo according to a democratic voting process similar to ones used by most industry work groups."

Despite the criticism, many vendors are willing to tout the battery life from the Productivity test as their overall MobileMark score. See these offers from Hewlett-Packard Co., Lenovo Group Ltd., and Dell Inc.

Only Acer Inc. (download PDF here) identified its MobileMark time as a productivity score. Asus Inc., Apple Inc. and Toshiba Corp. didn't mention MobileMark on their Web sites.

In late June, a class-action lawsuit was filed in U.S. District Court in San Jose, targeting Intel Corp. for, according to the San Jose Mercury News, "essentially rigging those tests to inflate the battery life of laptops powered by its chips."

Intel denied the claims, and noted that the same law firm, Girard Gibbs of San Francisco, unsuccessfully filed a separate class-action lawsuit against Intel several years earlier, according to the Mercury News.

Intel also disputes the claim that the public is being misled. "Anyone who criticizes consumers' intelligence when shopping for laptops is underestimating the consumers," an Intel spokesman told the Mercury News.

Carol Hess-Nickels, director of marketing for business notebooks at HP, took the same line. "I'd say we are pretty pleased with the benchmarks used today," she said in an interview last month several days before the lawsuit was filed. "I've not personally gotten complaints."

HP claims business netbooks such as the Mini 2140 and the Mini 5101 can run up to eight hours, or a full business day, on an optional, extended battery.

Lenovo, which has claimed as long as 7.5 hours of battery life for its laptops on extended batteries, acknowledges there is a problem, however.

"We don't really like the fact that something is supposed to get four hours and users routinely say, 'We divide that number by two and that's what we get,'" said Lenovo segment marketing manager David Critchley in an interview, also several days before the lawsuit's filing.

Dell appears to agree with Lenovo. "Customers expect the advertised battery life to reflect the way they really use the product," Ketan Pandya, head of AMD-based products at Dell, told Newsweek last month.

As a counterbalance, some magazine reviewers go overboard to turn off all of a laptop's power-saving features, Critchley said, which is equally inaccurate. "We put a lot of time and effort into our power manager," he said. "You'll see some significant gains from the way we handle sub-components."

AMD, which complained that MobileMark essentially discriminates against its chipsets because they are more graphically powerful than Intel's, is all for reforming MobileMark.

In a blog entitled "There has to be a better way," AMD suggests turning the widely used performance benchmark, 3DMark06, into "an active battery life test" that it argues would be a more accurate measure of average battery life.

Lenovo, although game, is more cautious. "If Dell, HP and Apple all said, 'Yea, verily,' Lenovo would be right there with them," Critchley said.

Join the PC World newsletter!

Error: Please check your email address.

Tags AMDLenovolaptop batteries

Struggling for Christmas presents this year? Check out our Christmas Gift Guide for some top tech suggestions and more.

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Eric Lai

Computerworld (US)

Most Popular Reviews

Follow Us

Best Deals on GoodGearGuide

Shopping.com

Latest News Articles

Resources

GGG Evaluation Team

Kathy Cassidy

STYLISTIC Q702

First impression on unpacking the Q702 test unit was the solid feel and clean, minimalist styling.

Anthony Grifoni

STYLISTIC Q572

For work use, Microsoft Word and Excel programs pre-installed on the device are adequate for preparing short documents.

Steph Mundell

LIFEBOOK UH574

The Fujitsu LifeBook UH574 allowed for great mobility without being obnoxiously heavy or clunky. Its twelve hours of battery life did not disappoint.

Andrew Mitsi

STYLISTIC Q702

The screen was particularly good. It is bright and visible from most angles, however heat is an issue, particularly around the Windows button on the front, and on the back where the battery housing is located.

Simon Harriott

STYLISTIC Q702

My first impression after unboxing the Q702 is that it is a nice looking unit. Styling is somewhat minimalist but very effective. The tablet part, once detached, has a nice weight, and no buttons or switches are located in awkward or intrusive positions.

Latest Jobs

Shopping.com

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?