Laptop battery life benchmarks are out of juice

Vendors AMD and Lenovo say laptop buyers are being misled

In an industry that gets off on throwing obscure benchmarks at buyers ( "pixel fill rate," anyone?), laptop battery life is one of the easiest to understand.

It's also long been one of the least useful, critics charge, due to the industry's deceptive use of the dominant standard, the MobileMark benchmark created by the Business Applications Performance Corp. (BAPCo), an industry consortium whose members include Intel Corp., Dell Inc., Hewlett-Packard Co., Advanced Micro Devices Inc. and others.

"Everyone in the industry knows this benchmark is wildly optimistic and that the actual battery life you'll get is often less than half what MobileMark suggests," wrote analyst Rob Enderle last month. "This is because MobileMark measures battery life much like you might measure gas mileage if you started the car, put it in neutral, and coasted down a long hill."

The latest MobileMark 2007 report measures laptop battery life under three scenarios: reading a document, watching a DVD movie, and doing a "representative" mix of productivity tasks, such as reading and composing documents, editing photos and encoding Flash videos (see page 13 of white paper).

But rather than using an average time based on all three measures, BAPCo designates its third scenario as the way most people use their laptop.

There are several problems with this, according to critics. First, doubters such as AMD say that the productivity test assumes that the notebook is idle 90% or more of the time.

Second, MobileMark's "productivity" scenario assumes that users, when active, are using only software such as Adobe Photoshop or Microsoft Office. They don't test usage of music or video applications such as iTunes or Windows Media Player, games or Web browsing. The test also assumes that Wi-Fi is turned off.

That seems particularly unjustified today, since netbooks are touted as on-the-go, Web-centric devices, or high-definition video-capable machines.

Finally, MobileMark 2007 allows PC vendors to set their laptop screen brightness at the lowest possible setting, provided it is no lower than 60 nits (a nit is a measure of brightness. The problem, again pointed out by AMD, is that 60 nits is quite dim, being only about one-fifth of most notebook PCs' maximum screen brightness.

According to an informal reader poll at Neowin.net, a Windows community site, fewer than 15% of respondents run their notebooks that dim.

BAPCo defended its MobileMark benchmarks. "The content of BAPCo benchmarks are vigorously debated and cooperatively developed by BAPCo members according to a long and rigorous process," the company said in an e-mailed statement. "As is the case with all BAPCo benchmarks, MobileMark 2007 was approved by BAPCo according to a democratic voting process similar to ones used by most industry work groups."

Despite the criticism, many vendors are willing to tout the battery life from the Productivity test as their overall MobileMark score. See these offers from Hewlett-Packard Co., Lenovo Group Ltd., and Dell Inc.

Only Acer Inc. (download PDF here) identified its MobileMark time as a productivity score. Asus Inc., Apple Inc. and Toshiba Corp. didn't mention MobileMark on their Web sites.

In late June, a class-action lawsuit was filed in U.S. District Court in San Jose, targeting Intel Corp. for, according to the San Jose Mercury News, "essentially rigging those tests to inflate the battery life of laptops powered by its chips."

Intel denied the claims, and noted that the same law firm, Girard Gibbs of San Francisco, unsuccessfully filed a separate class-action lawsuit against Intel several years earlier, according to the Mercury News.

Intel also disputes the claim that the public is being misled. "Anyone who criticizes consumers' intelligence when shopping for laptops is underestimating the consumers," an Intel spokesman told the Mercury News.

Carol Hess-Nickels, director of marketing for business notebooks at HP, took the same line. "I'd say we are pretty pleased with the benchmarks used today," she said in an interview last month several days before the lawsuit was filed. "I've not personally gotten complaints."

HP claims business netbooks such as the Mini 2140 and the Mini 5101 can run up to eight hours, or a full business day, on an optional, extended battery.

Lenovo, which has claimed as long as 7.5 hours of battery life for its laptops on extended batteries, acknowledges there is a problem, however.

"We don't really like the fact that something is supposed to get four hours and users routinely say, 'We divide that number by two and that's what we get,'" said Lenovo segment marketing manager David Critchley in an interview, also several days before the lawsuit's filing.

Dell appears to agree with Lenovo. "Customers expect the advertised battery life to reflect the way they really use the product," Ketan Pandya, head of AMD-based products at Dell, told Newsweek last month.

As a counterbalance, some magazine reviewers go overboard to turn off all of a laptop's power-saving features, Critchley said, which is equally inaccurate. "We put a lot of time and effort into our power manager," he said. "You'll see some significant gains from the way we handle sub-components."

AMD, which complained that MobileMark essentially discriminates against its chipsets because they are more graphically powerful than Intel's, is all for reforming MobileMark.

In a blog entitled "There has to be a better way," AMD suggests turning the widely used performance benchmark, 3DMark06, into "an active battery life test" that it argues would be a more accurate measure of average battery life.

Lenovo, although game, is more cautious. "If Dell, HP and Apple all said, 'Yea, verily,' Lenovo would be right there with them," Critchley said.

Join the newsletter!

Error: Please check your email address.
Rocket to Success - Your 10 Tips for Smarter ERP System Selection

Tags AMDLenovolaptop batteries

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Eric Lai

Computerworld (US)
Show Comments

Cool Tech

Breitling Superocean Heritage Chronographe 44

Learn more >

SanDisk MicroSDXC™ for Nintendo® Switch™

Learn more >

Toys for Boys

Family Friendly

Panasonic 4K UHD Blu-Ray Player and Full HD Recorder with Netflix - UBT1GL-K

Learn more >

Stocking Stuffer

Razer DeathAdder Expert Ergonomic Gaming Mouse

Learn more >

Christmas Gift Guide

Click for more ›

Most Popular Reviews

Latest Articles

Resources

PCW Evaluation Team

Edwina Hargreaves

WD My Cloud Home

I would recommend this device for families and small businesses who want one safe place to store all their important digital content and a way to easily share it with friends, family, business partners, or customers.

Walid Mikhael

Brother QL-820NWB Professional Label Printer

It’s easy to set up, it’s compact and quiet when printing and to top if off, the print quality is excellent. This is hands down the best printer I’ve used for printing labels.

Ben Ramsden

Sharp PN-40TC1 Huddle Board

Brainstorming, innovation, problem solving, and negotiation have all become much more productive and valuable if people can easily collaborate in real time with minimal friction.

Sarah Ieroianni

Brother QL-820NWB Professional Label Printer

The print quality also does not disappoint, it’s clear, bold, doesn’t smudge and the text is perfectly sized.

Ratchada Dunn

Sharp PN-40TC1 Huddle Board

The Huddle Board’s built in program; Sharp Touch Viewing software allows us to easily manipulate and edit our documents (jpegs and PDFs) all at the same time on the dashboard.

George Khoury

Sharp PN-40TC1 Huddle Board

The biggest perks for me would be that it comes with easy to use and comprehensive programs that make the collaboration process a whole lot more intuitive and organic

Featured Content

Product Launch Showcase

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?