Project shows big differences in mobile 'Net performance

New mobile Web measures show wide differences in Web performance for handheld users

A new mobile market research project now measures and ranks mobile Web sites, trying to identify how well they perform for handheld device users.

Intended to be an ongoing weekly measure of mobile Web performance, the mobile Web performance index, from Keynote Competitive Research, uses a trio of metrics to evaluate 10 popular online destinations. The index doesn't compare the wireless network metrics to wired broadband metrics, but it does show wide variance in how well the mobile sites perform.

The three metrics are:

  • Response time in seconds -- the average time it took for the mobile Web site to download to the smartphone.
  • Download speed in byte/second -- the average speed in byte/sec for the site to download.
  • Success rate -- the percentage of time the mobile site successfully downloaded the complete content to the mobile phone.

The nine sites surveyed were a mix of types and content, likely to be popular with mobile users: CNN, ESPN, Facebook, Fandango, Google, MapQuest, MSN, Schwab (online brokerage firm), The Weather Channel and Yahoo.

The initial data covers the period from November 19-26.

Response times varied from just less than 7 seconds, for Google and Facebook, to just more than 15 seconds for ESPN. To get an idea how long that is, count "one thousand one, one thousand two" and so on till you reach 15. You probably won't bother. Part of the variances was related to the size of the main page download: Google downloaded just 6,684 bytes, but ESPN downloaded 26,635.

But the byte/second metric showed a very different ranking, often reversing the previous ranking. In many cases, the sites with the largest Web downloads in bytes were also had the highest bytes per second. Facebook was last with an average of 693 byte/sec for a 4,663-byte download. But CNN was No.1 with 2,368 byte/sec, for a 24,211-byte download, followed by ESPN with 1,756 byte/sec, for a 26,635-byte download. Google was mid-way in the pack: 1,009 byte/sec, for a 6,684-byte download.

The sites were much closer to each other in their success rates, again the percentage of time each site successfully downloaded the full content to the mobile phone. Facebook and Google tied for first with a success rate of 99.23%. CNN and Yahoo were nearly tied for second, with 98.99% and 98.97% respectively. But even the last place site, MSN, had a 97% success rate.

Keynote used its own hosted testing service, the Mobile Application Perspective application, which uses a library of 1,000 device simulations to monitor, measure and evaluate mobile content performance. Cellular modems connect to live wireless networks, and download the target over the carrier's network, to the emulation software.

In this case, Keynote emulated the Motorola RAZR smartphone, which a Keynote statement says is the most popular in the United States. Each modem takes one measurement at each site every hour across all locations for a week. The measurements are aggregated at the end of the week across all locations and carriers to come up with an index for each of the three metrics. Keynote has also launched a mobile index for European mobile sites.

According to a Keynote statement, all three metrics were constantly changing, sometimes for the better, sometimes for the worse, during the course of the week-long test. "Clearly, companies are still struggling to perfect their mobile offerings," according to a Keynote statement.

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.
Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

John Cox

Network World
Show Comments

Cool Tech

Bang and Olufsen Beosound Stage - Dolby Atmos Soundbar

Learn more >

Toys for Boys

Sony WF-1000XM3 Wireless Noise Cancelling Headphones

Learn more >

Nakamichi Delta 100 3-Way Hi Fi Speaker System

Learn more >

ASUS ROG, ACRONYM partner for Special Edition Zephyrus G14

Learn more >

Family Friendly

Philips Sonicare Diamond Clean 9000 Toothbrush

Learn more >

Mario Kart Live: Home Circuit for Nintendo Switch

Learn more >

Stocking Stuffer

SunnyBunny Snowflakes 20 LED Solar Powered Fairy String

Learn more >

Teac 7 inch Swivel Screen Portable DVD Player

Learn more >

Christmas Gift Guide

Click for more ›

Brand Post

Shining a light on creativity

MSI has long pushed the boundaries of invention with its ever-evolving range of laptops but it has now pulled off a world first with the new MSI Creative 17.

Most Popular Reviews

Latest Articles

Resources

PCW Evaluation Team

Tom Pope

Dynabook Portégé X30L-G

Ultimately this laptop has achieved everything I would hope for in a laptop for work, while fitting that into a form factor and weight that is remarkable.

Tom Sellers

MSI P65

This smart laptop was enjoyable to use and great to work on – creating content was super simple.

Lolita Wang

MSI GT76

It really doesn’t get more “gaming laptop” than this.

Jack Jeffries

MSI GS75

As the Maserati or BMW of laptops, it would fit perfectly in the hands of a professional needing firepower under the hood, sophistication and class on the surface, and gaming prowess (sports mode if you will) in between.

Taylor Carr

MSI PS63

The MSI PS63 is an amazing laptop and I would definitely consider buying one in the future.

Christopher Low

Brother RJ-4230B

This small mobile printer is exactly what I need for invoicing and other jobs such as sending fellow tradesman details or step-by-step instructions that I can easily print off from my phone or the Web.

Featured Content

Product Launch Showcase

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?