Wikipedia gears up for explosion in digital media

Bigger upload limits made possible by new servers, storage.

Wikipedia is gearing up for an explosion in digital content with new servers and storage designed to handle larger photo and video uploads.

Video file sizes are quickly reaching the dozens and hundreds of megabytes, and the proliferation of high-megapixel cameras means even small photos can take up a few megabytes, says Brion Vibber, CTO at the Wikimedia Foundation, which operates Wikipedia. Until early 2008, the user-generated encyclopedia's primary media file server had just 2TB of total space, Vibber says.

"For a long time, we just did not have the capacity [to handle very large media files]," he says.

Wikipedia has since scaled up from 2TB to 24TB and now 48TB of storage for its primary medial file server, and recently raised file upload limits from 20MB to 100MB. The amount of storage actually being used is about 5TB but that will grow quickly, Vibber says.

For example, users are uploading public domain classical music, and some recordings can last a half-hour. Documentary films that are out of copyright are also being uploaded, and some users are struggling to keep files under the 100MB limit, according to Vibber.

Vibber's long-term goal is to let users upload feature-length, high-quality videos, but in addition to capacity limits he says there are challenges related to getting files in the appropriate format and the physical movement of large files.

"There's no limit, and there's no practical limit," Vibber says. "The limits will get bigger and bigger to where it will be relatively easy for someone who has a legitimate need to upload a two-hour video of good quality."

Wikipedia's new servers and storage were supplied by Sun which provided donations and special pricing on Sun Fire x4500 and x4150 servers and StorageTek arrays. Wikipedia, which handles more than 10 billion page views a month, also uses Sun's open source MySQL as its primary database software.

Wikimedia operates a primary data center in Tampa, Fla., and caching centers in Amsterdam and South Korea, and has a total of 450 or so commodity servers from a mix of vendors, Vibber says.

The site's technology goals this year and next include improving usability, making it easier to edit articles and upload videos and photos, and integration with media file sites like Flickr, he says.

The donations from Sun came in the midst of a big fundraising push by Wikimedia, which raised more than US$6 million from 125,000 donors, allowing it to fund operations through the end of its fiscal year on June 30. Wikimedia is a nonprofit and the encyclopedia Web site is devoid of advertising, so the site relies on donations, grants and gifts. Wikipedia has a paid staff of 23 employees, about eight of which are in IT.

Join the newsletter!

Or
Error: Please check your email address.
Rocket to Success - Your 10 Tips for Smarter ERP System Selection

Tags wikipedia

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Jon Brodkin

Network World
Show Comments

Essentials

James Cook University - Master of Data Science Online Course

Learn more >

Mobile

Exec

Budget

Back To Business Guide

Click for more ›

Brand Post

Most Popular Reviews

Latest Articles

Resources

PCW Evaluation Team

Andrew Teoh

Brother MFC-L9570CDW Multifunction Printer

Touch screen visibility and operation was great and easy to navigate. Each menu and sub-menu was in an understandable order and category

Louise Coady

Brother MFC-L9570CDW Multifunction Printer

The printer was convenient, produced clear and vibrant images and was very easy to use

Edwina Hargreaves

WD My Cloud Home

I would recommend this device for families and small businesses who want one safe place to store all their important digital content and a way to easily share it with friends, family, business partners, or customers.

Walid Mikhael

Brother QL-820NWB Professional Label Printer

It’s easy to set up, it’s compact and quiet when printing and to top if off, the print quality is excellent. This is hands down the best printer I’ve used for printing labels.

Ben Ramsden

Sharp PN-40TC1 Huddle Board

Brainstorming, innovation, problem solving, and negotiation have all become much more productive and valuable if people can easily collaborate in real time with minimal friction.

Sarah Ieroianni

Brother QL-820NWB Professional Label Printer

The print quality also does not disappoint, it’s clear, bold, doesn’t smudge and the text is perfectly sized.

Featured Content

Product Launch Showcase

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?