Wikipedia gears up for explosion in digital media

Bigger upload limits made possible by new servers, storage.

Wikipedia is gearing up for an explosion in digital content with new servers and storage designed to handle larger photo and video uploads.

Video file sizes are quickly reaching the dozens and hundreds of megabytes, and the proliferation of high-megapixel cameras means even small photos can take up a few megabytes, says Brion Vibber, CTO at the Wikimedia Foundation, which operates Wikipedia. Until early 2008, the user-generated encyclopedia's primary media file server had just 2TB of total space, Vibber says.

"For a long time, we just did not have the capacity [to handle very large media files]," he says.

Wikipedia has since scaled up from 2TB to 24TB and now 48TB of storage for its primary medial file server, and recently raised file upload limits from 20MB to 100MB. The amount of storage actually being used is about 5TB but that will grow quickly, Vibber says.

For example, users are uploading public domain classical music, and some recordings can last a half-hour. Documentary films that are out of copyright are also being uploaded, and some users are struggling to keep files under the 100MB limit, according to Vibber.

Vibber's long-term goal is to let users upload feature-length, high-quality videos, but in addition to capacity limits he says there are challenges related to getting files in the appropriate format and the physical movement of large files.

"There's no limit, and there's no practical limit," Vibber says. "The limits will get bigger and bigger to where it will be relatively easy for someone who has a legitimate need to upload a two-hour video of good quality."

Wikipedia's new servers and storage were supplied by Sun which provided donations and special pricing on Sun Fire x4500 and x4150 servers and StorageTek arrays. Wikipedia, which handles more than 10 billion page views a month, also uses Sun's open source MySQL as its primary database software.

Wikimedia operates a primary data center in Tampa, Fla., and caching centers in Amsterdam and South Korea, and has a total of 450 or so commodity servers from a mix of vendors, Vibber says.

The site's technology goals this year and next include improving usability, making it easier to edit articles and upload videos and photos, and integration with media file sites like Flickr, he says.

The donations from Sun came in the midst of a big fundraising push by Wikimedia, which raised more than US$6 million from 125,000 donors, allowing it to fund operations through the end of its fiscal year on June 30. Wikimedia is a nonprofit and the encyclopedia Web site is devoid of advertising, so the site relies on donations, grants and gifts. Wikipedia has a paid staff of 23 employees, about eight of which are in IT.

Join the newsletter!

Error: Please check your email address.
Rocket to Success - Your 10 Tips for Smarter ERP System Selection

Tags wikipedia

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Jon Brodkin

Network World
Show Comments

Most Popular Reviews

Latest Articles

Resources

PCW Evaluation Team

Ben Ramsden

Sharp PN-40TC1 Huddle Board

Brainstorming, innovation, problem solving, and negotiation have all become much more productive and valuable if people can easily collaborate in real time with minimal friction.

Sarah Ieroianni

Brother QL-820NWB Professional Label Printer

The print quality also does not disappoint, it’s clear, bold, doesn’t smudge and the text is perfectly sized.

Ratchada Dunn

Sharp PN-40TC1 Huddle Board

The Huddle Board’s built in program; Sharp Touch Viewing software allows us to easily manipulate and edit our documents (jpegs and PDFs) all at the same time on the dashboard.

George Khoury

Sharp PN-40TC1 Huddle Board

The biggest perks for me would be that it comes with easy to use and comprehensive programs that make the collaboration process a whole lot more intuitive and organic

David Coyle

Brother PocketJet PJ-773 A4 Portable Thermal Printer

I rate the printer as a 5 out of 5 stars as it has been able to fit seamlessly into my busy and mobile lifestyle.

Kurt Hegetschweiler

Brother PocketJet PJ-773 A4 Portable Thermal Printer

It’s perfect for mobile workers. Just take it out — it’s small enough to sit anywhere — turn it on, load a sheet of paper, and start printing.

Featured Content

Product Launch Showcase

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?