Amazon Elastic MapReduce
Based on Hadoop, MapReduce equips users with potent distributed data-processing tools
- Doesn't take long to get the hang of
- Currently available in the US region only
You'll want to be familiar with the Apache Hadoop framework before you jump into Elastic MapReduce. It doesn't take long to get the hang of it, though. Most developers can have a MapReduce application running within a few hours.
Have you got a few hundred gigabytes of data that need processing? Perhaps a dump of radio telescope data that could use some combing through by a squad of processors running Fourier transforms? Or maybe you're convinced some statistical analysis will reveal a pattern hidden in several years of stock market information? Unfortunately, you don't happen to have a grid of distributed processors to run your application, much less the time to construct a parallel processing infrastructure.
Well, cheer up: Amazon has added Elastic MapReduce to its growing list of cloud-based Web services. Currently in beta, Elastic MapReduce uses Amazon's Elastic Compute Cloud (EC2) and Simple Storage Service (S3) to implement a virtualized distributed processing system based on Apache Hadoop.
Hadoop's internal architecture is the MapReduce framework. The mechanics of MapReduce are well documented in a paper by J. Dean and S. Ghemawat [PDF], and a full treatment is beyond the scope of this article. Instead, I'll illustrate by example.
Suppose you have a set of 10 words and you want to count the number of times those words appear in a collection of e-books. Your input data is a set of key/value pairs, the value being a line of text from one of the books and the key being the concatenation of the book's name and the line's number. This set might comprise a few megabytes big -- or gigabytes. MapReduce doesn't much care about size.
You write a routine that reads this input, a pair at a time, and produces another key/value pair as output. The output key is a word (from the original set of 10) and the associated value is the number of times that word appears in the line. (Zero values are not emitted.) This routine is the map part of map/reduce. Its output is referred to as the intermediate key/value pairs.
The intermediate key/value pairs are fed to another function (another "step" in the parlance of MapReduce). For this step, you write a routine that iterates through the intermediate data, sums up the values, and returns a single pair whose key is the word and whose value is the grand total. You don't have to worry about grouping the results of like keys (i.e., gathering all the intermediate key/values for a given word), because Hadoop does that grouping for you in the background.
Join the newsletter!
Most Popular Reviews
- 1 LG G6 Plus: Full, in-depth review
- 2 First Look: Nikon D850
- 3 OnePlus 5: Full, in-depth review
- 4 Nokia 8: Full, in-depth review
- 5 Panasonic Ultra HD OLED TV Review
Latest News Articles
- Polycom Renew Their Most Iconic Teleconferencing Solution
- Western Digital Announces Acquisition of Upthere
- Western Digital to acquire Tegile
- NBN hooks up 6 million homes and businesses
- ASUS Announces Two New Entries into the VivoBook Range with the VivoBook 14 and VivoBook 15
PCW Evaluation Team
The print quality also does not disappoint, it’s clear, bold, doesn’t smudge and the text is perfectly sized.
The Huddle Board’s built in program; Sharp Touch Viewing software allows us to easily manipulate and edit our documents (jpegs and PDFs) all at the same time on the dashboard.
The biggest perks for me would be that it comes with easy to use and comprehensive programs that make the collaboration process a whole lot more intuitive and organic
I rate the printer as a 5 out of 5 stars as it has been able to fit seamlessly into my busy and mobile lifestyle.
It’s perfect for mobile workers. Just take it out — it’s small enough to sit anywhere — turn it on, load a sheet of paper, and start printing.
The HP OfficeJet 250 Mobile Printer is a great device that fits perfectly into my fast paced and mobile lifestyle. My first impression of the printer itself was how incredibly compact and sleek the device was.
- Dishonored - Death Of The Outsider review:
- LG G6 Plus: Full, in-depth review
- MSI GE73 VR Raider Gaming Laptop review
- What's the difference between an Intel Core i3, i5 and i7?
- Laser vs. inkjet printers: which is better?
- FTSocial Media Executive / Specialist (Facebook) - Online PokerNSW
- TPBusiness Process Flow AnalystNSW
- FTOffice 365 SMEOther
- FTTraining Coordinator - Sydney CBD (Urgent!!)Other
- FTSenior Software EngineerOther
- FTBiztalk DeveloperSA
- FTGun Java Developers wantedVIC
- CCSenior Business Analyst - InfrastructureVIC
- CCService Management AnalystNSW
- FTCloud Infrastructure ArchitectQLD
- FTIT Project ManagerOther
- FTSenior Project Manager - Go to Market / Channel ReadinessOther
- FTApplication Security AnalystOther
- FT.NET DeveloperWA
- TPAnalyst ProgrammerSA
- CCFilenet Developer - BrisbaneVIC
- CCMiddleware SpecialistNSW
- FTDevOps Engineer - TelecommunicationsOther
- FTUser Experience (UX) SpecialistACT
- FTSolutions Architect (Enterprise Infrastructure)ACT
- FTMid - Senior Web DeveloperWA
- FTBusiness AnalystOther
- FTSolution Architect - Enterprise VoiceVIC
- FTSAP Integration ManagerOther
- CCDigital Business AnalystACT