Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
courses:mapreduce-tutorial:step-5 [2012/01/24 22:08] straka |
courses:mapreduce-tutorial:step-5 [2012/01/24 22:36] straka |
||
---|---|---|---|
Line 1: | Line 1: | ||
====== MapReduce Tutorial : Basic reducer ====== | ====== MapReduce Tutorial : Basic reducer ====== | ||
+ | |||
+ | The interesting part of a MR job is the reducer -- after all mappers produce the (key, value) pairs, for every unique key and all its values a '' | ||
+ | |||
+ | The '' | ||
<file perl reducer.pl> | <file perl reducer.pl> | ||
- | # | ||
- | |||
package Mapper; | package Mapper; | ||
use Moose; | use Moose; | ||
with ' | with ' | ||
- | + | ||
sub map { | sub map { | ||
my ($self, $key, $value, $context) = @_; | my ($self, $key, $value, $context) = @_; | ||
- | + | ||
$context-> | $context-> | ||
} | } | ||
- | + | ||
+ | package Reducer; | ||
+ | use Moose; | ||
+ | with ' | ||
+ | |||
+ | sub reduce { | ||
+ | my ($self, $key, $values, $context) = @_; | ||
+ | |||
+ | while ($values-> | ||
+ | $context-> | ||
+ | } | ||
+ | } | ||
package Main; | package Main; | ||
use Hadoop:: | use Hadoop:: | ||
- | + | ||
my $runner = Hadoop:: | my $runner = Hadoop:: | ||
mapper => Mapper-> | mapper => Mapper-> | ||
- | | + | |
- | output_format => ' | + | |
- | output_compression => 0); | + | |
- | + | ||
$runner-> | $runner-> | ||
</ | </ | ||
+ | |||
+ | ===== Exercise 1 ===== | ||
+ | |||
+ | Run a MR job on / | ||
+ | |||
+ | {{: | ||
+ | |||
+ | ===== Exercise 2 ===== | ||
+ | |||
+ | Run a MR job on / | ||