The interesting part of a MR job is the reducer – after all mappers produce the (key, value) pairs, for every unique key and all its values a reduce function is called. The reduce function can output (key, value) pairs, which are written to disk.

The reduce is similar to map, but instead of one value it gets an iterator, which can enumerate all values:

reducer.pl

package Mapper;
use Moose;
with 'Hadoop::Mapper';
 
sub map {
  my ($self, $key, $value, $context) = @_;
 
  $context->write($key, $value);
}
 
package Reducer;
use Moose;
with 'Hadoop::Reducer';
 
sub reduce {
  my ($self, $key, $values, $context) = @_;
 
  while ($values->next) {
    $context->write($key, $values->value);
  }
}
 
package Main;
use Hadoop::Runner;
 
my $runner = Hadoop::Runner->new(
  mapper => Mapper->new(),
  reducer => Reducer->new());
 
$runner->run();

Run a MR job on /home/straka/wiki/cs-text-small, which counts occurences of every word in the article texts.

Solution.pl

Run a MR job on /home/straka/wiki/cs-text-small, which generates an inverted index. Inverted index contains for each word all its occurrences, each occurrence is pair (article of occurrence, position of occurrence).