Differences

This shows you the differences between two versions of the page.

--- courses:mapreduce-tutorial:step-10 [2012/01/25 19:34]
straka
+++ courses:mapreduce-tutorial:step-10 [2012/01/25 22:12]
straka
@@ Line 3: / Line 3: @@
 Sometimes the reduce is a binary operation, which is associative and commutative, e.g. ''+''. In that case it is inefficient to produce all the (key, value) pairs in the mappers and send them through the network.
-Instead, reducer can be executed right after the map, on //some portion// of values belonging to the same key. Only the results are then sent through the network.
+Instead, reducer can be executed right after the map, on //some portion// of values belonging to the same key. Only the aggregated results are then sent through the network.
-A Hadoop job can have such locally executed reducer, called //combiner//. If a combiner is specified, the output of a mapper is processed by a combiner before sending the pairs to reducer. The combiner may be invoked 0, 1 or multiple times, usually when the data are written to disk.
+A Hadoop job can have such locally executed reducer, called a //combiner//. If a combiner is specified, the output of a mapper is processed by a combiner before sending the pairs to reducer. The combiner may be invoked 0, 1 or multiple times, usually when the data are written to disk.
 Typically, the combiner is the same as the reducer of a MR job.
-<code perl>
+<file perl>
 package Mapper;
 ...
@@ Line 25: / Line 25: @@
   input_format => 'KeyValueTextInputFormat');
 ...
-</code>
+</file>
-===== Excersise =====
+===== Exercise =====
-Compare the effect of adding the combiner to a MR job which counts occurences of words of ''/home/straka/wiki/cs-text-medium'': {{:courses:mapreduce-tutorial:step-5-solution1.txt|wc-without-combiner.pl}} and {{:courses:mapreduce-tutorial:step-10.txt|wc-with-combiner.pl}}.
+Compare the effect of adding the combiner to a MR job which counts occurrences of words in ''/home/straka/wiki/cs-text-medium'': {{:courses:mapreduce-tutorial:step-5-solution1.txt|wc-without-combiner.pl}} and {{:courses:mapreduce-tutorial:step-10.txt|wc-with-combiner.pl}}.
 How would you explain the results?

[ Back to the navigation ] [ Back to the content ]

Institute of Formal and Applied Linguistics Wiki

Differences