Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
courses:mapreduce-tutorial:step-10 [2012/01/25 19:06] straka |
courses:mapreduce-tutorial:step-10 [2012/01/25 22:12] straka |
||
---|---|---|---|
Line 3: | Line 3: | ||
Sometimes the reduce is a binary operation, which is associative and commutative, | Sometimes the reduce is a binary operation, which is associative and commutative, | ||
- | Instead, reducer can be executed right after the map, on //some portion// of values belonging to the same key. Only the results are then sent through the network. | + | Instead, reducer can be executed right after the map, on //some portion// of values belonging to the same key. Only the aggregated |
- | A Hadoop job can have such locally executed reducer, called // | + | A Hadoop job can have such locally executed reducer, called |
Typically, the combiner is the same as the reducer of a MR job. | Typically, the combiner is the same as the reducer of a MR job. | ||
- | <code perl> | + | <file perl> |
package Mapper; | package Mapper; | ||
... | ... | ||
Line 25: | Line 25: | ||
input_format => ' | input_format => ' | ||
... | ... | ||
- | </code> | + | </file> |
- | ===== Excersise | + | ===== Exercise |
+ | |||
+ | Compare the effect of adding the combiner to a MR job which counts occurrences of words in ''/ | ||
+ | |||
+ | How would you explain the results? | ||
- | Compare the effect of adding the combiner to a MR job which counts occurences of words: {{: |