Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
courses:mapreduce-tutorial:step-5 [2012/01/28 12:07] majlis Scripts were removed. |
courses:mapreduce-tutorial:step-5 [2012/01/29 23:42] majlis |
||
---|---|---|---|
Line 3: | Line 3: | ||
The interesting part of a Hadoop job is the //reducer// -- after all mappers produce the (key, value) pairs, for every unique key and all its values a '' | The interesting part of a Hadoop job is the //reducer// -- after all mappers produce the (key, value) pairs, for every unique key and all its values a '' | ||
- | The '' | + | The '' |
<file perl> | <file perl> | ||
Line 39: | Line 39: | ||
As before, Hadoop silently handles failures. It can happen that even a successfully finished mapper needs to be executed again -- if the machine, where its output data were stored, gets disconnected from the network. | As before, Hadoop silently handles failures. It can happen that even a successfully finished mapper needs to be executed again -- if the machine, where its output data were stored, gets disconnected from the network. | ||
+ | |||
+ | ===== Types of keys and values ===== | ||
+ | |||
+ | Currently in the Perl API, the keys and values are both strings, which are stored and loaded using UTF-8 format. If you need more complex structures, you have to serialize and deserialize them by yourselves. | ||
+ | |||
+ | The Java API offers a wide range of types, including user-defined types, to be used for keys and values. | ||
===== Exercise 1 ===== | ===== Exercise 1 ===== | ||
- | Run a Hadoop job on ''/ | + | Run a Hadoop job on ''/ |
+ | wget --no-check-certificate ' | ||
+ | # NOW EDIT THE FILE | ||
+ | # $EDITOR step-5-exercise1.pl | ||
+ | rm -rf step-5-out-ex1; | ||
+ | less step-5-out-ex1/ | ||
+ | |||
+ | ==== Solution ==== | ||
+ | You can also download the solution {{: | ||
+ | wget --no-check-certificate ' | ||
+ | rm -rf step-5-out-sol1; | ||
+ | less step-5-out-sol1/ | ||
===== Exercise 2 ===== | ===== Exercise 2 ===== | ||
- | Run a Hadoop job on ''/ | + | Run a Hadoop job on ''/ |
+ | wget --no-check-certificate ' | ||
+ | # NOW EDIT THE FILE | ||
+ | # $EDITOR step-5-exercise2.pl | ||
+ | rm -rf step-5-out-ex2; | ||
+ | less step-5-out-ex2/ | ||
+ | |||
+ | ==== Solution ==== | ||
+ | You can also download the solution {{: | ||
+ | wget --no-check-certificate ' | ||
+ | rm -rf step-5-out-sol2; | ||
+ | less step-5-out-sol2/ | ||
+ | ---- | ||
+ | < | ||
+ | <table style=" | ||
+ | <tr> | ||
+ | <td style=" | ||
+ | <td style=" | ||
+ | <td style=" | ||
+ | </tr> | ||
+ | </ | ||
+ | </ |