Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
courses:mapreduce-tutorial:step-3 [2012/01/28 11:09] majlis Odstraneny soubory s priklady |
courses:mapreduce-tutorial:step-3 [2012/01/30 13:51] straka |
||
---|---|---|---|
Line 8: | Line 8: | ||
<file perl> | <file perl> | ||
- | # | + | package |
- | + | ||
- | package Mapper; | + | |
use Moose; | use Moose; | ||
with ' | with ' | ||
Line 20: | Line 18: | ||
} | } | ||
- | package | + | package |
use Hadoop:: | use Hadoop:: | ||
my $runner = Hadoop:: | my $runner = Hadoop:: | ||
- | mapper => Mapper-> | + | mapper => My::Mapper-> |
input_format => ' | input_format => ' | ||
output_format => ' | output_format => ' | ||
output_compression => 0); | output_compression => 0); | ||
- | $runner-> | + | $runner-> |
</ | </ | ||
Line 37: | Line 35: | ||
perl script.pl run input_directory output_directory | perl script.pl run input_directory output_directory | ||
All files in input_directory are processes. The output_directory must not exist. | All files in input_directory are processes. The output_directory must not exist. | ||
+ | |||
+ | === Standard input and output === | ||
+ | Standard input and standard output of the Perl script are used to communicate with the Hadoop framework. Therefore you should use standard error output if you want to print some debugging output using '' | ||
===== Exercise ===== | ===== Exercise ===== | ||
- | To check that your Hadoop environment works, try running a MR job on ''/ | + | To check that your Hadoop environment works, try running a MR job on ''/ |
wget --no-check-certificate ' | wget --no-check-certificate ' | ||
- | rm -rf step-3-output; perl step-3-exercise.pl run / | + | |
+ | # $EDITOR step-3-exercise.pl | ||
+ | | ||
+ | less step-3-out-ex/ | ||
+ | |||
+ | ==== Solution ==== | ||
+ | You can also download the solution {{: | ||
+ | wget --no-check-certificate ' | ||
+ | # NOW VIEW THE FILE | ||
+ | # $EDITOR step-3-solution.pl | ||
+ | rm -rf step-3-out-sol; | ||
+ | less step-3-out-sol/ | ||
+ | ---- | ||
+ | < | ||
+ | <table style=" | ||
+ | <tr> | ||
+ | <td style=" | ||
+ | <td style=" | ||
+ | <td style=" | ||
+ | </tr> | ||
+ | </ | ||
+ | </ |