Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
courses:mapreduce-tutorial:step-3 [2012/01/28 12:05] majlis Scripts were reuploaded. |
courses:mapreduce-tutorial:step-3 [2012/01/30 13:51] straka |
||
---|---|---|---|
Line 8: | Line 8: | ||
<file perl> | <file perl> | ||
- | # | + | package |
- | + | ||
- | package Mapper; | + | |
use Moose; | use Moose; | ||
with ' | with ' | ||
Line 20: | Line 18: | ||
} | } | ||
- | package | + | package |
use Hadoop:: | use Hadoop:: | ||
my $runner = Hadoop:: | my $runner = Hadoop:: | ||
- | mapper => Mapper-> | + | mapper => My::Mapper-> |
input_format => ' | input_format => ' | ||
output_format => ' | output_format => ' | ||
output_compression => 0); | output_compression => 0); | ||
- | $runner-> | + | $runner-> |
</ | </ | ||
Line 38: | Line 36: | ||
All files in input_directory are processes. The output_directory must not exist. | All files in input_directory are processes. The output_directory must not exist. | ||
+ | === Standard input and output === | ||
+ | Standard input and standard output of the Perl script are used to communicate with the Hadoop framework. Therefore you should use standard error output if you want to print some debugging output using '' | ||
===== Exercise ===== | ===== Exercise ===== | ||
Line 43: | Line 43: | ||
To check that your Hadoop environment works, try running a MR job on ''/ | To check that your Hadoop environment works, try running a MR job on ''/ | ||
wget --no-check-certificate ' | wget --no-check-certificate ' | ||
+ | # NOW EDIT THE FILE | ||
+ | # $EDITOR step-3-exercise.pl | ||
rm -rf step-3-out-ex; | rm -rf step-3-out-ex; | ||
- | less step-3-out-ex/ | + | less step-3-out-ex/ |
| | ||
==== Solution ==== | ==== Solution ==== | ||
You can also download the solution {{: | You can also download the solution {{: | ||
wget --no-check-certificate ' | wget --no-check-certificate ' | ||
+ | # NOW VIEW THE FILE | ||
+ | # $EDITOR step-3-solution.pl | ||
rm -rf step-3-out-sol; | rm -rf step-3-out-sol; | ||
- | less step-3-out-sol/ | + | less step-3-out-sol/ |
---- | ---- |