Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
courses:mapreduce-tutorial:step-3 [2012/01/28 11:35] majlis Added links to previous and next chapter. |
courses:mapreduce-tutorial:step-3 [2012/01/30 13:51] straka |
||
---|---|---|---|
Line 8: | Line 8: | ||
<file perl> | <file perl> | ||
- | # | + | package |
- | + | ||
- | package Mapper; | + | |
use Moose; | use Moose; | ||
with ' | with ' | ||
Line 20: | Line 18: | ||
} | } | ||
- | package | + | package |
use Hadoop:: | use Hadoop:: | ||
my $runner = Hadoop:: | my $runner = Hadoop:: | ||
- | mapper => Mapper-> | + | mapper => My::Mapper-> |
input_format => ' | input_format => ' | ||
output_format => ' | output_format => ' | ||
output_compression => 0); | output_compression => 0); | ||
- | $runner-> | + | $runner-> |
</ | </ | ||
Line 37: | Line 35: | ||
perl script.pl run input_directory output_directory | perl script.pl run input_directory output_directory | ||
All files in input_directory are processes. The output_directory must not exist. | All files in input_directory are processes. The output_directory must not exist. | ||
+ | |||
+ | === Standard input and output === | ||
+ | Standard input and standard output of the Perl script are used to communicate with the Hadoop framework. Therefore you should use standard error output if you want to print some debugging output using '' | ||
===== Exercise ===== | ===== Exercise ===== | ||
- | To check that your Hadoop environment works, try running a MR job on ''/ | + | To check that your Hadoop environment works, try running a MR job on ''/ |
wget --no-check-certificate ' | wget --no-check-certificate ' | ||
+ | # NOW EDIT THE FILE | ||
+ | # $EDITOR step-3-exercise.pl | ||
rm -rf step-3-out-ex; | rm -rf step-3-out-ex; | ||
- | less step-3-out-ex/ | + | less step-3-out-ex/ |
| | ||
==== Solution ==== | ==== Solution ==== | ||
You can also download the solution {{: | You can also download the solution {{: | ||
wget --no-check-certificate ' | wget --no-check-certificate ' | ||
+ | # NOW VIEW THE FILE | ||
+ | # $EDITOR step-3-solution.pl | ||
rm -rf step-3-out-sol; | rm -rf step-3-out-sol; | ||
- | less step-3-out-sol/ | + | less step-3-out-sol/ |
---- | ---- |