[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki

[ Back to the navigation ]


This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
courses:mapreduce-tutorial:step-10 [2012/01/28 16:44]
majlis Commands for execution were added.
courses:mapreduce-tutorial:step-10 [2012/01/31 09:38] (current)
straka Change Perl commandline syntax.
Line 10: Line 10:
 <file perl> <file perl>
-package Mapper;+package My::Mapper;
 ... ...
-package Reducer;+package My::Reducer;
 ... ...
-package Main;+package main;
 use Hadoop::Runner; use Hadoop::Runner;
 my $runner = Hadoop::Runner->new( my $runner = Hadoop::Runner->new(
-  mapper => Mapper->new(), +  mapper => My::Mapper->new(), 
-  combiner => Reducer->new(), # Specify the combiner. +  combiner => My::Reducer->new(), # Specify the combiner. 
-  reducer => Reducer->new(),+  reducer => My::Reducer->new(),
   input_format => 'KeyValueTextInputFormat');   input_format => 'KeyValueTextInputFormat');
 ... ...
Line 31: Line 31:
 Compare the effect of adding the combiner to a MR job which counts occurrences of words in ''/home/straka/wiki/cs-text-medium'': {{:courses:mapreduce-tutorial:step-5-solution1.txt|step-10-wc-without-combiner.pl}} and {{:courses:mapreduce-tutorial:step-10.txt|step-10-wc-with-combiner.pl}}. Compare the effect of adding the combiner to a MR job which counts occurrences of words in ''/home/straka/wiki/cs-text-medium'': {{:courses:mapreduce-tutorial:step-5-solution1.txt|step-10-wc-without-combiner.pl}} and {{:courses:mapreduce-tutorial:step-10.txt|step-10-wc-with-combiner.pl}}.
   wget --no-check-certificate 'https://wiki.ufal.ms.mff.cuni.cz/_media/courses:mapreduce-tutorial:step-5-solution1.txt' -O 'step-10-wc-without-combiner.pl'   wget --no-check-certificate 'https://wiki.ufal.ms.mff.cuni.cz/_media/courses:mapreduce-tutorial:step-5-solution1.txt' -O 'step-10-wc-without-combiner.pl'
-  rm -rf step-10-out-wout; time perl step-10-wc-without-combiner.pl run /home/straka/wiki/cs-text-medium/ step-10-out-wout+  # NOW VIEW THE FILE 
 +  # $EDITOR step-10-wc-without-combiner.pl 
 +  rm -rf step-10-out-wout; time perl step-10-wc-without-combiner.pl /home/straka/wiki/cs-text-medium/ step-10-out-wout
   wget --no-check-certificate 'https://wiki.ufal.ms.mff.cuni.cz/_media/courses:mapreduce-tutorial:step-10.txt' -O 'step-10-wc-with-combiner.pl'   wget --no-check-certificate 'https://wiki.ufal.ms.mff.cuni.cz/_media/courses:mapreduce-tutorial:step-10.txt' -O 'step-10-wc-with-combiner.pl'
-  rm -rf step-10-out-with; time perl step-10-wc-with-combiner.pl run /home/straka/wiki/cs-text-medium/ step-10-out-with+  # NOW VIEW THE FILE 
 +  # $EDITOR step-10-wc-with-combiner.pl 
 +  rm -rf step-10-out-with; time perl step-10-wc-with-combiner.pl /home/straka/wiki/cs-text-medium/ step-10-out-with
 How would you explain the results? How would you explain the results?

[ Back to the navigation ] [ Back to the content ]