[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki

[ Back to the navigation ]

You are here: start » courses » mapreduce-tutorial » step-12

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
courses:mapreduce-tutorial:step-12 [2012/01/25 21:29]
straka 		courses:mapreduce-tutorial:step-12 [2012/01/31 09:39] (current)
straka Change Perl commandline syntax.
Line 14: Line 14:
 If a MR job runs without reducers, the output of mappers is written to output directory without further processing. In this case, environmental variable ''HADOOP_WORK_OUTPUT_PATH'' is present even in a mapper and the files created in this directory are copied to the job output directory. If a MR job runs without reducers, the output of mappers is written to output directory without further processing. In this case, environmental variable ''HADOOP_WORK_OUTPUT_PATH'' is present even in a mapper and the files created in this directory are copied to the job output directory.
  
 +
 +===== Exercise =====
 +Change the word counting script {{:courses:mapreduce-tutorial:step-5-solution1.txt|step-12-exercise.pl}} to produce results in reducers manually using the mentioned environmental variables, and execute it using four reducers.
 +
 +  wget --no-check-certificate 'https://wiki.ufal.ms.mff.cuni.cz/_media/courses:mapreduce-tutorial:step-5-solution1.txt' -O 'step-12-exercise.pl'
 +  # NOW EDIT THE FILE
 +  # $EDITOR step-12-exercise.pl
 +  rm -rf step-12-out-ex; perl step-12-exercise.pl -c 4 -r 4 /home/straka/wiki/cs-text-medium/ step-12-out-ex
 +  less step-12-out-ex/part-*
 +
 +==== Solution ====
 +You can also download the solution {{:courses:mapreduce-tutorial:step-12-solution.txt|step-12-solution.pl}} and check the correct output.
 +  wget --no-check-certificate 'https://wiki.ufal.ms.mff.cuni.cz/_media/courses:mapreduce-tutorial:step-12-solution.txt' -O 'step-12-solution.pl'
 +  # NOW VIEW THE FILE
 +  # $EDITOR step-12-solution.pl
 +  rm -rf step-12-out-sol; perl step-12-solution.pl -c 4 -r 4 /home/straka/wiki/cs-text-medium/ step-12-out-sol
 +  less step-12-out-sol/vystup-*
 +
 +----
 +
 +<html>
 +<table style="width:100%">
 +<tr>
 +<td style="text-align:left; width: 33%; "></html>[[step-11|Step 11]]: Initialization and cleanup of MR tasks, performance of combiners.<html></td>
 +<td style="text-align:center; width: 33%; "></html>[[.|Overview]]<html></td>
 +<td style="text-align:right; width: 33%; "></html>[[step-13|Step 13]]: Sorting.<html></td>
 +</tr>
 +</table>
 +</html>

[ Back to the navigation ] [ Back to the content ]