[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki


[ Back to the navigation ]

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
courses:mapreduce-tutorial:step-12 [2012/01/25 22:15]
straka
courses:mapreduce-tutorial:step-12 [2012/01/31 09:39] (current)
straka Change Perl commandline syntax.
Line 13: Line 13:
 ===== Reduce-less jobs ===== ===== Reduce-less jobs =====
 If a MR job runs without reducers, the output of mappers is written to output directory without further processing. In this case, environmental variable ''HADOOP_WORK_OUTPUT_PATH'' is present even in a mapper and the files created in this directory are copied to the job output directory. If a MR job runs without reducers, the output of mappers is written to output directory without further processing. In this case, environmental variable ''HADOOP_WORK_OUTPUT_PATH'' is present even in a mapper and the files created in this directory are copied to the job output directory.
 +
  
 ===== Exercise ===== ===== Exercise =====
-Change the word counting script {{:courses:mapreduce-tutorial:step-5-solution1.txt|wc.pl}} to produce results in reducers manually using the mentioned environmental variables, and execute it using four reducers.+Change the word counting script {{:courses:mapreduce-tutorial:step-5-solution1.txt|step-12-exercise.pl}} to produce results in reducers manually using the mentioned environmental variables, and execute it using four reducers. 
 + 
 +  wget --no-check-certificate 'https://wiki.ufal.ms.mff.cuni.cz/_media/courses:mapreduce-tutorial:step-5-solution1.txt' -O 'step-12-exercise.pl' 
 +  # NOW EDIT THE FILE 
 +  # $EDITOR step-12-exercise.pl 
 +  rm -rf step-12-out-ex; perl step-12-exercise.pl -c 4 -r 4 /home/straka/wiki/cs-text-medium/ step-12-out-ex 
 +  less step-12-out-ex/part-* 
 + 
 +==== Solution ==== 
 +You can also download the solution {{:courses:mapreduce-tutorial:step-12-solution.txt|step-12-solution.pl}} and check the correct output. 
 +  wget --no-check-certificate 'https://wiki.ufal.ms.mff.cuni.cz/_media/courses:mapreduce-tutorial:step-12-solution.txt' -O 'step-12-solution.pl' 
 +  # NOW VIEW THE FILE 
 +  # $EDITOR step-12-solution.pl 
 +  rm -rf step-12-out-sol; perl step-12-solution.pl -c 4 -r 4 /home/straka/wiki/cs-text-medium/ step-12-out-sol 
 +  less step-12-out-sol/vystup-*
  
-{{:courses:mapreduce-tutorial:step-12-solution.txt|Solution.pl}}+----
  
 +<html>
 +<table style="width:100%">
 +<tr>
 +<td style="text-align:left; width: 33%; "></html>[[step-11|Step 11]]: Initialization and cleanup of MR tasks, performance of combiners.<html></td>
 +<td style="text-align:center; width: 33%; "></html>[[.|Overview]]<html></td>
 +<td style="text-align:right; width: 33%; "></html>[[step-13|Step 13]]: Sorting.<html></td>
 +</tr>
 +</table>
 +</html>

[ Back to the navigation ] [ Back to the content ]