[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki


[ Back to the navigation ]

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
courses:mapreduce-tutorial:step-4 [2012/01/25 21:52]
straka
courses:mapreduce-tutorial:step-4 [2012/01/31 09:40] (current)
straka Change Perl commandline syntax.
Line 3: Line 3:
 Sometimes it is useful to count events differently than outputting them as (key, value) pairs. For that reason Hadoop offers simple counter framework. Sometimes it is useful to count events differently than outputting them as (key, value) pairs. For that reason Hadoop offers simple counter framework.
  
-Hadoop maintains a collection of pre-defined and user-defined counters. Every counter is identified by its group name and counter name. To increment a counter, the following code can be used:+Hadoop maintains a collection of pre-defined and user-defined counters. Every counter is identified by its group name and counter name. The group name and counter name is an arbitrary string -- for example group "Documents" and name "Number of words". To increment a counter, the following code can be used:
  
 <code perl> <code perl>
Line 17: Line 17:
 ===== Exercise ===== ===== Exercise =====
  
-Run a Hadoop job on ''/home/straka/wiki/cs-text'', which uses counters to count the number of articles according to their first letter (ignoring the case and possible diacritics).+Run a Hadoop job on ''/home/straka/wiki/cs-text-medium'', which uses counters to count the number of articles according to their first letter (ignoring the case). You can download the template {{:courses:mapreduce-tutorial:step-4-exercise.txt|step-4-exercise.pl}}  and execute it. 
 +  wget --no-check-certificate 'https://wiki.ufal.ms.mff.cuni.cz/_media/courses:mapreduce-tutorial:step-4-exercise.txt' -O 'step-4-exercise.pl' 
 +  # NOW EDIT THE FILE 
 +  # $EDITOR step-4-exercise.pl 
 +  rm -rf step-4-out-ex; perl step-4-exercise.pl /home/straka/wiki/cs-text-medium/ step-4-out-ex 
 +  less step-4-out-ex/part-*
  
-{{.:step-4-solution.txt|Solution.pl}}+==== Solution ==== 
 +You can also download the solution {{:courses:mapreduce-tutorial:step-4-solution.txt|step-4-solution.pl}} and check the correct output. 
 +  wget --no-check-certificate 'https://wiki.ufal.ms.mff.cuni.cz/_media/courses:mapreduce-tutorial:step-4-solution.txt' -O 'step-4-solution.pl' 
 +  # NOW VIEW THE FILE 
 +  # $EDITOR step-3-solution.pl 
 +  rm -rf step-4-out-sol; perl step-4-solution.pl /home/straka/wiki/cs-text-medium/ step-4-out-sol 
 +  less step-4-out-sol/part-*
  
 +----
 +
 +<html>
 +<table style="width:100%">
 +<tr>
 +<td style="text-align:left; width: 33%; "></html>[[step-3|Step 3]]: Basic mapper.<html></td>
 +<td style="text-align:center; width: 33%; "></html>[[.|Overview]]<html></td>
 +<td style="text-align:right; width: 33%; "></html>[[step-5|Step 5]]: Basic reducer.<html></td>
 +</tr>
 +</table>
 +</html>

[ Back to the navigation ] [ Back to the content ]