Differences

This shows you the differences between two versions of the page.

--- courses:mapreduce-tutorial:step-3 [2012/01/28 12:03]
majlis
+++ courses:mapreduce-tutorial:step-3 [2012/01/30 00:37]
straka Improving package names of Perl programs.
@@ Line 8: / Line 8: @@
 <file perl>
-#!/usr/bin/perl
+package My::Mapper;
-package Mapper;
 use Moose;
 with 'Hadoop::Mapper';
@@ Line 20: / Line 18: @@
 }
-package Main;
+package main;
 use Hadoop::Runner;
 my $runner = Hadoop::Runner->new(
-  mapper => Mapper->new(),
+  mapper => My::Mapper->new(),
   input_format => 'TextInputFormat',
   output_format => 'TextOutputFormat',
   output_compression => 0);
-$runner->run();
+$runner->run();  # Parse arguments in @ARGV and run the Hadoop job.
 </file>
@@ Line 38: / Line 36: @@
 All files in input_directory are processes. The output_directory must not exist.
+===== Exercise =====
+To check that your Hadoop environment works, try running a MR job on ''/home/straka/wiki/cs-text-medium'', which outputs only articles with names beginning with an ''A'' (ignoring the case). You can download the template {{:courses:mapreduce-tutorial:step-3-exercise.txt|step-3-exercise.pl}}  and execute it.
+  wget --no-check-certificate 'https://wiki.ufal.ms.mff.cuni.cz/_media/courses:mapreduce-tutorial:step-3-exercise.txt' -O 'step-3-exercise.pl'
+  # NOW EDIT THE FILE
+  # $EDITOR step-3-exercise.pl
+  rm -rf step-3-out-ex; perl step-3-exercise.pl run /home/straka/wiki/cs-text-medium/ step-3-out-ex
+  less step-3-out-ex/part-*
+==== Solution ====
+You can also download the solution {{:courses:mapreduce-tutorial:step-3-solution.txt|step-3-solution.pl}} and check the correct output.
+  wget --no-check-certificate 'https://wiki.ufal.ms.mff.cuni.cz/_media/courses:mapreduce-tutorial:step-3-solution.txt' -O 'step-3-solution.pl'
+  # NOW VIEW THE FILE
+  # $EDITOR step-3-solution.pl
+  rm -rf step-3-out-sol; perl step-3-solution.pl run /home/straka/wiki/cs-text-medium/ step-3-out-sol
+  less step-3-out-sol/part-*
 ----

[ Back to the navigation ] [ Back to the content ]

Institute of Formal and Applied Linguistics Wiki

Differences