[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki

• ÚFAL Wiki Home
• ÚFAL Home
• Sitemap
• RSS
• ÚFAL calendar
• Recent changes
• Login
• Hledat:

[ Back to the navigation ]

Differences

This shows you the differences between two versions of the page.

View differences: Side by SideInline Go

Link to this comparison view

Both sides previous revision Previous revision 2012/01/31 09:38 straka Change Perl commandline syntax.2012/01/30 10:28 majlis 2012/01/30 00:33 straka 2012/01/28 16:44 majlis Commands for execution were added.2012/01/28 16:00 majlis Added links to previous and next chapter.2012/01/28 12:32 majlis 2012/01/25 22:12 straka 2012/01/25 19:34 straka 2012/01/25 19:33 straka 2012/01/25 19:33 straka 2012/01/25 19:30 straka 2012/01/25 19:27 straka 2012/01/25 19:09 straka 2012/01/25 19:08 straka 2012/01/25 19:06 straka 2012/01/25 19:03 straka 2012/01/25 19:01 straka 2012/01/25 18:37 straka 2012/01/25 15:46 straka vytvořeno Go Next revision		Previous revision 2012/01/31 09:38 straka Change Perl commandline syntax.2012/01/30 10:28 majlis 2012/01/30 00:33 straka 2012/01/28 16:44 majlis Commands for execution were added.2012/01/28 16:00 majlis Added links to previous and next chapter.2012/01/28 12:32 majlis 2012/01/25 22:12 straka 2012/01/25 19:34 straka 2012/01/25 19:33 straka 2012/01/25 19:33 straka 2012/01/25 19:30 straka 2012/01/25 19:27 straka 2012/01/25 19:09 straka 2012/01/25 19:08 straka 2012/01/25 19:06 straka 2012/01/25 19:03 straka 2012/01/25 19:01 straka 2012/01/25 18:37 straka 2012/01/25 15:46 straka vytvořeno Go Next revision Both sides next revision
courses:mapreduce-tutorial:step-10 [2012/01/25 19:01] straka		courses:mapreduce-tutorial:step-10 [2012/01/25 19:34] straka
Line 11:		Line 11:
	<code perl>		<code perl>
	package Mapper;		package Mapper;
-	use Moose;	+	...
-	with 'Hadoop::Mapper';	+
-		+
-	sub map {	+
-	my ($self, $key, $value, $context) = @_;	+
-		+
-	foreach my $word (split /\W/, $value) {	+
-	next if not length $word;	+
-	$context->write($word, 1);	+
-	}	+
-	}	+
	package Reducer;		package Reducer;
-	use Moose;	+	...
-	with 'Hadoop::Reducer';	+
-		+
-	sub reduce {	+
-	my ($self, $key, $values, $context) = @_;	+
-		+
-	my $sum = 0;	+
-	while ($values->next) {	+
-	$sum += $values->value;	+
-	}	+
-		+
-	$context->write($key, $sum);	+
-	}	+
	package Main;		package Main;
Line 43:		Line 21:
	my $runner = Hadoop::Runner->new(		my $runner = Hadoop::Runner->new(
	mapper => Mapper->new(),		mapper => Mapper->new(),
-	combiner => Reducer->new(),	+	combiner => Reducer->new(), # Specify the combiner.
	reducer => Reducer->new(),		reducer => Reducer->new(),
	input_format => 'KeyValueTextInputFormat');		input_format => 'KeyValueTextInputFormat');
		+	...
		+	</code>
-	$runner->run();	+	===== Excersise =====
-	</code>	+
		+	Compare the effect of adding the combiner to a MR job which counts occurences of words of ''/home/straka/wiki/cs-text-medium'': {{:courses:mapreduce-tutorial:step-5-solution1.txt|wc-without-combiner.pl}} and {{:courses:mapreduce-tutorial:step-10.txt|wc-with-combiner.pl}}.
		+	How would you explain the results?

---

[ Back to the navigation ] [ Back to the content ]