[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki

• ÚFAL Wiki Home
• ÚFAL Home
• Sitemap
• RSS
• ÚFAL calendar
• Recent changes
• Login
• Hledat:

[ Back to the navigation ]

Differences

This shows you the differences between two versions of the page.

View differences: Side by SideInline Go

Link to this comparison view

Both sides previous revision Previous revision 2012/01/31 15:55 straka 2012/01/31 09:37 straka Change Perl commandline syntax.2012/01/31 09:36 straka The number of reducers must be specified in the exercise.2012/01/30 00:38 straka Improving package names of Perl programs.2012/01/29 23:54 majlis 2012/01/29 21:12 straka 2012/01/29 21:04 straka 2012/01/29 21:01 straka 2012/01/29 20:58 straka 2012/01/28 17:40 straka 2012/01/28 15:55 majlis Added links to previous and next chapter.2012/01/28 15:53 majlis Commands for execution were added.2012/01/28 15:36 majlis 2012/01/25 22:09 straka 2012/01/25 18:37 straka 2012/01/25 15:54 straka 2012/01/25 15:54 straka 2012/01/25 15:29 straka 2012/01/25 15:17 straka 2012/01/25 15:16 straka 2012/01/25 15:00 straka 2012/01/25 14:54 straka 2012/01/25 14:47 straka 2012/01/25 14:46 straka vytvořeno Go		2012/01/31 15:55 straka 2012/01/31 09:37 straka Change Perl commandline syntax.2012/01/31 09:36 straka The number of reducers must be specified in the exercise.2012/01/30 00:38 straka Improving package names of Perl programs.2012/01/29 23:54 majlis 2012/01/29 21:12 straka 2012/01/29 21:04 straka 2012/01/29 21:01 straka 2012/01/29 20:58 straka 2012/01/28 17:40 straka 2012/01/28 15:55 majlis Added links to previous and next chapter.2012/01/28 15:53 majlis Commands for execution were added.2012/01/28 15:36 majlis 2012/01/25 22:09 straka 2012/01/25 18:37 straka 2012/01/25 15:54 straka 2012/01/25 15:54 straka 2012/01/25 15:29 straka 2012/01/25 15:17 straka 2012/01/25 15:16 straka 2012/01/25 15:00 straka 2012/01/25 14:54 straka 2012/01/25 14:47 straka 2012/01/25 14:46 straka vytvořeno Go Next revision Both sides next revision
courses:mapreduce-tutorial:step-8 [2012/01/25 14:54] straka		courses:mapreduce-tutorial:step-8 [2012/01/25 15:00] straka
Line 9:		Line 9:
	===== Multiple reducers =====		===== Multiple reducers =====
		+	Then number of reducers is specified by the job, default number is one. As the outputs of reducers are not merged, there are as many output files as reducers.
		+	To use multiple reducers, the MR job must be executed by a cluster (even with one computer), not locally. The number of reducers is specified by ''-r'' flag:
		+	perl script.pl [-j cluster_master | -c cluster_size [-w sec_to_wait]] [-r number_of_reducers]
		+
		+	==== Partitioning ====
		+	When there are multiple reducers, it is important how the (key, value) pairs are distributed between the reducers.
		+
		+	By default, (key, value) pair is sent to reducer number //hash(key) modulo number_of_reducers//. This guarantees that for one key, all its values are processed by unique reducer.
		+
		+	To override the default behaviour, MR job can specify a //partitioner//.

---

[ Back to the navigation ] [ Back to the content ]