[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki

• ÚFAL Wiki Home
• ÚFAL Home
• Sitemap
• RSS
• ÚFAL calendar
• Recent changes
• Login
• Hledat:

[ Back to the navigation ]

Differences

This shows you the differences between two versions of the page.

View differences: Side by SideInline Go

Link to this comparison view

2013/02/08 14:36 popel Milan improved our Hadoop2012/01/31 12:43 straka 2012/01/31 09:41 straka Change Perl commandline syntax.2012/01/29 23:52 majlis 2012/01/29 23:51 majlis 2012/01/29 20:52 straka 2012/01/29 20:52 straka 2012/01/29 20:51 straka 2012/01/28 15:33 majlis 2012/01/26 23:11 straka 2012/01/25 22:01 straka 2012/01/25 00:51 straka 2012/01/25 00:51 straka 2012/01/25 00:48 straka 2012/01/25 00:45 straka 2012/01/24 19:05 straka vytvořeno Go Next revision		Previous revision 2013/02/08 14:36 popel Milan improved our Hadoop2012/01/31 12:43 straka 2012/01/31 09:41 straka Change Perl commandline syntax.2012/01/29 23:52 majlis 2012/01/29 23:51 majlis 2012/01/29 20:52 straka 2012/01/29 20:52 straka 2012/01/29 20:51 straka 2012/01/28 15:33 majlis 2012/01/26 23:11 straka 2012/01/25 22:01 straka 2012/01/25 00:51 straka 2012/01/25 00:51 straka 2012/01/25 00:48 straka 2012/01/25 00:45 straka 2012/01/24 19:05 straka vytvořeno Go Next revision Both sides next revision
courses:mapreduce-tutorial:step-7 [2012/01/24 19:05] straka vytvořeno		courses:mapreduce-tutorial:step-7 [2012/01/25 00:48] straka
Line 1:		Line 1:
-	====== MapReduce Tutorial : ======	+	====== MapReduce Tutorial : Dynamic Hadoop cluster for several computations ======
		+
		+	When multiple MR jobs should be executed, it would be better to reuse the cluster instead of allocating a new one for every computation.
		+
		+	A cluster can be created using
		+	/home/straka/hadoop/bin/hadoop-cluster -c number_of_machines -w sec_to_run_the_cluster_for
		+	The syntax is the same as in ''perl script.pl run''.
		+
		+	The associated SGE job name is HadoopCluster. The running job can be stopped by either removing ''HadoopCluster.c$SGE_JOBID'' file or deleting the SGE job using ''qdel''.
		+
		+	===== Using a running cluster =====
		+	Running cluster is identified by its master. When running a Perl MR job, existing cluster can be used by
		+	perl script.pl run -jt hostname_of_cluster_master:9001 ...

---

[ Back to the navigation ] [ Back to the content ]