Differences

This shows you the differences between two versions of the page.

--- courses:mapreduce-tutorial:step-7 [2012/01/25 00:45]
straka
+++ courses:mapreduce-tutorial:step-7 [2012/01/25 00:51]
straka
@@ Line 4: / Line 4: @@
 A cluster can be created using
-  /home/straka/hadoop/bin/hadoop-cluster -c number_of_machines [-w sec_to_run_the_cluster_for]
+  /home/straka/hadoop/bin/hadoop-cluster -c number_of_machines -w sec_to_run_the_cluster_for
 The syntax is the same as in ''perl script.pl run''.
-The associated
+The associated SGE job name is HadoopCluster. The running job can be stopped by either removing ''HadoopCluster.c$SGE_JOBID'' file or deleting the SGE job using ''qdel''.
+===== Using a running cluster =====
+Running cluster is identified by its master. When running a Perl MR job, existing cluster can be used by
+  perl script.pl run -jt hostname_of_cluster_master:9001 ...
+===== Example =====
+Try running the same script {{:courses:mapreduce-tutorial:step-6.txt|wordcount.pl}} as in the last step, by creating the cluster and submitting the job to it:
+  /home/straka/hadoop/bin/hadoop-cluster -c 1 -w 7200
+  perl wordcount.pl -jt hostname_of_cluster_master:9001 -Dmapred.max.split.size=1000000 /home/straka/wiki/cs-text-medium some_output_directory

Institute of Formal and Applied Linguistics Wiki