Differences
This shows you the differences between two versions of the page.
Next revision | Previous revision Next revision Both sides next revision | ||
courses:mapreduce-tutorial:step-7 [2012/01/24 19:05] straka vytvořeno |
courses:mapreduce-tutorial:step-7 [2012/01/25 22:01] straka |
||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== MapReduce Tutorial : ====== | + | ====== MapReduce Tutorial : Dynamic Hadoop cluster for several computations |
+ | |||
+ | When multiple Hadoop jobs should be executed, it is better to reuse the cluster instead of allocating a new one for every computation. | ||
+ | |||
+ | A cluster can be created using | ||
+ | / | ||
+ | The syntax is the same as in '' | ||
+ | |||
+ | The associated SGE job name is HadoopCluster. The running job can be stopped by either removing '' | ||
+ | |||
+ | ===== Using a running cluster ===== | ||
+ | Running cluster is identified by its master. When running a Hadoop job using Perl API, existing cluster can be used by | ||
+ | perl script.pl run -jt cluster_master: | ||
+ | |||
+ | ===== Example ===== | ||
+ | |||
+ | Try running the same script {{: | ||
+ | / | ||
+ | perl wordcount.pl run -jt cluster_master: |