Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
courses:mapreduce-tutorial:step-6 [2012/01/25 00:03] straka |
courses:mapreduce-tutorial:step-6 [2012/01/25 00:28] straka |
||
---|---|---|---|
Line 8: | Line 8: | ||
When a distributed MR computations is executed, it submits a job to SGE cluster, with the name of the Perl script. The SGE cluster creates 3 files in current directory: | When a distributed MR computations is executed, it submits a job to SGE cluster, with the name of the Perl script. The SGE cluster creates 3 files in current directory: | ||
- | * '' | + | * '' |
* '' | * '' | ||
* '' | * '' | ||
When the computation ends and is waiting because of the '' | When the computation ends and is waiting because of the '' | ||
+ | |||
+ | ===== Web interface ===== | ||
+ | |||
+ | The cluster master provides a web interface on port 50030 (the port may change in the future). The cluster master address can be found at the first line of '' | ||
+ | |||
+ | The web interface provides a lot of useful informations: | ||
+ | * running, failed and successfully completed jobs | ||
+ | * for running job, current progress and counters of the whole job and also of each mapper and reducer is available | ||
+ | * for any job, the counters and outputs of all mappers and reducers | ||
+ | * for any job, all Hadoop settings | ||
+ | |||
+ | ===== Example ===== | ||
+ | |||
+ | Try running the {{: | ||
+ | perl wordcount.pl -c 1 -w 300 -Dmapred.max.split.size=1000000 / | ||
+ | and explore the web interface. | ||
+ | |||
+ | If you cannot access directly the '' | ||
+ | ssh -N -L 50030: | ||
+ | to create a tunnel from local port 50030 to machine '' |