Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision | Next revision Both sides next revision | ||
|
courses:mapreduce-tutorial:step-6 [2012/01/30 13:46] straka |
courses:mapreduce-tutorial:step-6 [2012/01/31 09:41] straka Change Perl commandline syntax. |
||
|---|---|---|---|
| Line 4: | Line 4: | ||
| So far all our Hadoop jobs were executed locally. But all of them can be executed on multiple machines. It suffices to add parameter '' | So far all our Hadoop jobs were executed locally. But all of them can be executed on multiple machines. It suffices to add parameter '' | ||
| - | perl script.pl | + | perl script.pl -c number_of_machines [-w sec_to_wait_after_job_completion] input_directory output_directory |
| This commands creates a cluster of specified number of machines. Every machine is able to run two mappers and two reducers simultaneously. In order to be able to observe the counters, status and error logs of the computation after it ends, parameter '' | This commands creates a cluster of specified number of machines. Every machine is able to run two mappers and two reducers simultaneously. In order to be able to observe the counters, status and error logs of the computation after it ends, parameter '' | ||
| Line 35: | Line 35: | ||
| Try running the {{: | Try running the {{: | ||
| wget --no-check-certificate ' | wget --no-check-certificate ' | ||
| - | rm -rf step-6-out; perl step-6-wordcount.pl | + | rm -rf step-6-out; perl step-6-wordcount.pl -c 1 -w 600 -Dmapred.max.split.size=1000000 / |
| and explore the web interface. | and explore the web interface. | ||