Differences
This shows you the differences between two versions of the page.
| Both sides previous revision
Previous revision
|
Next revision
Both sides next revision
|
courses:mapreduce-tutorial:step-7 [2012/01/26 23:11]
straka |
courses:mapreduce-tutorial:step-7 [2012/01/28 15:33]
majlis |
| ===== Example ===== | ===== Example ===== |
| |
| Try running the same script {{:courses:mapreduce-tutorial:step-6.txt|wordcount.pl}} as in the last step, this time by creating the cluster and submitting the job to it: | Try running the same script {{:courses:mapreduce-tutorial:step-6.txt|step-7-wordcount.pl}} as in the last step, this time by creating the cluster and submitting the job to it: |
| | wget --no-check-certificate 'https://wiki.ufal.ms.mff.cuni.cz/_media/courses:mapreduce-tutorial:step-6.txt' -O 'step-7-wordcount.pl' |
| /net/projects/hadoop/bin/hadoop-cluster -c 1 -w 600 | /net/projects/hadoop/bin/hadoop-cluster -c 1 -w 600 |
| perl wordcount.pl run -jt cluster_master:9001 -Dmapred.max.split.size=1000000 /home/straka/wiki/cs-text-medium some_output_directory | rm -rf step-7-out-sol; perl step-7-wordcount.pl run -jt cluster_master:9001 -Dmapred.max.split.size=1000000 /home/straka/wiki/cs-text-medium step-7-out-sol |
| | less less step-7-out-sol/part-* |
| |
| | |
| | ---- |
| | |
| | <html> |
| | <table style="width:100%"> |
| | <tr> |
| | <td style="text-align:left; width: 33%; "></html>[[step-6|Step 6]]: Running on cluster.<html></td> |
| | <td style="text-align:center; width: 33%; "></html>[[.|Overview]]<html></td> |
| | <td style="text-align:right; width: 33%; "></html>[[step-8|Step 8]]: Multiple mappers, reducers and partitioning.<html></td> |
| | </tr> |
| | </table> |
| | </html> |
