Both sides previous revision
Previous revision
|
Next revision
Both sides next revision
|
courses:mapreduce-tutorial:step-31 [2012/02/06 08:26] straka |
courses:mapreduce-tutorial:step-31 [2012/02/06 08:28] straka |
| |
To run on a cluster with //C// machines using //C// mappers: | To run on a cluster with //C// machines using //C// mappers: |
rm -rf step-31-out; /net/projects/hadoop/bin/hadoop -c C `/net/projects/hadoop/bin/compute-splitsize /net/projects/hadoop/examples/inputs/numbers-small C` Sum.jar /net/projects/hadoop/examples/inputs/numbers-small step-31-out | rm -rf step-31-out; /net/projects/hadoop/bin/hadoop Sum.jar -c C `/net/projects/hadoop/bin/compute-splitsize /net/projects/hadoop/examples/inputs/numbers-small C` /net/projects/hadoop/examples/inputs/numbers-small step-31-out |
less step-31-out/part-* | less step-31-out/part-* |
| |
# $EDITOR Statistics.java | # $EDITOR Statistics.java |
make -f /net/projects/hadoop/java/Makefile Statistics.java | make -f /net/projects/hadoop/java/Makefile Statistics.java |
rm -rf step-31-out; /net/projects/hadoop/bin/hadoop Statistics.jar | rm -rf step-31-out; /net/projects/hadoop/bin/hadoop Statistics.jar -c C `/net/projects/hadoop/bin/compute-splitsize /net/projects/hadoop/examples/inputs/numbers-small C` /net/projects/hadoop/examples/inputs/numbers-small step-31-out |
less step-31-out/part-* | less step-31-out/part-* |
| |
| ===== Exercise 2 ===== |
| |
| Implement an AllReduce job on ''/net/projects/hadoop/examples/inputs/numbers-small'', which computes |
| |
| You can download the template {{:courses:mapreduce-tutorial:step-31-exercise2.txt|Median.java}} and execute it using: |
| wget --no-check-certificate 'https://wiki.ufal.ms.mff.cuni.cz/_media/courses:mapreduce-tutorial:step-31-exercise2.txt' -O Median.java |
| # NOW VIEW THE FILE |
| # $EDITOR Median.java |
| make -f /net/projects/hadoop/java/Makefile Median.java |
| rm -rf step-31-out; /net/projects/hadoop/bin/hadoop Median.jar -c C `/net/projects/hadoop/bin/compute-splitsize /net/projects/hadoop/examples/inputs/numbers-small C` /net/projects/hadoop/examples/inputs/numbers-small step-31-out |
| less step-31-out/part-* |
| |
| Solution: {{:courses:mapreduce-tutorial:step-31-solution2.txt|Median.java}}. |
| |
| ===== Exercise 3 ===== |
| |
| Implement an AllReduce job on ''/net/projects/hadoop/examples/inputs/numbers-small'', which computes |
| |
| You can download the template {{:courses:mapreduce-tutorial:step-31-exercise3.txt|Median.java}} and execute it using: |
| wget --no-check-certificate 'https://wiki.ufal.ms.mff.cuni.cz/_media/courses:mapreduce-tutorial:step-31-exercise3.txt' -O KMeans.java.java |
| # NOW VIEW THE FILE |
| # $EDITOR KMeans.java.java |
| make -f /net/projects/hadoop/java/Makefile KMeans.java.java |
| rm -rf step-31-out; /net/projects/hadoop/bin/hadoop KMeans.java.jar -c C `/net/projects/hadoop/bin/compute-splitsize /net/projects/hadoop/examples/inputs/numbers-small C` /net/projects/hadoop/examples/inputs/numbers-small step-31-out |
| less step-31-out/part-* |
| |
| Solution: {{:courses:mapreduce-tutorial:step-31-solution3.txt|KMeans.java}}. |
| |