Both sides previous revision
Previous revision
Next revision
|
Previous revision
Next revision
Both sides next revision
|
courses:mapreduce-tutorial [2012/01/27 00:45] straka |
courses:mapreduce-tutorial [2012/01/27 23:31] straka |
| |
=== MapReduce extended === | === MapReduce extended === |
From now on, it is best to run MR jobs using a one-machine cluster. Running the scripts locally without any cluster has several disadvantages, most notably having only one reducer per job. | From now on, it is best to run MR jobs using a one-machine cluster -- create a one-machine cluster using ''hadoop-cluster'' for 3h (10800s) and run jobs using ''-jt cluster_master''. Running the scripts locally without any cluster has several disadvantages, most notably having only one reducer per job. |
* [[.:mapreduce-tutorial:Step 8]]: Multiple mappers, reducers and partitioning. | * [[.:mapreduce-tutorial:Step 8]]: Multiple mappers, reducers and partitioning. |
* [[.:mapreduce-tutorial:Step 9]]: Hadoop properties. | * [[.:mapreduce-tutorial:Step 9]]: Hadoop properties. |
* [[.:mapreduce-tutorial:Step 24]]: Mappers, running Java Hadoop jobs. | * [[.:mapreduce-tutorial:Step 24]]: Mappers, running Java Hadoop jobs. |
* [[.:mapreduce-tutorial:Step 25]]: Reducers, combiners and partitioners. | * [[.:mapreduce-tutorial:Step 25]]: Reducers, combiners and partitioners. |
* [[.:mapreduce-tutorial:Step 26]]: Counters, compression. | * [[.:mapreduce-tutorial:Step 26]]: Job configuration, counters and job context. |
* [[.:mapreduce-tutorial:Step 27]]: Reusing Mapper and Reducer code. | * [[.:mapreduce-tutorial:Step 27]]: Reusing Mapper and Reducer code. |
| |