• Part 1: Monday January 30, 14:00-17:00, lab SU2
• Part 2: Tuesday January 31, 14:00-17:00, lab SU2

• Introduction

Today we will be using the Perl API (there is no need to study it now, the tutorial will explain it).

• Step 1: Setting the environment.

• Step 2: Input and output format, testing data.
• Step 3: Basic mapper.
• Step 4: Counters.
• Step 5: Basic reducer.

• Step 6: Running on cluster.
• Step 7: Dynamic Hadoop cluster for several computations.

From now on, it is best to run MR jobs using a one-machine cluster. Running the scripts locally without any cluster has several disadvantages, most notably having only one reducer per job.

• Step 8: Multiple mappers, reducers and partitioning.
• Step 9: Hadoop properties.
• Step 10: Combiners.
• Step 11: Initialization and cleanup of MR tasks, performance of combiners.
• Step 12: Additional output from mappers and reducers.

Exercises in this section can be made in any order, but it is recommended to try solving all of them. The Perl API reference may come handy.

• Step 13: Sorting
• Step 14: N-gram language model
• Step 15: K-means clustering

Today we will be using the Java API.

• Step 21: Preparing the environment.
• Step 22: Optional – Setting Eclipse.

• Step 23: Predefined formats and types.
• Step 24: Mappers, running Java Hadoop jobs.
• Step 25: Reducers, combiners and partitioners.
• Step 26: Counters, compression.
• Step 27: Reusing Mapper and Reducer code.

• Custom input format – WholeFile and WholeFileAsPath
• Custom data type – Pair<A, B>

• Further information