This is an old revision of the document!
Table of Contents
MapReduce Tutorial : Counters and job configuration
Counters
As in the Perl API, a mapper or a reducer can increment various counters by using context.getCounter(“Group”, “Name”).increment(value)
:
public void map(Text key, Text value, Context context) throws IOException, InterruptedException { ... context.getCounter("Group", "Name").increment(value); ... }
The getCounter
method returns a Counter object, so if a counter is incremented frequently, the getCounter
method can be called only once:
public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException { ... Counter values = context.getCounter("Reducer", "Number of values"); for (IntWritable value : values) { ... values.increment(1); } }
Job configuration
The job properties can be set:
- on the command line – the ToolRunner parses options in format
-Dname=value
- using the Job object.
See the brief list of Hadoop properties.
There is one