[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki


[ Back to the navigation ]

This is an old revision of the document!


Table of Contents

MapReduce Tutorial : Counters, compression and job configuration

Counters

As in the Perl API, a mapper or a reducer can increment various counters by using context.getCounter(“Group”, “Name”).increment(value):

public void map(Text key, Text value, Context context) throws IOException, InterruptedException {
  ...
  context.getCounter("Group", "Name").increment(value);
  ...
}

The getCounter method returns a Counter object, so if a counter is incremented frequently, the getCounter method can be called only once:

public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException {
  ...
  Counter values = context.getCounter("Reducer", "Number of values");
  for (IntWritable value : values) {
    ...
    values.increment(1);
  }
}

Compression

The output files can be compressed using

  FileOutputFormat.setCompressOutput(job, true);

Job configuration

The job properties can be set:

Apart from already mentioned brief list of Hadoop properties, there is one important Java-specific property:


[ Back to the navigation ] [ Back to the content ]