Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
courses:mapreduce-tutorial:step-26 [2012/01/28 15:41] straka |
courses:mapreduce-tutorial:step-26 [2012/01/28 20:26] straka |
||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== MapReduce Tutorial : Counters and job configuration ====== | + | ====== MapReduce Tutorial : Counters, compression |
===== Counters ===== | ===== Counters ===== | ||
Line 22: | Line 22: | ||
} | } | ||
</ | </ | ||
+ | |||
+ | ===== Compression ===== | ||
+ | |||
+ | The output files can be compressed using | ||
+ | <code java> | ||
+ | FileOutputFormat.setCompressOutput(job, | ||
+ | </ | ||
+ | | ||
+ | The default compression format is '' | ||
+ | <code java> | ||
+ | import org.apache.hadoop.io.compress.*; | ||
+ | |||
+ | ... | ||
+ | FileOutputFormat.setOutputCompressorClass(GzipCodec.class); | ||
+ | FileOutputFormat.setOutputCompressorClass(BZip2Codec.class); | ||
+ | </ | ||
+ | |||
+ | Of course, any of these formats is decompressed transparently when the file is being read. | ||
===== Job configuration ===== | ===== Job configuration ===== | ||
Line 36: | Line 54: | ||
Apart from already mentioned [[.: | Apart from already mentioned [[.: | ||
+ | * **'' |