| Both sides previous revision
Previous revision
Next revision
|
Previous revision
Last revision
Both sides next revision
|
courses:mapreduce-tutorial:perl-api [2012/01/25 22:05]
straka |
courses:mapreduce-tutorial:perl-api [2012/01/27 17:48]
straka |
| |
| has 'hadoop_prefix' => (isa => 'Str', default => '/SGE/HADOOP/active'); | has 'hadoop_prefix' => (isa => 'Str', default => '/SGE/HADOOP/active'); |
| has 'keep_env' => (isa => 'ArrayRef[Str]', default => sub { ["PERLLIB", "PERL5LIB"] }); | has 'copy_environment' => (isa => 'ArrayRef[Str]', default => sub { [] }); |
| |
| sub run(); | sub run(); |
| * ''output_compression'' -- Bool flag controlling the compression of output | * ''output_compression'' -- Bool flag controlling the compression of output |
| * ''hadoop_prefix'' -- the prefix of Hadoop instalation. Default value is fine in UFAL cluster. | * ''hadoop_prefix'' -- the prefix of Hadoop instalation. Default value is fine in UFAL cluster. |
| * ''keep_env'' -- which environment variables are preserved when running perl mappers, reducers, combiners and partitioners | * ''copy_environment'' -- which environment variables are preserved when running perl mappers, reducers, combiners and partitioners. Needed only when running job using ''-jt'' -- both local execution and execution using ''-c'' option retain all environmental variables. |
| |
| ==== Command line arguments supported by Hadoop::Runner::run() ==== | ==== Command line arguments supported by Hadoop::Runner::run() ==== |
| sub cleanup {} | sub cleanup {} |
| </file> | </file> |
| * ''sub map($self, $key, $value, $context)'' -- executed for every (key, value) input pair. The variable '$content' has following methods: | * ''sub map($self, $key, $value, $context)'' -- executed for every (key, value) input pair. The variable ''$content'' has following methods: |
| * ''$content%%->%%write($key, $value)'' -- output the (''$key'', ''$value'') pair | * ''$content%%->%%write($key, $value)'' -- output the (''$key'', ''$value'') pair |
| * ''$content%%->%%counter($group, $name, $increment)'' -- increases the counter ''$name'' in the group ''$group'' by ''$increment'' | * ''$content%%->%%counter($group, $name, $increment)'' -- increases the counter ''$name'' in the group ''$group'' by ''$increment'' |
| * ''$values%%->%%value()'' -- returns the current value, undef if there is any. | * ''$values%%->%%value()'' -- returns the current value, undef if there is any. |
| * ''$values%%->%%next()'' -- advance to next value. Returns true if there is any, false otherwise. | * ''$values%%->%%next()'' -- advance to next value. Returns true if there is any, false otherwise. |
| * At the beginning there is no current value, the first value should be obtained by calling 'next'. | * At the beginning there is no current value, the first value should be obtained by calling ''next''. |
| * ''sub reduce($self, $key, $values, $context)'' -- the variable ''$content'' has following methods: | * ''sub reduce($self, $key, $values, $context)'' -- the variable ''$content'' has following methods: |
| * ''$content%%->%%write($key, $value)'' -- output the (''$key'', ''$value'') pair | * ''$content%%->%%write($key, $value)'' -- output the (''$key'', ''$value'') pair |
