Differences

This shows you the differences between two versions of the page.

--- courses:mapreduce-tutorial:making-your-job-configurable [2012/02/05 21:40]
straka
+++ courses:mapreduce-tutorial:making-your-job-configurable [2012/02/05 21:45] (current)
straka
@@ Line 2: / Line 2: @@
 Sometimes it is desirable for a Hadoop job to be configurable without recompiling/rewriting the source. This can be achieved:
-  * both Perl and Java: use environment variables -- all environment variables are shared by the Hadoop job
+  * Java: use [[.:step-9|Hadoop properties]]:
-  * Java only: use [[.:step-9|Hadoop properties]]:
+    - when running the job, use ''/net/projects/hadoop/bin/hadoop job.jar -Dname1=value1 -Dname2=value2 ... input output''.
-    - when running the job, use ''/net/projects/hadoop/bin/hadoop job.jar -Dname1=value1 -Dname2=value2 ... input output''
+    - in the job, use ''job.getConfiguration.get("name", default)'' to get the ''value'' as ''String'', or use one of ''getInt, getLong, getFloat, getRange, getFile, getStrings, ...''.
-    - in the job, use ''job.getConfiguration.get("name", default)'' to get the ''value'' as ''String'', or use one of ''getInt, getLong, getFloat, getRange, getFile, getStrings, ...''
+  * Perl: use environment variables:
+    - when constructing [[.:perl-api#hadooprunner|Hadoop::Runner]], use ''copy_environment %%=>%% ['VARIABLE1', 'VARIABLE2', ... ]''.
+    - run the job using ''VARIABLE1=value1 VARIABLE2=value2 ... perl script.pl input output''.
+    - in the job, use ''$ENV{VARIABLE1}'' to access the value.

[ Back to the navigation ] [ Back to the content ]

Institute of Formal and Applied Linguistics Wiki

Differences