Differences

This shows you the differences between two versions of the page.

--- courses:mapreduce-tutorial:making-your-job-configurable [2012/02/05 19:16]
straka vytvořeno
+++ courses:mapreduce-tutorial:making-your-job-configurable [2012/02/05 21:45] (current)
straka
@@ Line 1: / Line 1: @@
-====== MapReduce Tutorial : ======
+====== MapReduce Tutorial : Making your job configurable ======
+Sometimes it is desirable for a Hadoop job to be configurable without recompiling/rewriting the source. This can be achieved:
+  * Java: use [[.:step-9|Hadoop properties]]:
+    - when running the job, use ''/net/projects/hadoop/bin/hadoop job.jar -Dname1=value1 -Dname2=value2 ... input output''.
+    - in the job, use ''job.getConfiguration.get("name", default)'' to get the ''value'' as ''String'', or use one of ''getInt, getLong, getFloat, getRange, getFile, getStrings, ...''.
+  * Perl: use environment variables:
+    - when constructing [[.:perl-api#hadooprunner|Hadoop::Runner]], use ''copy_environment %%=>%% ['VARIABLE1', 'VARIABLE2', ... ]''.
+    - run the job using ''VARIABLE1=value1 VARIABLE2=value2 ... perl script.pl input output''.
+    - in the job, use ''$ENV{VARIABLE1}'' to access the value.

Institute of Formal and Applied Linguistics Wiki