[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki

• ÚFAL Wiki Home
• ÚFAL Home
• Sitemap
• RSS
• ÚFAL calendar
• Recent changes
• Login
• Hledat:

[ Back to the navigation ]

Differences

This shows you the differences between two versions of the page.

View differences: Side by SideInline Go

Link to this comparison view

Both sides previous revision Previous revision 2012/01/30 15:25 straka 2012/01/30 14:47 straka 2012/01/29 15:57 straka 2012/01/29 15:55 straka 2012/01/29 15:39 straka 2012/01/29 15:38 straka 2012/01/29 15:38 straka 2012/01/28 10:25 majlis 2012/01/27 17:08 majlis 2012/01/27 17:06 majlis 2012/01/26 23:07 straka 2012/01/26 23:07 straka 2012/01/24 20:06 straka 2012/01/24 20:04 straka 2012/01/24 19:24 straka 2012/01/24 19:03 straka 2012/01/24 08:56 straka 2012/01/23 20:56 straka 2012/01/23 20:56 straka 2012/01/23 20:55 straka vytvořeno Go Next revision		Previous revision 2012/01/30 15:25 straka 2012/01/30 14:47 straka 2012/01/29 15:57 straka 2012/01/29 15:55 straka 2012/01/29 15:39 straka 2012/01/29 15:38 straka 2012/01/29 15:38 straka 2012/01/28 10:25 majlis 2012/01/27 17:08 majlis 2012/01/27 17:06 majlis 2012/01/26 23:07 straka 2012/01/26 23:07 straka 2012/01/24 20:06 straka 2012/01/24 20:04 straka 2012/01/24 19:24 straka 2012/01/24 19:03 straka 2012/01/24 08:56 straka 2012/01/23 20:56 straka 2012/01/23 20:56 straka 2012/01/23 20:55 straka vytvořeno Go
courses:mapreduce-tutorial:step-1 [2012/01/29 15:38] straka		courses:mapreduce-tutorial:step-1 [2012/01/30 15:25] (current) straka
Line 1:		Line 1:
	====== MapReduce Tutorial : Setting the environment ======		====== MapReduce Tutorial : Setting the environment ======
-	===== Hadoop instalation =====	+	===== Requirements =====
-	===== The Perl API =====	+	The tutorial expects you to be logged to a computer in the UFAL cluster and be able to submit jobs using SGE. In this environment, Hadoop is installed in ''/SGE/HADOOP/active''.
	To use the Perl MapReduce API, you need		To use the Perl MapReduce API, you need
Line 19:		Line 19:
-	===== The Hadoop package =====	+	==== The Hadoop package ====
	The custom Hadoop package is available in ''/net/projects/hadoop/perl'', just add		The custom Hadoop package is available in ''/net/projects/hadoop/perl'', just add
	export PERLLIB="$PERLLIB:/net/projects/hadoop/perl/"		export PERLLIB="$PERLLIB:/net/projects/hadoop/perl/"
	export PERL5LIB="$PERL5LIB:/net/projects/hadoop/perl"		export PERL5LIB="$PERL5LIB:/net/projects/hadoop/perl"
	to ''.profile'', ''.bash_profile'', ''.bashrc'' or type it in the shell.		to ''.profile'', ''.bash_profile'', ''.bashrc'' or type it in the shell.
-
	echo -e "\n#MR Tutorial - Hadoop" >> ~/.bashrc		echo -e "\n#MR Tutorial - Hadoop" >> ~/.bashrc
	echo 'export PERLLIB="$PERLLIB:/net/projects/hadoop/perl/"' >> ~/.bashrc		echo 'export PERLLIB="$PERLLIB:/net/projects/hadoop/perl/"' >> ~/.bashrc
	echo 'export PERL5LIB="$PERL5LIB:/net/projects/hadoop/perl"' >> ~/.bashrc		echo 'export PERL5LIB="$PERL5LIB:/net/projects/hadoop/perl"' >> ~/.bashrc
		+
		+	===== When not logged in UFAL cluster =====
		+
		+	**If you are not logged in the UFAL cluster, you will need:**
		+	* local Hadoop installation
		+	- download ''http://www.apache.org/dist/hadoop/common/hadoop-1.0.0/hadoop-1.0.0.tar.gz''
		+	- unpack it
		+	- edit ''conf/hadoop-env.sh'' file and make sure there is valid line <code>export JAVA_HOME=/path/to/your/jdk</code>
		+	* the repository ''hadoop'' containing the Perl API and Java extensions.
		+	* when using Perl API, set ''hadoop_prefix'' to point to your Hadoop installation
		+	* when using Java API, one of the ''Makefile''s contain absolute path to the ''hadoop'' repository -- please correct it
		+	When using local Hadoop installation, you must run all jobs either locally in a single thread or start a local cluster and use ''-jt'' for the jobs to use it (see [[.:step-7#using-a-running-cluster]]).
	----		----

---

[ Back to the navigation ] [ Back to the content ]