[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki


[ Back to the navigation ]

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
courses:mapreduce-tutorial:if-things-go-wrong [2012/02/06 13:51]
straka
courses:mapreduce-tutorial:if-things-go-wrong [2012/02/06 13:55] (current)
straka
Line 1: Line 1:
 ====== MapReduce Tutorial : If things go wrong ====== ====== MapReduce Tutorial : If things go wrong ======
  
-A lot can go wrong in the process of submitting ​and running ​Hadoop job:+A lot can go wrong in the process of creating cluster ​and submitting the Hadoop job:
   * ''​Hadoop::​Runner.pm''​ module not found: The Perl Hadoop package is not configured, see [[.:​step-1|Setting the environment]].   * ''​Hadoop::​Runner.pm''​ module not found: The Perl Hadoop package is not configured, see [[.:​step-1|Setting the environment]].
   * ''​ipc.Client:​ Retrying connect to server: IP_ADDRESS:​PORT. Already tried ? time(s)'':​ The jobtracker cannot be contacted. If using ''​-jt jobtracker:​port''​ flag, check, that the jobtracker address is correct.   * ''​ipc.Client:​ Retrying connect to server: IP_ADDRESS:​PORT. Already tried ? time(s)'':​ The jobtracker cannot be contacted. If using ''​-jt jobtracker:​port''​ flag, check, that the jobtracker address is correct.
   * ''/​net/​projects/​hadoop/​bin/​hadoop-cluster''​ fails to start a cluster: Look where the jobtracker was scheduled by SGE using ''​qstat''​. Login to that machine and investigate logs in ''/​var/​log/​hadoop/​$USER/​$SGE_JOBID/''​.   * ''/​net/​projects/​hadoop/​bin/​hadoop-cluster''​ fails to start a cluster: Look where the jobtracker was scheduled by SGE using ''​qstat''​. Login to that machine and investigate logs in ''/​var/​log/​hadoop/​$USER/​$SGE_JOBID/''​.
 +
 +If the cluster works, but your job crashes, you can:
 +  * run the computation locally in single threaded mode (i.e., without specifying ''​-c''​ and ''​-jt''​ flag). This is more useful for Hadoop jobs written in Java, because then you can use a debugger. When using Perl API, new subprocess are created for Perl tasks anyway.
 +  * use the web interface to see Java exceptions / Perl return codes for each failed task attempt.
 +  * use standard error output for log messages. You can access the stderr logs of all Hadoop tasks using the web interface.

[ Back to the navigation ] [ Back to the content ]