[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki


[ Back to the navigation ]

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
grid [2019/04/17 09:24]
vodrazka fixed broken links
grid [2020/01/27 20:31]
popel link http://aic.ufal.mff.cuni.cz
Line 190: Line 190:
  
 ''​qsub **-b** y''​ ''​qsub **-b** y''​
-Treat ''​script.sh''​ (or whatever is the name of the command you execute) as a binary, i.e. don't search for [[#​in-script options]] within the file, don't transfer it to the qmaster and then to the execution node. This makes the execution a bit faster and it may prevent some rare but hard-to-detect errors caused SGE interpreting the script. The script must be available on the execution node via NFS, Lustre (which is our case), etc. With ''​-b y''​ (shortcut for ''​-b yes''​),​ ''​script.sh''​ can be script or a binary. With ''​-b n''​ (which is the default for ''​qsub''​),​ ''​script.sh''​ must be a script (text file).+Treat ''​script.sh''​ (or whatever is the name of the command you execute) as a binary, i.e. don't search for [[#​in-script options]] within the file, don't transfer it to the qmaster and then to the execution node. This makes the execution a bit faster and it may prevent some rare but hard-to-detect errors caused SGE interpreting the script. The script must be available on the execution node via NFS, Lustre (which is our case), etc. With ''​-b y''​ (shortcut for ''​-b yes''​),​ ''​script.sh''​ can be an executable ​script or a binary ​(and you must provide full path, e.g. ''​./​script.sh''​). With ''​-b n''​ (which is the default for ''​qsub''​),​ ''​script.sh''​ must be a script (text file).
  
 ''​qsub **-M** popel@ufal.mff.cuni.cz,​rosa@ufal.mff.cuni.cz **-m** beas''​ ''​qsub **-M** popel@ufal.mff.cuni.cz,​rosa@ufal.mff.cuni.cz **-m** beas''​
Line 329: Line 329:
 ===== Other ===== ===== Other =====
   * There is a **great course [[http://​ufal.mff.cuni.cz/​courses/​npfl102|Data intensive computing]]**,​ see the 2016 handouts if you missed the course. It covers the usage of [[http://​spark.apache.org/​|Spark]] (MapReduce/​Hadoop alternative,​ but better) and HDFS (Hadoop filesystem).   * There is a **great course [[http://​ufal.mff.cuni.cz/​courses/​npfl102|Data intensive computing]]**,​ see the 2016 handouts if you missed the course. It covers the usage of [[http://​spark.apache.org/​|Spark]] (MapReduce/​Hadoop alternative,​ but better) and HDFS (Hadoop filesystem).
-  * This course had used a special **DLRC (Demo LRC) cluster** (students had to login with ''​ssh -p 11422 ufallab.ms.mff.cuni.cz''​ and special NPFL102-only LDAP logins) with six virtual machines on one physical. During the years when NPFL102 is not taught (e.g. 2017), the DLRC cluster has just one virtual machine. 
   * **Note:** soma hadoop basics and a lot of NoSQL technologies are covered by [[https://​is.cuni.cz/​studium/​predmety/​index.php?​do=predmet&​kod=NDBI040|Big Data Management and NoSQL Databases]]   * **Note:** soma hadoop basics and a lot of NoSQL technologies are covered by [[https://​is.cuni.cz/​studium/​predmety/​index.php?​do=predmet&​kod=NDBI040|Big Data Management and NoSQL Databases]]
 +  * There is a special cluster for Mgr (and Bc) students (but not for PhD and UFAL members): http://​aic.ufal.mff.cuni.cz/​
   * You can use environment variables ''​$JOB_ID'',​ ''​$JOB_NAME''​.   * You can use environment variables ''​$JOB_ID'',​ ''​$JOB_NAME''​.
   * One job can submit other jobs (but be careful with recursive:​-)). A job submitted to the CPU cluster may submit GPU jobs (to the ''​qpu.q''​ queue).   * One job can submit other jobs (but be careful with recursive:​-)). A job submitted to the CPU cluster may submit GPU jobs (to the ''​qpu.q''​ queue).

[ Back to the navigation ] [ Back to the content ]