Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
grid [2017/09/27 20:45] popel [Rules] |
grid [2017/10/02 16:56] popel [Advanced usage] |
||
---|---|---|---|
Line 70: | Line 70: | ||
# We have used two handy qsub parameters: | # We have used two handy qsub parameters: | ||
# -cwd ... the script is executed in the current directory (the default is your home) | # -cwd ... the script is executed in the current directory (the default is your home) | ||
- | # -j y ... stdout and stderr outputs are merged and redirected to a file ('' | + | # -j y ... stdout and stderr outputs are merged and redirected to a file ('' |
# We have also provided two parameters for our script " | # We have also provided two parameters for our script " | ||
# The qsub prints something like | # The qsub prints something like | ||
Line 120: | Line 120: | ||
* If your job needs more than one CPU (on a single machine) for most of the time, reserve the given number of CPU cores (and SGE slots) with < | * If your job needs more than one CPU (on a single machine) for most of the time, reserve the given number of CPU cores (and SGE slots) with < | ||
* If you are sure your job needs less than 1GB RAM, then you can skip this. Otherwise, if you need e.g. 8 GiB, you must always use '' | * If you are sure your job needs less than 1GB RAM, then you can skip this. Otherwise, if you need e.g. 8 GiB, you must always use '' | ||
- | * Be kind to your colleagues. If you are going to submit jobs that effectively occupy **more than one fifth of our cluster for more than several hours**, check if the cluster is free (with '' | + | * Be kind to your colleagues. If you are going to submit jobs that effectively occupy **more than one fifth of our cluster for more than several hours**, check if the cluster is free (with '' |
Line 135: | Line 135: | ||
===== Advanced usage ===== | ===== Advanced usage ===== | ||
+ | |||
+ | '' | ||
+ | This way your job is submitted to the Troja queue. The default is '' | ||
+ | '' | ||
+ | '' | ||
+ | '' | ||
+ | However, usually you should specify just the queue (troja-all.q vs. ms-all.q), not a particular machine, and instead use '' | ||
+ | |||
+ | '' | ||
+ | See '' | ||
'' | '' | ||
Define a priority of your job as a number between -1024 and 0. Only SGE admins may use a number higher than 0. The default is 0, i.e. the highest possible priority. SGE uses the priority to decide when to start which pending job in the queue (it computes a real number called '' | Define a priority of your job as a number between -1024 and 0. Only SGE admins may use a number higher than 0. The default is 0, i.e. the highest possible priority. SGE uses the priority to decide when to start which pending job in the queue (it computes a real number called '' | ||
+ | |||
'' | '' | ||
- | redirect std{out, | + | redirect std{out, |
'' | '' | ||
Line 174: | Line 185: | ||
'' | '' | ||
By default, all the resource requirements (specified with '' | By default, all the resource requirements (specified with '' | ||
+ | |||
+ | '' | ||
+ | This causes qsub to wait for the job to complete before exiting (with the same exit code as the job). Useful in scripts. | ||
'' | '' | ||
You can change some properties of already submitted jobs, which are still waiting in the queue (// | You can change some properties of already submitted jobs, which are still waiting in the queue (// | ||
- | '' | + | '' |
Find out all the gory details which are missing here. You'll have to do it one day anyway:-). | Find out all the gory details which are missing here. You'll have to do it one day anyway:-). | ||
Line 187: | Line 201: | ||
=== qunhold === | === qunhold === | ||
'' | '' | ||
+ | |||
+ | === sshcwd === | ||
+ | This is useful not only when sshing to sol machines. Add the following lines to your '' | ||
+ | |||
+ | < | ||
+ | function sshcwd () { | ||
+ | # save the current history so that it is available | ||
+ | # immediately on the remote machine | ||
+ | history -a; | ||
+ | # setup the working directory by setting WD | ||
+ | ssh -X -Y -C -t $@ " | ||
+ | } | ||
+ | |||
+ | # use WD to setup the working directory | ||
+ | if [ -n " | ||
+ | echo " | ||
+ | cd $WD; | ||
+ | fi | ||
+ | |||
+ | alias sol1=" | ||
+ | </ | ||
=== In-script options === | === In-script options === | ||
Line 202: | Line 237: | ||
=== Array jobs === | === Array jobs === | ||
- | If you have a set of tasks (of the same type) and want to run them on multiple machines, | + | If you have a set of tasks (of the same type) and want to run them on multiple machines, |
TODO | TODO | ||
+ | |||
===== Monitorování úloh ===== | ===== Monitorování úloh ===== | ||
Line 243: | Line 279: | ||
Pokud chcete submittovaný program pouštět ve svém oblíbeném prostředí (např. nastavení '' | Pokud chcete submittovaný program pouštět ve svém oblíbeném prostředí (např. nastavení '' | ||
- | |||
- | ==== Jiný shell ==== | ||
- | |||
- | Abych mohl poslat nějakou úlohu do fronty, musím pro ni vyrobit vlastní skript. Budiž, vyrobil jsem vlastní skript: | ||
- | |||
- | < | ||
- | #!/bin/bash | ||
- | program > log.out 2> log.err | ||
- | </ | ||
- | |||
- | Když tento skript spustím, stane se očekávané. Přesměrují se výstupy z daného programu do souborů a je to. | ||
- | |||
- | Když takový skript submitnu, program se **nespustí**. V logu zjistím, že (standardní chybový) výstup shellu, který pouštěl můj skript praví kryptickou zprávu " | ||
- | |||
- | Nebudu vás napínat, zde je vysvětlení: | ||
- | |||
- | Takto SGE přinutíte, | ||
- | |||
- | < | ||
- | qsub -S /bin/bash skript | ||
- | </ | ||
- | |||
- | Jinou možností je přesměrovat stderr a stdout pomocí syntaxe csh: | ||
- | |||
- | < | ||
- | ( command > | ||
- | </ | ||
- | |||
==== bashrc a podobné nesmí nic vypisovat na konzoli ==== | ==== bashrc a podobné nesmí nic vypisovat na konzoli ==== | ||
Line 297: | Line 305: | ||
hard resource_list: | hard resource_list: | ||
hard resource_list: | hard resource_list: | ||
- | |||
- | ==== Jak rezervovat více jader na stejném stroji pro 1 job ==== | ||
- | |||
- | < | ||
- | qsub -pe smp <pocet jader> | ||
- | </ | ||