| Both sides previous revision
Previous revision
|
Next revision
Both sides next revision
|
grid [2018/03/24 11:33]
popel [Advanced usage] |
grid [2018/03/24 12:58]
ufal |
| |
| If you have a set of tasks (of the same type) and want to run them on multiple machines, use ''qsub -t''. | If you have a set of tasks (of the same type) and want to run them on multiple machines, use ''qsub -t''. |
| * ''-t 1-n'' start array job with n jobs numbered 1 ... n | * ''-t 1-n'' start array job with //n// tasks numbered //1 ... n// |
| * environmental variable ''SGE_TASK_ID'' | * environmental variable ''SGE_TASK_ID'' |
| * output and error files ''$JOB_NAME.[eo]$JOB_ID.$TASK_ID'' | * output and error files ''$JOB_NAME.[eo]$JOB_ID.$TASK_ID'' |
| * ''-t m-n[:s]'' start array job with jobs m, m + s, ..., n | * ''-t m-n[:s]'' start array job with tasks //m, m + s, ..., n// |
| * environmental variables ''SGE_TASK_FIRST, SGE_TASK_LAST, SGE_TASK_STEPSIZE'' | * environmental variables ''SGE_TASK_FIRST, SGE_TASK_LAST, SGE_TASK_STEPSIZE'' |
| * ''-tc j'' run at most j jobs simultaneously | * ''-tc j'' run at most //j// tasks simultaneously |
| * ''-hold_jid_ad comma_separated_job_list'' array jobs that must finish before this job starts; task //i// of the current job depends only on task //i// of the specified jobs | * ''-hold_jid_ad comma_separated_job_list'' array jobs that must finish before this job starts; task //i// of the current job depends only on task //i// of the specified jobs |
| |
| If you use ''-tc'', then SGE can handle array jobs of virtually any size. It only starts as many jobs as specified in ''-tc'' at any time, and each scheduling interval (15 seconds in our current configuration) it starts new jobs if less than the specified ''-tc'' limit are running. However, note that it means the maximum throughput is 4*"-tc" jobs per minute, so the individual array jobs need to run for at least tens of seconds for this to be effective. | If you use ''-tc'', then SGE can handle array jobs of virtually any size. It only starts as many tasks as specified in ''-tc'' at any time, and each scheduling interval (15 seconds in our current configuration) it starts new tasks if less than the specified ''-tc'' limit are running. However, note that it means the maximum throughput is //4 * tc// tasks per minute, so the individual array job tasks need to run for at least tens of seconds for this to be effective. |
| |
| The advantage of array jobs over [[#qunhold]] is that it does not overload the SGE job queue. Also if you start an array job, the others can see that it is an array job, how many individual tasks there are and how many of them have already finished. | The advantage of array jobs over [[#qunhold]] is that it does not overload the SGE job queue. Also if you start an array job, the others can see that it is an array job, how many individual tasks there are and how many of them have already finished. |
