Changes between Version 38 and Version 39 of Public/User_Guide/Batch_system
- Timestamp:
- Jul 28, 2020, 5:33:52 PM (4 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
Public/User_Guide/Batch_system
v38 v39 321 321 == Workflows == 322 322 323 The version of Slurm installed on the system enables workflows (chains of jobs) with the possibility of having some overlap between the dependent jobs. This allows them to exchange data over the network rather writing and reading it using a common file system.323 The version of Slurm installed on the system enables workflows (chains of jobs) with the possibility of having some overlap between the dependent jobs. This allows them to exchange data over the network rather than writing and reading it using a common file system. 324 324 325 325 Workflows can be submitted in two ways: … … 329 329 An example project that uses all the features discussed is provided [https://gitlab.version.fz-juelich.de/deamicis1/mpi_connect_test/-/tree/test_zia_workflows here]. 330 330 331 The following simple example script helps understandingthe mechanism of new {{{delay}}} switch for workflows.331 The following simple example script helps to understand the mechanism of new {{{delay}}} switch for workflows. 332 332 333 333 {{{ … … 353 353 }}} 354 354 355 In the above {{{sbatch}}} script, the usage of {{{--delay}}} can be seen. The option takes values in minutes , and allows to delay the subsequent job of by a user-defined number of minutes from the start of the first job in the job pack. After submission of this job pack (which uses the same syntax as a heterogeneous job), Slurm divides it into separate jobs. Also, Slurm ensures that the delay is respected by using reservations, rather than the usual scheduler.355 In the above {{{sbatch}}} script, the usage of {{{--delay}}} can be seen. The option takes values in minutes and allows us to delay the subsequent job of by a user-defined number of minutes from the start of the first job in the job pack. After submission of this job pack (which uses the same syntax as a heterogeneous job), Slurm divides it into separate jobs. Also, Slurm ensures that the delay is respected by using reservations, rather than the usual scheduler. 356 356 357 357 Here is the example execution of this script. … … 425 425 }}} 426 426 427 Note the {{{delay}}} values for the second and third job in the script are equal. Also, note the usage of the environment variable {{{SLURM_JOB_NAME}}} in the script to decide which command to run in which job. Currently this is the only Slurm environment variable which allows to differentiate components of a heterogeneous job (and workflow) in a static manner (i.e. not using job-dependent variables like the job ID or the node lists).427 Note the {{{delay}}} values for the second and third job in the script are equal. **The {{{delay}}} value for the 4th job ({{{-J third}}}) is relative to the start of the first job and not from the start of middle 2 jobs. So it will start after 2 minutes of the start time of the middle jobs.** Also, note the usage of the environment variable {{{SLURM_JOB_NAME}}} in the script to decide which command to run in which job. The example execution leads to the following: 428 428 429 429 The example execution leads to the following: