Changes between Version 38 and Version 39 of Public/User_Guide/Batch_system


Ignore:
Timestamp:
Jul 28, 2020, 5:33:52 PM (4 years ago)
Author:
Zia Ul Huda
Comment:

Legend:

Unmodified
Added
Removed
Modified
  • Public/User_Guide/Batch_system

    v38 v39  
    321321== Workflows ==
    322322
    323 The version of Slurm installed on the system enables workflows (chains of jobs) with the possibility of having some overlap between the dependent jobs. This allows them to exchange data over the network rather writing and reading it using a common file system.
     323The version of Slurm installed on the system enables workflows (chains of jobs) with the possibility of having some overlap between the dependent jobs. This allows them to exchange data over the network rather than writing and reading it using a common file system.
    324324
    325325Workflows can be submitted in two ways:
     
    329329An example project that uses all the features discussed is provided [https://gitlab.version.fz-juelich.de/deamicis1/mpi_connect_test/-/tree/test_zia_workflows here].
    330330
    331 The following simple example script helps understanding the mechanism of new {{{delay}}} switch for workflows.
     331The following simple example script helps to understand the mechanism of new {{{delay}}} switch for workflows.
    332332
    333333{{{
     
    353353}}}
    354354
    355 In the above {{{sbatch}}} script, the usage of {{{--delay}}} can be seen. The option takes values in minutes, and allows to delay the subsequent job of by a user-defined number of minutes from the start of the first job in the job pack. After submission of this job pack (which uses the same syntax as a heterogeneous job), Slurm divides it into separate jobs. Also, Slurm ensures that the delay is respected by using reservations, rather than the usual scheduler.
     355In the above {{{sbatch}}} script, the usage of {{{--delay}}} can be seen. The option takes values in minutes and allows us to delay the subsequent job of by a user-defined number of minutes from the start of the first job in the job pack. After submission of this job pack (which uses the same syntax as a heterogeneous job), Slurm divides it into separate jobs. Also, Slurm ensures that the delay is respected by using reservations, rather than the usual scheduler.
    356356
    357357Here is the example execution of this script.
     
    425425}}}
    426426
    427 Note the {{{delay}}} values for the second and third job in the script are equal. Also, note the usage of the environment variable {{{SLURM_JOB_NAME}}} in the script to decide which command to run in which job. Currently this is the only Slurm environment variable which allows to differentiate components of a heterogeneous job (and workflow) in a static manner (i.e. not using job-dependent variables like the job ID or the node lists).
     427Note the {{{delay}}} values for the second and third job in the script are equal. **The {{{delay}}} value for the 4th job ({{{-J third}}}) is relative to the start of the first job and not from the start of middle 2 jobs. So it will start after 2 minutes of the start time of the middle jobs.** Also, note the usage of the environment variable {{{SLURM_JOB_NAME}}} in the script to decide which command to run in which job. The example execution leads to the following:
    428428
    429429The example execution leads to the following: