Information about the batch system

Ressource allocation

• Start an interactive session for example on one DEEP Cluster Node for 2.5 hours:

→ qsub -I -X -l nodes=1:ppn=16:cluster,walltime=02:30:00

• Start an interactive session for example on one DEEP Cluster Node plus one DEEP Booster Node for 2.5 hours:

→ qsub -I -X -l nodes=1:ppn=16:cluster+1:ppn=244:booster,walltime=02:30:00

• Leave the interactive session:

→ exit

• Submit a batch job:

→ qsub job_script.sh

• Specify the nodes you want to use:

→ -l nodes=x:ppn=y:cluster will allocate DEEP Cluster Nodes
→ -l nodes=x:ppn=y:booster will allocate DEEP Booster Nodes
→ -l nodes=x:ppn=y:sdv will allocate SDV Cluster Nodes
→ -l nodes=x:ppn=y:knl will allocate (randomly) SDV KNL Nodes
→ -l nodes=x:ppn=y:knl:extoll will allocate SDV KNL Nodes with EXTOLL
→ -l nodes=x:ppn=y:knl:nvme will allocate SDV KNL Nodes with NVMe

• The following job script can be used as a template for your own jobs:

#!/bin/bash
#PBS -S /bin/bash
#PBS -l nodes=[number of nodes]:ppn=[number of processes per node]:[nodetype]
#PBS -l walltime=[hours]:[minutes]:[seconds]
#PBS -e [path to your errorfiles]
#PBS -o [path to your outputfiles]
#PBS -m e
#PBS -M [your mail address]
#PBS -N [name of your program]
#PBS -d [path to your working directory] 
#PBS -v LD_LIBRARY_PATH
module load parastation
[load all the modules you need]
mpiexec -np [number of processes] [executable]

Some useful commands

• pbstop → Shows system load and job overview.
  
• pbsnodes -nl → Shows information on nodes, that are currently not available.
  
• showres → Shows reservations. (Only on deepm)
  
• showq → Lists active, idle and blocked jobs. (Only on deepm)
  
• qstat -u USER → Shows jobs of a certain user.
  
• qstat -f JOBID → Shows detailed information about a certain job (job id can be obtained by qstat -u).
  
• showstart JOBID → Provides an estimated start time for a certain job. (Only on deepm)