The system can be used through the SLURM based batch system that is also used for the DEEP-EST Cluster Module and (most of) the remaining compute nodes.You can request cluster nodes on the SDV with an interactive session like this:
srun --partition=sdv -N 4 -n 8 --pty /bin/bash -i srun ./hello_cluster Hello world from process 6 of 8 on deeper-sdv07 Hello world from process 7 of 8 on deeper-sdv07 Hello world from process 3 of 8 on deeper-sdv05 Hello world from process 4 of 8 on deeper-sdv06 Hello world from process 0 of 8 on deeper-sdv04 Hello world from process 2 of 8 on deeper-sdv05 Hello world from process 5 of 8 on deeper-sdv06 Hello world from process 1 of 8 on deeper-sdv04
When using a batch script, you have to adapt the —partition option within your script: —partition=sdv
The home filesystems on the SDV are provided via GPFS/NFS and hence the same as on the DEEP System. The local storage system of the SDV running BeeGFS is available at
/sdv-work
This is NOT the same storage being used on the DEEP-EST system. Both, the DEEP-EST System and the DEEP-ER SDV have their own local storage. On the DEEP-EST nodes it is
mounted to /work on the deeper-sdv nodes it can be found in /sdv-work. The DEEP-EST storage in /workcan accessed, but with 1 GbE only. Hence, it should not be used for performance relevant applications since it is much slower than the local storages (see NVMe) and the DEEP-ER SDV storage (/sdv-work).
During job startup, all files of non-priviledged users within /nvme are removed. If you want to keep your files across consecutive jobs on a particular SDV node, add a list of filenames to $HOME/.nvme_keep:
/nvme/tmp/myfile.A /nvme/tmp/myfile.B
This will keep the files /nvme/tmp/myfile.{A,B} across two or more job runs in a row.
The SDV Cluster nodes are connected via EXTOLL tourmalet (100 Gb/s). Please use
module load extoll
to run jobs on multiple nodes.