System usage

The system can be used through the PBS based batch system that is also used for the DEEP Cluster and Booster. You can request cluster nodes on the SDV with an interactive session like this:

kreutz@deepl:~ > qsub -I -l nodes=2:ppn=24:sdv,walltime=01:00:00
qsub: waiting for job 76649.deepm to start
qsub: job 76649.deepm ready

kreutz@deeper-sdv16:~ >

When using a batch script, you have to adapt the -l option within your script.

Filesystems and local storage

The home filesystems on the SDV are provided via GPFS/NFS and hence the same as on the DEEP System. The local storage system of the SDV running BeeGFS is available at

/sdv-work

This is NOT the same storage being used on the DEEP system. Both, the DEEP System and the DEEP-ER SDV have their own local storage. On the DEEP nodes it is mounted to /work on the deeper-sdv nodes it can be found in /sdv-work. In addition both systems provide a work filesystem via GPFS, which is called /gpfs-work and shared for both systems. But it should not be used for performance relevant applications since it is much slower than the local storages.

Using the EXTOLL network

Since the LD_LIBRARY_PATH attracts incompatible EXTOLL libraries it as to be adapted manually. The following setup should work:

module purge
module load gcc parastation/gcc-5.1.4-1_1_g064e3f7
export LD_LIBRARY_PATH=/usr/local/parastation/pscom/lib64:${LD_LIBRARY_PATH}
export LD_LIBRARY_PATH=/direct/Software/Extoll/SDV/lib:$LD_LIBRARY_PATH

Using /nvme

During job startup, all files of non-priviledged users within /nvme are removed. If you want to keep your files across consecutive jobs on a particular SDV node, add a list of filenames to $HOME/.nvme_keep:

/nvme/tmp/myfile.A
/nvme/tmp/myfile.B

This will keep the files /nvme/tmp/myfile.{A,B} across two or more job runs in a row.