File Systems

The following table summarizes the characteristics of the file systems available in the DEEP and DEEP-ER systems:

Mount Point	User can write/read to/from	Cluster	Type	Global / Local	SW Version	Stripe Pattern Details	Maximum Measured Performance (see footnotes)	Other
/home[a-b]	/home[a-b]/$USER	DEEP, SDV	GPFS exported via NFS	Global				Home directory; not suitable for performance relevant applications or benchmarking
/gpfs-work	/gpfs-work/$USER	DEEP, SDV	GPFS exported via NFS	Global				GPFS work file system; not suitable for performance relevant applications or benchmarking
/work	/work/$USER	DEEP	BeeGFS	Global	2015.03.!r11	Type: RAID0, Chunksize: 512K, Number of storage targets: desired: 4	2170 MiB/s write, 2111 MiB/s read ~21000 ops/s create ![1]	Work file system
/sdv-work	/sdv-work/$USER	SDV	BeeGFS	Global	2015.03.!r10	Type: RAID0, Chunksize: 512K, Number of storage targets: desired: 4	425 MiB/s write, 67 MiB/s read 15202 ops/s create, 5111 ops/s remove ![2]	Work file system
/nvme	/nvme/tmp	SDV	NVMe device	Local		Block size: 4K	1145 MiB/s write, 3108 MiB/s read 139148 ops/s create, 62587 ops/s remove ![2]	1 NVMe device available at each SDV compute node
/mnt/beeond	/mnt/beeond	SDV	BeeGFS On Demand running on the NVMe	Local	2015.03.!r10	Block size: 512K	1130 MiB/s write, 2447 MiB/s read 12511 ops/s create, 18424 ops/s remove ![2]	1 BeeOND instance running on each NVMe device

Stripe Pattern Details

It is possible to query this information from the deep login node, for instance:

manzano@deep $ fhgfs-ctl --getentryinfo /work/manzano
Path: /manzano
Mount: /work
EntryID: 1D-53BA4FF8-3BD3
Metadata node: deep-fs02 [ID: 15315]
Stripe pattern details:
+ Type: RAID0
+ Chunksize: 512K
+ Number of storage targets: desired: 4

manzano@deep $ beegfs-ctl --getentryinfo /sdv-work/manzano
Path: /manzano
Mount: /sdv-work
EntryID: 0-565C499C-1
Metadata node: deeper-fs01 [ID: 1]
Stripe pattern details:
+ Type: RAID0
+ Chunksize: 512K
+ Number of storage targets: desired: 4

Or like this:

manzano@deep $ stat -f /work/manzano
  File: "/work/manzano"
    ID: 0        Namelen: 255     Type: fhgfs
Block size: 524288     Fundamental block size: 524288
Blocks: Total: 120178676  Free: 65045470   Available: 65045470
Inodes: Total: 0          Free: 0

manzano@deep $ stat -f /sdv-work/manzano
  File: "/sdv-work/manzano"
    ID: 0        Namelen: 255     Type: fhgfs
Block size: 524288     Fundamental block size: 524288
Blocks: Total: 120154793  Free: 110378947  Available: 110378947
Inodes: Total: 0          Free: 0

See ​http://www.beegfs.com/wiki/Striping for more information.

Additional infos

Detailed information on the BeeGFS Configuration can be found ​here.

Detailed information on the BeeOND Configuration can be found ​here.

Detailed information on the Storage Configuration can be found ​here.

Detailed information on the Storage Performance can be found ​here.

Footnotes

![1] Performance tests (IOR and mdtest) reports are available in the BSCW under DEEP-ER → Work Packages (WPs) → WP4 → T4.5 - Performance measurement and evaluation of I/O software → Jülich DEEP Cluster → Benchmarking reports:

• ​https://bscw.zam.kfa-juelich.de/bscw/bscw.cgi/1382059

![2] Test results and parameters used stored in JUBE:

user@deep $ cd /usr/local/deep-er/sdv-benchmarks/synthetic/ior
user@deep $ jube2 result benchmarks

user@deep $ cd /usr/local/deep-er/sdv-benchmarks/synthetic/mdtest
user@deep $ jube2 result benchmarks