| 1 | = File Systems = |
| 2 | |
| 3 | The following table summarizes the characteristics of the file systems available in the DEEP and DEEP-ER systems: |
| 4 | |
| 5 | || '''Mount Point''' || '''User can write/read to/from''' || '''Cluster''' || '''Type''' || '''Global / Local''' || '''SW Version''' || '''Stripe Pattern Details''' || '''Maximum Measured Performance[[BR]](see footnotes)''' || '''Other''' || |
| 6 | || /home[a-b] || /home[a-b]/$USER || DEEP, SDV || GPFS exported via NFS || Global || || || || Home directory;[[BR]]not suitable for performance relevant applications[[BR]]or benchmarking || |
| 7 | || /gpfs-work || /gpfs-work/$USER || DEEP, SDV || GPFS exported via NFS || Global || || || || GPFS work file system;[[BR]]not suitable for performance relevant applications[[BR]]or benchmarking || |
| 8 | || /work || /work/$USER || DEEP || BeeGFS || Global || 2015.03.!r11 || Type: RAID0,[[BR]]Chunksize: 512K,[[BR]]Number of storage targets: desired: 4 || 2170 MiB/s write, 2111 MiB/s read[[BR]]~21000 ops/s create[[BR]]![1] || Work file system || |
| 9 | || /sdv-work || /sdv-work/$USER || SDV || BeeGFS || Global || 2015.03.!r10 || Type: RAID0,[[BR]]Chunksize: 512K,[[BR]]Number of storage targets: desired: 4 || 425 MiB/s write, 67 MiB/s read[[BR]]15202 ops/s create, 5111 ops/s remove[[BR]]![2] || Work file system || |
| 10 | || /nvme || /nvme/tmp || SDV || NVMe device || Local || || Block size: 4K || 1145 MiB/s write, 3108 MiB/s read[[BR]]139148 ops/s create, 62587 ops/s remove[[BR]]![2] || 1 NVMe device available at each SDV compute node || |
| 11 | || /mnt/beeond || /mnt/beeond || SDV || BeeGFS On Demand running on the NVMe || Local || 2015.03.!r10 || Block size: 512K || 1130 MiB/s write, 2447 MiB/s read[[BR]]12511 ops/s create, 18424 ops/s remove[[BR]]![2] || 1 BeeOND instance running on each NVMe device || |
| 12 | |
| 13 | == Stripe Pattern Details == |
| 14 | |
| 15 | It is possible to query this information from the deep login node, for instance: |
| 16 | |
| 17 | {{{ |
| 18 | manzano@deep $ fhgfs-ctl --getentryinfo /work/manzano |
| 19 | Path: /manzano |
| 20 | Mount: /work |
| 21 | EntryID: 1D-53BA4FF8-3BD3 |
| 22 | Metadata node: deep-fs02 [ID: 15315] |
| 23 | Stripe pattern details: |
| 24 | + Type: RAID0 |
| 25 | + Chunksize: 512K |
| 26 | + Number of storage targets: desired: 4 |
| 27 | |
| 28 | manzano@deep $ beegfs-ctl --getentryinfo /sdv-work/manzano |
| 29 | Path: /manzano |
| 30 | Mount: /sdv-work |
| 31 | EntryID: 0-565C499C-1 |
| 32 | Metadata node: deeper-fs01 [ID: 1] |
| 33 | Stripe pattern details: |
| 34 | + Type: RAID0 |
| 35 | + Chunksize: 512K |
| 36 | + Number of storage targets: desired: 4 |
| 37 | }}} |
| 38 | |
| 39 | Or like this: |
| 40 | |
| 41 | {{{ |
| 42 | manzano@deep $ stat -f /work/manzano |
| 43 | File: "/work/manzano" |
| 44 | ID: 0 Namelen: 255 Type: fhgfs |
| 45 | Block size: 524288 Fundamental block size: 524288 |
| 46 | Blocks: Total: 120178676 Free: 65045470 Available: 65045470 |
| 47 | Inodes: Total: 0 Free: 0 |
| 48 | |
| 49 | manzano@deep $ stat -f /sdv-work/manzano |
| 50 | File: "/sdv-work/manzano" |
| 51 | ID: 0 Namelen: 255 Type: fhgfs |
| 52 | Block size: 524288 Fundamental block size: 524288 |
| 53 | Blocks: Total: 120154793 Free: 110378947 Available: 110378947 |
| 54 | Inodes: Total: 0 Free: 0 |
| 55 | }}} |
| 56 | |
| 57 | See http://www.beegfs.com/wiki/Striping for more information. |
| 58 | |
| 59 | == Additional infos == |
| 60 | |
| 61 | Detailed information on the '''BeeGFS Configuration''' can be found [https://trac.version.fz-juelich.de/deep-er/wiki/BeeGFS here]. |
| 62 | |
| 63 | Detailed information on the '''BeeOND Configuration''' can be found [https://trac.version.fz-juelich.de/deep-er/wiki/BeeOND here]. |
| 64 | |
| 65 | Detailed information on the '''Storage Configuration''' can be found [https://trac.version.fz-juelich.de/deep-er/wiki/local_storage here]. |
| 66 | |
| 67 | Detailed information on the '''Storage Performance''' can be found [https://trac.version.fz-juelich.de/deep-er/wiki/SDV_AdminGuide/3_Benchmarks here]. |
| 68 | |
| 69 | == Footnotes == |
| 70 | |
| 71 | ![1] Performance tests (IOR and mdtest) reports are available in the BSCW under DEEP-ER -> Work Packages (WPs) -> WP4 -> T4.5 - Performance measurement and evaluation of I/O software -> Jülich DEEP Cluster -> Benchmarking reports: |
| 72 | * https://bscw.zam.kfa-juelich.de/bscw/bscw.cgi/1382059 |
| 73 | |
| 74 | ![2] Test results and parameters used stored in JUBE: |
| 75 | |
| 76 | {{{ |
| 77 | user@deep $ cd /usr/local/deep-er/sdv-benchmarks/synthetic/ior |
| 78 | user@deep $ jube2 result benchmarks |
| 79 | |
| 80 | user@deep $ cd /usr/local/deep-er/sdv-benchmarks/synthetic/mdtest |
| 81 | user@deep $ jube2 result benchmarks |
| 82 | }}} |