Changes between Version 10 and Version 11 of Public/User_Guide/Information_on_software


Ignore:
Timestamp:
Jan 21, 2020, 9:45:01 AM (4 years ago)
Author:
Jochen Kreutz
Comment:

JK: updated to reflect current SW stack.

Legend:

Unmodified
Added
Removed
Modified

  • Public/User_Guide/Information_on_software

    v10 v11  
     1{{{#!comment JK: Removed since this is the only level 1 header
    12= General information =
     3}}}
    24[[PageOutline]]
    35
     6{{{#!comment JK: I think we should focus on the new stack
    47== Usage of modules ==
    58
    … …  
    2326
    2427The modules which are currently loaded can be listed with the command “module list”. To unload a module just type “module unload” followed by the name of the module.
    25 
    26 
    27 == Information on new software stack and lmod ==
    28 
    29 Since the second half of 2018 a new software stack managed via Easybuild is available on the system (more information on Easybuild is available [https://easybuild.readthedocs.io/en/latest/ here]).
     28}}}
     29
     30= Information on software stack and using modules =
     31
     32== General information ==
     33
     34Since the second half of 2018 a new software stack managed via Easybuild is available on the system (more information on Easybuild is available [https://easybuild.readthedocs.io/en/latest/ here]). It provides a `module` command for loading software environments.
     35To see which modules are available use
     36
     37`module avail`
    3038
    3139The software made available on the new stack via Easybuild is arranged hierarchically, with only some core modules made available to the user at login. Typically, these core modules include the compilers, or some other standalone tools (i.e. JUBE). Loading such core modules via `module load` allows the list of available modules to increase, including the modules that are installed with the fore-mentioned core module as a dependency.
    3240
    33 A typical workflow would be the following. After logging in, the user would be in this situation (note: the examples below were produced with the old Haswell stack):
     41A typical workflow would be the following. After logging in, the user would be in this situation:
    3442
    3543{{{
    3644$ module avail
    3745
    38 --------------------------------------------------------------- /usr/local/software/haswell/Stages/2018a/modules/all/Compiler/GCCcore/5.5.0 ----------------------------------------------------------------
    39    Autotools/20170619    CMake/3.9.6     CMake/3.11.1        Doxygen/1.8.14    LLVM/5.0.1    Perl/5.26.1        Python/2.7.14        X11/20180131    flex/2.6.4               unzip/6.0
    40    Bazel/0.11.1          CMake/3.10.3    CMake/3.12.2 (D)    GMP/6.1.2         PAPI/5.6.0    PostgreSQL/10.3    Python/3.6.5  (D)    cURL/7.59.0     libspatialindex/1.8.5
    41 
    42 -------------------------------------------------------------------------- /usr/local/software/haswell/Stages/2018a/UI/Compilers ---------------------------------------------------------------------------
    43    Intel/2018.2.199-GCC-5.5.0
    44 
    45 ---------------------------------------------------------------------------- /usr/local/software/haswell/Stages/2018a/UI/Tools -----------------------------------------------------------------------------
    46    Advisor/2018_update3    EasyBuild/3.6.1    JUBE/2.2.1    Java/1.8.0_162    VTune/2018_update3    intel-para/2018a-mt    intel-para/2018a (D)    tbb/2018.2.199
     46--------------------------------------------------------------------------------------------------- /usr/local/software/skylake/Stages/2019a/modules/all/Compiler/GCCcore/8.3.0 ---------------------------------------------------------------------------------------------------
     47   Autotools/20180311                         CubeWriter/4.4.2           Keras/2.2.4-GPU-Python-3.6.8        Ninja/1.9.0                           Python/3.6.8                       (D)    X11/20190311                          numactl/2.0.12
     48   Bazel/0.20.0                               Doxygen/1.8.15             LLVM/8.0.0                          PAPI/5.7.0                            SciPy-Stack/2019a-Python-2.7.16           cURL/7.64.1                           scikit/2019a-Python-2.7.16
     49   CMake/3.14.0                               GEOS/3.7.1-Python-3.6.8    MPFR/4.0.2                          Perl/5.28.1                           SciPy-Stack/2019a-Python-3.6.8     (D)    dask/1.1.5-Python-2.7.16              scikit/2019a-Python-3.6.8  (D)
     50   Clang/8.0.0-GCC-8.3.0-CUDA-10.1.105 (g)    GMP/6.1.2                  Mesa/19.0.1                         PostgreSQL/11.2                       Tcl/8.6.9                                 dask/1.1.5-Python-3.6.8        (D)    unzip/6.0
     51   CubeGUI/4.4.3                              Graphviz/2.40.1            Meson/0.50.0-Python-3.6.8           PyTorch/1.1.0-GPU-Python-3.6.8 (g)    TensorFlow/1.13.1-GPU-Python-3.6.8 (g)    flex/2.6.4
     52   CubeLib/4.4.3                              HDF5/1.10.5-serial         NCCL/2.4.6-1-CUDA-10.1.105   (g)    Python/2.7.16                         UCX/1.5.1                                 h5py/2.9.0-serial-Python-3.6.8
     53
     54-------------------------------------------------------------------------------------------------------------- /usr/local/software/skylake/Stages/2019a/UI/Compilers --------------------------------------------------------------------------------------------------------------
     55   GCC/8.3.0    Intel/2019.3.199-GCC-8.3.0    PGI/19.3-GCC-8.3.0
     56
     57---------------------------------------------------------------------------------------------------------------- /usr/local/software/skylake/Stages/2019a/UI/Tools ----------------------------------------------------------------------------------------------------------------
     58   Advisor/2019_update3        EasyBuild/3.8.1        Inspector/2019_update3    Java/1.8         VTune/2019_update3                  intel-para/2019a-mt        pscom/5.3.1-1      tbb/2019.4.199
     59   CUDA/10.1.105        (g)    EasyBuild/3.9.1 (D)    JUBE/2.2.2                UCX/1.6.1 (D)    cuDNN/7.5.1.10-CUDA-10.1.105 (g)    intel-para/2019a    (D)    rpmrebuild/2.14
     60
     61--------------------------------------------------------------------------------------------------------------------------- /usr/local/software/extoll ----------------------------------------------------------------------------------------------------------------------------
     62   extoll    openmpi/1.6.1_extoll
    4763
    4864  Where:
     65   g:  built for GPU
    4966   D:  Default Module
    5067
    51 }}}
    52 
    53 Loading the Intel compiler (Intel/2018.2.199-GCC-5.5.0 module), allows the list of available tools to be expanded:
    54 
    55 {{{
    56 module load Intel/2018.2.199-GCC-5.5.0
    57 module avail
    58 
    59 ------------------------------------------------------- /usr/local/software/haswell/Stages/2018a/modules/all/Compiler/mpi/intel/2018.2.199-GCC-5.5.0 -------------------------------------------------------
    60    ParaStationMPI/5.2.1-1-mt    ParaStationMPI/5.2.1-1 (D)
    61 
    62 --------------------------------------------------------- /usr/local/software/haswell/Stages/2018a/modules/all/Compiler/intel/2018.2.199-GCC-5.5.0 ---------------------------------------------------------
    63    mkl-dnn/0.13
    64 
    65 --------------------------------------------------------------- /usr/local/software/haswell/Stages/2018a/modules/all/Compiler/GCCcore/5.5.0 ----------------------------------------------------------------
    66    Autotools/20170619    CMake/3.9.6     CMake/3.11.1        Doxygen/1.8.14    LLVM/5.0.1    Perl/5.26.1        Python/2.7.14        X11/20180131    flex/2.6.4               unzip/6.0
    67    Bazel/0.11.1          CMake/3.10.3    CMake/3.12.2 (D)    GMP/6.1.2         PAPI/5.6.0    PostgreSQL/10.3    Python/3.6.5  (D)    cURL/7.59.0     libspatialindex/1.8.5
    68 
    69 -------------------------------------------------------------------------- /usr/local/software/haswell/Stages/2018a/UI/Compilers ---------------------------------------------------------------------------
    70    Intel/2018.2.199-GCC-5.5.0 (L)
    71 
    72 ---------------------------------------------------------------------------- /usr/local/software/haswell/Stages/2018a/UI/Tools -----------------------------------------------------------------------------
    73    Advisor/2018_update3    EasyBuild/3.6.1    JUBE/2.2.1    Java/1.8.0_162    VTune/2018_update3    intel-para/2018a-mt    intel-para/2018a (D)    tbb/2018.2.199
     68Use "module spider" to find all possible modules.
     69Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".
     70
     71}}}
     72
     73It is recommended to use the Intel compiler and tool chain on the DEEP-EST system unless there are good reasons to move to the GCC or PGI compilers.
     74Loading the Intel compiler (Intel/2019.3.199-GCC-8.3.0), allows the list of available tools to be expanded:
     75
     76{{{
     77$ module load Intel/2019.3.199-GCC-8.3.0
     78$ module avail
     79
     80------------------------------------------------------------------------------------------ /usr/local/software/skylake/Stages/2019a/modules/all/Compiler/mpi/intel/2019.3.199-GCC-8.3.0 -------------------------------------------------------------------------------------------
     81   IntelMPI/2019.3.199    ParaStationMPI/5.2.2-1-mt    ParaStationMPI/5.2.2-1 (D)
     82
     83-------------------------------------------------------------------------------------------- /usr/local/software/skylake/Stages/2019a/modules/all/Compiler/intel/2019.3.199-GCC-8.3.0 ---------------------------------------------------------------------------------------------
     84   GSL/2.5    HDF5/1.10.5-serial (D)
     85
     86--------------------------------------------------------------------------------------------------- /usr/local/software/skylake/Stages/2019a/modules/all/Compiler/GCCcore/8.3.0 ---------------------------------------------------------------------------------------------------
     87   Autotools/20180311                         CubeWriter/4.4.2           Keras/2.2.4-GPU-Python-3.6.8        Ninja/1.9.0                           Python/3.6.8                       (D)    X11/20190311                          numactl/2.0.12
     88   Bazel/0.20.0                               Doxygen/1.8.15             LLVM/8.0.0                          PAPI/5.7.0                            SciPy-Stack/2019a-Python-2.7.16           cURL/7.64.1                           scikit/2019a-Python-2.7.16
     89   CMake/3.14.0                               GEOS/3.7.1-Python-3.6.8    MPFR/4.0.2                          Perl/5.28.1                           SciPy-Stack/2019a-Python-3.6.8     (D)    dask/1.1.5-Python-2.7.16              scikit/2019a-Python-3.6.8  (D)
     90   Clang/8.0.0-GCC-8.3.0-CUDA-10.1.105 (g)    GMP/6.1.2                  Mesa/19.0.1                         PostgreSQL/11.2                       Tcl/8.6.9                                 dask/1.1.5-Python-3.6.8        (D)    unzip/6.0
     91   CubeGUI/4.4.3                              Graphviz/2.40.1            Meson/0.50.0-Python-3.6.8           PyTorch/1.1.0-GPU-Python-3.6.8 (g)    TensorFlow/1.13.1-GPU-Python-3.6.8 (g)    flex/2.6.4
     92   CubeLib/4.4.3                              HDF5/1.10.5-serial         NCCL/2.4.6-1-CUDA-10.1.105   (g)    Python/2.7.16                         UCX/1.5.1                                 h5py/2.9.0-serial-Python-3.6.8
     93
     94-------------------------------------------------------------------------------------------------------------- /usr/local/software/skylake/Stages/2019a/UI/Compilers --------------------------------------------------------------------------------------------------------------
     95   GCC/8.3.0    Intel/2019.3.199-GCC-8.3.0 (L)    PGI/19.3-GCC-8.3.0
     96
     97---------------------------------------------------------------------------------------------------------------- /usr/local/software/skylake/Stages/2019a/UI/Tools ----------------------------------------------------------------------------------------------------------------
     98   Advisor/2019_update3        EasyBuild/3.8.1        Inspector/2019_update3    Java/1.8         VTune/2019_update3                  intel-para/2019a-mt        pscom/5.3.1-1      tbb/2019.4.199
     99   CUDA/10.1.105        (g)    EasyBuild/3.9.1 (D)    JUBE/2.2.2                UCX/1.6.1 (D)    cuDNN/7.5.1.10-CUDA-10.1.105 (g)    intel-para/2019a    (D)    rpmrebuild/2.14
     100
     101--------------------------------------------------------------------------------------------------------------------------- /usr/local/software/extoll ----------------------------------------------------------------------------------------------------------------------------
     102   extoll    openmpi/1.6.1_extoll
    74103
    75104  Where:
     105   g:  built for GPU
    76106   L:  Module is loaded
    77107   D:  Default Module
    78 }}}
    79 
    80 After further loading an MPI library, the user can then access all parallel tools that require MPI:
    81 
    82 {{{
    83 module load ParaStationMPI/5.2.1-1
    84 module avail
    85 
    86 
    87 ---------------------------------------------------- /usr/local/software/haswell/Stages/2018a/modules/all/MPI/intel/2018.2.199-GCC-5.5.0/psmpi/5.2.1-1 -----------------------------------------------------
    88    Boost/1.66.0-Python-2.7.14        Extrae/3.5.2    FTI/1.2    HDF5/1.8.20            PETSc/3.9.0_int8        R/3.4.3          SciPy-Stack/2018a-Python-3.6.5    imkl/2018.2.199
    89    Boost/1.66.0-Python-3.6.5  (D)    FFTW/3.3.7      GSL/2.4    PETSc/3.9.0_complex    PETSc/3.9.0      (D)    SIONlib/1.7.1    TensorFlow/1.8.0-Python-3.6.5     netCDF/4.6.1
    90 
    91 ------------------------------------------------------- /usr/local/software/haswell/Stages/2018a/modules/all/Compiler/mpi/intel/2018.2.199-GCC-5.5.0 -------------------------------------------------------
    92    ParaStationMPI/5.2.1-1-mt    ParaStationMPI/5.2.1-1 (L,D)
    93 
    94 --------------------------------------------------------- /usr/local/software/haswell/Stages/2018a/modules/all/Compiler/intel/2018.2.199-GCC-5.5.0 ---------------------------------------------------------
    95    mkl-dnn/0.13
    96 
    97 --------------------------------------------------------------- /usr/local/software/haswell/Stages/2018a/modules/all/Compiler/GCCcore/5.5.0 ----------------------------------------------------------------
    98    Autotools/20170619    CMake/3.9.6     CMake/3.11.1        Doxygen/1.8.14    LLVM/5.0.1    Perl/5.26.1        Python/2.7.14        X11/20180131    flex/2.6.4               unzip/6.0
    99    Bazel/0.11.1          CMake/3.10.3    CMake/3.12.2 (D)    GMP/6.1.2         PAPI/5.6.0    PostgreSQL/10.3    Python/3.6.5  (D)    cURL/7.59.0     libspatialindex/1.8.5
    100 
    101 -------------------------------------------------------------------------- /usr/local/software/haswell/Stages/2018a/UI/Compilers ---------------------------------------------------------------------------
    102    Intel/2018.2.199-GCC-5.5.0 (L)
    103 
    104 ---------------------------------------------------------------------------- /usr/local/software/haswell/Stages/2018a/UI/Tools -----------------------------------------------------------------------------
    105    Advisor/2018_update3    EasyBuild/3.6.1    JUBE/2.2.1    Java/1.8.0_162    VTune/2018_update3    intel-para/2018a-mt    intel-para/2018a (D)    tbb/2018.2.199
    106 
    107   Where:
    108    L:  Module is loaded
    109    D:  Default Module
    110 }}}
     108
     109
     110}}}
     111
     112To see which modules are currently loaded, use the `module list` command.
    111113
    112114It can be seen from the previous examples that with the new software stack, the command `module avail` does not allow the user to visualize all the modules actually installed on the system, but rather the ones made available by the system with the modules previously loaded up to that point. In order to explore the full hierarchy of modules, the command
    … …  
    114116`module spider`
    115117
    116 must be used. After enabling the new stack on a new shell, it's possible to search for a certain module (Extrae, for instance) with the following instructions:
    117 
    118 {{{
    119 $ ml spider Extrae
    120 
    121 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    122   Extrae: Extrae/3.5.2
    123 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
     118must be used. After logging in to a new shell, it's possible to search for a certain module (Extrae, for instance) with the following instructions:
     119
     120{{{
     121$ module spider Extrae
     122
     123------------------------------------------------------------------------------------------------------------------------------------------
     124  Extrae:
     125------------------------------------------------------------------------------------------------------------------------------------------
    124126    Description:
    125       Extrae is the core instrumentation package developed by the Performance Tools group at BSC. Extrae is capable of instrumenting applications based on MPI, OpenMP, pthreads, CUDA1, OpenCL1, and
    126       StarSs1 using different instrumentation approaches. The information gathered by Extrae typically includes timestamped events of runtime calls, performance counters and source code references.
    127       Besides, Extrae provides its own API to allow the user to manually instrument his or her application.
    128 
    129 
    130     You will need to load all module(s) on any one of the lines below before the "Extrae/3.5.2" module is available to load.
    131 
    132       Intel/2018.2.199-GCC-5.5.0  ParaStationMPI/5.2.1-1
     127      Extrae is the core instrumentation package developed by the Performance Tools group at BSC. Extrae is capable of instrumenting
     128      applications based on MPI, OpenMP, pthreads, CUDA1, OpenCL1, and StarSs1 using different instrumentation approaches. The
     129      information gathered by Extrae typically includes timestamped events of runtime calls, performance counters and source code
     130      references. Besides, Extrae provides its own API to allow the user to manually instrument his or her application.
     131
     132     Versions:
     133        Extrae/3.7.0
     134        Extrae/3.7.1
     135
     136}}}
     137
     138To get information on how to load a certain module including a list of modules that need to be loaded, in order to be able to load a certain Extrae version, you have to use the `module spider` command with that version:
     139
     140{{{
     141
     142$ module spider Extrae/3.7.0
     143
     144------------------------------------------------------------------------------------------------------------------------------------------
     145  Extrae: Extrae/3.7.0
     146------------------------------------------------------------------------------------------------------------------------------------------
     147    Description:
     148      Extrae is the core instrumentation package developed by the Performance Tools group at BSC. Extrae is capable of instrumenting
     149      applications based on MPI, OpenMP, pthreads, CUDA1, OpenCL1, and StarSs1 using different instrumentation approaches. The
     150      information gathered by Extrae typically includes timestamped events of runtime calls, performance counters and source code
     151      references. Besides, Extrae provides its own API to allow the user to manually instrument his or her application.
     152
     153
     154    You will need to load all module(s) on any one of the lines below before the "Extrae/3.7.0" module is available to load.
     155
     156      Intel/2019.3.199-GCC-8.3.0  ParaStationMPI/5.2.2-1
     157      Intel/2019.3.199-GCC-8.3.0  ParaStationMPI/5.2.2-1-mt
    133158 
    134159    Help:
    … …  
    148173       - Homepage: http://www.bsc.es/computer-sciences/performance-tools
    149174       - Site contact: sc@fz-juelich.de
    150 }}}
    151 
    152 The description provided by the `module spider` command includes also a list of modules that need to be loaded, in order to be able to load the Extrae module.
    153 
    154 The `module spider` command is part of a new implementation of the old `module` command, namely `lmod`. Together with this added functionality, lmod provides also a wrapper around the `module` command: the `ml` command. This wrapper has some user-friendly features:
     175
     176}}}
     177
     178The `module spider` command is part of a new implementation of the old `module` command, namely `lmod`, which is provided with Easybuild. Together with this added functionality, lmod provides also a wrapper around the `module` command: the `ml` command. This wrapper has some user-friendly features:
    155179- `ml` alone lists all the currently loaded modules;
    156180- `ml +module` (or `ml module`) allows to load a specific module (e.g. `ml Intel` loads the default version of the Intel module);
    … …  
    182206
    183207
    184 == Information on compilation with !ParaStation MPI wrappers ==
    185 
    186 When compiling software using the !ParaStation MPI compiler wrappers, by default the Intel compiler will be used:
     208== MPI programs - compilation with !ParaStation MPI ==
     209
     210It is recommended to use the !ParaStation MPI installation. When compiling software using the !ParaStation MPI compiler wrappers, by default the Intel compiler will be used:
    187211
    188212{{{
    … …  
    192216}}}
    193217
    194 In order to use GCC rather than the Intel compiler, this must be done:
     218In order to use GCC rather than the Intel compiler, use one of the following:
    195219
    196220{{{
    … …  
    226250
    227251
    228 == Using the old stacks ==
     252= Using the old software stacks (for SDV Hardware) =
    229253
    230254As of July 2019, a new software stack compiled for Skylake is loaded by default on the DEEP-EST system.
    … …  
    250274
    251275
    252 === Information relative to the legacy stack only ===
    253 
    254 ==== Compilers ====
     276== Information relative to the legacy stack only ==
     277
     278=== Compilers ===
    255279
    256280There are several compilers available, but as it is highly recommended to use the Intel compiler on the KNC system it might be best to also use it on the DEEP system.\\
    … …  
    261285
    262286
    263 ==== Profiling and analysis tools ====
     287=== Profiling and analysis tools ===
    264288
    265289* Intel VTune:    module load VTune
    … …  
    271295* Scalasca: module load UNITE scalasca
    272296
     297
     298{{{#!comment JK: moved to Parastation MPI Wrapper section
    273299== MPI programs ==
    274 It is recommended to use the !ParaStation MPI installation.
     300It is recommended to use the !ParaStation MPI installation.
     301}}}