Changes between Version 25 and Version 26 of Public/User_Guide/News

Timestamp:

Jan 21, 2022, 11:36:18 AM (2 years ago)

Author:

Jochen Kreutz

Comment:

general update to reflect current system status

Public/User_Guide/News

@@ -5 +5 @@
 This is a summary of the latest news concerning the system. For a list of known problems related to the system, please refer to [wiki:Public/User_Guide/PaS this page].
 
-''Last update: 2021-12-10''
+''Last update: 2022-01-21''
 
-[[span(style=color: #FF0000, System will be in maintenance in CW50 (Tuesday, 2021-12-14 to Thursday 2021-12-16))]]
-
+[[span(style=color: #FF0000, Moving to Rocky Linux 8.5 on compute nodes, please expect limited system access. See also [wiki:News#SystemSystemsoftware System software] section)]]
 {{{#!comment 
 [[span(style=color: #FF0000, System will be in maintenance in CW37 (Monday, 2020-09-07 to Friday 2020-09-11))]]
@@ -15 +14 @@
 == System hardware ==
 
-{{{#!comment
+
 === CM nodes ===
-}}}
+
+- the cluster nodes have direct EBR IB access to the SSSM storage nodes now (without using the IB <-> 40 GbE gateway)
+
 
 === ESB nodes ===
 
-- the first ESB rack (nodes `dp-esb[01-25]`) is planned to be revised to use IB again (instead of Extoll interconnect)
-  - date for the HW intervention not yet fixed, but due to delivery time unlikely to be performed this year 
+- all ESB nodes (`dp-esb[01-75]`) are using EDR Infiniband interconnect (no Extoll anymore)
+- SSSM and AFSM file servers can be directly accessed through IB
+
+
 
 === DAM nodes ===
 
-- along with first ESB rack also the DAM nodes will move to IB (instead of using 40 GbE and Extoll) 
+- DAM nodes are using EDR Infiniband (instead of using 40 GbE and Extoll) now
+- SSSM and AFSM file servers can be directly accessed through IB
+
 
 === Network Federation Gateways ===
 
-- with aiming for an "all IB" solution the NFGWs will become obsolete
+- with the all IB solution NFGWs are not needed anymore ! this also affects heterogenous jobs
 
-- current status is
-  - 2x NF-GW EDR/Extoll (1 x Fabri3, 1 x Tourmalet)
-  - 2x NF-GW 40GbE/Extoll ( 1 x Fabri3, 1 x Tourmalet)
-  - 2x NF-GW EDR/40GbE
-  - the NF-GWs equipped with Fabri3 PCIe cards are not in operation
-  - for an example on how to use the gateway nodes and for further information, 
-    please refer to the [wiki:/Public/User_Guide/Batch_system#HeterogeneousjobswithMPIcommunicationacrossmodules batchsystem] wiki page.
+- `dp-nfgw[01,04]` (IB EDR <-> 40GbE) still present, but not in use anymore
+
+- remaining NFGWs are being used for BXI testing now: `dp-nfgw[02,03,05,06]`
+ 
 
 
-=== Global resources ===
+
+=== SDV ===
+
+- 4 node BXI test setup has been installed using the former GW nodes
+
+- FPGA test nodes available for using FPGAs with oneAPI, OpenCL:
+  - Arria10: deeper-sdv[09,10]
+  - Stratix10: dp-sdv-esb[01,02]
+
+
+
+{{{#!comment obsolete with removal of Extoll
 
 ==== NAM ====
@@ -47 +60 @@
 - a NAM SW implementation has been done, a test environment on the DAM has been set up on dp-dam[09-16].
 - for more information on NAM usage and an example, please refer to [wiki:Public/User_Guide/TAMPI_NAM NAM with TAMPI] page
+}}}
 
 === File Systems ===
 
-**please refer to the**  [wiki:Filesystems Filesystems] **overview**
+**please also refer to the**  [wiki:Filesystems Filesystems] **overview**
 
-- recent changes:
-  - a new All Flash Storage Module (AFSM) is in place and provides fast work file system mounted to `/work` on compute nodes and login node (`deepv`)
-    - the older System Services and Storage Module (SSSM) still serves the /usr/local file system
-    - SSSM storage has been rebuilt for performance reasons
-  - BeeGFS servers and clients have been updated
-  - BeeGFS (`/work`) user quotas is in place now (see section "User management")
-  - It is possible to access the `$ARCHIVE` file system from the `deepv` login node under `/arch`.  
-    See hint in the MOTD for efficient usage of the archive filesystem
+- the All Flash Storage Module (AFSM) provides a fast work file system mounted to `/afsm` (symbolic link to `/work`) on all compute nodes (CM, DAM, ESB) and the login node (`deepv`)
+- the older System Services and Storage Module (SSSM) work file system is obsolete, but still available at (`/work_old`) for data migration
+- SSSM still serves the /usr/local/software file system, but
+  - starting from Rocky 8 image `/usr/local` will be a local file system
+  - `/usr/local/software` will be shared and provided by the SSSM storage
+  - in addition to the !Easybuild software stack the shared `/usr/local/software` filesystem will contain manually installed software in a `legacy` subfolder
   
 
 == System software ==
 
-=== SW updates ===
+- ParaStation update (psmgmt) to 5.1.45-3 has been performed
 
-- transition from CentOS to Rocky Linux currently being investigated
-  - a second login node running rocky linux 8.4 has been provided for testing:
-    `ssh -l your_judoor_id zam906`
-  - few CM nodes can be used as rocky linux testbed: `dp-cn[47-50]`
-  - please get in contact with `niessen(at)par-tec.com` if you would like to get access
+=== OS 
+
+- the decision was taken to use one Rocky linux image for all compute nodes
+
+- transition from CentOS to Rocky Linux 8.5 is ongoing (starting on the CM and then continue with ESB and DAM)
+
+  - login node `deepv`has moved to Rocky linux 8 as well
 
 
-- new SLURM version has been installed: 20.11.8.         
-  - please use `--interactive` flag for interactive sessions now (see MOTD hint)
-
-- 2021 Easybuild stage is being set up
-  - as of 2021-09-08 the default stage is `2020` (was `2019a` before)
-
-- latest Intel oneAPI version is available in /usr/local/intel/oneapi
+=== !EasyBuild
 
 
-=== User management ===
-
-==== BeeGFS Quotas ====
-
-- a quota for the BeeGFS file system (mounted to /work) has been implemented
-  - no need to activate thresholds yet
+- 2022 !EasyBuild stage is being set up
+  - expect to have a first version (basic components like compilers etc.) available in Feburary 2022 
 
 
-
-
-