Goal: Introduce the LMOD Module System and the concept of dependencies.
| Table of Contents | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...
BUT We can demonstrate the compiler/python dependency issue since we don’t have this use-case on MedBow.
...
What do we have available?
Compilers and eco systems: GNU family, Intel’s oneAPI, Nvidia’s hpc-sdk
Languages: C/C+, Fortran, Go, Java, Julia, Perl, Python, R, Ruby, Rust
Scientific libraries and toolkits: Built with a specific compiler: GNU by default
Standalone applications and utilities: Installed using:
Conda
Containers: running Singularity (not Docker)
We can create a Singularity image from a Docker image.
Binaries/Executables
Check the MedicineBow Software Pages
...
Page.
...
LMOD: An Environment Module System
...
What’s available?
| Code Block |
|---|
[bin# Go back to your home folder. [...]$ cd ~ [~]$ cd intro_to_hpcmodules/ # Modules currently loaded. []$ ml []$Currently moduleLoaded availModules: []$ module load1) gcc/12.2.0 []$ ml []$ module avail # What is different compared to the first time we called ml?slurm/latest (S) 2) arcc/1.0 (S) # What's available within my 'current' environment. # With respect to what modules I 'currently' have loaded. []$ module loadavail gcc/11..2.0 # What happened?. |
| Info |
|---|
Notice: There are various categories of available software: containers, compilers, conda environments, Linux binaries… |
| Info |
|---|
Modules have a name and a version: |
...
What’s available? Compiler tree
| Info |
|---|
Some software has to be built with respect to a compiler. By default we build using the GNU family of compilers for C/C++ and Fortran. |
| Code Block |
|---|
[]$ module load gcc/1113.2.0 #[]$ Whatml happened?Currently DueLoaded toModules: MODULEPATH changes, the following have been reloaded: 11) slurm/latest (S) 3) gmp/6.2.1 25) mpfrmpc/41.3.1.0 37) zlibzstd/1.5.5 2.12) arcc/1.0 (S) 4) mpfr/4.2.0 6) zstdzlib-ng/2.1.5.2 The following have been reloaded with a version change: 1) gcc/12.2.0 => gcc/11.2.0 # You can only have one compiler loaded at a time within a session. []$ ml4 8) gcc/13.2.0 |
| Info |
|---|
The There are libraries that need to be loaded for gcc to work. They are loaded automatically so you don’t have to. |
| Code Block |
|---|
[] module avail
# What has changed? |
| Info |
|---|
We can now wee all the modules available that have been compiled with |
| Note |
|---|
These are only available if/when the |
...
Change Compiler
| Code Block |
|---|
[]# module load nvhpc-sdk/24.3 Lmod is automatically replacing "gcc/13.2.0" with "nvhpc-sdk/24.3". # What happened? []$ ml Currently Loaded Modules: 1) slurm/latest (S) 2) arcc/1.0 (S) 3) nvhpc-sdk/24.3 []$ module avail # Modules relating to nvhpc-sdk/24.3 are now available. |
| Note |
|---|
You can only have one module compiler loaded at a time. |
...
Purge Your Modules
| Code Block |
|---|
[]$ module purge purge # What has changed? []$ ml #[]$ What do you notice?module avail |
| Info |
|---|
Use the |
| Note |
|---|
The more modules you have loaded, the more dependencies you will have loaded and the more likely you are to forget what you have loaded… |
...
What’s available? Modules loaded by default
| Code Block |
|---|
[]$ module purge
The following modules were not unloaded:
(Use "module --force purge" to unload all):
1) slurm/latest 2) arcc/1.0
[]$ ml
Currently Loaded Modules:
1) slurm/latest (S) 2) arcc/1.0 (S)
Where:
S: Module is Sticky, requires --force to unload or purge |
...
Looking for Modules: module spider
| Info |
|---|
Remember: module avail only shows what is available to be loaded with respect to your current environment and what you currently have loaded. It does not show you everything. |
| Code Block |
|---|
[]$ module spider openmpi
----------------------------------------------------------------------------
openmpi: openmpi/4.1.6
----------------------------------------------------------------------------
Other possible modules matches:
nvhpc-openmpi3
You will need to load all module(s) on any one of the lines below before the "openmpi/4.1.6" module is available to load.
arcc/1.0 gcc/13.2.0
... |
...
Module load/spider: Dependencies
| Code Block |
|---|
[]$ module load pythonopenmpi/34.101.6 Lmod has detected the following error: These module(s) or extension(s) exist but cannot be loaded as requested: "pythonopenmpi/34.101.6" Try: "module spider pythonopenmpi/34.101.6" to see how to load the module(s). # What dependencies does openmpi have? And what is currently loaded? []$ module load gcc/13.2.0 []$ module load openmpi/4.1.6 # What do you notice? []$ ml []$ module avail |
| Info |
|---|
Due to dependencies, there can be a hierarchy of core modules that need to be loaded to make other libraries and software available. and what is loaded. For example, some libraries, such as Typically their version number will be prefixed with Try: |
...
What’s different between these command-lines?
| Code Block |
|---|
[]$ module spider python/3.10.6 []$ module spider python/3.10.812.0 []$ module spider python2 |
...
What’s different between these command-lines? Dependencies
| Code Block |
|---|
[]$ module spider python/3.10.6 ---------------------------------------- python:[]$ module spider python/3.1012.6 ----------------------------------------0 ... You will need to load all module(s) on any one of the lines below before the "python/3.1012.60" module is available to load. arcc/1.0 gcc/1213.2.0 Help: The Python programming language. []$ module spider python/3.10.8 python2 ---------------------------------------------------------------------------- pythonpython2: pythonpython2/32.107.818 ---------------------------------------------------------------------------- You will need to load all module(s) on any one of the lines below before the "pythonpython2/32.107.818" module is available to load. arcc/1.0 gcc/11.2.0 Help: The Python programming language. |
...