Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »

Goal: Introduce the LMOD Module System and the concept of dependencies.

What do we have available?

  • Compilers and eco systems: GNU family, Intel’s oneAPI, Nvidia’s hpc-sdk

  • Languages: C/C+, Fortran, Go, Java, Julia, Perl, Python, R, Ruby, Rust

  • Scientific libraries and toolkits: Built with a specific compiler: GNU by default

  • Standalone applications and utilities: Installed using:

    • Conda

    • Containers: running Singularity (not Docker)

      • We can create a Singularity image from a Docker image.

    • Binaries/Executables

Check the MedicineBow Software Page.

LMOD: An Environment Module System

What’s available?

# Go back to your home folder.
[...]$ cd ~
[~]$ cd intro_to_modules/

# Modules currently loaded.
[]$ ml
Currently Loaded Modules:
  1) slurm/latest (S)   2) arcc/1.0 (S)

# What's available within my 'current' environment.
# With respect to what modules I 'currently' have loaded.
[]$ module avail
...

Notice: There are various categories of available software: containers, compilers, conda environments, Linux binaries…

Modules have a name and a version: <module-name>/<module-version>

What’s available? Compiler tree

Some software has to be built with respect to a compiler.

By default we build using the GNU family of compilers for C/C++ and Fortran.

[]$ module load gcc/13.2.0
[]$ ml
Currently Loaded Modules:
  1) slurm/latest (S)   3) gmp/6.2.1    5) mpc/1.3.1       7) zstd/1.5.5
  2) arcc/1.0     (S)   4) mpfr/4.2.0   6) zlib-ng/2.1.4   8) gcc/13.2.0

The gmp, mpc, mpfr, zlib-ng and zstd are know as dependencies.

There are libraries that need to be loaded for gcc to work.

They are loaded automatically so you don’t have to.

[] module avail
# What has changed?

We can now wee all the modules available that have been compiled with gcc/13.2.0

These are only available if/when the gcc/13.2.0 module is loaded.

Change Compiler

Let’s see what happens to our environment if we load a different compiler.

[]# module load nvhpc-sdk/24.3
Lmod is automatically replacing "gcc/13.2.0" with "nvhpc-sdk/24.3".

# What happened?
[]$ ml
Currently Loaded Modules:
  1) slurm/latest (S)   2) arcc/1.0 (S)   3) nvhpc-sdk/24.3

[]$ module avail
# Modules relating to nvhpc-sdk/24.3 are now available.

You can only have one module compiler loaded at a time.

Compilers of the same name, but different versions (gcc/12.2.0 vs gcc/13.2.0) are considered different.

Purge Your Modules

[]$ purge

# What has changed?
[]$ ml
[]$ module avail

Use the purge command to reset your environment to the initial defaults.

The more modules you have loaded, the more dependencies you will have loaded and the more likely you are to forget what you have loaded…

What’s available? Modules loaded by default

[]$ module purge
The following modules were not unloaded:
  (Use "module --force purge" to unload all):
  1) slurm/latest   2) arcc/1.0

[]$ ml
Currently Loaded Modules:
  1) slurm/latest (S)   2) arcc/1.0 (S)
  Where:
   S:  Module is Sticky, requires --force to unload or purge

Looking for Modules: module spider

Remember: module avail only shows what is available to be loaded with respect to your current environment and what you currently have loaded. It does not show you everything.

[]$ module spider openmpi
----------------------------------------------------------------------------
  openmpi: openmpi/4.1.6
----------------------------------------------------------------------------
     Other possible modules matches:
        nvhpc-openmpi3

    You will need to load all module(s) on any one of the lines below before the "openmpi/4.1.6" module is available to load.

      arcc/1.0  gcc/13.2.0
...

Module load/spider: Dependencies

[]$ module load openmpi/4.1.6
Lmod has detected the following error: These module(s) or extension(s) exist
but cannot be loaded as requested: "openmpi/4.1.6"
   Try: "module spider openmpi/4.1.6" to see how to load the module(s).

# What dependencies does openmpi have? And what is currently loaded?
[]$ module load gcc/13.2.0
[]$ module load openmpi/4.1.6

# What do you notice?
[]$ ml
[]$ module avail

Due to dependencies, there can be a hierarchy of core modules that need to be loaded to make other libraries and software available.

and what is loaded. For example, some libraries, such as fftw, need to be built with gcc/13.2.0 and use the openmpi/4.1.6 library.

Typically their version number will be prefixed with -ompi.

Try: []$ module spider fftw

What’s different between these command-lines?

[]$ module spider python/3.10.6
[]$ module spider python/3.12.0
[]$ module spider python2

What’s different between these command-lines? Dependencies

[]$ module spider python/3.10.6
[]$ module spider python/3.12.0
...
    You will need to load all module(s) on any one of the lines below before the "python/3.12.0" module is available to load.
      arcc/1.0  gcc/13.2.0

[]$ module spider python2
----------------------------------------------------------------------------
  python2: python2/2.7.18
----------------------------------------------------------------------------
    You will need to load all module(s) on any one of the lines below before the "python2/2.7.18" module is available to load.
      arcc/1.0

Prev

Your Basic Environment 

Workshop Home

Workshop

Next

Setting Up Environments

 

 

  • No labels