Versions Compared

Old Version 2

changes.mady.by.user Simon Alexander

Saved on

compared with

New Version Current

changes.mady.by.user Simon Alexander

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Table of Contents
stylenone

...

Do I Need to Load Modules slurm/latest and arcc/1.0?

As mentioned earlier these two modules will be loaded automatically on the start of any session.

As long as you do not --force purge nor unload these, they stay within your session and essentially you do not need to re-load them.

The arcc/<version> module can be considered as a dependency for everything, which is why ARCC loads it. But there is no harm if you wish to include it within any scripts.

...

Dependencies: Remember

Info

  • If a library/application has been installed with a compiler, you need to load that compiler first.

  • If a library/application has been built with openmpi, you need to load that next. Libraries will be post fixed with -ompi.

  • Software based on containers and conda environments do not have dependencies.

  • Some Linux binaries might have might have dependencies. These will be automatically loaded and specified in the module spider help details.

...

Note

Remember: Use module spider to find if something has dependencies.

...

Are There Modules For Python Packages?

Info

Yes, there are, try performing a module spider py- and see what is listed.

Note

These have not been explicitly installed by ARCC, but are dependencies that have been installed with respect to more high-level software that ARCC has installed using Spack.

We do not create modules for individual python packages. There are 1000s of packages, and each would need to be installed for the specific version of that package, as well as having a specific version of Python.

So, although there is a module py-numpy/1.26.1 it will only:

  • work with the version of Python that was used in it’s installation - which in this case is python/3.10.6.

  • only provides the numpy python package and no other.

  • you will need to install other python packages yourself if needed.

Regards installing python packages, we encourage users to perform this themselves. Look at the following workshops for various ways you can do this:

...

Are There Modules For R Packages?

Info

Yes, there might be, try performing a module spider r- and see what is listed.

Note

These have not been explicitly installed by ARCC, but are dependencies that have been installed with respect to more high-level software that ARCC has installed using Spack.

We do not create modules for individual R packages. There are 1000s of packages, and each would need to be installed for the specific version of that package, as well as having a specific version of R.

So, although there might be a module it will only:

  • work with the version of R that was used in it’s installation.

  • only provides that R package and no other.

  • you will need to install other R packages yourself if needed.

Regards install R packages, we encourage users to perform this themselves. Look at the following workshops for various ways you can do this:

...

Can Not Find a Previous Module I Used

Info

There are a number of scenarios where this might occur.

Info

Using an old script from a previous cluster:

Typically when we setup a new cluster is will have similar software available, but using newer versions, and potentially newer dependency versions.

For example, say someone was using samtools:

On Beartooth:

Code Block
module load gcc/12.2.0 samtools/1.16.1

On MedicineBow:

Code Block
module load gcc/14.2.0 samtools/1.20

Remember: Use the module spider command to look up versions and dependencies.

Expand
titleOn Beartooth:
Code Block
[]$ module spider samtools
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  samtools:
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
     Versions:
        samtools/1.14
        samtools/1.16.1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "samtools" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:
     $ module spider samtools/1.16.1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

[]$ module spider samtools/1.16.1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  samtools: samtools/1.16.1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    You will need to load all module(s) on any one of the lines below before the "samtools/1.16.1" module is available to load.
      arcc/1.0  gcc/12.2.0
    Help:
      SAM Tools provide various utilities for manipulating alignments in the
      SAM format, including sorting, merging, indexing and generating
      alignments in a per-position format
Expand
titleOn MedicineBow
Code Block
[]$ module spider samtools
----------------------------------------------------------------------------
  samtools: samtools/1.20
----------------------------------------------------------------------------
    This module can be loaded directly: module load samtools/1.20
    Additional variants of this module can also be loaded after loading the following modules:
      arcc/1.0  gcc/14.2.0
    Help:
      samtools : 1.20
      SAM Tools provide various utilities for manipulating alignments in the
      SAM format, including sorting, merging, indexing and generating
      alignments in a per-position format.
      https://www.htslib.org/

[]$ module spider samtools/1.20
----------------------------------------------------------------------------
  samtools: samtools/1.20
----------------------------------------------------------------------------
    This module can be loaded directly: module load samtools/1.20
    Additional variants of this module can also be loaded after loading the following modules:
      arcc/1.0  gcc/14.2.0
    Help:
      samtools : 1.20
      SAM Tools provide various utilities for manipulating alignments in the
      SAM format, including sorting, merging, indexing and generating
      alignments in a per-position format.
      https://www.htslib.org/

...

Module Names Can Update

Note

Due to the complicated nature of how some applications/libraries are installed, and how they require different variations of an underlying library, some times module names need to be updated to differentiate these difference versions. For example:

Code Block
[]$ module spider hdf5
----------------------------------------------------------------------------
  hdf5:
----------------------------------------------------------------------------
     Versions:
        hdf5/1.14.3_x86_64__hl_False-ompi
        hdf5/1.14.3_x86_64__hl_True-ompi

ARCC can not predict when this might occur.

If you have previously been able to load a module on a Cluster, but are now getting a warning about it, it might be that the module name has updated. Perform a module spider and check/confirm if this has occurred.

...

Prev

Using Modules: Considerations and Defaults

Workshop Home

Intro to using the Module System

Next

Required Software is Not Available

...