/
Using Modules: FAQs

Using Modules: FAQs

Owned by Simon Alexander
Last updated: Sep 30, 2024
4 min read

Goal: Present a list of commonly asked questions that might not be addressed specifically during the workshop.

Do I Need to Load Modules slurm/latest and arcc/1.0?

As mentioned earlier these two modules will be loaded automatically on the start of any session.

As long as you do not --force purge nor unload these, they stay within your session and essentially you do not need to re-load them.

The arcc/<version> module can be considered as a dependency for everything, which is why ARCC loads it. But there is no harm if you wish to include it within any scripts.

Dependencies: Remember

  • If a library/application has been installed with a compiler, you need to load that compiler first.

  • If a library/application has been built with openmpi, you need to load that next. Libraries will be post fixed with -ompi.

  • Software based on containers and conda environments do not have dependencies.

  • Some Linux binaries might have might have dependencies. These will be automatically loaded and specified in the module spider help details.

image-20240930-145042.png
Relationships with Dependencies

Remember: Use module spider to find if something has dependencies.

Are There Modules For Python Packages?

Yes, there are, try performing a module spider py- and see what is listed.

These have not been explicitly installed by ARCC, but are dependencies that have been installed with respect to more high-level software that ARCC has installed using Spack.

We do not create modules for individual python packages. There are 1000s of packages, and each would need to be installed for the specific version of that package, as well as having a specific version of Python.

So, although there is a module py-numpy/1.26.1 it will only:

  • work with the version of Python that was used in it’s installation - which in this case is python/3.10.6.

  • only provides the numpy python package and no other.

  • you will need to install other python packages yourself if needed.

Regards installing python packages, we encourage users to perform this themselves. Look at the following workshops for various ways you can do this:

Are There Modules For R Packages?

Yes, there might be, try performing a module spider r- and see what is listed.

These have not been explicitly installed by ARCC, but are dependencies that have been installed with respect to more high-level software that ARCC has installed using Spack.

We do not create modules for individual R packages. There are 1000s of packages, and each would need to be installed for the specific version of that package, as well as having a specific version of R.

So, although there might be a module it will only:

  • work with the version of R that was used in it’s installation.

  • only provides that R package and no other.

  • you will need to install other R packages yourself if needed.

Regards install R packages, we encourage users to perform this themselves. Look at the following workshops for various ways you can do this:

Can Not Find a Previous Module I Used

There are a number of scenarios where this might occur.

Using an old script from a previous cluster:

Typically when we setup a new cluster is will have similar software available, but using newer versions, and potentially newer dependency versions.

For example, say someone was using samtools:

On Beartooth:

module load gcc/12.2.0 samtools/1.16.1

On MedicineBow:

module load gcc/14.2.0 samtools/1.20

Remember: Use the module spider command to look up versions and dependencies.

[]$ module spider samtools -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- samtools: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Versions: samtools/1.14 samtools/1.16.1 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- For detailed information about a specific "samtools" package (including how to load the modules) use the module's full name. Note that names that have a trailing (E) are extensions provided by other modules. For example: $ module spider samtools/1.16.1 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- []$ module spider samtools/1.16.1 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- samtools: samtools/1.16.1 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- You will need to load all module(s) on any one of the lines below before the "samtools/1.16.1" module is available to load. arcc/1.0 gcc/12.2.0 Help: SAM Tools provide various utilities for manipulating alignments in the SAM format, including sorting, merging, indexing and generating alignments in a per-position format
[]$ module spider samtools ---------------------------------------------------------------------------- samtools: samtools/1.20 ---------------------------------------------------------------------------- This module can be loaded directly: module load samtools/1.20 Additional variants of this module can also be loaded after loading the following modules: arcc/1.0 gcc/14.2.0 Help: samtools : 1.20 SAM Tools provide various utilities for manipulating alignments in the SAM format, including sorting, merging, indexing and generating alignments in a per-position format. https://www.htslib.org/ []$ module spider samtools/1.20 ---------------------------------------------------------------------------- samtools: samtools/1.20 ---------------------------------------------------------------------------- This module can be loaded directly: module load samtools/1.20 Additional variants of this module can also be loaded after loading the following modules: arcc/1.0 gcc/14.2.0 Help: samtools : 1.20 SAM Tools provide various utilities for manipulating alignments in the SAM format, including sorting, merging, indexing and generating alignments in a per-position format. https://www.htslib.org/

Module Names Can Update

Due to the complicated nature of how some applications/libraries are installed, and how they require different variations of an underlying library, some times module names need to be updated to differentiate these difference versions. For example:

[]$ module spider hdf5 ---------------------------------------------------------------------------- hdf5: ---------------------------------------------------------------------------- Versions: hdf5/1.14.3_x86_64__hl_False-ompi hdf5/1.14.3_x86_64__hl_True-ompi

ARCC can not predict when this might occur.

If you have previously been able to load a module on a Cluster, but are now getting a warning about it, it might be that the module name has updated. Perform a module spider and check/confirm if this has occurred.

Prev

Using Modules: Considerations and Defaults

Workshop Home

Intro to using the Module System

Next

Required Software is Not Available

 

Related content