Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 13 Next »

Goal: Understand how to load modules, reset your environment by purging, and potential dependency issues.

Setup Python environment

[]$ module purge
[]$ module load gcc/13.2.0
[]$ module load python/3.12.0
[]$ python –version

# Single line:
# Order matters:
[]$ module purge
[]$ module load python/3.12.0 gcc/13.2.0
vs
[]$ module purge
[]$ module load gcc/13.2.0 python/3.12.0

What’s happened to the PATH environment variable?

[]$ module purge
[]$ echo $PATH
[]$ module load gcc/13.2.0 python/3.12.0
[]$ echo $PATH
/apps/u/spack/gcc/13.2.0/python/3.12.0-ovfqpv2/bin:/apps/u/spack/gcc/13.2.0/util-linux-uuid/2.38.1-x54sns7/bin:
/apps/u/spack/gcc/13.2.0/sqlite/3.43.2-vkxcwyj/bin:/apps/u/spack/gcc/13.2.0/openssl/3.1.3-shbr5xo/bin:
/apps/u/spack/gcc/13.2.0/gettext/0.22.3-54ib6sn/bin:/apps/u/spack/gcc/13.2.0/tar/1.34-drux4fh/bin:
/apps/u/spack/gcc/13.2.0/pigz/2.7-6mr4kci/bin:/apps/u/spack/gcc/13.2.0/libxml2/2.10.3-5toq4pi/bin:
/apps/u/spack/gcc/13.2.0/xz/5.4.1-uainak7/bin:/apps/u/spack/gcc/13.2.0/libiconv/1.17-3dj22ny/bin:
/apps/u/spack/gcc/13.2.0/gdbm/1.23-idyayjv/bin:/apps/u/spack/gcc/13.2.0/readline/8.2-j6zrtuz/bin:
/apps/u/spack/gcc/13.2.0/ncurses/6.4-y257enz/bin:/apps/u/spack/gcc/13.2.0/expat/2.5.0-3aw2urq/bin:
/apps/u/spack/gcc/13.2.0/bzip2/1.0.8-7q5awx3/bin:/apps/u/spack/gcc/11.4.1/gcc/13.2.0-sxnvmb2/bin:
/apps/u/spack/gcc/13.2.0/zstd/1.5.5-ibvf7gj/bin:/apps/s/arcc/1.0/bin:
/apps/s/slurm/latest/bin:/home/<username>/.local/bin:/home/<username>/bin:/usr/share/Modules/bin:
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

[]$ which python
/apps/u/spack/gcc/13.2.0/python/3.12.0-ovfqpv2/bin/python

What’s Happening to My Environment?

Loading modules does not change anything in your /home nor /project storage space - nothing is being installed.

What is being updated are the environment variables within your current session. These are not persistent across other sessions.

Remember, that the order that the system will look within paths is from left to right.

The /apps/u/spack/gcc/13.2.0/python/3.12.0-ovfqpv2/bin path is before the /usr/bin/ so it is picking up python/3.12.0 before the system’s version.

Can we use the R language?

[]$ module purge

# Is R available?
[]$ module avail

[]$ module spider r
# What do we see and why?

# Does this work? 
[]$ module load r/4.4.0

# How do we fix it?

Can we use the R language? Fixed

[]$ module purge
[]$ module load gcc/13.2.0 r/4.4.0

# How many dependencies have automatically been loaded?
[]$ ml

[]$ R --version
R version 4.4.0 (2024-04-24) -- "Puppy Cup"

Can we also use the Python language?

[]$ module purge
[]$ module load gcc/13.2.0 r/4.4.0

[]$ module load python/3.12.0
-------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: python/3.10.6 (required by: glib/2.78.0, xcb-proto/1.15.2, gobject-introspection/1.76.1)
-------------------------------------------------------------------------------
The following have been reloaded with a version change:
  1) python/3.10.6 => python/3.12.0

[]$  R --version
R version 4.4.0 (2024-04-24) -- "Puppy Cup"
[]$ python --version
Python 3.12.0

Notice: Loading R automatically loaded python/3.10.6.

After loading a module, consider calling ml to see what dependencies have also been loaded.

Loading python/3.12.0 replaced this version.

Load Another Compiler:

[]$ module load gcc/13.2.0 r/4.4.0
[]$ R --version
R version 4.4.0 (2024-04-24) -- "Puppy Cup"
...
[]$ module load oneapi/2024.1.0
Lmod is automatically replacing "gcc/13.2.0" with "oneapi/2024.1.0".
Inactive Modules:
  1) cairo/1.16.0                    30) libxcb/1.14
...
 29) libxau/1.0.8                    58) xtrans/1.4.0
Due to MODULEPATH changes, the following have been reloaded:
  1) berkeley-db/18.1.40     6) libiconv/1.17     11) pigz/2.7         16) zlib-ng/2.1.4
...
  5) libbsd/0.11.7          10) perl/5.38.0       15) xz/5.4.1

[]$ R --version
-bash: R: command not found

Remember:

  • Only one compiler/version can be loaded into your environment at a time.

  • Can only load languages/applications built with the same compiler. 

  • But, even this can introduce dependency issues.

Will changing the version of Python affect R? In this case probably not.

But if underlying versions of libraries are changing - then maybe - and that’s the best we can say…

Remember: The more complicated your environments, the more dependencies there’ll be, the more potential for dependency hell.

Prev

The Module System and What’s Available

Workshop Home

Intro to using the Module System

Next

Using Modules: Considerations and Defaults

 

  • No labels