Goal: Understand how to load modules, reset your environment by purging, and potential dependency issues.
...
Code Block |
---|
[]$ module load gcc/13.2.0 python/3.12.0 []$ echo $PATH /apps/u/spack/gcc/13.2.0/python/3.12.0-ovfqpv2/bin: /apps/u/spack/gcc/13.2.0/util-linux-uuid/2.38.1-x54sns7/bin: ... /apps/u/spack/gcc/13.2.0/sqlitezstd/31.435.25-vkxcwyjibvf7gj/bin: /apps/us/spack/gcc/13.2.0/openssl/3.1.3-shbr5xoarcc/1.0/bin: /apps/us/spackslurm/gcc/13.2.0/gettext/0.22.3-54ib6snlatest/bin:/home/<username>/.local/bin:/appshome/u<username>/spackbin:/gcc/13.2.0/tar/1.34-drux4fhusr/share/Modules/bin: /appsusr/ulocal/spack/gcc/13.2.0/pigz/2.7-6mr4kcibin:/usr/bin:/appsusr/ulocal/spacksbin:/gcc/13.2.0/libxml2/2.10.3-5toq4pi/bin:usr/sbin []$ which python /apps/u/spack/gcc/13.2.0/xzpython/53.412.10-uainak7ovfqpv2/bin:/apps/u/spack/gcc/13.2.0/libiconv/1.17-3dj22ny/bin: /apps/u/spack/gcc/13.2.0/gdbm/1.23-idyayjv/bin:/python |
Info |
---|
Loading modules updates a number of environment variables exposing the compiler/application/software we want to use. |
...
What’s Happening to My Environment?
Info |
---|
Loading modules does not change anything in your What is being updated are the environment variables within your current session. These are not persistent across other sessions. |
Info |
---|
You can have two or more separate/independent sessions opened at the same time, each with different modules loaded - but all still sharing the same file storage system. |
Info |
---|
Remember: The order that the system will look within paths is from left to right. The
path is before the |
...
Can we use the R language?
Info |
---|
Can we use R within an environment? |
Code Block |
---|
[]$ module purge
[]$ module avail |
Info |
---|
Do we see R available within the default list of available modules? No. Can we search for it? |
Code Block |
---|
[]$ module spider r |
Info |
---|
From what is returned by Can we load it? |
Code Block |
---|
[]$ module load r/4.4.0 |
Note |
---|
How do we fix this? |
...
Can we use the R language? Fixed
Info |
---|
Notice from the |
Info |
---|
Loading modules updates a number of environment variables exposing the compiler/application/software we want to use. |
What’s Happening to My Environment?
Info |
---|
Loading modules does not change anything in your What is being updated are the environment variables within your current session. These are not persistent across other sessions. You can have two or more sessions opened at the same time, each with different modules loaded - but all still sharing the same file storage system. |
Info |
---|
Remember, that the order that the system will look within paths is from left to right. The /apps/u/spack/. Until |
Code Block |
---|
[]$ module purge
[]$ module load gcc/13.2.0 r/4.4.0 |
Info |
---|
Once |
Code Block |
---|
[]$ ml
Currently Loaded Modules:
1) slurm/latest (S) 42) libxau/1.0.8
...
40) libpthread-stubs/0.4 81) r/4.4.0
41) xproto/7.0.31 |
Info |
---|
A lot! The module system hides the loading of all these additional libraries. |
Info |
---|
Let’s check R is loaded: |
Code Block |
---|
[]$ R --version
R version 4.4.0 (2024-04-24) -- "Puppy Cup" |
...
Can we also use the Python language?
Info |
---|
We know we can only have one compiler loaded at a time. Can we have more than one language loaded at a time? First load |
Code Block |
---|
Code Block |
[]$ module purge []$ module load gcc/13.2.0 r/python/3 4.12.0-ovfqpv2/bin path is before the /usr/bin/ so it is picking up 4.0 []$ module load python/3.12.0before the system’s version. |
Can we use the R language?
Info |
---|
Can we use R within an environment? |
Code Block |
---|
[]$ module purge
[]$ module avail |
Info |
---|
Do we see R available within the default list of available modules? No. Can we search for it? |
Code Block |
---|
[]$ module spider r |
Info |
---|
From what is returned by module spider, can you see why it isn’t currently available? Can we load it? |
Code Block |
---|
[]$ module load r/4.4.0 |
Info |
---|
How do we fix this? |
Can we use the R language? Fixed
Info |
---|
The module Until this module has been loaded, we will not see |
Code Block |
---|
[]$ module purge
[]$ module load gcc/13.2.0 r/4.4.0 |
Info |
---|
Once |
Code Block |
---|
[]$ ml
Currently Loaded Modules:
1) slurm/latest (S) 42) libxau/1.0.8
...
40) libpthread-stubs/0.4 81) r/4.4.0
41) xproto/7.0.31 |
Info |
---|
A lot! The module system hides the loading of all these additional libraries. |
Info |
---|
Let’s check R is loaded: |
Code Block |
---|
[]$ R --version
R version 4.4.0 (2024-04-24) -- "Puppy Cup" |
Can we also use the Python language?
Info |
---|
We know we can only have one compiler loaded at a time. Can we have more than one language loaded at a time? First load |
Code Block |
---|
[]$ module purge
[]$ module load gcc/13.2.0 r/4.4.0
[]$ module load python/3.12.0
-------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: python/3.10.6 (required by: glib/2.78.0, xcb-proto/1.15.2, gobject-introspection/1.76.1)
-------------------------------------------------------------------------------
The following have been reloaded with a version change:
1) python/3.10.6 => python/3.12.0
[]$ R --version
R version 4.4.0 (2024-04-24) -- "Puppy Cup"
[]$ python --version
Python 3.12.0 |
Info |
---|
Notice: Loading R automatically loaded After loading a module, consider calling Loading |
Note |
---|
Does replacing Potentially: We do not know how this is being used and thus can’t say definitively if it does/doesn’t affect the running of R. We would suggest airing on the side of caution and check everything is running as expected if you notice modules being replaced. |
Load Another Compiler:
Info |
---|
After loading R and Python, what happens if I replace the |
[]$ module purge []$ module load gcc/13.2.0 r/4.4.0 []$ ------------------------------------------------------------------------------- The following dependent module(s) are not currently loaded: python/3.10.6 (required by: glib/2.78.0, xcb-proto/1.15.2, gobject-introspection/1.76.1) ------------------------------------------------------------------------------- The following have been reloaded with a version change: 1) python/3.10.6 => python/3.12.0 []$ R --version R version 4.4.0 (2024-04-24) -- "Puppy Cup" ... []$ python --version Python 3.1012.60 |
Info |
---|
Load the Intel oneApi compiler and see what happens to your environment: |
Code Block |
[]$ module load oneapi/2024.1.0
Lmod is automatically replacing "gcc/13.2.0" with "oneapi/2024.1.0".
Inactive Modules:
1) cairo/1.16.0 30) libxcb/1.14
...
29) libxau/1.0.8 58) xtrans/1.4.0
Due to MODULEPATH changes, the following have been reloaded:
1) berkeley-db/18.1.40 6) libiconv/1.17 11) pigz/2.7 16) zlib-ng/2.1.4
...
5) libbsd/0.11.7 10) perl/5.38 Notice: Loading R automatically loaded After loading a module, consider calling Loading |
Note |
---|
Question: Does replacing Potentially: We do not know how this is being used and thus can’t say definitively if it does/doesn’t affect the running of R. We would suggest airing on the side of caution and check everything is running as expected if you notice modules being replaced. |
...
Load Another Compiler:
Info |
---|
After loading R and Python, what happens if you replace the |
Code Block |
---|
[]$ module purge
[]$ module load gcc/13.2.0 r/4.4.0
[]$ R --version
R version 4.4.0 (2024-04-24) -- "Puppy Cup"
...
[]$ python --version
Python 3.10.6 |
Info |
---|
Load the Intel |
Code Block |
---|
[]$ module load oneapi/2024.1.0 Lmod is automatically replacing "gcc/13.2.0" with "oneapi/2024.1.0". Inactive Modules: 1) cairo/1.16.0 15) xz/5.4.1 []$ R --version -bash: R: command not found []$ python --version Python 3.9.18 []$ which python /usr/bin/python |
Info |
---|
Remember: You can only have one compiler loaded at a time. Replacing one compiler with another will also remove and modules that have this first compiler as a dependency. So, with the above, R has been removed and is no longer available, and Python has been removed and reverted back to the System version. |
But R is still in my Module List?
Info |
---|
But, what can we see if we list the modules available? |
Code Block |
---|
[]$ ml Currently Loaded Modules30) libxcb/1.14 ... 29) libxau/1.0.8 58) xtrans/1.4.0 Due to MODULEPATH changes, the following have been reloaded: 1) slurm/latestberkeley-db/18.1.40 (S)6) libiconv/1.17 8 11) xzpigz/5.4.12.7 1516) berkeleyzlib-dbng/182.1.4 ...40 25) arcclibbsd/10.11.07 (S) 910) libxml2perl/25.1038.30 16) readline/8.2 315) oneapixz/20245.4.1.0 []$ R --version -bash: R: 10) ncurses/6.4 17) gdbm/1.23 4) zlib-ng/2.1.4 11) pigz/2.7 18) perl/5.38.0 5) zstd/1.5.5 12) tar/1.34command not found []$ python --version Python 3.9.18 []$ which python /usr/bin/python |
Info |
---|
Remember: You can only have one compiler loaded at a time. Replacing one compiler with another will affect modules that have the first compiler as a dependency. So, with the above, R has been made inactive and is no longer available, and Python has also been made inactive and reverts back to the System version. |
...
But R is still in my Module List?
Info |
---|
What can you see if you list the modules loaded? |
Code Block |
---|
[]$ ml Currently Loaded Modules: 1) slurm/latest (S) 8) xz/5.4.1 1915) libmdberkeley-db/18.1.40 ..0.4 6) bzip2/1.0.8 13) gettext/0.22.3 20) libbsd/0.11.7 7) libiconv/1.17 14) pkgconf/1.9.5 Where: S: Module is Sticky, requires --force to unload or purge Inactive Modules: ... 13) python/3.10.6 42) libtirpc/1.3.3 ... 29) pixman/0.42.2 58) r/4.4.0 |
Note |
---|
Yes, both R and Python are listed. But these are listed as |
Info |
---|
We would suggest, as a good practice, always perform a |
...
Note |
---|
Will changing the version of Python affect R? In this case probably not. But if underlying versions of libraries are changing - then maybe - and that’s the best we can say… Remember: The more complicated your environments, the more dependencies there’ll be, the more potential for dependency hell. |
...
Prev | Workshop Home | Next |
...