Goal:
What do we have available?
Compilers and eco systems: GNU family, Intel’s oneAPI, Nvidia’s hpc-sdk
Languages: C/C+, Fortran, Go, Java, Julia, Perl, Python, R, Ruby, Rust
Scientific libraries and toolkits: Built with a specific compiler: GNU by default
Standalone applications and utilities: Installed using:
Conda
Containers: running Singularity (not Docker)
We can create a Singularity image from a Docker image.
Binaries/Executables
Check Software Pages
Setting up your environment
What’s available?
[bin]$ cd ~ [~]$ cd intro_to_hpc/ []$ ml []$ module avail []$ module load gcc/12.2.0 []$ ml []$ module avail # What is different compared to the first time we called ml? []$ module load gcc/11.2.0 # What happened?
What’s available? Compiler tree
[]$ module load gcc/11.2.0 # What happened? Due to MODULEPATH changes, the following have been reloaded: 1) gmp/6.2.1 2) mpfr/4.1.0 3) zlib/1.2.12 4) zstd/1.5.2 The following have been reloaded with a version change: 1) gcc/12.2.0 => gcc/11.2.0 # You can only have one compiler loaded at a time within a session. []$ ml []$ module avail []$ module purge []$ ml # What do you notice?
What’s available? Modules loaded by default
[]$ module purge The following modules were not unloaded: (Use "module --force purge" to unload all): 1) slurm/latest 2) arcc/1.0 []$ ml Currently Loaded Modules: 1) slurm/latest (S) 2) arcc/1.0 (S) Where: S: Module is Sticky, requires --force to unload or purge
Module load/spider: Dependencies
[]$ module load python/3.10.6 Lmod has detected the following error: These module(s) or extension(s) exist but cannot be loaded as requested: "python/3.10.6" Try: "module spider python/3.10.6" to see how to load the module(s).
What’s different between these command-lines?
[]$ module spider python/3.10.6 []$ module spider python/3.10.8
What’s different between these command-lines? Dependencies
[]$ module spider python/3.10.6
----------------------------------------
python: python/3.10.6
----------------------------------------
You will need to load all module(s) on any one of the lines below before the "python/3.10.6" module is available to load.
arcc/1.0 gcc/12.2.0
Help:
The Python programming language.
[]$ module spider python/3.10.8
----------------------------------------
python: python/3.10.8
----------------------------------------
You will need to load all module(s) on any one of the lines below before the "python/3.10.8" module is available to load.
arcc/1.0 gcc/11.2.0
Help:
The Python programming language.
Setup Python environment
[]$ module purge []$ module load gcc/12.2.0 []$ module load python/3.10.6 []$ python –version # Single line: # Order matters: []$ module purge []$ module load python/3.10.6 gcc/12.2.0 vs []$ module load gcc/12.2.0 python/3.10.6
What’s happened to the PATH environment variable?
[]$ module purge []$ echo $PATH []$ module load gcc/12.2.0 python/3.10.6 []$ echo $PATH []$ which python /apps/u/spack/gcc/12.2.0/python/3.10.6-7ginwsd/bin/python
Can we use the R language?
[]$ module purge []$ r []$ R # Can we find an 'R' module? []$ module avail []$ module spider r # What do we see and why? []$ module load r/4.2.2 # How do we fix it?
Can we use the R language? Fixed
[]$ module purge []$ r []$ R # Can we find an 'R' module? []$ module avail []$ module spider r # What do we see and why? []$ module load r/4.2.2 # How do we fix it? []$ module load gcc/12.2.0 r/4.2.2 []$ ml []$ R --version
Can we also use the Python language?
[]$ module load python/3.10.6 []$ python --version Python 3.10.6 # Where’s the gcc/12.2.0? # What happens if we: []$ module load gcc/11.2.0 python/3.10.8 []$ python –-version []$ R --version
Remember:
Only one compiler/version can be loaded into your environment at a time.
Can only load languages/applications built with the same compiler.
But, even this can introduce dependency issues.