Versions Compared

Old Version 2

changes.mady.by.user Simon Alexander

Saved on

compared with

New Version Current

changes.mady.by.user Simon Alexander

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Info

Yes, but…

Remember, performing a module load is basically extending various environment variables, including PATH. So, in principle if you have loaded a module that provides access to say some command-line utility, scripts and/or libraries, then these are still available.

But, will they function as expected? Maybe…

Since you have a Conda environment activated, then whatever is within it is being used. For example, maybe the Conda environment has been built with Python version 3.9 - but what happens if the module you have loaded is running scripts that have been built to use Python version 3.12 and are not compatible with 3.9? Will they run?

The moral of this answer is that in principle you can, but you need to test that there are no dependency issues that might occur.

Also, regards reproducibility, you can share the Conda environment, but how will you share the module load to say work on another/different cluster/desktop environment?

...

What is the /cluster/medicine/ path I'm Seeing?

Note

Question: Why am I seeing absolute paths of the form /cluster/medicinebow/project/<project-name>/ and not just /project/<project-name>/? to activate my Conda environments?

I thought /project/ was from the root?

Info

This is to do with how our clusters are setup and access the shared file system, and an advance topic.

When you’re on a login node/compute node you can list what is local to that machine - and in that respect, performing the following:

Code Block
[@mblog1 ~]$ ls /
afs     apps  boot     dev  gscratch  install  lib64  misc  net  proc     root  sbin      srv  sys  usr  vast
alcova  bin   cluster  etc  home      lib      media  mnt   opt  project  run   software  SSH  tmp  var

You are listing what is from the / root local on this machine.

Remember: From the HPC cluster perspective, we are using a shared file system that can be accessed from all servers/nodes on that cluster.

If you perform a long form list, you can actually see that /home, /project and /gscratch are symbolic links (notice the l at the beginning of the permissions column) whose full paths are actually from /cluster/medicine/ from the perspective of the file system.

Code Block
[salexan5@mblog1 ~]$ ls -l /
...
lrwxrwxrwx     1 root root    24 May 20 12:23 gscratch -> /cluster/medbow/gscratch
lrwxrwxrwx     1 root root    20 May 20 12:23 home -> /cluster/medbow/home
...
lrwxrwxrwx     1 root root    23 May 20 12:23 project -> /cluster/medbow/project
...

So, when we list the absolute paths for these various files we are returning their path with respect to the file system, not the local machine.

...

Prev

Example 04: Application: Qiime Ecosystem

Workshop Home

Intro to Using Conda

Next

Conda Workshop: Summary

...