/
Reproducibility and Sharing

Reproducibility and Sharing

Goal: Introduce how conda environments can be reproduced and shared.

Based on Managing Environments.


In the following examples, we assume you already have a miniconda3 module already loaded.


Clone an Environment

Existing Conda environments can be cloned (essentially copied).

[]$ conda create --help ... options: ... --clone ENV Create a new environment as a copy of an existing local environment. ...

Example

[]$ conda create -p py_env2 --clone py_env Retrieving notices: ...working... done Source: /home/<username>/.conda/envs/py_env Destination: /cluster/medbow/project/<project-name>/<username>/software/py_env2 Packages: 39 Files: 1 ... # # To activate this environment, use # $ conda activate /cluster/medbow/project/<project-name>/<username>/software/py_env2

Notice: The source (environment being cloned) and the destination (new cloned environment) are listed.

In the above, we are using the -p option so the destination is within the current working directory, and cloning from an existing environment that was created within the default environment location - so only its name (not absolute path) is required.

[]$ pwd /project/<project-name>/<username>/software []$ ls slim_env_4.2.2 []$ conda create -p py_env2 --clone py_env Retrieving notices: ...working... done Source: /home/<username>/.conda/envs/py_env Destination: /cluster/medbow/project/<project-name>/<username>/software/py_env2 Packages: 39 Files: 1 Downloading and Extracting Packages: Downloading and Extracting Packages: Preparing transaction: done Verifying transaction: done Executing transaction: done # # To activate this environment, use # # $ conda activate /cluster/medbow/project/<project-name>/<username>/software/py_env2 # # To deactivate an active environment, use # # $ conda deactivate []$ ls py_env2 slim_env_4.2.2

Using Cloned Environment

Notice how you need to activate/use this cloned environment:

[]$ cat ~/.conda/environments.txt /home/<username>/.conda/envs/py_env /project/<project-name>/<username>/conda/envs/r_env /cluster/medbow/project/<project-name>/<username>/software/slim_env_4.2.2 /cluster/medbow/project/<project-name>/<username>/software/py_env2 []$ conda info --env # conda environments: # base /apps/u/opt/linux/miniconda3/24.3.0 /cluster/medbow/project/<project-name>/<username>/software/py_env2 /cluster/medbow/project/<project-name>/<username>/software/slim_env_4.2.2 py_env /home/<username>/.conda/envs/py_env r_env /project/<project-name>/<username>/conda/envs/r_env

Need to define the full path to activate.

[]$ conda activate /cluster/medbow/project/<project-name>/<username>/software/py_env2 (/cluster/medbow/project/<project-name>/<username>/software/py_env2) []$

Conda env command

The env sub-command provides the a process that can be used to export into a readable text file a description of the entire Conda environment that can then used to create (essentially import) to replicate.

[]$ conda env --help usage: conda env [-h] command ... positional arguments: command config Configure a conda environment. create Create an environment based on an environment definition file. export Export a given environment list List the Conda environments. remove Remove an environment. update Update the current environment based on environment file. options: -h, --help Show this help message and exit.

Export an Environment

Within an active environment we can export into a readable file a description of the entire Conda environment. This describes all the system related libraries as well as anything that that was explicitly conda install-ed.

[]$ conda activate r_env (r_env) []$ conda env export > r_env.yml (r_env) []$ conda deactivate []$ cat r_env.yml name: r_env channels: - conda-forge - defaults dependencies: - _libgcc_mutex=0.1=main ... - zstd=1.5.5=hc292b87_2 prefix: /project/<project-name>/<username>/conda/envs/r_env

Import an Environment

Using the env create sub-command we can then define a file to read/import that describes the environment we want to create - essentially cloning it.

[]$ cd /project/<project-name>/<username>/software/ []$ conda env create -p r_env2 --file r_env.yml ... # # To activate this environment, use # # $ conda activate /cluster/medbow/project/<project-name>/<username>/software/r_env2 ...

 

Related content