Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Goal: Understand some of the basic conda configurations and where environments are stored.

Table of Contents
stylenone

...

Where is my Environment?

When we created the py_env environment, you would have noticed within the output:

Code Block
environment location: /home/<username>/.conda/envs/py_env
Code Block
[~]$ cd .conda
[.conda]$ ls
aau_token  environments.txt  envs  pkgs

# Lists the known/named created conda environments.
[.conda]$ cat environments.txt
/home/<username>/.conda/envs/py_env

# Where conda environments are created:
[.conda]$ ls envs/
py_env

# Where downloaded packages are cached before being unpacked/installed.
[.conda]$ ls pkgs/
bottleneck-1.3.7-py312ha883a20_0        mkl_random-1.2.4-py312hdb19cb5_0.conda   pip-24.0-py312h06a4308_0.conda
...

...

How Large is my Environment?

Code Block
[.conda]$ du -d 1 -h
1.2G    ./envs
247M    ./pkgs
1.4G    .

This one single conda environment (Python, numpy and pandas) is >1.2G

The size of the original packages downloaded in nearly 250M

Issue:

  • You home folder has limited storage - you will quickly fill this up with more and larger conda environments?

  • Can we change these default locations?

...

Conda Info

Code Block
[]$ module purge
[]$ module load miniconda3/24.3.0
[]$ conda info
...
       user config file : /home/<username>/.condarc
          conda version : 24.3.0

       base environment : /apps/u/opt/linux/miniconda3/24.3.0  (read only)

           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64   # Channels within that packages will be searched for.
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /apps/u/opt/linux/miniconda3/24.3.0/pkgs
                          /home/<username>/.conda/pkgs                   # Where downloaded packages will be stored.
       envs directories : /home/<username>/.conda/envs                   # Where environments will be created.
                          /apps/u/opt/linux/miniconda3/24.3.0/envs       # Read only.
Expand
titleFull Info Details
Code Block
[~]$ module load miniconda3/24.3.0
[~]$ conda info

     active environment : None
            shell level : 0
       user config file : /home/<username>/.condarc
 populated config files :
          conda version : 24.3.0
    conda-build version : not installed
         python version : 3.12.2.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=zen4
                          __conda=24.3.0=0
                          __glibc=2.34=0
                          __linux=5.14.0=0
                          __unix=0=0
       base environment : /apps/u/opt/linux/miniconda3/24.3.0  (read only)
      conda av data dir : /apps/u/opt/linux/miniconda3/24.3.0/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /apps/u/opt/linux/miniconda3/24.3.0/pkgs
                          /home/<username>/.conda/pkgs
       envs directories : /home/<username>/.conda/envs
                          /apps/u/opt/linux/miniconda3/24.3.0/envs
               platform : linux-64
             user-agent : conda/24.3.0 requests/2.31.0 CPython/3.12.2 Linux/5.14.0-362.24.1.el9_3.x86_64 rhel/9.3 glibc/2.34 solver/libmamba conda-libmamba-solver/24.1.0 libmambapy/1.5.8 aau/0.4.4 c/. s/.
                UID:GID : 10275911:10275911
             netrc file : None
           offline mode : False

...

.condarc

Info

The Conda Runtime Configuration file, an optional .yaml file that allows you to configure many aspects of conda, such as which channels it searches for packages, proxy settings, and environment directories

Note

This file is not created by default, you will need to create it. In the example below we use vim to create it.

Info
Code Block
# In your home folder:
[~]$ vim .condarc
envs_dirs:
  - /project/<project-name>/<username>/conda/envs

pkgs_dirs:
  - /project/<project-name>/<username>/conda/pkgs

always_yes: true
Info
  • The final option “Choose the yes option whenever asked to proceed, such as when installing.“

...

Check our Configuration Updates

Code Block
[~]$ conda info
...
          package cache : /project/<project-name>/<username>/conda/pkgs
       envs directories : /project/<project-name>/<username>/conda/envs
                          /home/<username>/.conda/envs
                          /apps/u/opt/linux/miniconda3/24.3.0/envs

...