Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Next »

Goal: Understand some of the basic conda configurations and where environments are stored.

Where is my Environment?

When we created the py_env environment, you would have noticed within the output:

environment location: /home/salexan5/.conda/envs/py_env
[salexan5@mblog1 ~]$ cd .conda
[salexan5@mblog1 .conda]$ ls
aau_token  environments.txt  envs  pkgs

# Lists the known/named created conda environments.
[salexan5@mblog1 .conda]$ cat environments.txt
/home/salexan5/.conda/envs/py_env

# Where conda environments are created:
[salexan5@mblog1 .conda]$ ls envs/
py_env

# Where downloaded packages are cached before being unpacked/installed.
[salexan5@mblog1 .conda]$ ls pkgs/
bottleneck-1.3.7-py312ha883a20_0        mkl_random-1.2.4-py312hdb19cb5_0.conda   pip-24.0-py312h06a4308_0.conda
...

How Large is my Environment?

[salexan5@mblog1 .conda]$ du -d 1 -h
1.2G    ./envs
247M    ./pkgs
1.4G    .

This one single conda environment (Python, numpy and pandas) is >1.2G

The size of the original packages downloaded in nearly 250M

Issue:

  • You home folder has limited storage - you will quickly fill this up with more and larger conda environments?

  • Can we change these default locations?

Conda Info

[salexan5@mblog1 ~]$ module load miniconda3
[salexan5@mblog1 ~]$ conda info
...
       user config file : /home/salexan5/.condarc
          conda version : 24.3.0

       base environment : /apps/u/opt/linux/miniconda3/24.3.0  (read only)

           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64   # Channels within that packages will be searched for.
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /apps/u/opt/linux/miniconda3/24.3.0/pkgs
                          /home/salexan5/.conda/pkgs                     # Where downloaded packages will be stored.
       envs directories : /home/salexan5/.conda/envs                     # Where environments will be created.
                          /apps/u/opt/linux/miniconda3/24.3.0/envs       # Read only.
 Full Info Details
[salexan5@mblog1 ~]$ module load miniconda3
[salexan5@mblog1 ~]$ conda info

     active environment : None
            shell level : 0
       user config file : /home/salexan5/.condarc
 populated config files :
          conda version : 24.3.0
    conda-build version : not installed
         python version : 3.12.2.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=zen4
                          __conda=24.3.0=0
                          __glibc=2.34=0
                          __linux=5.14.0=0
                          __unix=0=0
       base environment : /apps/u/opt/linux/miniconda3/24.3.0  (read only)
      conda av data dir : /apps/u/opt/linux/miniconda3/24.3.0/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /apps/u/opt/linux/miniconda3/24.3.0/pkgs
                          /home/salexan5/.conda/pkgs
       envs directories : /home/salexan5/.conda/envs
                          /apps/u/opt/linux/miniconda3/24.3.0/envs
               platform : linux-64
             user-agent : conda/24.3.0 requests/2.31.0 CPython/3.12.2 Linux/5.14.0-362.24.1.el9_3.x86_64 rhel/9.3 glibc/2.34 solver/libmamba conda-libmamba-solver/24.1.0 libmambapy/1.5.8 aau/0.4.4 c/. s/.
                UID:GID : 10275911:10275911
             netrc file : None
           offline mode : False

.condarc

The Conda Runtime Configuration file, an optional .yaml file that allows you to configure many aspects of conda, such as which channels it searches for packages, proxy settings, and environment directories

See: Using the .condarc conda configuration file and Settings.

Lets update the environment creation location and cached package folder:

# In your home folder:
[salexan5@mblog1 ~]$ vim .condarc
envs_dirs:
  - /project/arcc/salexan5/conda/envs

pkgs_dirs:
  - /project/arcc/salexan5/conda/pkgs

always_yes: true

The final option “Choose the yes option whenever asked to proceed, such as when installing.“

Check our Configuration Updates

[salexan5@mblog1 ~]$ conda info
...
          package cache : /project/arcc/salexan5/conda/pkgs
       envs directories : /project/arcc/salexan5/conda/envs
                          /home/salexan5/.conda/envs
                          /apps/u/opt/linux/miniconda3/24.3.0/envs

Example 01: Python, Numpy and Pandas Environment

 

Example 02: R and TidyVerse

  • No labels