Goal: Create Demonstrate a basic conda environment creation workflow by creating a Python environment that contains the numpy and pandas packages.
...
General Process
Code Block |
[salexan5@mblog1 ~]$ module load miniconda3/24.3.0
[salexan5@mblog1 ~ Info |
---|
Below demonstrates the general process for: Setting up your environment to use Conda via a module load. How to create a conda environment. How to activate/start your environment and install packages. How to then use what you’ve installed. And then how to deactivate/finish using your environment.
|
Code Block |
---|
[]$ module purge
[]$ module load miniconda3/24.3.0
[]$ conda search python
[salexan5@mblog1 ~]$ conda create -n py_env
[salexan5@mblog1 ~]$ conda activate py_env
(py_env) [salexan5@mblog1 ~]$ conda install python=3.12.4
(py_env) [salexan5@mblog1 ~]$ python --version
Python 3.12.4
(py_env) [salexan5@mblog1 ~]$ conda search numpy
(py_env) [salexan5@mblog1 ~]$ conda install numpy
(py_env) [salexan5@mblog1 ~]$ python -c "import numpy; print(numpy.__version__)"
1.26.4
(py_env) [salexan5@mblog1 ~]$ conda deactivate
[salexan5@mblog1 ~]$ |
...
Search for Packages
Code Block |
[salexan5@mblog1 ~ Info |
---|
Conda enables you look up and search for packages and what versions are available. |
Code Block |
---|
[]$ conda search python
Loading channels: done
# Name Version Build Channel
python 2.7.13 hac47a24_15 pkgs/main
...
python 3.12.3 h996f2a0_1 pkgs/main
python 3.12.4 h5148396_1 pkgs/main |
Info |
---|
Notice: Although Python version 2 has been deprecated, it is still used for old packages/modules/scripts. You can create a conda environment that provides this old, no longer supported version. This is how ARCC provides this version on the cluster. |
Code Block |
---|
(py_env) [salexan5@mblog1 ~]$ conda search numpy
Loading channels: done
# Name Version Build Channel
numpy 1.9.3 py27_nomklhbee5d10_3 pkgs/main
...
numpy 1.26.4 py39heeff2f4_0 pkgs/main |
Create an Environment
Code Block |
---|
[salexan5@mblog1 ~]$ conda create -n py_env
Channels:
- defaults
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /home/salexan5/.conda/envs/py_env
Proceed ([y]/n)? y |
Info |
---|
Note the location that the environment will be saved to: environment location: /home/salexan5/.conda/envs/py_env |
Create an Environment: Proceed
Code Block |
---|
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate py_env
#
# To deactivate an active environment, use
#
# $ conda deactivate |
Info |
---|
Note the command required to activate your environment: conda activate py_env This will be required every time you wish to use it. |
Activate an Environment
Code Block |
---|
[salexan5@mblog1 ~]$ conda activate py_env
(py_env) [salexan5@mblog1 ~]$ |
Info |
---|
Note how the command line prompt has changed once the environment has been activated: (py_env) [...] This indicates the name of the conda environment that is currently active. |
Conda Install a Version of Python
Code Block |
---|
(py_env) [salexan5@mblog1 ~]$ conda install python=3.12.4 |
...
title | Output of what is happening: |
---|
...
Create an Environment
Info |
---|
Use the create sub command to create your environment. The -n option will create your environment (which is just a folder) in the default configured location. Out the box, this will be under: /home/<username>/.conda/envs/ |
Code Block |
---|
[]$ conda create -n py_env
Channels:
- defaults
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /home/salexan5/.conda/envs/py_env
Proceed ([y]/n)? y |
Info |
---|
Note the location that the environment will be saved to: environment location: /home/salexan5/.conda/envs/py_env |
...
Create an Environment: Proceed
Code Block |
---|
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate py_env
#
# To deactivate an active environment, use
#
# $ conda deactivate |
Info |
---|
Note the command required to activate your environment: conda activate py_env This will be required every time you wish to use it. |
...
Activate an Environment
Info |
---|
Before you can use a conda environment it must be activated. Every time you want to use it, you must remember to activate it. |
Code Block |
---|
[]$ conda activate py_env
(py_env) []$ |
Info |
---|
Note: See how the command line prompt has changed once the environment has been activated: (py_env) [...] This indicates the name of the conda environment that is currently active. |
...
Conda Install a Version of Python
Info |
---|
Once a conda environment is active, you can start to use it, and one aspect of this is to install conda packages. |
Code Block |
---|
(py_env) []$ conda install python=3.12.4 |
Expand |
---|
title | Output of what is happening: |
---|
|
Code Block |
---|
(py_env) []$ conda install python=3.12.4
Channels:
- defaults
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /home/salexan5/.conda/envs/py_env
added / updated specs:
- python=3.12.4
The following packages will be downloaded:
package | build
---------------------------|-----------------
pip-24.0 | py312h06a4308_0 3.3 MB
setuptools-69.5.1 | py312h06a4308_0 1.3 MB
wheel-0.43.0 | py312h06a4308_0 142 buildKB
---------------------------|-----------------
pip-24.0----------------
| py312h06a4308_0 3.3 MB setuptools-69.5.1 Total: | py312h06a4308_0 1.34.7 MB
The following wheel-0.43.0 NEW packages will be INSTALLED:
_libgcc_mutex | py312h06a4308_0 142 KB
------------------------------------------------------------
pkgs/main/linux-64::_libgcc_mutex-0.1-main
_openmp_mutex pkgs/main/linux-64::_openmp_mutex-5.1-1_gnu
bzip2 pkgs/main/linux-64::bzip2-1.0.8-h5eee18b_6
ca-certificates pkgs/main/linux-64::ca-certificates-2024.3.11-h06a4308_0
expat Total: 4.7 MB
The following NEW packages will be INSTALLED:
_libgcc_mutex pkgs/main/linux-64::_libgcc_mutex-0.1-main
_openmp_mutexpkgs/main/linux-64::expat-2.6.2-h6a678d5_0
ld_impl_linux-64 pkgs/main/linux-64::ld_impl_linux-64-2.38-h1181459_1
libffi pkgs/main/linux-64::_openmp_mutex-5.1-1_gnu
bzip2 libffi-3.4.4-h6a678d5_1
libgcc-ng pkgs/main/linux-64::bzip2-1libgcc-ng-11.2.0.8-h5eee18b_6-h1234567_1
libgomp ca-certificates pkgs/main/linux-64::calibgomp-certificates-202411.32.110-h06a4308h1234567_01
expat libstdcxx-ng pkgs/main/linux-64::expatlibstdcxx-ng-11.2.6.20-h6a678d5_0h1234567_1
libuuid ld_impl_linux-64 pkgs/main/linux-64::ld_impl_linux-64-2.38-h1181459_1::libuuid-1.41.5-h5eee18b_0
libffi ncurses pkgs/main/linux-64::libffincurses-36.4.4-h6a678d5_10
openssl libgcc-ng pkgs/main/linux-64::libgccopenssl-ng-113.20.014-h1234567_1h5eee18b_0
pip libgomp pkgs/main/linux-64::libgomppip-1124.2.0-h1234567_1py312h06a4308_0
python libstdcxx-ng pkgs/main/linux-64::libstdcxxpython-ng-113.212.04-h1234567h5148396_1
libuuidreadline pkgs/main/linux-64::libuuidreadline-18.41.52-h5eee18b_0
ncurses setuptools pkgs/main/linux-64::ncursessetuptools-669.5.41-h6a678d5py312h06a4308_0
opensslsqlite pkgs/main/linux-64::opensslsqlite-3.045.143-h5eee18b_0
piptk pkgs/main/linux-64::piptk-248.6.014-py312h06a4308h39e8969_0
pythontzdata pkgs/main/linux-64noarch::python-3.12.4-h5148396_1tzdata-2024a-h04d1e81_0
wheel readline pkgs/main/linux-64::readlinewheel-80.43.20-h5eee18bpy312h06a4308_0
xz setuptools pkgs/main/linux-64::setuptoolsxz-695.54.16-py312h06a4308h5eee18b_01
zlib sqlite pkgs/main/linux-64::sqlitezlib-31.452.313-h5eee18b_01
tk Proceed ([y]/n)? y
Downloading and Extracting Packages:
Preparing transaction: done
Verifying transaction: done
Executing pkgs/main/linux-64::tk-8.6.14-h39e8969_0
tzdata pkgs/main/noarch::tzdata-2024a-h04d1e81_0
wheeltransaction: done |
|
Info |
---|
Note all the addition dependencies/libraries that are being installed into the environment |
Lets check the version installed, within are active environment.
Code Block |
---|
(py_env) []$ python --version
Python 3.12.4 |
...
Conda Install the numpy Package
Info |
---|
Let’s first check if numpy is available as a package, and if it is, which versions are available. |
Code Block |
---|
(py_env) []$ conda search numpy
Loading channels: done
# Name pkgs/main/linux-64::wheel-0.43.0-py312h06a4308_0 xzVersion Build Channel
numpy pkgs/main/linux-64::xz-5.4.6-h5eee18b_1 zlib pkgs/main/linux-64::zlib-1.2.13-h5eee18b_1 Proceed ([y]/n)? y
Downloading and Extracting Packages:
Preparing transaction: done
Verifying transaction: done
Executing transaction: done |
Info |
---|
Note all the addition dependencies/libraries that are being installed into the environment |
Lets check the version installed, within are active environment.
Code Block |
---|
(py_env) [salexan5@mblog1 ~]$ python --version
Python 3.12.4 |
Conda Install the numpy Package
1.9.3 py27_nomklhbee5d10_3 pkgs/main
...
numpy 1.26.4 py39heeff2f4_0 pkgs/main |
...
Expand |
---|
title | Output of what is happening: |
---|
|
|
...
]$ conda install numpy
Channels:
- defaults
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /home/salexan5/.conda/envs/py_env
added / updated specs:
- numpy
The following packages will be downloaded:
package | build
---------------------------|-----------------
mkl-service-2.4.0 | py312h5eee18b_1 66 KB
mkl_fft-1.3.8 | py312h5eee18b_0 204 KB
mkl_random-1.2.4 | py312hdb19cb5_0 284 KB
numpy-1.26.4 | py312hc5e2394_0 11 KB
numpy-base-1.26.4 | py312h0da6c21_0 7.7 MB
------------------------------------------------------------
Total: 8.2 MB
The following NEW packages will be INSTALLED:
blas pkgs/main/linux-64::blas-1.0-mkl
intel-openmp pkgs/main/linux-64::intel-openmp-2023.1.0-hdb19cb5_46306
mkl pkgs/main/linux-64::mkl-2023.1.0-h213fc3f_46344
mkl-service pkgs/main/linux-64::mkl-service-2.4.0-py312h5eee18b_1
mkl_fft pkgs/main/linux-64::mkl_fft-1.3.8-py312h5eee18b_0
mkl_random pkgs/main/linux-64::mkl_random-1.2.4-py312hdb19cb5_0
numpy pkgs/main/linux-64::numpy-1.26.4-py312hc5e2394_0
numpy-base pkgs/main/linux-64::numpy-base-1.26.4-py312h0da6c21_0
tbb pkgs/main/linux-64::tbb-2021.8.0-hdb19cb5_0
Proceed ([y]/n)? y
Downloading and Extracting |
|
...
Packages:
Preparing transaction: done
Verifying transaction: done
|
|
...
Executing transaction: done |
|
...
Info |
---|
Again, note all the addition dependencies/libraries that are being installed into the environment If no package version is defined, typically it’ll install the latest version. |
Lets check the version installed, within are active environment.
...
Info |
---|
(Again) Note: See all the addition dependencies/libraries that are being installed into the environment If no package version is defined, typically it’ll install the latest version. |
Lets check the version installed, within are active environment.
Code Block |
---|
(py_env) []$ python -c "import numpy; print(numpy.__version__)"
1.26.4 |
Note |
---|
Packages are being updated on a frequent basis. When this workshop was created numpy/1.26.4 was the latest version. As of 20240815, version 2.0.1 is available, and in a few weeks a newer version will likely be available. If you do not define a version, then the latest will be installed.
|
...
Conda Deactivate your Environment
Code Block |
---|
(py_env) [salexan5@mblog1 ~]$ conda deactivate
[salexan5@mblog1 ~]$ |
Info |
---|
Note how the command line prompt has changed, reverting back to before being activated: [...] The conda environment is no longer activate and can not be used. |
Code Block |
---|
[salexan5@mblog1 ~]$ python --version
Python 3.12.2
[salexan5@mblog1 ~]$ python -c "import numpy; print(numpy.__version__)"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy' |
...
(Re-)Using the Environment
Info |
---|
To re-use a conda environment, you must activate it again. But, you do not have to re-create it. Creation only happens once. |
Code Block |
---|
[salexan5@mblog1 ~[]$ module purge
[]$ module load miniconda3/24.3.0
[salexan5@mblog1 ~]$ conda activate py_env
(py_env) [salexan5@mblog1 ~]$ python --version
Python 3.12.4
(py_env) [salexan5@mblog1 ~]$ python -c "import numpy; print(numpy.__version__)"
1.26.4
(py_env) [salexan5@mblog1 ~]$ conda deactivate
[salexan5@mblog1 ~]$ |
Info |
---|
Note how the conda environment must be activated to use it. |
Adding to an Existing Environment
Code Block |
---|
[salexan5@mblog1 ~ : The conda environment must be activated to use it. |
...
Adding to an Existing Environment
Info |
---|
Although we only need to create a conda environment once, we can add/update it every time we use it. Let’s go back into are existing conda environment and add the pandas package. |
Code Block |
---|
[]$ module purge
[]$ module load miniconda3/24.3.0
[salexan5@mblog1 ~]$ conda activate py_env
(py_env) [salexan5@mblog1 ~]$ conda install pandas
(py_env) [salexan5@mblog1 ~]$ python py_test.py
Python: 3.12.4 | packaged by Anaconda, Inc. | (main, Jun 18 2024, 15:12:24) [GCC 11.2.0]
Numpy: 1.26.4
Pandas: 2.2.2
(py_env) [salexan5@mblog1 ~]$ conda deactivate |
Expand |
---|
|
Code Block |
---|
import sys
import numpy
import pandas
print("Python: " + str(sys.version))
print("Numpy: " + str(numpy.__version__))
print("Pandas: " + str(pandas.__version__)) |
|
Expand |
---|
title | Installation Output: |
---|
|
Code Block |
---|
(py_env) [salexan5@mblog1 ~]$ conda install pandas
Channels:
- defaults
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /home/salexan5/.conda/envs/py_env
added / updated specs:
- pandas
The following packages will be downloaded:
package | build
---------------------------|-----------------
bottleneck-1.3.7 | py312ha883a20_0 140 KB
numexpr-2.8.7 | py312hf827012_0 149 KB
pandas-2.2.2 | py312h526ad5a_0 15.4 MB
python-dateutil-2.9.0post0 | py312h06a4308_2 318 KB
pytz-2024.1 | py312h06a4308_0 220 KB
------------------------------------------------------------
Total: 16.2 MB
The following NEW packages will be INSTALLED:
bottleneck pkgs/main/linux-64::bottleneck-1.3.7-py312ha883a20_0
numexpr pkgs/main/linux-64::numexpr-2.8.7-py312hf827012_0
pandas pkgs/main/linux-64::pandas-2.2.2-py312h526ad5a_0
python-dateutil pkgs/main/linux-64::python-dateutil-2.9.0post0-py312h06a4308_2
python-tzdata pkgs/main/noarch::python-tzdata-2023.3-pyhd3eb1b0_0
pytz pkgs/main/linux-64::pytz-2024.1-py312h06a4308_0
six pkgs/main/noarch::six-1.16.0-pyhd3eb1b0_1
Proceed ([y]/n)? y
Downloading and Extracting Packages:
Preparing transaction: done
Verifying transaction: done
Executing transaction: done |
|
...