Goal: Understand how Conda’s pip works with a User’s Python Pip pip package Installs.
...
| Table of Contents | ||
|---|---|---|
|
...
| Info |
|---|
Miniconda does ship with a default version of Python. |
| Code Block |
|---|
[salexan5@mblog1 conda]$ cd /project/arcc<project-name>/salexan5<username>/conda [salexan5@mblog1 conda]$ module purge []$ module load miniconda3/24.3.0 [salexan5@mblog1 conda]$ conda create -p py_env ## Package Plan ## environment location: /cluster/medbow/project/arcc<project-name>/salexan5<username>/conda/py_env # To activate this environment, use # $ conda activate /cluster/medbow/project/arcc<project-name>/salexan5<username>/conda/py_env [salexan5@mblog1 conda]$ conda activate /cluster/medbow/project/arcc<project-name>/salexan5<username>/conda/py_env (/cluster/medbow/project/arcc<project-name>/salexan5<username>/conda/py_env) [salexan5@mblog1 conda]$ python --version Python 3.12.2 (/cluster/medbow/project/arcc<project-name>/salexan5<username>/conda/py_env) [salexan5@mblog1 conda]$ which python /apps/u/opt/linux/miniconda3/24.3.0/bin/python |
...
If a package is not available from conda or Anaconda.org, you may be able to find and install the package via conda-forge or with another package manager like
pip.Pip packages do not have all the features of conda packages and we recommend first trying to install any package with conda. If the package is unavailable through conda, try finding and installing it with conda-forge.
If you still cannot install the package, you can try installing it with pip. The differences between pip and conda packages cause certain unavoidable limits in compatibility but conda works hard to be as compatible with pip as possible.
Note:
Both pip and conda are included in Anaconda and Miniconda, so you do not need to install them separately.
It is possible to have pip installed outside a conda environment or inside a conda environment.
| Note |
|---|
Not every Python package can be |
...
Try Installing numpy into our Conda environment
| Code Block |
|---|
(/cluster/medbow/project/arcc/salexan5/conda/py_env) [salexan5@mblog1 conda]$ pipmodule installpurge numpy[]$ Defaultingmodule to user installation because normal site-packages is not writeable Requirement already satisfied: numpy in /home/salexan5/.local/lib/python3.12/site-packages (1.26.4) (load miniconda3/24.3.0 []$ conda activate /cluster/medbow/project/<project-name>/<username>/conda/py_env (/cluster/medbow/project/arcc<project-name>/salexan5<username>/conda/py_env) [salexan5@mblog1 conda]$ python ~/py_test.py Python: 3.12.2 | packaged by Anaconda, Inc. | (main, Feb 27 2024, 17:35:02) [GCC 11.2.0] Numpy: 1.26.4 |
What’s happened?
| Info |
|---|
Conda’s pip will look under the userbase for existing packages. In this case |
Confirm what is Currently Installed/Available
Check for conda installed packages:
| Code Block |
|---|
# This list should be empty since we have essentially a blank conda environment
# since we have not conda installed anything.
(/cluster/medbow/project/arcc/salexan5/conda/py_env) [salexan5@mblog1 conda]$ conda list
# packages in environment at /cluster/medbow/project/arcc/salexan5/conda/py_env:
#
# Name Version Build Channel |
Check for pip installed packages:
| Code Block |
|---|
pip install numpy
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: numpy in /home/<username>/.local/lib/python3.12/site-packages (1.26.4)
(/cluster/medbow/project/<project-name>/<username>/conda/py_env) [salexan5@mblog1 conda]$ python ~/py_test.py
Python: 3.12.2 | packaged by Anaconda, Inc. | (main, Feb 27 2024, 17:35:02) [GCC 11.2.0]
Numpy: 1.26.4 |
What’s happened?
| Info |
|---|
Conda’s pip will look under the userbase for existing packages. In this case |
...
Confirm what is Currently Installed/Available
Check for conda installed packages:
| Code Block |
|---|
# This list should be empty since we have essentially a blank conda environment # since we have not conda installed anything. (/cluster/medbow/project/arcc<project-name>/salexan5<username>/conda/py_env) [salexan5@mblog1 conda]$ pipconda list -v# Packagepackages in environment at /cluster/medbow/project/<project-name>/<username>/conda/py_env: # # Name Version Location Version Build Channel |
Check for pip installed packages:
| Code Block |
|---|
(/cluster/medbow/project/<project-name>/<username>/conda/py_env) []$ pip list -v Package Version Location Installer ----------------------- ----------- -------- Installer ----------------------- ----------- ------------------------ --------- anaconda-anon-usage ------------------------------ --------- anaconda-anon-usage 0.4.4 /apps/u/opt/linux/miniconda3/24.3.0/lib/python3.12/site-packages conda archspec 0.2.3 /apps/u/opt/linux/miniconda3/24.3.0/lib/python3.12/site-packages conda ... numpy 1.26.4 /home/salexan5<username>/.local/lib/python3.12/site-packages pip ... pip 23.3.1 /apps/u/opt/linux/miniconda3/24.3.0/lib/python3.12/site-packages ... |
...
| Code Block |
|---|
(/cluster/medbow/project/arcc<project-name>/salexan5<username>/conda/py_env) [salexan5@mblog1 conda]$ pip install numpy==2.0.0 Defaulting to user installation because normal site-packages is not writeable ... Installing collected packages: numpy Attempting uninstall: numpy Found existing installation: numpy 1.26.4 Uninstalling numpy-1.26.4: Successfully uninstalled numpy-1.26.4 Successfully installed numpy-2.0.0 (/cluster/medbow/project/arcc<project-name>/salexan5<username>/conda/py_env) [salexan5@mblog1 conda]$ pip list -v Package Version Location Installer ----------------------- ----------- ---------------------------------------------------------------- --------- ... numpy 2.0.0 /home/salexan5<username>/.local/lib/python3.12/site-packages pip ... |
| Note |
|---|
The |
...
| Code Block |
|---|
(/cluster/medbow/project/arcc<project-name>/salexan5<username>/conda/py_env) [salexan5@mblog1 conda]$ echo $CONDA_PREFIX /cluster/medbow/project/arcc<project-name>/salexan5<username>/conda/py_env (/cluster/medbow/project/arcc<project-name>/salexan5<username>/conda/py_env_3.12.4) [salexan5@mblog2 conda]$ export PYTHONUSERBASE=$CONDA_PREFIX (/cluster/medbow/project/arcc<project-name>/salexan5<username>/conda/py_env) [salexan5@mblog1 conda]$ pip list -v Package Version Location Installer ----------------------- -------- ------------------------------------------------------------------------------- --------- ... numpy 2.0.0 /cluster/medbow/project/arcc<project-name>/salexan5<username>/conda/py_env/lib/python3.12/site-packages pip ... |
...
| Code Block |
|---|
(/cluster/medbow/project/arcc<project-name>/salexan5<username>/conda/py_env) [salexan5@mblog1 conda]$ pwd /project/arcc<project-name>/salexan5<username>/conda (/cluster/medbow/project/arcc<project-name>/salexan5<username>/conda/py_env) [salexan5@mblog1 conda]$ ls py_env/lib/python3.12/site-packages/ numpy numpy-2.0.0.dist-info numpy.libs |
...
| Note |
|---|
Starting a new session, notice which version of |
| Code Block |
|---|
[salexan5@mblog2 ~]$ module purge []$ module load miniconda3/24.3.0 [salexan5@mblog2 ~]$ conda activate /project/arcc<project-name>/salexan5<username>/conda/py_env (/project/arcc<project-name>/salexan5<username>/conda/py_env) [salexan5@mblog2 ~]$ pip list -v Package Version Location Installer ----------------------- ----------- ---------------------------------------------------------------- --------- ... numpy 2.0.0 /home/salexan5<username>/.local/lib/python3.12/site-packages pip ... (/project/arcc<project-name>/salexan5<username>/conda/py_env) [salexan5@mblog2 ~]$ export PYTHONUSERBASE=$CONDA_PREFIX (/project/arcc<project-name>/salexan5<username>/conda/py_env) [salexan5@mblog2 ~]$ pip list -v Package Version Location Installer ----------------------- -------- ---------------------------------------------------------------- --------- ... numpy 2.0.0 /project/arcc/salexan5/conda/py_env/lib/python3.12/site-packages pip ... |
...
Python Pip Installs on the Cluster
...
/project/<project-name>/<username>/conda/py_env/lib/python3.12/site-packages pip
... |
...
Remember: Conda Environments, Pip Installs and Package Locations
| Info |
|---|
Remember: If you do not set |
...
Prev | Workshop Home | Next |