Goal: Understand where pip installs packages within a user’s home folder with respect to different versions of Python.
Available Versions of Python
...
| Table of Contents | ||
|---|---|---|
|
...
Python on the System
Python is available on the System
| Code Block |
|---|
[salexan5@mblog2 ~]$ python --version
Python 3.9.18
[salexan5@mblog1 ~]$ which python
/usr/bin/python |
| Note |
|---|
Note:
|
...
Python as a Module
Different versions are available via loading a module
| Code Block |
|---|
[salexan5@mblog2 ~]$ module spider python
----------------------------------------------------------------------------
python:
----------------------------------------------------------------------------
Versions:
python/3.10.6
python/3.12.0
Other possible modules matches:
python2
[salexan5@mblog2 ~]$ module load gcc/13.2.0 python/3.10.6
[salexan5@mblog2 ~]$ python --version
Python 3.10.6
[salexan5@mblog2 ~]$ which pip
/apps/u/spack/gcc/13.2.0/python/3.10.6-jh3qs5v/bin/pip |
| Info |
|---|
Note:
|
...
Pip Install Numpy - Python 3.10.6
| Code Block |
|---|
# Using python/3.10.6
[salexan5@mblog2 ~]$ pip install numpy
Defaulting to user installation because normal site-packages is not writeable
Collecting numpy
Downloading numpy-2.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.9/60.9 kB 2.0 MB/s eta 0:00:00
Downloading numpy-2.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (19.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.3/19.3 MB 131.3 MB/s eta 0:00:00
Installing collected packages: numpy
Successfully installed numpy-2.0.0
[notice] A new release of pip is available: 24.0 -> 24.1
[notice] To update, run: pip install --upgrade pip |
| Expand | ||
|---|---|---|
| ||
|
...
Use Python 3.12.0 - Is numpy available?
| Code Block |
|---|
[salexan5@mblog2 ~]$ module purge
[salexan5@mblog2 ~]$ module load gcc/13.2.0 python/3.12.0
[salexan5@mblog2 ~]$ python --version
Python 3.12.0
[salexan5@mblog2 ~]$ which python
/apps/u/spack/gcc/13.2.0/python/3.12.0-ovfqpv2/bin/python
[salexan5@mblog2 ~]$ python py_test.py
Traceback (most recent call last):
File "/cluster/medbow/home/salexan5/py_test.py", line 2, in <module>
import numpy
ModuleNotFoundError: No module named 'numpy |
| Note |
|---|
But didn’t we previously install the |
...
Python System Configuration
sysconfig: The sysconfig module provides access to Python’s configuration information like the list of installation paths and the configuration variables relevant for the current platform.
| Code Block |
|---|
[salexan5@mblog2 ~]$ module purge
[salexan5@mblog2 ~]$ module load gcc/13.2.0 python/3.10.6
[salexan5@mblog2 ~]$ python -m sysconfig
Platform: "linux-x86_64"
Python version: "3.10"
Current installation scheme: "posix_prefix"
Paths:
...
scripts = "/apps/u/spack/gcc/13.2.0/python/3.10.6-jh3qs5v/bin"
stdlib = "/apps/u/spack/gcc/13.2.0/python/3.10.6-jh3qs5v/lib/python3.10"
...
Variables:
...
py_version = "3.10.6"
...
py_version_short = "3.10"
...
userbase = "/home/salexan5/.local"
# Full list has over 700 lines! |
| View file | ||
|---|---|---|
|
...
What is the Installation Scheme?
Prefix scheme: The “prefix scheme” is useful when you wish to use one Python installation to perform the build/install (i.e., to run the setup script), but install modules into the third-party module directory of a different Python installation (or something that looks like a different Python installation). If this sounds a trifle unusual, it is—that’s why the user and home schemes come before. However, there are at least two known cases where the prefix scheme will be useful.
posix_prefix:
Path | Installation directory |
|---|---|
stdlib |
|
scripts |
|
data |
|
| Info |
|---|
The prefix will be the |
...
What is under the userbase?
| Code Block |
|---|
[salexan5@mblog2 ~]$ ls .local
bin lib share
[salexan5@mblog2 ~]$ ls -al .local/bin
total 1
drwxr-xr-x 2 salexan5 salexan5 4096 Jun 24 11:16 .
drwxr-xr-x 2 salexan5 salexan5 4096 Jun 24 11:16 ..
-rwxr-xr-x 1 salexan5 salexan5 257 Jun 24 11:16 f2py
-rwxr-xr-x 1 salexan5 salexan5 257 Jun 24 11:16 numpy-config
[salexan5@mblog2 ~]$ ls .local/lib/
python3.10
[salexan5@mblog2 ~]$ ls .local/lib/python3.10/site-packages/
numpy numpy-2.0.0.dist-info numpy.libs
# ~/.local/lib/python3.10 |
...
Why is they no lib/python3.12?
| Note |
|---|
Because you have not pip installed anything for this particular version. |
| Info |
|---|
When using python/3.12.0 it is looking under ~/.local/lib/python3.12 for pip installed packages. Since we have not pip installed numpy using version 3.12.0, no package exists under this folder. |
...
Pip Install Numpy - Python 3.12.0
| Code Block |
|---|
[salexan5@mblog2 ~]$ module purge
[salexan5@mblog2 ~]$ module load gcc/13.2.0 python/3.12.0
[salexan5@mblog2 ~]$ pip install numpy==1.26.4
[salexan5@mblog2 ~]$ python py_test.py
Python: 3.12.0 (main, Apr 30 2024, 11:30:37) [GCC 13.2.0]
Numpy: 1.26.4
[salexan5@mblog2 ~]$ ls .local/lib/
python3.10 python3.12
[salexan5@mblog2 ~]$ ls .local/lib/python3.12/site-packages/
numpy numpy-1.26.4.dist-info numpy.libs |
...
What has changed under the .local/bin folder?
| Code Block |
|---|
# Previously when installing numpy for python/3.10.6
[salexan5@mblog2 ~]$ ls -al .local/bin
total 1
-rwxr-xr-x 1 salexan5 salexan5 257 Jun 24 11:16 f2py
-rwxr-xr-x 1 salexan5 salexan5 257 Jun 24 11:16 numpy-config
[salexan5@mblog2 ~]$ cat .local/bin/f2py
#!/apps/u/spack/gcc/13.2.0/python/3.10.6-jh3qs5v/bin/python
...
# Now after installing numpy for python/3.12.0
[salexan5@mblog2 ~]$ ls -al .local/bin/
-rwxr-xr-x 1 salexan5 salexan5 257 Jun 24 11:45 f2py
-rwxr-xr-x 1 salexan5 salexan5 257 Jun 24 11:16 numpy-config
[salexan5@mblog2 ~]$ cat .local/bin/f2py
#!/apps/u/spack/gcc/13.2.0/python/3.12.0-ovfqpv2/bin/python
... |
| Note |
|---|
The |
...
...