Miniconda

Overview

Miniconda is a small, bootstrap version of Anaconda that includes only conda, Python, the packages they depend on, and a small number of other useful packages, including pip, zlib and a few others. It can also be used to start with a minimal set of installed packages, and then install only the required ones (to save space).

Using

Use the module name miniconda3 to discover versions available and to load the application.

Training:

ARCC’s training pages are currently under review/revision.

Install Packages into a Miniconda Environment in Your Home Directory

The following commands are useful for creating and using environments in your home directory:

Installing Python Packages

After creating and activating an environment, packages available through Anaconda can be installed into that environment using the conda install command. For example, NumPy can be installed by:

There are also packages available through different channels. For example, PyNIO and PyNGL from NCAR are available through the conda-forge channel, and can be installed by:

Searching for Packages in Channels

To search if a package is available in the current channels, e.g. the package demultiplex:

To search if a package is available from a particular channel (the conda-forge channel in this example):

If a package isn't available from a conda channel but is available as a PyPi package (usually installed with pip), it can still be installed into a conda environment.

Examples

Example: Installing a conda Package

Using the above commands, here is an example of creating an environment and installing the NumPy, PyNIO and PyNGL packages into it:

1. Load Miniconda (and dependecies):

2. Create and activate the environment:

3. Install the packages:

4. Deactivate environment:

It is also possible to interactively search/browse conda packages on the Anaconda Cloud site.

Example: Installing a PyPi Package

Using the above commands, here is an example of creating an environment and installing the PyPi demultiplex package into it:

1. Load Miniconda (and dependencies):

2. Create and activate the environment:

3. Install pip into this environment:

4. Update pip (in the conda environment):

5. Use pip (from the conda environment) to install the demultiplex package:

6. Deactivate environment:

Using Installed Python Packages

There are two ways to use Python packages installed in conda environments. For example, to use the demultiplex package installed above:

• Activate the demux environment, then a Python script with 'import demultiplex' will find the package, and when done running it, deactivate the environment.

• At the top of the Python script, use Python from the demux environment directly — this method does not require activating and deactivating the environment — the top line of the script would be:

where uname is replaced with your username, and demux is replaced by the name of your environment.

Example: Installing a C/C++ Package

Besides Python packages, conda can be used to install packages for various other languages such as R, C, and C++. For example, the GNU Scientific Library (GSL) is a numerical library of various mathematical functions that can be used in C and C++ programs. It is available as a conda package and can be installed by:

1. Load Miniconda (and dependecies):

2. Create and activate the environment:

3. Install the packages:

4. Deactivate environment:

After installing gsl into a conda environment as above, it is possible to use it in a C or C++ program by adding the necessary include and/or library paths to the paths that will be searched, e.g.:

• add to INCLUDE path: ~/.conda/envs/gsl/include/gsl

• add to LIBRARY path: ~/.conda/envs/gsl/lib

Note that the include and library paths will be different for different packages, and can be found by (from home directory) 'cd .conda/envs/NAME' where NAME is the name of the environment that the package was installed into, and then looking to see where the different types of files are.

Troubleshooting and Other Notes

• Python packages: All Python packages that will be needed in programs using a package will need to be installed into the same environment, as the instance of Python from this environment will be used to run the programs. To install additional packages, activate the environment, install the packages (with conda if possible, otherwise with pip), then deactivate the environment.

• Installing PyPi packages with pip: If an attempt to install a package gives an error about trying to write to a location to which there isn't permission, this is usually a result of a PATH issue.

  • It is important to use the instance of pip that was installed in the conda environment, and not one from another location that may be available in the path.

  • The current path can be seen by running 'echo $PATH'.

  • Issues with incorrect directories being included, or occurring in the wrong order, in a path can usually be resolved by logging out and back in, loading Miniconda, then activating the necessary environment.

• Conda environment dependencies:

  Within the Conda documentation on Managing environments, specifically the section on Building identical conda environments, it states:

  • You can use explicit specification files to build an identical conda environment on the same operating system platform, either on the same machine or on a different machine.

  • An explicit spec file is not usually cross platform, and therefore has a comment at the top such as # platform: osx-64 showing the platform where it was created. This platform is the one where this spec file is known to work. On other platforms, the packages specified might not be available or dependencies might be missing for some of the key packages already in the spec.

  • Conda does not check architecture or dependencies when installing from a spec file. To ensure that the packages work correctly, make sure that the file was created from a working environment, and use it on the same architecture, operating system, and platform, such as linux-64 or osx-64.

  The Miniconda documentation states:

  • System requirements License:

    • Operating system: Windows 8 or newer, 64-bit macOS 10.13+, or Linux, including Ubuntu, RedHat, CentOS 7+, and others.

Common Error Messages

Error: /lib64/libk5crypto.so.3: undefined symbol: EVP_KDF_ctrl, version OPENSSL_1_1_1b

This error is typically associated when trying to use git after performing a module load miniconda3/<version>. For example:

This is a know system issue as suggested here https://bugzilla.redhat.com/show_bug.cgi?id=1829790 and relates to trying to use a newer version of openssl than installed on the base system.

There are a number of potentially solutions depending on what your use case is:

1. Use a newer version of git than provided by the base system. Perform a module spider git to see available versions and load.

2. If you’re using an active conda environment, then you can explicitly install conda into it via conda install -c anaconda git.

Error: json.decoder.JSONDecodeError: Expecting value: line

This may occur while trying to install packages, or creating a new environment.
Solution: Try conda clean -i in your account then re-perform the conda install step.
The clean command removes unused packages and caches, with the -i option removing the index cache that might have become corrupted..

Further Information

• Conda User Guide

• Anaconda Package Repository