Miniconda

Overview

Miniconda is a small, bootstrap version of Anaconda that includes only conda, Python, the packages they depend on, and a small number of other useful packages, including pip, zlib and a few others. It can also be used to start with a minimal set of installed packages, and then install only the required ones (to save space).

Using

Use the module name miniconda3 to discover versions available and to load the application.

Check the required dependencies for a particular version. For example, on teton version 4.9.2 has a dependency on gcc/7.3.0, while 4.10.3 does not.

Training:

ARCC’s training pages are currently under review/revision.

Install Packages into a Miniconda Environment in Your Home Directory

The following commands are useful for creating and using environments in your home directory:

`conda info --envs`	show environments
`conda create -n NAME`	create an environment named NAME in your home related conda location.
`conda create -n NAME python=3.4`	create an environment named NAME, with specified Python version
`conda create -p NAME`	create an environment named NAME in your current location.
`conda activate NAME`	activate environment NAME
`conda install PKG`	install package PKG into the current environment
`conda install -c CHANNEL PKG`	install package PKG from channel CHANNEL into the current environment
`conda list`	show packages in current environment
`conda list -n NAME`	show packages in environment NAME
`conda deactivate`	deactivate current environment

Installing Python Packages

After creating and activating an environment, packages available through Anaconda can be installed into that environment using the conda install command. For example, NumPy can be installed by:

conda install numpy

There are also packages available through different channels. For example, PyNIO and PyNGL from NCAR are available through the conda-forge channel, and can be installed by:

conda install -c conda-forge pynio
conda install -c conda-forge pyngl

Searching for Packages in Channels

To search if a package is available in the current channels, e.g. the package demultiplex:

conda search demultiplex

To search if a package is available from a particular channel (the conda-forge channel in this example):

conda search -c conda-forge demultiplex

If a package isn't available from a conda channel but is available as a PyPi package (usually installed with pip), it can still be installed into a conda environment.

Examples

Example: Installing a `conda` Package

Using the above commands, here is an example of creating an environment and installing the NumPy, PyNIO and PyNGL packages into it:

Load Miniconda (and dependecies):
```
module load miniconda3/<version> 
```

Create and activate the environment:

conda create -n hdf python=2.7
conda activate hdf

Install the packages:

conda install numpy
conda install -c conda-forge pynio
conda install -c conda-forge pyngl

Deactivate environment:
```
conda deactivate
```

It is also possible to interactively search/browse conda packages on the Anaconda Cloud site.

Example: Installing a PyPi Package

Using the above commands, here is an example of creating an environment and installing the PyPi demultiplex package into it:

Load Miniconda (and dependencies):
```
module load miniconda3/<version>
```

Create and activate the environment:

conda create -n demux
conda activate demux

Install pip into this environment:
```
conda install pip
```
Update pip (in the conda environment):
```
pip install --upgrade pip
```
Use pip (from the conda environment) to install the demultiplex package:
```
pip install demultiplex
```
Deactivate environment:
```
conda deactivate
```

Using Installed Python Packages

There are two ways to use Python packages installed in conda environments. For example, to use the demultiplex package installed above:

Activate the demux environment, then a Python script with 'import demultiplex' will find the package, and when done running it, deactivate the environment.
At the top of the Python script, use Python from the demux environment directly — this method does not require activating and deactivating the environment — the top line of the script would be:

#!/home/uname/.conda/envs/demux/bin/python

where uname is replaced with your username, and demux is replaced by the name of your environment.

Example: Installing a C/C++ Package

Besides Python packages, conda can be used to install packages for various other languages such as R, C, and C++. For example, the GNU Scientific Library (GSL) is a numerical library of various mathematical functions that can be used in C and C++ programs. It is available as a conda package and can be installed by:

Load Miniconda (and dependecies):
```
module load miniconda3/<version>
```
Create and activate the environment:
```
conda create -n gsl
conda activate gsl
```
Install the packages:
```
conda install gsl
```
Deactivate environment:
```
conda deactivate
```

After installing gsl into a conda environment as above, it is possible to use it in a C or C++ program by adding the necessary include and/or library paths to the paths that will be searched, e.g.:

add to INCLUDE path: ~/.conda/envs/gsl/include/gsl
add to LIBRARY path: ~/.conda/envs/gsl/lib

Note that the include and library paths will be different for different packages, and can be found by (from home directory) 'cd .conda/envs/NAME' where NAME is the name of the environment that the package was installed into, and then looking to see where the different types of files are.

Troubleshooting and Other Notes

Python packages: All Python packages that will be needed in programs using a package will need to be installed into the same environment, as the instance of Python from this environment will be used to run the programs. To install additional packages, activate the environment, install the packages (with conda if possible, otherwise with pip), then deactivate the environment.
Installing PyPi packages with pip: If an attempt to install a package gives an error about trying to write to a location to which there isn't permission, this is usually a result of a PATH issue.
- It is important to use the instance of pip that was installed in the conda environment, and not one from another location that may be available in the path.
- The current path can be seen by running 'echo $PATH'.
- Issues with incorrect directories being included, or occurring in the wrong order, in a path can usually be resolved by logging out and back in, loading Miniconda, then activating the necessary environment.
Conda environment dependencies:
Within the Conda documentation on Managing environments, specifically the section on Building identical conda environments, it states:
- You can use explicit specification files to build an identical conda environment on the same operating system platform, either on the same machine or on a different machine.
- An explicit spec file is not usually cross platform, and therefore has a comment at the top such as # platform: osx-64 showing the platform where it was created. This platform is the one where this spec file is known to work. On other platforms, the packages specified might not be available or dependencies might be missing for some of the key packages already in the spec.
- Conda does not check architecture or dependencies when installing from a spec file. To ensure that the packages work correctly, make sure that the file was created from a working environment, and use it on the same architecture, operating system, and platform, such as linux-64 or osx-64.
The Miniconda documentation states:
- System requirements License:
  - Operating system: Windows 8 or newer, 64-bit macOS 10.13+, or Linux, including Ubuntu, RedHat, CentOS 7+, and others.

Common Error Messages

Error: /lib64/libk5crypto.so.3: undefined symbol: EVP_KDF_ctrl, version OPENSSL_1_1_1b

[jcook27@wilog01 multiqc]$ git clone https://github.com/ewels/MultiQC_TestData
Cloning into 'MultiQC_TestData'...
/usr/libexec/git-core/git-remote-https: symbol lookup error: /lib64/libk5crypto.so.3: undefined symbol: EVP_KDF_ctrl, version OPENSSL_1_1_1b

https://bugzilla.redhat.com/show_bug.cgi?id=1829790

This appears to be a bug related to conda and Red Hat.

Solution: Exiting the session and logging back in should restore functionality.

Error: json.decoder.JSONDecodeError: Expecting value: line
This may occur while trying to install packages, or creating a new environment.
Solution: Try conda clean -i in your account then re-perform the conda install step.
The clean command removes unused packages and caches, with the -i option removing the index cache that might have become corrupted..