Versions Compared

Old Version 19

changes.mady.by.user Lisa Stafford

Saved on

compared with

New Version 20

changes.mady.by.user Lisa Stafford

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Table of Contents
stylenone

...

Why

...

use Conda?

Also:

...

  1. By default, local installs will install to one central location within your home directory (pip under ~/.local, install.packages() under ~/R)

    1. Package installations are separated by kernel software versions, but if conflicts exist within the overall install location, packages are overwritten to make the software and dependencies “fit” with your most recently requested installation. This can change installation and available packages for your user profile, breaking older installations and software that you still want to use.

    2. Installing in your $HOME directory makes packages and versions of software in your /home available to you regardless of whether you want them at the time, or not.

    3. This causes software conflicts between versions native to the HPC system, those you may want to load off and on with modules, and those you’ve installed in $HOME, meaning that loaded modules or native HPC software may not work properly or crash due to underlying dependencies can be superseded by packages in $HOME.

...

Dependency Hell is not Limited to Python: Examples in R

tidyverse.png

Also:

...

In these steps, we assume you’ve already created your desired Conda environment. To learn how to create a Conda environment, please see our Conda module.

You should also return to the documentation on exporting conda environments to a python kernel if you have not completed this training.

With our Conda environment already installed and configured, we can now set it up to be used as a jupyter kernel. (To learn about how to make your own Conda environments, see our training on Conda)

  1. Open a Command Line Terminal on the HPC resource.

  2. Load Miniconda

  3. Activate your Conda environment

  4. Install your kernel:
    conda install ipykernel for python kernels

  5. Set your environment to be recognized as an available kernel
    python -m ipykernel install --user --name=<kernelname>

For Python:

Code Block
languagebash
#open command line interface
module load miniconda3
conda activate <insert_environment_name_or_path_here>
conda install ipykernel
python -m ipykernel install --user --name=mypythonkernel

...

In these steps, we assume you’ve already created your desired Conda environment. To learn how to create a Conda environment, please see our Conda module.

Note: A more in depth description with background for creating an R kernel is available at in our R and RStudio Training Materials in the section re: Create an R Kernel for Jupyter

With our Conda environment already installed and configured, we can now set it up to be used as a jupyter kernel. (To learn about how to make your own Conda environments, see our training on Conda)

  1. Open a Command Line Terminal on the HPC resource.

  2. Module Load Miniconda

  3. Activate your Conda environment

  4. Install your kernel:
    conda install r-irkernel for R kernels

  5. Set your environment to be recognized as an available kernel

    1. note R location in conda

    2. Load R: R

    3. install kernel: R prompt:
      (conda_env) []$ R > IRkernel::installspec(name,displaynameinstall.packages('IRkernel')

    4. Create specific R version kernel folder in ~/.local/share/jupyter/kernels/

    5. Copy config from <conda-env> location appending the following to path after <conda-env> name with /lib/R/library/IRkernel/kernelspec/* to kernels folder in your ~/.local/share/jupyter/kernels/ folder.

For R:

Code Block
languagebash
#Step# Step (1) open command line interface:
#Step# Step (2) Load miniconda:
module load miniconda3
#Step# Step (3) activate conda environment: 
conda activate <insert_environment_name_or_path_here>
# #StepStep (4) install kernel in conda environment:
conda install r-irkernel
# #StepStep (5a) Note R location in conda:
which R
/project/<project_name>/software/conda/r/rtest/<conda_env_path>/bin/R
#Launch# RStep (step 5b) Launch R:
R
#Install
# Step (5c) Install kernel inwithin R prompt (step 5biii):
> install.packages('IRkernel') 
> IRkernel::installspec(name='r4.4.1', displayname='R
4.4.1 Kernel')
> q()
#Step (5c5d): make #Changea directory tofor jupyteryour kernel directory:
cdmkdir ~/.local/share/jupyter/kernels/
#Step (5d): make a directory for your kernel:
mkdir r4.4.1
#change directory to new directory:
cd r4.4.1
0
#Step (5e) Copy kernelspec from conda to .local:
cp /project/<project_name>/software/conda/r/rtest/<conda_env_name_or_path>/lib/R/library/IRkernel/kernelspec/* ~/.local/share/jupyter/kernels/r4.4.10/.

...

Running a Console Kernel

...