Extend Conda Environment to Jupyter Kernel
- Simon Alexander
Goal: Demonstrate extending a Python related conda environment in to Juypter kernel.
What is Jupyter and What is a Kernel?
Jupyter is a large umbrella project that covers many different software offerings and tools, including the popular Jupyter Notebook and JupyterLab web-based notebook authoring and editing applications.
Kernels are programming language specific processes that run independently and interact with the Jupyter Applications and their user interfaces.
They can be considered as the computational engine for notebooks.
The kernel process is a standalone process that runs in the background and executes the code that you write in your notebooks. The kernel process is responsible for running the code and returning the results to the frontend.
ARCC provides the Jupyter with OnDemand workshop.
General Process
The general process involves updating the conda environment to include kernel related packages, and then configuring the kernel spec to allow it to be picked up by the Jupyter service.
Activate you Python Conda Environment.
Conda install the
ipykernelpackage.Deactivate your Conda environment.
Copy the created
kernelspecinto your home.local/share/jupyter/kernels/folder.Update the kernel.json.
The process creates a kernel for an individual user from the previously created shared Conda environment.
Each individual from the shared space will need to follow this process to create their own kernel.
Install the ipykernel package
[]$ module purge
[]$ module load miniconda3/24.3.0
[]$ conda activate /cluster/medbow/project/<project-name>/software/tensorflow/2.16
(/cluster/medbow/project/<project-name>/software/tensorflow/2.16) []$ export PYTHONUSERBASE=$CONDA_PREFIX
(/cluster/medbow/project/<project-name>/software/tensorflow/2.16) []$ conda install ipykernel
...
(/cluster/medbow/project/<project-name>/software/tensorflow/2.16) []$ python -m ipykernel install --user --name=TF2.16-local
(/cluster/medbow/project/<project-name>/software/tensorflow/2.16) []$ conda deactivate
[]$ The Related Kernel Details
() (/project/<project-name>/software/tensorflow/2.16) []$ python -m ipykernel install --user --name=TF2.16-local
Installed kernelspec TF2.16-local in /cluster/medbow/home/<username>/.local/share/jupyter/kernels/tf2.16-local[]$ ls /cluster/medbow/home/<username>/.local/share/jupyter/kernels/tf2.16-local/
kernel.json logo-32x32.png logo-64x64.png logo-svg.svg
[]$ cat /cluster/medbow/home/<username>/.local/share/jupyter/kernels/tf2.16-local/kernel.json
{
"argv": [
"/project/<project-name>/software/tensorflow/2.16/bin/python",
"-Xfrozen_modules=off",
"-m",
"ipykernel_launcher",
"-f",
"{connection_file}"
],
"display_name": "TF2.16-local",
"language": "python",
"metadata": {
"debugger": true
}
}Start Jupyter
Within a Notebook
Within a Jupyter notebook cell try:
Add label