Create an R Kernel for a Jupyter Notebook
- Simon Alexander
Goal: Detail how to update an R Conda environment so it can be used as a kernel within ARCC’s Jupyter service.
What is Jupyter and What is a Kernel?
Jupyter is a large umbrella project that covers many different software offerings and tools, including the popular Jupyter Notebook and JupyterLab web-based notebook authoring and editing applications.
Kernels are programming language specific processes that run independently and interact with the Jupyter Applications and their user interfaces.
They can be considered as the computational engine for notebooks.
The kernel process is a standalone process that runs in the background and executes the code that you write in your notebooks. The kernel process is responsible for running the code and returning the results to the frontend.
ARCC provides the Jupyter with OnDemand workshop.
General Process
The general process involves updating the Conda environment to include kernel related packages, and then configuring the kernel spec to allow it to be picked up by the Jupyter service.
Activate you R Conda Environment.
Start R.
Install the
IRkernelpackage.Exit R.
Deactivate your Conda environment.
Copy the created
kernelspecinto your home~/.local/share/jupyter/kernels/folder.Update the
kernel.json.
Install the IRkernel package
Activate you R Conda Environment and start R.
[]$ module load miniconda3/24.3.0
[]$ conda activate /project/<project-name>/software/conda-envs/r_4.3.3_env/
(/project/<project-name>/software/conda-envs/r_4.3.3_env) []$ R
> install.packages('IRkernel')
...
also installing the dependencies ‘fastmap’, ‘fansi’, ‘utf8’, ‘htmltools’, ‘pillar’, ‘base64enc’, ‘repr’, ‘evaluate’, ‘IRdisplay’, ‘pbdZMQ’, ‘crayon’, ‘jsonlite’, ‘uuid’, ‘digest’
...
* DONE (IRkernel)Created kernelspec folder
Installing the IRkernel package will create a kernel spec that we can use.
This can be found under the Conda environment location, under: lib/R/library/IRkernel/kernelspec/
[]$ ls /project/<project-name>/software/conda-envs/r_4.3.3_env/lib/R/library/IRkernel/kernelspec/
kernel.js kernel.json logo-64x64.png logo-svg.svg[salexan5@mblog2 ~]$ cat /project/<project-name>/software/conda-envs/r_4.3.3_env/lib/R/library/IRkernel/kernelspec/kernel.json
{"argv": ["R", "--slave", "-e", "IRkernel::main()", "--args", "{connection_file}"],
"display_name":"R",
"language":"R"
}Configure Your Jupyter Environment
If you haven’t used the Jupyter service, then you might not have, and thus will need to create the following folders:
.local/share/jupyter/kernels/
Copy the created kernelspec into your home:
[~]$ cd .local/share/jupyter/kernels/
[]$ mkdir r_4.3.3
[]$ cd r_4.3.3
[]$ cp /project/<project-name>/software/conda-envs/r_4.3.3_env/lib/R/library/IRkernel/kernelspec/* .Update the kernel.json file to:
point to the Conda environment’s version of R.
give this kernel a unique display name.
[]$ cat kernel.json
{"argv": ["/project/<project-name>/software/conda-envs/r_4.3.3_env/bin/R", "--slave", "-e", "IRkernel::main()", "--args", "{connection_file}"],
"display_name":"R_4.3.3 (local)",
"language":"R"
}Start Jupyter
From OnDemand start a Jupyter session.
Notice how the newly configured kernel is available.
Within a Notebook
Within a cell try:
.libPaths()# Cell Output:
'/cluster/medbow/project/<project-name>/software/conda-envs/r_4.3.3_env/lib/R/library'write.table(installed.packages()[,c(1,2,3:4)])# Cell Output:
"Package" "LibPath" "Version" "Priority"
"base" "base" "/cluster/medbow/project/<project-name>/software/conda-envs/r_4.3.3_env/lib/R/library" "4.3.3" "base"
...
"vctrs" "vctrs" "/cluster/medbow/project/<project-name>/software/conda-envs/r_4.3.3_env/lib/R/library" "0.6.5" NANote the library path is that of the Conda environment: /project/arcc/software/conda-envs/r_4.3.3_env/lib/R/library