Where are R Packages Installed on the Cluster?
- Simon Alexander
Goal: Understand where R installs packages and where libraries are located, as well as inspecting general R system configuration.
- 1 Terminology Package vs Library
- 2 Load an R Environment via the Module System
- 3 Inspect R Environment Configuration
- 4 Where are Packages Installed
- 5 How to Install Packages (from within R)
- 6 How to Install Packages (from outside of R)
- 7 Install tidyr Package
- 8 What’s Installed?
- 9 Remember: R Versions and Library Locations
Terminology Package vs Library
R-Bloggers: Packages v. Libraries in R: Packages are collections of R functions, data, and compiled code in a well-defined format. The directory where packages are stored is called the library:
A package is a directory of files which extend R, either a source package (the master files of a package), or a tarball containing the files of a source package, or an installed package, the result of running
R CMD INSTALLon a source package. On some platforms there are also binary packages, a zip file or tarball containing the files of an installed package which can be unpacked rather than installing from sources.A package is not a library. The latter is used in two senses in R documentation. The first is a directory into which packages are installed, e.g.
/usr/lib/R/library: in that sense it is sometimes referred to as a library directory or library tree (since the library is a directory which contains packages as directories, which themselves contain directories). …
Load an R Environment via the Module System
[]$ module purge
[]$ module load gcc/14.2.0 r/4.4.0
[]$ R --version
R version 4.4.0 (2024-04-24) -- "Puppy Cup"
...
# Run a simple function.
[]$ R -e "print(version[['version.string']])"
R version 4.4.0 (2024-04-24) -- "Puppy Cup"
...
> print(version[['version.string']])
[1] "R version 4.4.0 (2024-04-24)"
# Run an R script.
[]$ Rscript r_test.R
[1] "R version 4.4.0 (2024-04-24)"Inspect R Environment Configuration
[]$ R
R version 4.4.0 (2024-04-24) -- "Puppy Cup"
...
# List ALL environment variables:
> Sys.getenv()
# Get single environment variable.
> Sys.getenv("R_HOME")
[1] "/apps/u/spack/gcc/14.2.0/r/4.4.0-w7xoohc/rlib/R"
# Get a list of environment variables.
> Sys.getenv(c("R_PLATFORM", "R_HOME", "R_LIBS_USER"))
R_PLATFORM "x86_64-pc-linux-gnu"
R_HOME "/apps/u/spack/gcc/14.2.0/r/4.4.0-w7xoohc/rlib/R"
R_LIBS_USER "/cluster/medbow/home/salexan5/R/x86_64-pc-linux-gnu-library/4.4"Notice that User Libraries are stored under your home, in: R/x86_64-pc-linux-gnu-library/4.4
In general, under ~/R/<R_PLATFORM>/versionX.Y/
Get R Related Environment variables
Quick and dirty way to list all R_* environment variables.
[salexan5@mblog2 ~]$ R -e "Sys.getenv()" | grep R_
LMOD_FAMILY_COMPILER_VERSION
...
R_HOME /apps/u/spack/gcc/14.2.0/r/4.4.0-w7xoohc/rlib/R
R_INCLUDE_DIR /apps/u/spack/gcc/14.2.0/r/4.4.0-w7xoohc/rlib/R/include
R_LIBS_SITE /apps/u/spack/gcc/14.2.0/r/4.4.0-w7xoohc/rlib/R/site-library
R_LIBS_USER /cluster/medbow/home/<username>/R/x86_64-pc-linux-gnu-library/4.4
...
R_PLATFORM x86_64-pc-linux-gnu
...
R_SHARE_DIR /apps/u/spack/gcc/14.2.0/r/4.4.0-w7xoohc/rlib/R/share
...
R_UNZIPCMD /usr/bin/unzip
R_ZIPCMD /usr/bin/zip
thunderer_FAMILY_COMPILER_VERSIONWhere are Packages Installed
> help(".libPaths")
.Library package:base R Documentation
Search Paths for Packages
Description:
‘.libPaths’ gets/sets the library trees within which packages are
looked for.[]$ R -e ".libPaths()"
...
> .libPaths()
[1] "/cluster/medbow/home/<username>/R/x86_64-pc-linux-gnu-library/4.4"
[2] "/apps/u/spack/gcc/14.2.0/r/4.4.0-w7xoohc/rlib/R/library"
# Compare against:
R_LIBS_USER: /cluster/medbow/home/<username>/R/x86_64-pc-linux-gnu-library/4.4
R_LIBS_SITE: /apps/u/spack/gcc/14.2.0/r/4.4.0-w7xoohc/rlib/R/site-library/cluster/medbow/home/<username>/R/x86_64-pc-linux-gnu-library/4.4
This is where, for this R platform and version, your packages will be installed.
/apps/u/spack/gcc/14.2.0/r/4.4.0-w7xoohc/rlib/R/library
This is where the base packages for this R platform/version are installed.
You will not have permissions to install into this location.
[salexan5@mblog2 ~]$ ls /apps/u/spack/gcc/14.2.0/r/4.4.0-w7xoohc/rlib/R/library
base compiler datasets graphics grDevices grid methods parallel
splines stats stats4 tcltk tools translations utilsHow to Install Packages (from within R)
# Within R:
> help(install.packages)
install.packages package:utils R Documentation
Install Packages from Repositories or Local Files
Description:
Download and install packages from CRAN-like repositories or from
local files.
Usage:
install.packages(pkgs, lib, repos = getOption("repos"),
contriburl = contrib.url(repos, type),
method, available = NULL, destdir = NULL,
dependencies = NA, type = getOption("pkgType"),
configure.args = getOption("configure.args"),
configure.vars = getOption("configure.vars"),
clean = FALSE, Ncpus = getOption("Ncpus", 1L),
verbose = getOption("verbose"),
libs_only = FALSE, INSTALL_opts, quiet = FALSE,
keep_outputs = FALSE, ...)
...How to Install Packages (from outside of R)
[]$ R CMD INSTALL --help
Usage: R CMD INSTALL [options] pkgs
Install the add-on packages specified by pkgs. The elements of pkgs can
be relative or absolute paths to directories with the package
sources, or to gzipped package 'tar' archives. The library tree
to install to can be specified via '--library'. By default, packages are
installed in the library tree rooted at the first directory in
.libPaths() for an R session run in the current environment.
Options:
-h, --help print short help message and exit
-v, --version print INSTALL version info and exit
-c, --clean remove files created during installation
...Note: “The elements of pkgs can be relative or absolute paths to directories with the package sources, or to gzipped package 'tar' archives.”
i.e. You have the package already downloaded.
Install tidyr Package
# Within R:
> install.packages("tidyr")
Installing package into ‘/cluster/medbow/home/<username>/R/x86_64-pc-linux-gnu-library/4.4’
(as ‘lib’ is unspecified)
--- Please select a CRAN mirror for use in this session ---
...
also installing the dependencies ‘utf8’, ‘generics’, ‘pillar’, ‘R6’, ‘stringi’, ‘fansi’, ‘pkgconfig’, ‘withr’,
‘cli’, ‘dplyr’, ‘glue’, ‘lifecycle’, ‘magrittr’, ‘purrr’, ‘rlang’, ‘stringr’, ‘tibble’, ‘tidyselect’, ‘vctrs’, ‘cpp11’
...
* DONE (tidyr)Lets check:
[salexan5@mblog2 ~]$ ls /home/<username>/R/x86_64-pc-linux-gnu-library/4.4
cli cpp11 dplyr fansi generics glue lifecycle magrittr pillar pkgconfig
purrr R6 rlang stringi stringr tibble tidyr tidyselect utf8 vctrs withrWhat’s Installed?
# Within R:
> write.table(installed.packages()[,c(1,2,3:4)])
"Package" "LibPath" "Version" "Priority"
"cli" "cli" "/cluster/medbow/home/<username>/R/x86_64-pc-linux-gnu-library/4.4" "3.6.3" NA
"cpp11" "cpp11" "/cluster/medbow/home/<username>/R/x86_64-pc-linux-gnu-library/4.4" "0.4.7" NA
...
"tcltk" "tcltk" "/apps/u/spack/gcc/14.2.0/r/4.4.0-w7xoohc/rlib/R/library" "4.4.0" "base"
"tools" "tools" "/apps/u/spack/gcc/14.2.0/r/4.4.0-w7xoohc/rlib/R/library" "4.4.0" "base"
"utils" "utils" "/apps/u/spack/gcc/14.2.0/r/4.4.0-w7xoohc/rlib/R/library" "4.4.0" "base"Note what is identified as base.
Remember: R Versions and Library Locations
Use Case: If projects/scripts are using say: module load r/4.3.x:
The scripts in folder01/folder02 will both use/share the R packages under ~/R/x86_64-pc-linux-gnu-library/4.3
If you update an R package due to a need for a script in folder01, then this new R package will also be used by the scripts in folder02.
Is this intended? Does it course an issue for scripts in folder02? You need to be aware and manage.
Similarly: If projects/scripts are using: module load r/4.4.x:
The scripts in folder03/folder04 will both use/share the R packages under ~/R/x86_64-pc-linux-gnu-library/4.4
If you update an R package due to modifying a script in folder03, then this new R package will also be used by the scripts in folder04.
They will not be using any r/4.3.x related packages - that’s a different library location.
In general: Using r/x.y.z packages will be found under ~/R/x86_64-pc-linux-gnu-library/X.Y