OpenFOAM

Note: There are two flavors of OpenFOAM:

OpenFOAM.org variant released by the OpenFOAM Foundation Ltd.
OpenFOAM.com variant by OpenCFD Ltd an affiliate of ESI Group.

When using OpenFOAM and contacting ARCC, please make sure you know which which version you require. We appreciate it can be confusing, but in the past we have had users who within a discussion thread pointed to pages across both sites and where unsure which version they actually required.

Using

OpenFOAM is not provided as a module. The typical use-case we see from researchers is that they have focused research that extends existing OpenFOAM source code/solvers and thus need to rebuild after their modifications.

Multicore

A number of the OpenFOAM commands can run across multiple nodes. Inspect the usage of the command using the form <command> -help and look for the parallel option.

These commands must be proceeded with srun.

Building

Below are provided examples/instructions on building various versions. If the libraries we use to not meet your particular case then please contact us and we can assist you with your alternative.

Build within an Interactive Session

Building can take hours to complete and can use multiple cores - because of this do not build OpenFOAM on the login nodes. Instead perform within an interactive salloc session.

You can either define an explicit number of cores to build with, or try and use ALL the cores on a compute node:

# Define a number of cores:
[]$ salloc -A <account> -t 4:00:00 -c <num_of_cores>
...
[<compute node>]$ ./Allwmake -j <num_of_cores>

# Ask for ALL the cores on a compute node:
[]$ salloc -A <account> -t 4:00:00 --exclusive
...
[<compute node>]$ ./Allwmake -j

Building with Third Party libraries: Both Com/Org versions

Both the OpenFOAM and Third Party un-tarred folders need to within the same parent folder.

Typical Process:

[]$ salloc -A <account> -t 4:00:00 -c 16
[]$ ls
OpenFOAM-<version>      ThirdParty-<version>

[]$ cd OpenFoam-<version>
[]$ module load gcc/12.2.0 openmpi/4.1.4
[]$ source etc/bashrc

# .com
[]$ ./Allwmake -l -j 16

# .org
[]$ ./Allwmake -j 16

# Build commands typically found in:
[]$ ls platforms/linux64GccDPInt32Opt/bin/

Building using pre-built libraries (i.e. without the Third Party Libraries)

This assumes you do not have the Third Party libraries un-tarred in the same parent folder as OpenFOAM.

Build Com v2312

[]$ salloc -A account -t 4:00:00 -c 16
[]$ cd OpenFoam-<version>
[]$ module load gcc/12.2.0 openmpi/4.1.4 fftw/3.3.10-ompi adios2/2.8.3-ompi cgal/5.5.1 petsc/3.18.2-ompi
[]$ source etc/bashrc
No completions for /project/arcc/software/openfoam/source/2312/OpenFOAM-v2312/platforms/linux64GccDPInt32Opt/bin
[ignore if OpenFOAM is not yet compiled]
[]$ ./Allwmake -l -j 16
Logging wmake -all output to 'log.linux64GccDPInt32Opt'
Compiling enabled on 16 cores
...

ARCC Wiki