OpenFOAM.org released by the OpenFOAM Foundation Ltd. It is this version that we have installed upon request.
OpenFOAM is free, open-source software for CFD from the OpenFOAM Foundation. According to Wiki OpenFOAM (for "Open-source Field Operation And Manipulation") is a C++ toolbox for the development of customized numerical solvers, and pre-/post-processing utilities for the solution of continuum mechanics problems, most prominently including computational fluid dynamics (CFD).
Using
OpenFOAM is not provided as a module. The typical use-case we see from researchers is that they have focused research that extends existing OpenFOAM source code and thus need to rebuild after their modifications.
Below are provided examples/instructions on building various versions, across some of our clusters. If the libraries we use to not meet your particular case then please contact us and we can assist you with your alternative.
If you decide to add the source command to your .bashrc file, then you will need to module load your libraries before this command in the .bashrc, otherwise you might see errors of the form mpicc command not found. Within the build examples below, as an alternative they call source ... and export WM_PROJECT_DIR ... from the command line.
Multicore
A number of the OpenFOAM commands can run across multiple nodes. Inspect the usage of the command using the form <command> -help and look for the parallel option.
These commands must be proceeded with srun.
Beartooth
Here are the basic steps for building OpenFOAM 6. Further instructions can be found Download v6 | Source Pack.
# This example assumes using a fake project in /project/openfoam_fake/
[user@blog1 openfoam_fake]$ git clone https://github.com/OpenFOAM/OpenFOAM-6
[user@blog1 openfoam_fake]$ cd OpenFOAM-6
[user@blog1 OpenFOAM-6]$ module load gcc/12.2.0 openmpi/4.1.4 scotch/6.1.3-ompi
# Do not source before loading the modules otherwise it wil complain about not finding mpicc
[user@blog1 OpenFOAM-6]$ source /project/openfoam_fake/OpenFOAM-6/etc/bashrc
[user@blog1 OpenFOAM-6]$ export WM_PROJECT_DIR=/project/openfoam_fake/OpenFOAM-6
# Create an interactive session in which to build OpenFOAM.
# Due to the size and lengthof the build, do NOT build on the login nodes.
# Obviously replace 'openfoam_fake' with your project.
[user@blog1 OpenFOAM-6]$ salloc -A openfoam_fake -t 2:00:00 -c 16
# Using 16 cores, the complete build will take nealy one hour
[user@m001 OpenFOAM-6]$ ./Allwmake -j 16
...
g++ -std=c++11 -m64 -Dlinux64 -DWM_ARCH_OPTION=64 -DWM_DP -DWM_LABEL_SIZE=32 -Wall -Wextra -Wold-style-cast -Wnon-virtual-dtor -Wno-unused-parameter -Wno-invalid-offsetof -Wno-attributes -O3 -DNoRepository -ftemplate-depth-100 -IfoamToVTK/lnInclude -I/project/openfoam_fake/OpenFOAM-6/src/lagrangian/basic/lnInclude -I/project/openfoam_fake//OpenFOAM-6/src/finiteVolume/lnInclude -I/project/openfoam_fake/OpenFOAM-6/src/dynamicMesh/lnInclude -I/project/openfoam_fake/OpenFOAM-6/src/meshTools/lnInclude -IlnInclude -I. -I/project/openfoam_fake/OpenFOAM-6/src/OpenFOAM/lnInclude -I/project/openfoam_fake/OpenFOAM-6/src/OSspecific/POSIX/lnInclude -fPIC -Xlinker --add-needed -Xlinker --no-as-needed /project/openfoam_fake/OpenFOAM-6/platforms/linux64GccDPInt32Opt/applications/utilities/postProcessing/dataConversion/foamToVTK/foamToVTK.o -L/project/openfoam_fake/OpenFOAM-6/platforms/linux64GccDPInt32Opt/lib \
-lfoamToVTK -ldynamicMesh -llagrangian -lgenericPatchFields -lOpenFOAM -ldl \
-lm -o /project/openfoam_fake/OpenFOAM-6/platforms/linux64GccDPInt32Opt/bin/foamToVTK
[user@m001 OpenFOAM-6]$
# Delivered Scripts
[user@m001 OpenFOAM-6]$ ls bin/
# Built Scripts
[user@m001 OpenFOAM-6]$ ls platforms/linux64GccDPInt32Opt/bin/
Teton
Note: These are directions for using the de-commissioned Teton cluster. You will need to update the modules to work on the Beartooth cluster - but the overall process will be similar
The following instructions provides an example of building version 9 on Teton, with additional instructions and download from this page: https://openfoam.org/download/9-source/
Below is an example using the arcc-t01 user under the arccanetrain project. It demonstrates the required steps and provides the output you should expect to see.
You will obviously have your own username, and project. You can change paths where appropriate to install either in your own home or project folder.
Installation
# Decide where you wish to install OpenFOAM.
# Below is installing within the user's home folder.
[arcc-t01@tlog1 ~]$ mkdir OpenFOAM
[arcc-t01@tlog1 ~]$ cd OpenFOAM/
# Download the source code.
[arcc-t01@tlog1 OpenFOAM]$ wget http://dl.openfoam.org/source/9
# Notice that the download file is simply called '9'.
[arcc-t01@tlog1 OpenFOAM]$ ls -al
-rw-rw-r-- 1 arcc-t01 arcc-t01 44965713 Feb 23 13:49 9
# Extract the contents of the file.
[arcc-t01@tlog1 OpenFOAM]$ tar -xf 9
[arcc-t01@tlog1 OpenFOAM]$ ls
9 OpenFOAM-9-version-9
# Although the untarred file is called 'OpenFOAM-9-version-9' previous attempts at building have failed
# due to not being able to find the folder.
# Thus the reason for renaming (moving) it.
[arcc-t01@tlog1 OpenFOAM]$ mv OpenFOAM-9-version-9/ OpenFOAM-9/
# Notice under the etc/ folder there is the bashrc file that is used to setup your environment to be able to build OpenFOAM.
[arcc-t01@tlog1 OpenFOAM]$ cd OpenFOAM-9/etc/
[arcc-t01@tlog1 etc]$ pwd
/home/arcc-t01/OpenFOAM/OpenFOAM-9/etc
[arcc-t01@tlog1 etc]$ ls
bashrc caseDicts cellModels codeTemplates config.csh config.sh controlDict cshrc paraFoam README.org templates thermoData
# The next step is to load the required compiler, openmpi and scotch libraries and dependencies.
[arcc-t01@tlog1 OpenFOAM-9]$ module load gcc/11.2.0
[arcc-t01@tlog1 OpenFOAM-9]$ module load openmpi/4.1.2
[arcc-t01@tlog1 OpenFOAM-9]$ module load scotch/6.1.1
# Do not source the OpenFOAM related bashrc script until after the module loads.
# This is because as part of the sourcing it is looking for libraries, such as mpi.
[arcc-t01@tlog1 OpenFOAM-9]$ source /home/arcc-t01/OpenFOAM/OpenFOAM-9/etc/bashrc
# Since we want to use multiple cores to speed up the building,
# we should create an interactive session to build within.
# NYou will need to replace 'arccanetrain' with the project you have.
[arcc-t01@tlog1 OpenFOAM-9]$ salloc -A arccanetrain -t 3:00:00 -c 16
salloc: Granted job allocation 2247253
# On a standard moran node this will take 35/40 mins.
[arcc-t01@m073 OpenFOAM-9]$ ./Allwmake -j 16
Compiling enabled on 16 cores
Allwmake /home/arcc-t01/OpenFOAM/OpenFOAM-9
gcc -m64 -Dlinux64 -DWM_ARCH_OPTION=64 -DWM_DP -DWM_LABEL_SIZE=32 -Wall -O3 -fPIC dirToString.c -o /pfs/tc1/home/arcc-t01/OpenFOAM/OpenFOAM-9/wmake/platforms/linux64Gcc/dirToString
...
g++ -std=c++14 -m64 -Dlinux64 -DWM_ARCH_OPTION=64 -DWM_DP -DWM_LABEL_SIZE=32 -Wall -Wextra -Wold-style-cast -Wnon-virtual-dtor -Wno-unused-parameter -Wno-invalid-offsetof -Wno-attributes -O3 -DNoRepository -ftemplate-depth-100 -IfoamToVTK/lnInclude -I/pfs/tc1/home/arcc-t01/OpenFOAM/OpenFOAM-9/src/lagrangian/basic/lnInclude -I/pfs/tc1/home/arcc-t01/OpenFOAM/OpenFOAM-9/src/finiteVolume/lnInclude -I/pfs/tc1/home/arcc-t01/OpenFOAM/OpenFOAM-9/src/dynamicMesh/lnInclude -I/pfs/tc1/home/arcc-t01/OpenFOAM/OpenFOAM-9/src/meshTools/lnInclude -I/pfs/tc1/home/arcc-t01/OpenFOAM/OpenFOAM-9/src/fileFormats/lnInclude -IlnInclude -I. -I/pfs/tc1/home/arcc-t01/OpenFOAM/OpenFOAM-9/src/OpenFOAM/lnInclude -I/pfs/tc1/home/arcc-t01/OpenFOAM/OpenFOAM-9/src/OSspecific/POSIX/lnInclude -fPIC -fuse-ld=bfd -Xlinker --add-needed -Xlinker --no-as-needed /pfs/tc1/home/arcc-t01/OpenFOAM/OpenFOAM-9/platforms/linux64GccDPInt32Opt/applications/utilities/postProcessing/dataConversion/foamToVTK/foamToVTK.o -L/pfs/tc1/home/arcc-t01/OpenFOAM/OpenFOAM-9/platforms/linux64GccDPInt32Opt/lib \
-lfoamToVTK -ldynamicMesh -llagrangian -lgenericPatchFields -lfileFormats -lOpenFOAM -ldl \
-lm -o /pfs/tc1/home/arcc-t01/OpenFOAM/OpenFOAM-9/platforms/linux64GccDPInt32Opt/bin/foamToVTK
# General commands/scripts delivered with the source
[arcc-t01@m073 OpenFOAM-9]$ ls bin
# The 10s of built commands.
[arcc-t01@tlog1 platforms]$ ls platforms/linux64GccDPInt32Opt/bin/
We are aware that some users will update their home folder’s .bashrc script (that is automatically called when you start a new session) to always have their sessions to pre load the modules and source the OpenFOAM bashrc file. This is a perfectly valid approach, but we ask that if you need to contact ARCC with any issues that you clearly communicate what changes you’ve had to your account.
Running a Test for OpenFOAM 9
Here we will demonstrate running just one of many examples that can be found in the tutorials folder.
# Setup Environment
[arcc-t01@tlog1 ~]$ mkdir OpenFOAM_Test
[arcc-t01@tlog1 ~]$ cd OpenFOAM_Test/
[arcc-t01@tlog1 OpenFOAM_Test]$ module load gcc/11.2.0 openmpi/4.1.2 scotch/6.1.1
[arcc-t01@tlog1 OpenFOAM_Test]$ source /home/arcc-t01/OpenFOAM/OpenFOAM-9/etc/bashrc
[arcc-t01@tlog1 OpenFOAM_Test]$ echo $FOAM_RUN
/home/arcc-t01/OpenFOAM/arcc-t01-9/run
[arcc-t01@tlog1 OpenFOAM_Test]$ echo $FOAM_TUTORIALS
/pfs/tc1/home/arcc-t01/OpenFOAM/OpenFOAM-9/tutorials
[arcc-t01@tlog1 OpenFOAM_Test]$ pwd
/home/arcc-t01/OpenFOAM_Test
[arcc-t01@tlog1 OpenFOAM_Test]$ cp -r $FOAM_TUTORIALS/incompressible/simpleFoam/pitzDaily .
[arcc-t01@tlog1 OpenFOAM_Test]$ cd pitzDaily/
#- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# Create an interactive session to run the test on.
[arcc-t01@tlog1 pitzDaily]$ salloc -A arccanetrain -t 3:00:00 -c 16
salloc: Granted job allocation 2249834
#- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# Test Step 01
[arcc-t01@m069 pitzDaily]$ ls
0 Allrun constant system
[arcc-t01@m069 pitzDaily]$ blockMesh
/*---------------------------------------------------------------------------*\
========= |
\\ / F ield | OpenFOAM: The Open Source CFD Toolbox
\\ / O peration | Website: https://openfoam.org
\\ / A nd | Version: 9
\\/ M anipulation |
\*---------------------------------------------------------------------------*/
Build : 9
Exec : blockMesh
Date : Feb 23 2022
Time : 15:07:05
Host : "m069.cluster"
PID : 5126
I/O : uncollated
Case : /pfs/tc1/home/arcc-t01/OpenFOAM_Test/pitzDaily
nProcs : 1
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster (fileModificationSkew 10)
allowSystemOperations : Allowing user-supplied system call operations
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time
Deleting polyMesh directory
"/pfs/tc1/home/arcc-t01/OpenFOAM_Test/pitzDaily/constant/polyMesh"
Reading "blockMeshDict"
Creating block mesh from
"system/blockMeshDict"
Creating block edges
No non-planar block faces defined
Creating topology blocks
Creating topology patches
Creating block mesh topology
Check topology
Basic statistics
Number of internal faces : 5
Number of boundary faces : 20
Number of defined boundary faces : 20
Number of undefined boundary faces : 0
Checking patch -> block consistency
Creating block offsets
Creating merge list .
Creating polyMesh from blockMesh
Creating patches
Creating cells
Creating points with scale 0.001
Block 0 cell size :
i : 0.00158284 .. 0.000791418
j : 0.000318841 .. 0.000420268
k : 0.001
Block 1 cell size :
i : 0.000528387 .. 0.00211355
j : 0.00112889 .. 0.000360188 0.00112841 .. 0.000361677 0.00112889 .. 0.000360188 0.00112841 .. 0.000361677
k : 0.001
Block 2 cell size :
i : 0.000528387 .. 0.00211355
j : 0.000318841 .. 0.000420268 0.000320918 .. 0.000419851 0.000318841 .. 0.000420268 0.000320918 .. 0.000419851
k : 0.001
Block 3 cell size :
i : 0.0020578 .. 0.00514451 0.00205699 .. 0.00514248 0.0020578 .. 0.00514451 0.00205699 .. 0.00514248
j : 0.000940741 .. 0.000940741 0.0009328 .. 0.0009328 0.000940741 .. 0.000940741 0.0009328 .. 0.0009328
k : 0.001
Block 4 cell size :
i : 0.0020466 .. 0.00511651 0.00204663 .. 0.00511656 0.0020466 .. 0.00511651 0.00204663 .. 0.00511656
j : 0.00112889 .. 0.000257962 0.00111936 .. 0.000255785 0.00112889 .. 0.000257962 0.00111936 .. 0.000255785
k : 0.001
There are no merge patch pairs edges
Writing polyMesh
----------------
Mesh Information
----------------
boundingBox: (-0.0206 -0.0254 -0.0005) (0.29 0.0254 0.0005)
nPoints: 25012
nCells: 12225
nFaces: 49180
nInternalFaces: 24170
----------------
Patches
----------------
patch 0 (start: 24170 size: 30) name: inlet
patch 1 (start: 24200 size: 57) name: outlet
patch 2 (start: 24257 size: 223) name: upperWall
patch 3 (start: 24480 size: 250) name: lowerWall
patch 4 (start: 24730 size: 24450) name: frontAndBack
End
[arcc-t01@m069 pitzDaily]$
#- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# Test Step 02
[arcc-t01@m069 pitzDaily]$ simpleFoam
/*---------------------------------------------------------------------------*\
========= |
\\ / F ield | OpenFOAM: The Open Source CFD Toolbox
\\ / O peration | Website: https://openfoam.org
\\ / A nd | Version: 9
\\/ M anipulation |
\*---------------------------------------------------------------------------*/
Build : 9
Exec : simpleFoam
Date : Feb 23 2022
Time : 15:07:32
Host : "m069.cluster"
PID : 5163
I/O : uncollated
Case : /pfs/tc1/home/arcc-t01/OpenFOAM_Test/pitzDaily
nProcs : 1
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster (fileModificationSkew 10)
allowSystemOperations : Allowing user-supplied system call operations
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time
Create mesh for time = 0
SIMPLE: Convergence criteria found
p: tolerance 0.01
U: tolerance 0.001
"(k|epsilon|omega|f|v2)": tolerance 0.001
Reading field p
Reading field U
Reading/calculating face flux field phi
Selecting incompressible transport model Newtonian
Selecting turbulence model type RAS
Selecting RAS turbulence model kEpsilon
RAS
{
model kEpsilon;
turbulence on;
printCoeffs on;
Cmu 0.09;
C1 1.44;
C2 1.92;
C3 0;
sigmak 1;
sigmaEps 1.3;
}
No MRF models present
No fvModels present
No fvConstraints present
Starting time loop
streamlines streamlines:
automatic track length specified through number of sub cycles : 5
streamlines streamlines write:
seeded 10 particles
Tracks:20
Total samples:114
Writing data to "/pfs/tc1/home/arcc-t01/OpenFOAM_Test/pitzDaily/postProcessing/sets/streamlines/0"
Writing data to "/pfs/tc1/home/arcc-t01/OpenFOAM_Test/pitzDaily/postProcessing/sets/streamlines/0"
Time = 1
smoothSolver: Solving for Ux, Initial residual = 1, Final residual = 0.0538101, No Iterations 1
smoothSolver: Solving for Uy, Initial residual = 1, Final residual = 0.030925, No Iterations 2
GAMG: Solving for p, Initial residual = 1, Final residual = 0.068427, No Iterations 17
time step continuity errors : sum local = 1.19733, global = 0.179883, cumulative = 0.179883
smoothSolver: Solving for epsilon, Initial residual = 0.232889, Final residual = 0.0114607, No Iterations 3
bounding epsilon, min: -1.98669 max: 1033.28 average: 35.8882
smoothSolver: Solving for k, Initial residual = 1, Final residual = 0.0454605, No Iterations 3
ExecutionTime = 0.26 s ClockTime = 1 s
...
...
Time = 287
smoothSolver: Solving for Ux, Initial residual = 0.000119475, Final residual = 1.19332e-05, No Iterations 5
smoothSolver: Solving for Uy, Initial residual = 0.000988822, Final residual = 6.43953e-05, No Iterations 6
GAMG: Solving for p, Initial residual = 0.00147862, Final residual = 0.00014116, No Iterations 3
time step continuity errors : sum local = 0.00595802, global = -0.000643205, cumulative = 1.18609
smoothSolver: Solving for epsilon, Initial residual = 0.000135797, Final residual = 8.98303e-06, No Iterations 3
smoothSolver: Solving for k, Initial residual = 0.000217753, Final residual = 1.3218e-05, No Iterations 4
ExecutionTime = 9.97 s ClockTime = 10 s
SIMPLE solution converged in 287 iterations
streamlines streamlines write:
seeded 10 particles
Tracks:20
Total samples:12040
Writing data to "/pfs/tc1/home/arcc-t01/OpenFOAM_Test/pitzDaily/postProcessing/sets/streamlines/287"
Writing data to "/pfs/tc1/home/arcc-t01/OpenFOAM_Test/pitzDaily/postProcessing/sets/streamlines/287"
End
[arcc-t01@m069 pitzDaily]$