masterdesky

Installing and running OpenFOAM on Linux

OpenFOAM (Open-source Field Operation And Manipulation) is an open source software, designed primarily to solve problems in three-dimensional continuum mechanics, especially in computational fluid dynamics, but the possibilities of OpenFOAM goes much beyond this. It also has several forks and three main branches of development. In this tutorial I cover the installation and basic usage of OpenFOAM. As an example problem, I detail the configuration of a simple water droplet simulation, which I created for one of my courses during my master studies.

OpenFOAM had an adventurous journey with some controversies and legal delicacies mixed in, which started even before its initial release in 2004. As a result, OpenFOAM currently has three official development branches maintained. The first one, maintained by OpenCFD Limited and distributed over https://www.openfoam.com since 2004. The second one, maintained by The OpenFOAM Foundation and distributed over https://www.openfoam.org since its acquisition of the OPENFOAM® trademark in 2011. And a third one called foam-extend, maintained by the Wikki Ltd. since 2019. In this tutorial I will showcase The OpenFOAM Foundation’s OpenFOAM, installed on a machine running a Linux Debian 10 environment.

This tutorial is based on OpenFOAM version 8.0, but the installation process is close to identical for other versions as well. At the time of writing, OpenFOAM is only packaged for Ubuntu (in case of v8 for Ubuntu versions 16.04LTS, 18.04LTS and 20.04LTS, for later versions it is probably different), but it can be installed on other 64-bit distributions of Linux too using Docker. The list of every tested and supported Linux version along with their full and up-to-date installation tutorials can be found on the official downloads page of the OpenFOAM project.

Prerequisites and installing OpenFOAM on Linux

Step 1. Installing Docker

On Linux distributions other than Ubuntu, we need to run OpenFOAM in a Docker environment. Docker is a platform for developing, shipping, and running applications in containers. It is a lightweight and open-source containerization technology, and it is widely used to automate the deployment of applications such as OpenFOAM. This section details the installation of Docker on a Linux machine.

Unfortunately, Docker requires that the user either has root access or added to the docker group on the machine. On a personal computer it is not a problem, but if you are working on a shared server, then you should ask its administrator to install Docker and add you to the docker group. Since it grants you an elevated access to the system, it should be always granted and used with the uttermost caution.

For the sake of reproducibility, I used the latest 19.03.13 version of Docker – at the time of writing – for this project, which requires Linux kernel version 3.10 or above. However, you should always use the latest stable version of Docker. The dependencies for the current version can be always checked in the documentations. The kernel version of your machine can be easily looked up with the terminal command

$ uname -r

If you made sure, that you have a supported Linux kernel version, then you can proceed to the installation. Docker provides a “convenience script”, suitable for not too serious production environments, which can be used to install the latest version of Docker on a Linux machine. The script can be downloaded and executed with the following command (which requires root access):

$ sudo apt update -y
$ curl -fsSL https://get.docker.com/ | sh

Step 2. Configure docker for user

Using Docker requires root privileges by default. Although several tutorials overlooks this, it is an extremely important and fundamental safety practice to log in to your personal computer with a non-root user account. Since Docker version 19.3, it became possible to run Docker without sudo by adding the selected user to a new Unix group called docker. This prevents the users messing with the system using root privileges, while still allowing them to run Docker containers. The following command adds the current user to the docker group:

$ sudo groupadd docker
$ sudo usermod -aG docker ${USER}

Where the variable ${USER} can be replaced with the name of the user, who we want to add to the docker group. Otherwise, it will expand to the name of the currently logged in user (i.e. you). For this change to take effect the user should log out and log back in.

Step 3. Installing OpenFOAM

The Docker-version of OpenFOAM v8 can be downloaded using wget from the official OpenFOAM download site. For convenient execution anywhere on the machine we can add it to our PATH environment variable. The following commands do everything for us:

$ export install_path=${HOME}/apps/openfoam
$ mkdir -p ${install_path}
$ sh -c "wget http://dl.openfoam.org/docker/openfoam8-linux -O ${install_path}/openfoam8-linux"
$ chmod 755 ${install_path}/openfoam8-linux

Line-by-line, they do the following:

  • export install_path=[...]: Just for convenience, I defined a variable to store the path of the OpenFOAM installation, because it is used three times in the next three commands. I dedicated a separate folder for OpenFOAM in my home directory inside a folder named apps, but you can place it and name the variable whatever you want.
  • mkdir [...]: Creates the directory for the OpenFOAM installation.
  • sh -c [...]: Downloads and launches the convenience script that downloads OpenFOAM v8 to the install_path location.
  • chmod [...]: Makes the downloaded script executable.

To persistently make the openfoam8-linux executable available from anywhere on the machine, add the install path to the end of .bashrc (or the appropriate shell configuration file) in your home directory:

# [...other things in bashrc...]
export PATH=${HOME}/apps/openfoam:${PATH}

Step 4. Launching OpenFOAM

In the previous step we installed the Docker version of openfoam8-linux executable. When it is launched, it starts up a Docker container that – by default – mounts the current working directory, where openfoam8-linux was launched. However, mounting the $HOME directory is forbidden.

As detailed in the official installation guide, “the mounted directory is represented in the container environment by the WM_PROJECT_USER_DIR environment variable, which is set to $HOME/OpenFOAM/${USER}-8 by default”. This means, that it is recommended to start openfoam8-linux from inside this $HOME/OpenFOAM/${USER}-8 directory. Here, the variable ${USER} stands for the username of the user, who executed the script:

$ mkdir -p ${HOME}/OpenFOAM/${USER}-8
$ cd ${HOME}/OpenFOAM/${USER}-8
$ openfoam8-linux

Step 5. Testing OpenFOAM

The official installation guide presents a way to test, whether OpenFOAM was successfully installed or not. All projects (including this test project) should be placed and executed inside the run directory, represented with the $FOAM_RUN variable.

Just for clarity, the > character are not part of the following commands, but serves the same purpose as the $ character in the terminal: to separate the actual commands from [session ID]: current-dir> part. For simplicity I omitted the current-dir part in all following commands, but they should be there in a real terminal session.

First we need to create this directory:

[session ID]: > mkdir -p $FOAM_RUN

The session ID here refers to the Docker session, which the application is launched in. In every new launch it will be a randomly generated line of alphabetical and numeral characters. It has no deeper meaning.

To run the test simulation, first we copy a simple test case from the OpenFOAM tutorials folder, and place it into a separate folder inside the run directory. After this step, we follow the most simple OpenFOAM workflow by first generating the mesh for the simulation using the blockMesh routine, running the simulation with the simpleFoam solver, and finally visualizing the generated snapshots of the simulation using the paraFoam tool. To do all of this, we need to run the following sequence of commands in the running OpenFOAM session:

[session ID]: > cd $FOAM_RUN
[session ID]: > cp -r $FOAM_TUTORIALS/incompressible/simpleFoam/pitzDaily .
[session ID]: > cd pitzDaily
[session ID]: > blockMesh
[session ID]: > simpleFoam
[session ID]: > paraFoam

The routine paraFoam opens the ParaView application, which is used as the primary visualization software for OpenFOAM.

Step 6. Exit Docker and OpenFOAM

If the steps above executed successfully and every component of OpenFOAM works, we can close OpenFOAM and the container using simply an exit command:

[session ID]: > exit

Project description for water droplet simulation

OpenFOAM is primarily designed for computational fluid dynamics (CFD) and it is optimized to solve various numerical problems in this field of research. This tutorial showcases an example project, where I used OpenFOAM to simulate a simple fluid dynamics problem for one of my university classes.

My choice was to simulate a single water droplet falling into a vessel with still water at the bottom. The phenomenon is such an unspectacular and everyday process that everyone is familiar with it and knows what to expect. We expect the water at the bottom of the vessel to splash back when the droplet hits the water surface, creating circular waves afterwards, which then bounce back and forth between the walls of the vessel. After the moments of the droplet hitting the water surface, it hollows out the water for a small amount of time, creating a “crater” and pushing away the water in every direction. However, this crater is quickly filled back in by the water as it immediately tries to find its level. At this scale, surface tension is also a significant factor. All this results in a rapid and circularly symmetric movement of water particles, culminating in their collision at the center. This collision throws some water upwards in a violent (but in a small-scale) “explosion”.

My sole goal was to setup the simulation in a way that it produces an aesthetic, good-looking and realistic simulation of the described phenomenon and then visualize it in a clear and meaningful manner.

To realize such simulation in OpenFOAM, a number of configurations had to be done beforehand. Namely the mesh definition and generation, the so-called field generation and its setup and tuning the general parameters of the simulation. In the following section I will describe how the corresponding configuration files (so-called dictionary files in OpenFOAM) were set up for my project.

III. Creating a mesh for a water droplet simulation in OpenFOAM

OpenFOAM offers the blockMesh utility for mesh generation. It is a versatile tool capable of the creation of meshes with arbitrary grading, curved edges and more as stated in the official user guide. The mesh for any simulation needs to be defined in the dictionary file named blockMeshDict. When blockMesh is executed it reads data from this file and then creates the appropriate mesh files called points, faces, cells and boundary in the same directory as blockMeshDict. In OpenFOAM there is nothing unusual in defining a mesh compared to other regular methods and techniques. However it offers us a wide variety of tools and pre-defined routines to easily fine-tune our mesh at will and to utilize OpenFOAM in plenty of complex use cases.

As it was already mentioned, the blockMeshDict is one of the dictionary files used for the configuration of OpenFOAM simulations and it operates using just a handful of keywords. The mesh definition itself simply consist of giving appropriate values to the data tables denoted by each of these keywords.

III.1. Keywords for the blockMeshDict

III.1.1. convertToMeters

The first keyword is convertToMeters, which is the scaling factor for the coordinate system. This keyword defines the length of a unit vector in meters. In my simulation I tried to explore a fairly small-scale phenomenon (compared to eg. the size of humans), therefore I defined the unit length as

convertToMeters                   0.025;

III.1.2. vertices

The second keyword is vertices, where one can define the 3 dimensional $(x,y,z)$ coordinates of the mesh’s vertices as an array of vectors. OpenFOAM will only simulate points, sprites and physical quantities on the inside of the defined mesh. Everything leaving this mesh during the simulation, disappears.

For the water droplet project I defined a very simple, square-shaped vessel, with an open top side and some atmosphere over it. The atmosphere needs to be specified, because OpenFOAM will simulate particles only inside the defined mesh, even if a block of it is simply air. The mesh is a rectangular prism with height $4$-times longer than its base. The vessel height is resides in the bottom $1/4$ quarter of the mesh, while the atmosphere takes up the upper $3/4$ of it.

My vertices array in this setup was the following:

vertices
(
    // Bottom plane
    (0 0 0)  // 0
    (4 0 0)  // 1
    (4 0 4)  // 2
    (0 0 4)  // 3
    // Middle plane
    (0 4 0)  // 4 over 0
    (4 4 0)  // 5 over 1
    (4 4 4)  // 6 over 2
    (0 4 4)  // 7 over 3
    // Upper plane
    (0 16 0) // 8 over 4
    (4 16 0) // 9 over 5
    (4 16 4) // 10 over 6
    (0 16 4) // 11 over 7
);

The “Bottom plane” and “Middle plane” refers to the bottom and the top side of the vessel (where the top is completely open), while “Upper plane” refers to the top plane of the prism, which the simulation runs in.

III.1.3. blocks

The third keyword blocks is responsible for the definition of resolution and grading of an arbitrary volume in the mesh. In the context of OpenFOAM a “block” is always a hexahedra and its vertices are defined using the mesh vertices, detailed in the previous section. (There is a simple method how a block with less than 8 distinct vertices can be defined, but now I won’t detail it here.)

An arbitrary number of blocks can be defined and each of the blocks needs to specified using the following three parameters:

Vertex numbering
The list of vertex indeces, defining the hexahedra. This is always denoted with the `hex` keyword, followed by the list of $8$ appropriate vertex indeces inside a parenthesis. The order of indeces given should be always counter-clockwise. The defined block will be enclosed between two sides of the mesh, corresponding respectively to the first $4$ and the second $4$ vertices given for this array.
Number of cells
This entry corresponds to the resolution of the block in each directions. Values given as $(x, y, z)$ denotes the number of cells inside the block in the $x$, $y$ and $z$ directions respectively.
Cell expansion ratios
The expansion ratio enables the mesh to be graded, or refined, in specified directions. The ratio is that of the width of the end cell $\delta_{e}$ along one edge of a block to the width of the start cell $\delta_{s}$ along that edge as shown on Fig. 4.5 in official user guide. Two keywords can be used here, namely `simpleGrading` and `edgeGrading`, which specify one of the two grading specifications available in blockMesh.

In my project two separate blocks were defined. The first one represents the square-shaped vessel with the open lid at the bottom of the mesh, while the second one is the free, open region over the vessel. The resolution of the mesh was completely uniform, and was $70 \times 70$ cells in the horizontal direction. Since the height of the mesh is 4-times its width, I set to have 280 cells along the vertical axis for the distribution of cells to be uniform. The definitions of my blocks can be seen below.

blocks
(
    hex (0 1 5 4 3 2 6 7) (70 70 70) simpleGrading (1 1 1)
    hex (4 5 9 8 7 6 10 11) (70 210 70) simpleGrading (1 1 1)
);

III.1.4. boundary

The boundary keyword is used to identify the patch faces of the mesh, define their types and order similar faces under the same, arbitrary keyword. The definition of a patch-group is made by specifying a user-selected name for the group and the type of the patches, and finally identify the faces for the patch group. This can be made similarly as the definition of blocks above.

Faces can be only defined between mesh vertices (same as for blocks). To define a mesh face we have to give the indeces of the corresponding mesh vertices in a counter-clockwise order. (Corresponding mesh vertices here means the vertices of the face itself.) For a given patch-group an arbitrary large number of faces can be assigned to, which really helps in grouping mesh faces of the same types.

My final list of patches for the water droplet project can be seen below. The keyword lowerSideWalls denotes the side walls of the vessel itself, while upperSideWalls stands for the borders of the upper $3/4$ of the mesh, namely the free, open region. Keyword lowerWall represents the bottom of the vessel, while the atmosphere defines the top plane of the mesh.

boundary
(
    lowerSideWalls
    {
        type wall;
        faces
        (
            (0 1 5 4)
            (1 2 6 5)
            (2 3 7 6)
            (3 0 4 7)
        );
    }
    upperSideWalls
    {
        type wall;
        faces
        (
            (4 5 9 8)
            (5 6 10 9)
            (6 7 11 10)
            (7 4 8 11)
        );
    }
    lowerWall
    {
        type wall;
        faces
        (
            (0 1 2 3)
        );
    }
    atmosphere
    {
        type patch;
        faces
        (
            (8 9 10 11)
        );
    }
);

III.1.5 Other keywords (edges and mergePatches)

These two other keywords can be used to fine-tune the mesh generation of any OpenFOAM simulation and they’re offering a number of unique tools for the user. However because I didn’t use them for my project, I won’t detail the usage of them here. Refer to the official user guide for further information.

III.2. Generating the mesh

After successfully defining the mesh, the appropriate (real) mesh files can be generated by using OpenFOAM’s built-in utility, executed in the root of the project’s folder:

[session ID]: project_water> blockMesh

IV. The configuration of other dictionary files for the simulation

IV.1. setFieldsDict

The second most important configuration file is the setFieldsDict dictionary. This file is responsible to place specific materials, or set specific physical quantities in an arbitrary region inside the created mesh. Therefore this dictionary is probably the heart of a simulation in OpenFOAM, since this is responsible for the definition of the actually simulated quantities and fluids.

My setup for setFieldsDict can be seen below. First I’ve filled the whole mesh with the scalar field alpha.water set to $0$ everywhere. This ensured, that by default there are absolutely none water inside the mesh. Second I’ve defined two regions filled with water:

First region (Water in vessel)
This region was definied using OpenFOAM's boxToCell keyword, which creates a rectangular water block in the bottom of the vessel with a base area of $0.1\,\mathrm{m} \times 0.1\,\mathrm{m}$ and with height of $0.04\,\mathrm{m}$.
Second region (Water droplet)
This region was definied using the sphereToCell keyword, which creates a shperical block of water over the center of the rectangular water surface. The center of the sphere is $0.38\,\mathrm{m}$ high, measured from the bottom of the vessel and its radius is $0.004\,\mathrm{m}$. The water droplet also has an initial velocity of $-2\,\mathrm{m}/\mathrm{s}$ along the vertical axis.
defaultFieldValues
(
    volScalarFieldValue alpha.water 0
);

regions
(
    boxToCell
    {
        box (0 0 0) (0.1 0.04 0.1);
        fieldValues
        (
            volScalarFieldValue alpha.water 1
        );
    }
    sphereToCell
    {
        centre (0.05 0.38 0.05);
        radius 0.004;
        fieldValues
        (
            volScalarFieldValue alpha.water 1
            volVectorFieldValue U (0 -2 0)
        );
    }
);

Finishing up the setFieldsDict configuration file, after generating the mesh, the fields can also be generated by executing the following command in the root of the project’s directory:

[session ID]: project_water> setFields

IV.2. Other dictionary files

There are $4$ other dictionary files besides blockMeshDict and setFieldsDict: controlDict, decomposeParDict, fvSchemes and fvSolution.

controlDict
This dictionary file controls the interval of the I/O and the length of the simulation. As stated in the official user guide, only the time control and writeInterval entries are mandatory, every other entry uses its default value if not specified. In my project I've set endTime to $1.5$ and writeInterval to $0.005$. This ensured, that $1.5/0.005 = 300$ checkpoint files were created. Using this database a $10$ seconds long simulation with the frame rate of $30$ FPS can be constructed.
decomposeParDict
This file is responsible to control the parameters needed for an OpenFOAM simulation to be run in parallel. I'll detail its usage in section V.
fvSchemes
This dictionary file sets the numerical schemes for terms and quantities, such as derivatives in equations, that are calculated during a simulation. In my project I'm using the template file found in the interFoam/damBreak tutorial without any changes.
fvSolution
Called as the "Solution and algorithm control", this dictionary specifies the details about the solver methods, tolerances and algorithms used in a simulation. Similarly to fvSchemes, I used the file from the interFoam/damBreak tutorial.

V. Running an OpenFOAM simulation parallel using MPI

Running simulations in parallel using OpenFOAM is based on the technique of domain decomposition. In this framework the mesh and the simulated fields are broken down into numerous smaller pieces and these parts then are assigned to different processor cores to work with. As detailed in the official user guide, the process of parallel computation involves: decomposition of mesh and fields; running the application in parallel; and, post-processing the decomposed case.

V.1. Decomposition

The decomposition is performed by the decomposePar utility. The slicing of the geometry is specified by the parameters given in the decomposeParDict dictionary, By default, an example is presented in the interFoam/damBreak tutorial for the user to use as a template. During my project I used this particular configuration file for the water droplet simulation too. The content of the decomposeParDict dictionary used by me can be seen down below.

The file below shows, that the original geometry is decomposed into $4$ parts, determined by the numberOfSubdomains keyword using the decomposition method simple. This means, that the geometry is simply sliced into parts along the cardinal directions. The number of slices in this case is determined by the simpleCoeffs structure. Here the value of n is (2 2 1), which indicates the geometry is sliced into $2$-$2$ parts along the $x$ and $y$ axis.

The distributed keyword determines whether the output files should be scattered on numerous disks or not, where the locations are specified by the roots() list. In my case I didn’t used this part of the decomposePar utility, so I set the value of distributed to no.

numberOfSubdomains 4;

method          simple;

simpleCoeffs
{
    n               (2 2 1);
    delta           0.001;
}

distributed     no;

roots           ( );

After the dictionary file was appropriately setup, the decomposition can be started by running the utility inside the root of the project folder:

[session ID]: project_water> decomposePar

V.2. Running the simulation in parallel

Running the decomposed simulation on several cores in parallel is done by using a software implementation of the Message Passing Interface (MPI) standard. In my case I’ve used specifically MPICH, but eg. openMPI or other implementations can be also utilized. Using MPI is cannot be called as straightforward and it also offers us an overwhelmingly wide variety of tools and techniques useful for parallel computation. Detailing the usage of MPI is far beyond the scope of this document, so I’ll only mention those details here that I’ve used during the project too. Any further informations regarding parallel computation in OpenFOAM can be accessed in the official user guide.

Since I’ve sliced the mesh and fields into $4$ different parts, the simulation has to be allocated equally on $4$ concurrent processor cores. It should be mentioned, that I’m running a simulation on a single computer (while MPI would be also capable of executing it on a computer cluster) and I’m writing the output files on a single SSD. This two factors makes running an OpenFOAM simulation in parallel much easier, because the only aspect I was needed to give my attention to is to choose the correct number of processors and appropriate numerical solver method for my simulation.

As I’ve mentioned above I need to run the simulation exactly on $4$ processor cores, which can be set by using the -np flag running MPI. For the water droplet simulation I’ve had to use OpenFOAM’s interFoam solver method, since I’m working with exactly two incompressible, isothermal immiscible fluids (namely water and air). Running a simulation with the interFoam solver in parallel using MPI with $4$ cores on a singular machine and also writing output files on a single SSD can be done by the following command executed in the root of the project’s directory:

[session ID]: project_water> mpirun -np 4 interFoam -parallel |& tee project_water.log &

In my case the creation of a $10$ seconds long simulation with a frame rate of $30$ FPS taken approximately $16+$ hours to finish. During the simulation, a file named project_water.log is also created containing the complete terminal output during the process. The output will be separated into $4$ separate set of time directories, each of the sets named as processor_{N}, where N denotes the index of the processor core, which processed the dataset found in that folder.

V.3. Reconstruction

After a case has been run in parallel, it can be now reconstructed. This step is done by merging the sets of time directories from each processor_{N} directory into a single set of time directories. The reconstructPar utility performs such a reconstruction by executing the command:

[session ID]: project_water> reconstructPar

When the data is distributed across several disks, it must be first copied to the local case directory for reconstruction.

VI. Results and visualization

It was already mentioned, that by default, OpenFOAM utilizes ParaView to visualize the output checkpoints of a simulation. After a simulation is finished, one can load the checkpoint files in ParaView by starting the utility paraFoam. This will open ParaView, and the setup of the visualization can be started.

[session ID]: project_water> paraFoam

In the project I’ve simulated the time evolution of a number of physical quantities, like speed of particles or pressure, but most importantly the already mentioned quantity, denoted by alpha.water. This is the so-called phase fraction of water and air and denotes the proportion of water and air in a given location. If $\texttt{alpha.water} = 1$, then the volume in question is filled with water to a $100\,\%$. Similarly, if $\texttt{alpha.water} = 0$, there are no water in that location. Finally, if the value of alpha.water is between $0$ and $1$, water and air are mixed in that volume by the given proportion.

On the animation below I’ve visualized exactly this quantity. There we can see the contour line, where the alpha.water field is at least $0.001$. The borders of the vessel is also rendered invisible. The animation can be accessed on YouTube: