The Collections and associated MatrixFree libraries adds optimisations to perform certain elemental operations collectively by applying an operator using either matrix-matrix or unrolled matrix free operations, rather than a sequence of matrix-vector multiplications. Certain operators benefit more than other from this treatment, so the following implementations are available:
All configuration relating to Collections is given in the
COLLECTIONS Xml element within the
NEKTAR XML element.
By default we now try to select the optimal choice of implementation when you first run a solver. If you run the solver in verbose mode you will observe an output of the form:
This shows the selected collection operation, in this case
MatrixFree, for the different
operators implmentations and the various approaches. Note that IterLocExp is equivalent to
NoCollection and IterStdExp is directly related to the IterPerExp option.
This choice of optimisation is then written into a file called
name of the user defined xml file. We note that the optimal choice is currently based on the
volumetric elements of the mesh (i.e. Tris and Quads in 2D and Tets, Pyramids, Prisms and
Hexs in 3D) and not on the boundary conditions. In the case of a parallel run the root process
will write the file based on the optimisation on this processor. In the case one type of element
is not on the root processor the output form the highest rank process with this element shape
will be outputted. Once this file is present it will be read directly rather than re-running the
COLLECTIONS section can be set manually within the
COLLECTIONS tag as shown in the
following example. Note this section can be added in either the input
Session.xml file or the
Session.opt file that is auto-generated.
Different implementations may be chosen for different element shapes and expansion orders.
ORDER sets the default implementation for any expansion orders not explicitly
The default implementation for all operators may be chosen through setting the
DEFAULT attribute of the
COLLECTIONS XML element to one of
StdMat sets up a standard matrix for
the element in the collection as the underlying operator. The following uses the
collated matrix-matrix type elemental operation for all operators and expansion
NoCollection option iterates over each expansion in the local region calling the local
operator which is implemented in a sum factorization method within the element. The
IterPerExp holds a standard expansion and then also holds an expanded copy of the
geometric factors within the collection operator.
SumFac is a sum factorization implementation
which undertakes each direction of the method over multiple elements in the collection. Finally
MatrixFree implements a vectorisation suitable version of the sum factorisation which has
minimal memory movement but requires some initial data re-orientation when vectorising over
The choice of implementation for each operator, for the given mesh and expansion orders, can
be selected selected automatically through an attribute in the
COLLECITON section. To enable
this, add the following to the Nektar++ session file:
This will collate elements from the given mesh and given expansion orders, run and time each
implementation strategy in turn, and select the fastest performing case. Note that the
selections will be mesh- and order- specific. The selections made via auto-tuning are output if
–verbose command-line switch is given.
The maximum number of elements within a single collection can be enforced using the