5.6 Processing large files in serial

When processing large files, it is not always convenient to run in parallel but process each parallel partition in serial, for example when interpolating a solution field from one mesh to another.

5.6.1 Using the nprocs and procid options

One option is to use the –nprocs and –procid command line options. For example, the following command will interpolate partition 2 of a decomposition into 10 partitions of fiile2.xml from file1.fld

This call will only therefore consider the interpolation process across one partition (namely, partition 2). To create the full interpolated field requires a loop over each of the partitions, which, in a bash shell can be run as

The resulting output will lie in a directory called file2.fld, with each of the different parallel partitions in files with names P0000000.fld, P0000001.fld, …, P0000009.fld. This is nearly a complete parallel field file. However, the Info.xml file, which contains the information about which elements lie in each partition, is missing. This can be generated by using the command

Note the final :info extension on the last argument is necessary to tell FieldConvert that you wish to generate an info file, but with the extension .xml. This syntax allows the routine not to get confused with the input/output XML files.

5.6.2 Using the –part-only and –part-only-overlapping options

Another approach to serially proessing a large file is to initially process the file into multiple partitions. This can be done with the –part-only option. So the command

will partition the mesh into 10 paritions and write each partition into a directory called file_xml. If you enter this directory you will find partitioned XML files P0000000.xml, P0000001.xml, …, P0000009.xml which can then be processed individually as outlined above.

There is also a –part-only-overlapping option, which can be run in the same fashion.

In this mode, the mesh is partitioned into 10 partitions in a similar manner, but the elements at the partition edges will now overlap, so that the intersection of each partition with its neighbours is non-empty. This is sometime helpful when, for example, producing a global isocontour which has been smoothed. Applying the smoothed isocontour extraction routine with the –part-only option will produce a series of isocontour where there will be a gap between partitions, as the smoother tends to shrink the isocontour within a partition. using the –part-only-overlapping option will still yield a shrinking isocontour, but the overlapping partitions help to overlap the partiiton boundaries.