Nektar::LibUtilities::FieldIOXml Class Reference

#include <FieldIOXml.h>

Inheritance diagram for Nektar::LibUtilities::FieldIOXml:

Public Member Functions
	FieldIOXml (LibUtilities::CommSharedPtr pComm, bool sharedFilesystem)
	Default constructor. More...
virtual	~FieldIOXml ()
void	ImportFieldDefs (DataSourceSharedPtr dataSource, std::vector< FieldDefinitionsSharedPtr > &fielddefs, bool expChild)
	Import field definitions from the target file. More...
void	ImportFieldData (DataSourceSharedPtr dataSource, const std::vector< FieldDefinitionsSharedPtr > &fielddefs, std::vector< std::vector< NekDouble > > &fielddata)
	Import field data from a target file. More...
void	WriteMultiFldFileIDs (const std::string &outfile, const std::vector< std::string > fileNames, std::vector< std::vector< unsigned int > > &elementList, const FieldMetaDataMap &fieldinfomap=NullFieldMetaDataMap)
	Write out a file containing element ID to partition mapping. More...
void	SetUpFieldMetaData (const std::string &outname, const std::vector< FieldDefinitionsSharedPtr > &fielddefs, const FieldMetaDataMap &fieldmetadatamap)
	Set up field meta data map. More...
void	ImportMultiFldFileIDs (const std::string &inFile, std::vector< std::string > &fileNames, std::vector< std::vector< unsigned int > > &elementList, FieldMetaDataMap &fieldmetadatamap)
	Read file containing element ID to partition mapping. More...
void	v_Import (const std::string &infilename, std::vector< FieldDefinitionsSharedPtr > &fielddefs, std::vector< std::vector< NekDouble > > &fielddata=NullVectorNekDoubleVector, FieldMetaDataMap &fieldinfomap=NullFieldMetaDataMap, const Array< OneD, int > &ElementIDs=NullInt1DArray)
	Import an XML format file. More...
virtual const std::string &	GetClassName () const
	Returns the class name. More...
Public Member Functions inherited from Nektar::LibUtilities::FieldIO
	FieldIO (LibUtilities::CommSharedPtr pComm, bool sharedFilesystem)
	Constructor for FieldIO base class. More...
virtual	~FieldIO ()
void	Write (const std::string &outFile, std::vector< FieldDefinitionsSharedPtr > &fielddefs, std::vector< std::vector< NekDouble > > &fielddata, const FieldMetaDataMap &fieldinfomap=NullFieldMetaDataMap, const bool backup=false)
	Write out the field information to the file outFile. More...
void	Import (const std::string &infilename, std::vector< FieldDefinitionsSharedPtr > &fielddefs, std::vector< std::vector< NekDouble > > &fielddata=NullVectorNekDoubleVector, FieldMetaDataMap &fieldinfomap=NullFieldMetaDataMap, const Array< OneD, int > &ElementIDs=NullInt1DArray)
	Read field information from the file infilename. More...
DataSourceSharedPtr	ImportFieldMetaData (const std::string &filename, FieldMetaDataMap &fieldmetadatamap)
	Import the metadata from a field file. More...

Static Public Member Functions
static FieldIOSharedPtr	create (LibUtilities::CommSharedPtr pComm, bool sharedFilesystem)
	Creates an instance of this class. More...
Static Public Member Functions inherited from Nektar::LibUtilities::FieldIO
static const std::string	GetFileType (const std::string &filename, CommSharedPtr comm)
	Determine file type of given input file. More...
static std::shared_ptr< FieldIO >	CreateDefault (const LibUtilities::SessionReaderSharedPtr session)
	Returns an object for the default FieldIO method. More...
static std::shared_ptr< FieldIO >	CreateForFile (const LibUtilities::SessionReaderSharedPtr session, const std::string &filename)
	Construct a FieldIO object for a given input filename. More...
static void	AddInfoTag (TagWriterSharedPtr root, const FieldMetaDataMap &fieldmetadatamap)
	Add provenance information to the field metadata map. More...

Static Public Attributes
static std::string	className
	Name of class. More...

Private Member Functions
virtual void	v_Write (const std::string &outFile, std::vector< FieldDefinitionsSharedPtr > &fielddefs, std::vector< std::vector< NekDouble > > &fielddata, const FieldMetaDataMap &fieldinfomap=NullFieldMetaDataMap, const bool backup=false)
	Write an XML file to outFile given the field definitions fielddefs, field data fielddata and metadata fieldmetadatamap. More...
virtual DataSourceSharedPtr	v_ImportFieldMetaData (const std::string &filename, FieldMetaDataMap &fieldmetadatamap)
	Import field metadata from filename and return the data source which wraps filename. More...

Additional Inherited Members
Protected Member Functions inherited from Nektar::LibUtilities::FieldIO
int	CheckFieldDefinition (const FieldDefinitionsSharedPtr &fielddefs)
	Check field definitions for correctness and return storage size. More...
virtual std::string	GetFileEnding () const
	Helper function that determines default file extension. More...
std::string	SetUpOutput (const std::string outname, bool perRank, bool backup=false)
	Set up the filesystem ready for output. More...
Protected Attributes inherited from Nektar::LibUtilities::FieldIO
LibUtilities::CommSharedPtr	m_comm
	Communicator to use when writing parallel format. More...
bool	m_sharedFilesystem
	Boolean dictating whether we are on a shared filesystem. More...

Detailed Description

Definition at line 201 of file FieldIOXml.h.

Constructor & Destructor Documentation

FieldIOXml()

Nektar::LibUtilities::FieldIOXml::FieldIOXml	(	LibUtilities::CommSharedPtr	pComm,
		bool	sharedFilesystem
	)

Default constructor.

Parameters

pComm	Communicator.
sharedFilesystem	True if the underlying filesystem is shared by the compute nodes.

Definition at line 62 of file FieldIOXml.cpp.

    : FieldIO(pComm, sharedFilesystem)
{
}

~FieldIOXml()

virtual Nektar::LibUtilities::FieldIOXml::~FieldIOXml ( )

inlinevirtual

Definition at line 219 of file FieldIOXml.h.

References LIB_UTILITIES_EXPORT, Nektar::LibUtilities::NullFieldMetaDataMap, Nektar::NullInt1DArray, and Nektar::LibUtilities::NullVectorNekDoubleVector.

    {
    }

Member Function Documentation

create()

static FieldIOSharedPtr Nektar::LibUtilities::FieldIOXml::create ( LibUtilities::CommSharedPtr  pComm, bool  sharedFilesystem  )

inlinestatic

Creates an instance of this class.

Definition at line 205 of file FieldIOXml.h.

References Nektar::MemoryManager< DataType >::AllocateSharedPtr().

    {
        return MemoryManager<FieldIOXml>::AllocateSharedPtr(pComm,
                                                            sharedFilesystem);
    }

GetClassName()

virtual const std::string& Nektar::LibUtilities::FieldIOXml::GetClassName ( ) const

inlinevirtual

Returns the class name.

Implements Nektar::LibUtilities::FieldIO.

Definition at line 259 of file FieldIOXml.h.

References LIB_UTILITIES_EXPORT, and Nektar::LibUtilities::NullFieldMetaDataMap.

    {
        return className;
    }

ImportFieldData()

void Nektar::LibUtilities::FieldIOXml::ImportFieldData	(	DataSourceSharedPtr	dataSource,
		const std::vector< FieldDefinitionsSharedPtr > &	fielddefs,
		std::vector< std::vector< NekDouble > > &	fielddata
	)

Import field data from a target file.

Parameters

dataSource	Target XML file
fielddefs	Field definitions for file
fielddata	On return, contains field data for each field.

Definition at line 1101 of file FieldIOXml.cpp.

References ASSERTL0, Nektar::LibUtilities::FieldIO::CheckFieldDefinition(), Nektar::LibUtilities::XmlDataSource::Get(), Nektar::LibUtilities::CompressData::GetCompressString(), WARNINGL0, and Nektar::LibUtilities::CompressData::ZlibDecodeFromBase64Str().

Referenced by v_Import().

{
    int cntdumps = 0;
    XmlDataSourceSharedPtr xml =
        std::static_pointer_cast<XmlDataSource>(dataSource);

    TiXmlElement *master =
        NULL; // Master tag within which all data is contained.

    master = xml->Get().FirstChildElement("NEKTAR");
    ASSERTL0(master, "Unable to find NEKTAR tag in file.");

    // Loop through all nektar tags, finding all of the element tags.
    while (master)
    {
        TiXmlElement *element = master->FirstChildElement("ELEMENTS");
        ASSERTL0(element, "Unable to find ELEMENTS tag within nektar tag.");
        while (element)
        {
            // Extract the body, which the "data".
            TiXmlNode *elementChild = element->FirstChild();
            ASSERTL0(elementChild,
                     "Unable to extract the data from the element tag.");
            std::string elementStr;
            while (elementChild)
            {
                if (elementChild->Type() == TiXmlNode::TINYXML_TEXT)
                {
                    elementStr += elementChild->ToText()->ValueStr();
                }
                elementChild = elementChild->NextSibling();
            }

            std::vector<NekDouble> elementFieldData;

            // Convert from base64 to binary.
            const char *CompressStr = element->Attribute("COMPRESSED");
            if (CompressStr)
            {
                WARNINGL0(boost::iequals(CompressStr,
                                         CompressData::GetCompressString()),
                          "Compressed formats do not match. "
                          "Expected: " +
                          CompressData::GetCompressString() +
                          " but got " + std::string(CompressStr));
            }

            ASSERTL0(Z_OK == CompressData::ZlibDecodeFromBase64Str(
                                 elementStr, elementFieldData),
                     "Failed to decompress field data.");
            fielddata.push_back(elementFieldData);

            int datasize = CheckFieldDefinition(fielddefs[cntdumps]);
            ASSERTL0(
                fielddata[cntdumps].size() ==
                    datasize * fielddefs[cntdumps]->m_fields.size(),
                "Input data is not the same length as header infoarmation");

            cntdumps++;

            element = element->NextSiblingElement("ELEMENTS");
        }
        master = master->NextSiblingElement("NEKTAR");
    }
}

ImportFieldDefs()

void Nektar::LibUtilities::FieldIOXml::ImportFieldDefs	(	DataSourceSharedPtr	dataSource,
		std::vector< FieldDefinitionsSharedPtr > &	fielddefs,
		bool	expChild
	)

Import field definitions from the target file.

Parameters

dataSource	Target XML file
fielddefs	Output vector that will contain read field definitions.
expChild	Determines if the field definitions are defined by <EXPANSIONS> or in <NEKTAR>.

Definition at line 768 of file FieldIOXml.cpp.

References Nektar::MemoryManager< DataType >::AllocateSharedPtr(), ASSERTL0, Nektar::LibUtilities::BasisTypeMap, Nektar::ErrorUtil::ewarning, Nektar::ParseUtils::GenerateSeqVector(), Nektar::ParseUtils::GenerateVector(), Nektar::LibUtilities::XmlDataSource::Get(), Nektar::LibUtilities::CompressData::GetCompressString(), Nektar::LibUtilities::kPointsTypeStr, CG_Iterations::loc, NEKERROR, Nektar::LibUtilities::ShapeTypeMap, Nektar::LibUtilities::SIZE_BasisType, Nektar::LibUtilities::SIZE_PointsType, Nektar::LibUtilities::SIZE_ShapeType, and WARNINGL0.

Referenced by v_Import().

{
    XmlDataSourceSharedPtr xml =
        std::static_pointer_cast<XmlDataSource>(dataSource);
    TiXmlElement *master =
        NULL; // Master tag within which all data is contained.

    master = xml->Get().FirstChildElement("NEKTAR");
    ASSERTL0(master, "Unable to find NEKTAR tag in file.");
    std::string strLoop   = "NEKTAR";
    TiXmlElement *loopXml = master;

    TiXmlElement *expansionTypes;
    if (expChild)
    {
        expansionTypes = master->FirstChildElement("EXPANSIONS");
        ASSERTL0(expansionTypes, "Unable to find EXPANSIONS tag in file.");
        loopXml = expansionTypes;
        strLoop = "EXPANSIONS";
    }

    // Loop through all nektar tags, finding all of the element tags.
    while (loopXml)
    {
        TiXmlElement *element = loopXml->FirstChildElement("ELEMENTS");

        while (element)
        {
            // Extract the attributes.
            std::string idString;
            std::string shapeString;
            std::string basisString;
            std::string homoLengthsString;
            std::string homoSIDsString;
            std::string homoZIDsString;
            std::string homoYIDsString;
            std::string numModesString;
            std::string numPointsString;
            std::string fieldsString;
            std::string pointsString;
            bool pointDef        = false;
            bool numPointDef     = false;
            TiXmlAttribute *attr = element->FirstAttribute();
            while (attr)
            {
                std::string attrName(attr->Name());
                if (attrName == "FIELDS")
                {
                    fieldsString.insert(0, attr->Value());
                }
                else if (attrName == "SHAPE")
                {
                    shapeString.insert(0, attr->Value());
                }
                else if (attrName == "BASIS")
                {
                    basisString.insert(0, attr->Value());
                }
                else if (attrName == "HOMOGENEOUSLENGTHS")
                {
                    homoLengthsString.insert(0, attr->Value());
                }
                else if (attrName == "HOMOGENEOUSSIDS")
                {
                    homoSIDsString.insert(0, attr->Value());
                }
                else if (attrName == "HOMOGENEOUSZIDS")
                {
                    homoZIDsString.insert(0, attr->Value());
                }
                else if (attrName == "HOMOGENEOUSYIDS")
                {
                    homoYIDsString.insert(0, attr->Value());
                }
                else if (attrName == "NUMMODESPERDIR")
                {
                    numModesString.insert(0, attr->Value());
                }
                else if (attrName == "ID")
                {
                    idString.insert(0, attr->Value());
                }
                else if (attrName == "POINTSTYPE")
                {
                    pointsString.insert(0, attr->Value());
                    pointDef = true;
                }
                else if (attrName == "NUMPOINTSPERDIR")
                {
                    numPointsString.insert(0, attr->Value());
                    numPointDef = true;
                }
                else if (attrName == "COMPRESSED")
                {
                    WARNINGL0(boost::iequals(attr->Value(),
                                             CompressData::GetCompressString()),
                              "Compressed formats do not "
                              "match. Expected: " +
                              CompressData::GetCompressString() +
                              " but got " + std::string(attr->Value()));
                }
                else if (attrName == "BITSIZE")
                {
                    // This information is for future compatibility
                    // issues, for example in case we end up using a 128
                    // bit machine. Currently just do nothing.
                }
                else
                {
                    std::string errstr("Unknown attribute: ");
                    errstr += attrName;
                    NEKERROR(ErrorUtil::ewarning, errstr.c_str());
                }

                // Get the next attribute.
                attr = attr->Next();
            }

            // Check to see if using strips formulation
            bool strips = false;
            if (shapeString.find("Strips") != std::string::npos)
            {
                strips = true;
            }

            // Check to see if homogeneous expansion and if so
            // strip down the shapeString definition
            int numHomoDir = 0;
            size_t loc;
            //---> This finds the first location of  'n'!
            if ((loc = shapeString.find_first_of("-")) != std::string::npos)
            {
                if (shapeString.find("Exp1D") != std::string::npos)
                {
                    numHomoDir = 1;
                }
                else // HomogeneousExp1D
                {
                    numHomoDir = 2;
                }

                shapeString.erase(loc, shapeString.length());
            }

            // Reconstruct the fielddefs.
            std::vector<unsigned int> elementIds;
            {
                bool valid =
                    ParseUtils::GenerateSeqVector(idString, elementIds);
                ASSERTL0(valid, "Unable to correctly parse the element ids.");
            }

            // Get the geometrical shape
            ShapeType shape = (ShapeType)0;
            bool valid = false;
            for (unsigned int j = 0; j < SIZE_ShapeType; j++)
            {
                if (ShapeTypeMap[j] == shapeString)
                {
                    shape = (ShapeType)j;
                    valid = true;
                    break;
                }
            }

            ASSERTL0(valid,
                     std::string("Unable to correctly parse the shape type: ")
                         .append(shapeString)
                         .c_str());

            // Get the basis
            std::vector<std::string> basisStrings;
            std::vector<BasisType> basis;
            valid = ParseUtils::GenerateVector(basisString, basisStrings);
            ASSERTL0(valid, "Unable to correctly parse the basis types.");
            for (std::vector<std::string>::size_type i = 0;
                 i < basisStrings.size();
                 i++)
            {
                valid = false;
                for (unsigned int j = 0; j < SIZE_BasisType; j++)
                {
                    if (BasisTypeMap[j] == basisStrings[i])
                    {
                        basis.push_back((BasisType)j);
                        valid = true;
                        break;
                    }
                }
                ASSERTL0(
                    valid,
                    std::string("Unable to correctly parse the basis type: ")
                        .append(basisStrings[i])
                        .c_str());
            }

            // Get homoLengths
            std::vector<NekDouble> homoLengths;
            if (numHomoDir)
            {
                valid = ParseUtils::GenerateVector(homoLengthsString,
                                                   homoLengths);
                ASSERTL0(valid, "Unable to correctly parse the number of "
                                "homogeneous lengths.");
            }

            // Get Homogeneous strips IDs
            std::vector<unsigned int> homoSIDs;
            if (strips)
            {
                valid = ParseUtils::GenerateVector(homoSIDsString, homoSIDs);
                ASSERTL0(valid,
                         "Unable to correctly parse homogeneous strips IDs.");
            }

            // Get Homogeneous points IDs
            std::vector<unsigned int> homoZIDs;
            std::vector<unsigned int> homoYIDs;

            if (numHomoDir == 1)
            {
                valid = ParseUtils::GenerateSeqVector(homoZIDsString,
                                                      homoZIDs);
                ASSERTL0(valid,
                         "Unable to correctly parse homogeneous planes IDs.");
            }

            if (numHomoDir == 2)
            {
                valid = ParseUtils::GenerateSeqVector(homoZIDsString,
                                                      homoZIDs);
                ASSERTL0(valid, "Unable to correctly parse homogeneous lines "
                                "IDs in z-direction.");
                valid = ParseUtils::GenerateSeqVector(homoYIDsString,
                                                      homoYIDs);
                ASSERTL0(valid, "Unable to correctly parse homogeneous lines "
                                "IDs in y-direction.");
            }

            // Get points type
            std::vector<PointsType> points;

            if (pointDef)
            {
                std::vector<std::string> pointsStrings;
                valid = ParseUtils::GenerateVector(pointsString, pointsStrings);
                ASSERTL0(valid, "Unable to correctly parse the points types.");
                for (std::vector<std::string>::size_type i = 0;
                     i < pointsStrings.size();
                     i++)
                {
                    valid = false;
                    for (unsigned int j = 0; j < SIZE_PointsType; j++)
                    {
                        if (kPointsTypeStr[j] == pointsStrings[i])
                        {
                            points.push_back((PointsType)j);
                            valid = true;
                            break;
                        }
                    }

                    ASSERTL0(valid,
                             std::string(
                                 "Unable to correctly parse the points type: ")
                                 .append(pointsStrings[i])
                                 .c_str());
                }
            }

            // Get numModes
            std::vector<unsigned int> numModes;
            bool UniOrder = false;

            if (strstr(numModesString.c_str(), "UNIORDER:"))
            {
                UniOrder = true;
            }

            valid = ParseUtils::GenerateVector(
                numModesString.substr(9), numModes);
            ASSERTL0(valid, "Unable to correctly parse the number of modes.");

            // Get numPoints
            std::vector<unsigned int> numPoints;
            if (numPointDef)
            {
                valid = ParseUtils::GenerateVector(numPointsString, numPoints);
                ASSERTL0(valid,
                         "Unable to correctly parse the number of points.");
            }

            // Get fields names
            std::vector<std::string> Fields;
            valid = ParseUtils::GenerateVector(fieldsString, Fields);
            ASSERTL0(valid, "Unable to correctly parse the number of fields.");

            FieldDefinitionsSharedPtr fielddef =
                MemoryManager<FieldDefinitions>::AllocateSharedPtr(shape,
                                                                   elementIds,
                                                                   basis,
                                                                   UniOrder,
                                                                   numModes,
                                                                   Fields,
                                                                   numHomoDir,
                                                                   homoLengths,
                                                                   strips,
                                                                   homoSIDs,
                                                                   homoZIDs,
                                                                   homoYIDs,
                                                                   points,
                                                                   pointDef,
                                                                   numPoints,
                                                                   numPointDef);

            fielddefs.push_back(fielddef);

            element = element->NextSiblingElement("ELEMENTS");
        }
        loopXml = loopXml->NextSiblingElement(strLoop);
    }
}

ImportMultiFldFileIDs()

void Nektar::LibUtilities::FieldIOXml::ImportMultiFldFileIDs	(	const std::string &	inFile,
		std::vector< std::string > &	fileNames,
		std::vector< std::vector< unsigned int > > &	elementList,
		FieldMetaDataMap &	fieldmetadatamap
	)

Read file containing element ID to partition mapping.

This function reads an XML file (usually called Info.xml) that contains the element IDs that are contained within each partition, as well as the field metadata map.

Parameters

inFile	Input multi-field file name.
fileNames	List of partition filenames.
elementList	Vector of element IDs that lie on each process.
fieldmetadatamap	Field metadata map that is read from inFile.

Definition at line 428 of file FieldIOXml.cpp.

References ASSERTL0, and Nektar::ParseUtils::GenerateSeqVector().

Referenced by Nektar::LibUtilities::PtsIO::Import(), and v_Import().

{
    boost::ignore_unused(fieldmetadatamap);

    TiXmlDocument doc(inFile);
    bool loadOkay = doc.LoadFile();

    std::stringstream errstr;
    errstr << "Unable to load file: " << inFile << std::endl;
    errstr << "Reason: " << doc.ErrorDesc() << std::endl;
    errstr << "Position: Line " << doc.ErrorRow() << ", Column "
           << doc.ErrorCol() << std::endl;
    ASSERTL0(loadOkay, errstr.str());

    // Handle on XML document
    TiXmlHandle docHandle(&doc);

    // Retrieve main NEKTAR tag - XML specification states one
    // top-level element tag per file.
    TiXmlElement *master = doc.FirstChildElement("NEKTAR");
    ASSERTL0(master, "Unable to find NEKTAR tag in file.");

    // Partition element tag name
    std::string strPartition = "Partition";

    // First attempt to get the first Partition element
    TiXmlElement *fldfileIDs = master->FirstChildElement(strPartition.c_str());
    if (!fldfileIDs)
    {
        // If this files try previous name
        strPartition = "MultipleFldFiles";
        fldfileIDs   = master->FirstChildElement("MultipleFldFiles");
    }
    ASSERTL0(fldfileIDs,
             "Unable to find 'Partition' or 'MultipleFldFiles' tag "
             "within nektar tag.");

    while (fldfileIDs)
    {
        // Read file name of partition file
        const char *attr = fldfileIDs->Attribute("FileName");
        ASSERTL0(attr,
                 "'FileName' not provided as an attribute of '" + strPartition +
                     "' tag.");
        fileNames.push_back(std::string(attr));

        const char *elementIDs = fldfileIDs->GetText();
        ASSERTL0(elementIDs, "Element IDs not specified.");

        std::string elementIDsStr(elementIDs);

        std::vector<unsigned int> idvec;
        ParseUtils::GenerateSeqVector(elementIDsStr, idvec);

        elementList.push_back(idvec);

        fldfileIDs = fldfileIDs->NextSiblingElement(strPartition.c_str());
    }
}

SetUpFieldMetaData()

void Nektar::LibUtilities::FieldIOXml::SetUpFieldMetaData	(	const std::string &	outname,
		const std::vector< FieldDefinitionsSharedPtr > &	fielddefs,
		const FieldMetaDataMap &	fieldmetadatamap
	)

Set up field meta data map.

This routine sets up the necessary information for the field metadata map before calling FieldIOXml::WriteMultiFldFileIDs, which involves each process sending its element ID list to the root processor. The root processor writes the Info.xml file.

Parameters

outname	Output directory.
fielddefs	Field definitions, needed to grab element IDs.
fieldmetadatamap	Field metadata map that is also written to the Info.xml file.

Definition at line 697 of file FieldIOXml.cpp.

References ASSERTL0, CellMLToNektar.pycml::format, Nektar::LibUtilities::FieldIO::GetFileEnding(), Nektar::LibUtilities::FieldIO::m_comm, Nektar::LibUtilities::PortablePath(), Nektar::LibUtilities::ReduceMax, and WriteMultiFldFileIDs().

Referenced by v_Write().

{
    ASSERTL0(!outname.empty(), "Empty path given to SetUpFieldMetaData()");

    unsigned int nprocs = m_comm->GetSize();
    unsigned int rank   = m_comm->GetRank();

    fs::path specPath(outname);

    // Compute number of elements on this process and share with other
    // processes. Also construct list of elements on this process from
    // available vector of field definitions.
    std::vector<size_t>       elmtnums(nprocs, 0);
    std::vector<unsigned int> idlist;
    for (size_t i = 0; i < fielddefs.size(); ++i)
    {
        elmtnums[rank] += fielddefs[i]->m_elementIDs.size();
        idlist.insert(idlist.end(),
                      fielddefs[i]->m_elementIDs.begin(),
                      fielddefs[i]->m_elementIDs.end());
    }
    m_comm->AllReduce(elmtnums, LibUtilities::ReduceMax);

    // Collate per-process element lists on root process to generate
    // the info file.
    if (rank == 0)
    {
        std::vector<std::vector<unsigned int> > ElementIDs(nprocs);

        // Populate the list of element ID lists from all processes
        ElementIDs[0] = idlist;
        for (size_t i = 1; i < nprocs; ++i)
        {
            std::vector<unsigned int> tmp(elmtnums[i]);
            m_comm->Recv(i, tmp);
            ElementIDs[i] = tmp;
        }

        // Set up output names
        std::vector<std::string> filenames;
        for (unsigned int i = 0; i < nprocs; ++i)
        {
            boost::format pad("P%1$07d.%2$s");
            pad % i % GetFileEnding();
            filenames.push_back(pad.str());
        }

        // Write the Info.xml file
        std::string infofile =
            LibUtilities::PortablePath(specPath / fs::path("Info.xml"));

        WriteMultiFldFileIDs(infofile, filenames, ElementIDs, fieldmetadatamap);
    }
    else
    {
        // Send this process's ID list to the root process
        m_comm->Send(0, idlist);
    }
}

v_Import()

void Nektar::LibUtilities::FieldIOXml::v_Import ( const std::string &  infilename, std::vector< FieldDefinitionsSharedPtr > &  fielddefs, std::vector< std::vector< NekDouble > > &  fielddata = NullVectorNekDoubleVector, FieldMetaDataMap &  fieldinfomap = NullFieldMetaDataMap, const Array< OneD, int > &  ElementIDs = NullInt1DArray  )

virtual

Import an XML format file.

Parameters

finfilename	Input filename
fielddefs	Field definitions of resulting field
fielddata	Field data of resulting field
fieldinfomap	Field metadata of resulting field
ElementIDs	If specified, contains the list of element IDs on this rank. The resulting field definitions will only contain data for the element IDs specified in this array.

Implements Nektar::LibUtilities::FieldIO.

Definition at line 504 of file FieldIOXml.cpp.

References Nektar::LibUtilities::XmlDataSource::create(), ImportFieldData(), ImportFieldDefs(), Nektar::LibUtilities::FieldIO::ImportFieldMetaData(), ImportMultiFldFileIDs(), Nektar::NullInt1DArray, Nektar::LibUtilities::NullVectorNekDoubleVector, and Nektar::LibUtilities::PortablePath().

{
    std::string infile = infilename;

    fs::path pinfilename(infilename);

    // Check to see whether infile is a directory and therefore read in parallel
    // or serial.
    if (fs::is_directory(pinfilename))
    {
        fs::path infofile("Info.xml");
        fs::path fullpath = pinfilename / infofile;
        infile            = PortablePath(fullpath);

        std::vector<std::string> filenames;
        std::vector<std::vector<unsigned int> > elementIDs_OnPartitions;

        ImportMultiFldFileIDs(
            infile, filenames, elementIDs_OnPartitions, fieldinfomap);

        // Load metadata
        ImportFieldMetaData(infile, fieldinfomap);

        if (ElementIDs == NullInt1DArray) // load all elements
        {
            for (int i = 0; i < filenames.size(); ++i)
            {
                fs::path pfilename(filenames[i]);
                fullpath                       = pinfilename / pfilename;
                std::string fname              = PortablePath(fullpath);
                DataSourceSharedPtr dataSource = XmlDataSource::create(fname);
                ImportFieldDefs(dataSource, fielddefs, false);
                if (fielddata != NullVectorNekDoubleVector)
                {
                    ImportFieldData(dataSource, fielddefs, fielddata);
                }
            }
        }
        else // only load relevant elements from partitions
        {
            int i, j;
            std::map<int, std::vector<int> > FileIDs;
            std::set<int> LoadFile;

            for (i = 0; i < elementIDs_OnPartitions.size(); ++i)
            {
                for (j = 0; j < elementIDs_OnPartitions[i].size(); ++j)
                {
                    FileIDs[elementIDs_OnPartitions[i][j]].push_back(i);
                }
            }

            for (i = 0; i < ElementIDs.num_elements(); ++i)
            {
                auto it = FileIDs.find(ElementIDs[i]);
                if (it != FileIDs.end())
                {
                    for (j = 0; j < it->second.size(); ++j)
                    {
                        LoadFile.insert(it->second[j]);
                    }
                }
            }

            for (auto &iter : LoadFile)
            {
                fs::path pfilename(filenames[iter]);
                fullpath                       = pinfilename / pfilename;
                std::string fname              = PortablePath(fullpath);
                DataSourceSharedPtr dataSource = XmlDataSource::create(fname);
                ImportFieldDefs(dataSource, fielddefs, false);
                if (fielddata != NullVectorNekDoubleVector)
                {
                    ImportFieldData(dataSource, fielddefs, fielddata);
                }
            }
        }
    }
    else
    {
        // serial format case
        DataSourceSharedPtr doc = ImportFieldMetaData(infilename, fieldinfomap);
        ImportFieldDefs(doc, fielddefs, false);
        if (fielddata != NullVectorNekDoubleVector)
        {
            ImportFieldData(doc, fielddefs, fielddata);
        }
    }
}

v_ImportFieldMetaData()

DataSourceSharedPtr Nektar::LibUtilities::FieldIOXml::v_ImportFieldMetaData ( const std::string &  filename, FieldMetaDataMap &  fieldmetadatamap  )

privatevirtual

Import field metadata from filename and return the data source which wraps filename.

Parameters

filename	Input filename.
fieldmetadatamap	Resulting field metadata from dataSource.

Implements Nektar::LibUtilities::FieldIO.

Definition at line 605 of file FieldIOXml.cpp.

References ASSERTL0, Nektar::LibUtilities::XmlDataSource::create(), Nektar::ErrorUtil::efatal, Nektar::LibUtilities::XmlDataSource::Get(), and NEKERROR.

{
    DataSourceSharedPtr doc    = XmlDataSource::create(filename);
    XmlDataSourceSharedPtr xml = std::static_pointer_cast<XmlDataSource>(doc);
    TiXmlElement *metadata     = 0;
    TiXmlElement *master       = 0; // Master tag within which all data is
                                    // contained.

    master = xml->Get().FirstChildElement("NEKTAR");
    ASSERTL0(master, "Unable to find NEKTAR tag in file.");
    std::string strLoop = "NEKTAR";

    // Retain original metadata structure for backwards compatibility
    // TODO: Remove old metadata format
    metadata = master->FirstChildElement("FIELDMETADATA");
    if (metadata)
    {
        TiXmlElement *param = metadata->FirstChildElement("P");

        while (param)
        {
            TiXmlAttribute *paramAttr = param->FirstAttribute();
            std::string attrName(paramAttr->Name());
            std::string paramString;

            if (attrName == "PARAM")
            {
                paramString.insert(0, paramAttr->Value());
            }
            else
            {
                NEKERROR(ErrorUtil::efatal,
                         "PARAM not provided as an attribute in "
                         "FIELDMETADATA section");
            }

            // Now read body of param
            std::string paramBodyStr;

            TiXmlNode *paramBody = param->FirstChild();

            paramBodyStr += paramBody->ToText()->Value();

            fieldmetadatamap[paramString] = paramBodyStr;
            param                         = param->NextSiblingElement("P");
        }
    }

    // New metadata format
    metadata = master->FirstChildElement("Metadata");
    if (metadata)
    {
        TiXmlElement *param = metadata->FirstChildElement();

        while (param)
        {
            std::string paramString = param->Value();
            if (paramString != "Provenance")
            {
                // Now read body of param
                if (param->NoChildren())
                {
                    fieldmetadatamap[paramString] = "";
                }
                else
                {
                    TiXmlNode *paramBody     = param->FirstChild();
                    std::string paramBodyStr = paramBody->ToText()->Value();
                    fieldmetadatamap[paramString] = paramBodyStr;
                }
            }
            param = param->NextSiblingElement();
        }
    }

    return doc;
}

v_Write()

void Nektar::LibUtilities::FieldIOXml::v_Write ( const std::string &  outFile, std::vector< FieldDefinitionsSharedPtr > &  fielddefs, std::vector< std::vector< NekDouble > > &  fielddata, const FieldMetaDataMap &  fieldmetadatamap = NullFieldMetaDataMap, const bool  backup = false  )

privatevirtual

Write an XML file to outFile given the field definitions fielddefs, field data fielddata and metadata fieldmetadatamap.

The writing strategy is as follows:

• Use FieldIO::SetUpOutput to construct the directory to contain each partition.
• The root processor writes an Info.xml file containing the field metadata and an index that describes which elements lie in which XML file.
• Each processor then writes an XML file containing the field definitions for that processor and output data in base64-encoded zlib-compressed format.

Parameters

outFile	Output filename.
fielddefs	Input field definitions.
fielddata	Input field data.
fieldmetadatamap	Field metadata.

Implements Nektar::LibUtilities::FieldIO.

Definition at line 87 of file FieldIOXml.cpp.

References Nektar::LibUtilities::FieldIO::AddInfoTag(), ASSERTL0, ASSERTL1, Nektar::LibUtilities::BasisTypeMap, Nektar::LibUtilities::FieldIO::CheckFieldDefinition(), Nektar::ParseUtils::GenerateSeqString(), Nektar::LibUtilities::CompressData::GetBitSizeStr(), Nektar::LibUtilities::CompressData::GetCompressString(), Nektar::LibUtilities::FieldIO::m_comm, SetUpFieldMetaData(), Nektar::LibUtilities::FieldIO::SetUpOutput(), Nektar::LibUtilities::ShapeTypeMap, and Nektar::LibUtilities::CompressData::ZlibEncodeToBase64Str().

{
    double tm0 = 0.0, tm1 = 0.0;
    if (m_comm->TreatAsRankZero())
    {
        tm0 = m_comm->Wtime();
    }

    // Check everything seems sensible
    ASSERTL1(fielddefs.size() == fielddata.size(),
             "Length of fielddefs and fielddata incompatible");
    for (int f = 0; f < fielddefs.size(); ++f)
    {
        ASSERTL1(fielddata[f].size() > 0,
                 "Fielddata vector must contain at least one value.");

        ASSERTL1(fielddata[f].size() ==
                     fielddefs[f]->m_fields.size() *
                         CheckFieldDefinition(fielddefs[f]),
                 "Invalid size of fielddata vector.");
    }

    // Prepare to write out data. In parallel, we must create directory and
    // determine the full pathname to the file to write out.  Any existing
    // file/directory which is in the way is removed.
    std::string filename = SetUpOutput(outFile, true, backup);
    SetUpFieldMetaData(outFile, fielddefs, fieldmetadatamap);

    // Create the file (partition)
    TiXmlDocument doc;
    TiXmlDeclaration *decl = new TiXmlDeclaration("1.0", "utf-8", "");
    doc.LinkEndChild(decl);

    TiXmlElement *root = new TiXmlElement("NEKTAR");
    doc.LinkEndChild(root);

    AddInfoTag(XmlTagWriterSharedPtr(new XmlTagWriter(root)), fieldmetadatamap);

    for (int f = 0; f < fielddefs.size(); ++f)
    {
        //---------------------------------------------
        // Write ELEMENTS
        TiXmlElement *elemTag = new TiXmlElement("ELEMENTS");
        root->LinkEndChild(elemTag);

        // Write FIELDS
        std::string fieldsString;
        {
            std::stringstream fieldsStringStream;
            bool first = true;
            for (std::vector<int>::size_type i = 0;
                 i < fielddefs[f]->m_fields.size();
                 i++)
            {
                if (!first)
                {
                    fieldsStringStream << ",";
                }
                fieldsStringStream << fielddefs[f]->m_fields[i];
                first = false;
            }
            fieldsString = fieldsStringStream.str();
        }
        elemTag->SetAttribute("FIELDS", fieldsString);

        // Write SHAPE
        std::string shapeString;
        {
            std::stringstream shapeStringStream;
            shapeStringStream << ShapeTypeMap[fielddefs[f]->m_shapeType];
            if (fielddefs[f]->m_numHomogeneousDir == 1)
            {
                shapeStringStream << "-HomogenousExp1D";
            }
            else if (fielddefs[f]->m_numHomogeneousDir == 2)
            {
                shapeStringStream << "-HomogenousExp2D";
            }

            if (fielddefs[f]->m_homoStrips)
            {
                shapeStringStream << "-Strips";
            }

            shapeString = shapeStringStream.str();
        }
        elemTag->SetAttribute("SHAPE", shapeString);

        // Write BASIS
        std::string basisString;
        {
            std::stringstream basisStringStream;
            bool first = true;
            for (std::vector<BasisType>::size_type i = 0;
                 i < fielddefs[f]->m_basis.size();
                 i++)
            {
                if (!first)
                {
                    basisStringStream << ",";
                }
                basisStringStream << BasisTypeMap[fielddefs[f]->m_basis[i]];
                first = false;
            }
            basisString = basisStringStream.str();
        }
        elemTag->SetAttribute("BASIS", basisString);

        // Write homogeneuous length details
        if (fielddefs[f]->m_numHomogeneousDir)
        {
            std::string homoLenString;
            {
                std::stringstream homoLenStringStream;
                bool first = true;
                for (int i = 0; i < fielddefs[f]->m_numHomogeneousDir; ++i)
                {
                    if (!first)
                        homoLenStringStream << ",";
                    homoLenStringStream
                        << fielddefs[f]->m_homogeneousLengths[i];
                    first = false;
                }
                homoLenString = homoLenStringStream.str();
            }
            elemTag->SetAttribute("HOMOGENEOUSLENGTHS", homoLenString);
        }

        // Write homogeneuous planes/lines details
        if (fielddefs[f]->m_numHomogeneousDir)
        {
            if (fielddefs[f]->m_homogeneousYIDs.size() > 0)
            {
                std::string homoYIDsString;
                {
                    std::stringstream homoYIDsStringStream;
                    bool first = true;
                    for (int i = 0; i < fielddefs[f]->m_homogeneousYIDs.size();
                         i++)
                    {
                        if (!first)
                        {
                            homoYIDsStringStream << ",";
                        }
                        homoYIDsStringStream
                            << fielddefs[f]->m_homogeneousYIDs[i];
                        first = false;
                    }
                    homoYIDsString = homoYIDsStringStream.str();
                }
                elemTag->SetAttribute("HOMOGENEOUSYIDS", homoYIDsString);
            }

            if (fielddefs[f]->m_homogeneousZIDs.size() > 0)
            {
                std::string homoZIDsString;
                {
                    std::stringstream homoZIDsStringStream;
                    bool first = true;
                    for (int i = 0; i < fielddefs[f]->m_homogeneousZIDs.size();
                         i++)
                    {
                        if (!first)
                        {
                            homoZIDsStringStream << ",";
                        }
                        homoZIDsStringStream
                            << fielddefs[f]->m_homogeneousZIDs[i];
                        first = false;
                    }
                    homoZIDsString = homoZIDsStringStream.str();
                }
                elemTag->SetAttribute("HOMOGENEOUSZIDS", homoZIDsString);
            }

            if (fielddefs[f]->m_homogeneousSIDs.size() > 0)
            {
                std::string homoSIDsString;
                {
                    std::stringstream homoSIDsStringStream;
                    bool first = true;
                    for (int i = 0; i < fielddefs[f]->m_homogeneousSIDs.size();
                         i++)
                    {
                        if (!first)
                        {
                            homoSIDsStringStream << ",";
                        }
                        homoSIDsStringStream
                            << fielddefs[f]->m_homogeneousSIDs[i];
                        first = false;
                    }
                    homoSIDsString = homoSIDsStringStream.str();
                }
                elemTag->SetAttribute("HOMOGENEOUSSIDS", homoSIDsString);
            }
        }

        // Write NUMMODESPERDIR
        std::string numModesString;
        {
            std::stringstream numModesStringStream;

            if (fielddefs[f]->m_uniOrder)
            {
                numModesStringStream << "UNIORDER:";
                // Just dump single definition
                bool first = true;
                for (std::vector<int>::size_type i = 0;
                     i < fielddefs[f]->m_basis.size();
                     i++)
                {
                    if (!first)
                    {
                        numModesStringStream << ",";
                    }
                    numModesStringStream << fielddefs[f]->m_numModes[i];
                    first = false;
                }
            }
            else
            {
                numModesStringStream << "MIXORDER:";
                bool first = true;
                for (std::vector<int>::size_type i = 0;
                     i < fielddefs[f]->m_numModes.size();
                     i++)
                {
                    if (!first)
                    {
                        numModesStringStream << ",";
                    }
                    numModesStringStream << fielddefs[f]->m_numModes[i];
                    first = false;
                }
            }

            numModesString = numModesStringStream.str();
        }
        elemTag->SetAttribute("NUMMODESPERDIR", numModesString);

        // Write ID
        // Should ideally look at ways of compressing this stream
        // if just sequential;
        std::string idString;
        {
            std::stringstream idStringStream;
            idString = ParseUtils::GenerateSeqString(fielddefs[f]->m_elementIDs);
        }
        elemTag->SetAttribute("ID", idString);
        elemTag->SetAttribute("COMPRESSED",
                              LibUtilities::CompressData::GetCompressString());

        // Add this information for future compatibility
        // issues, for exmaple in case we end up using a 128
        // bit machine.
        elemTag->SetAttribute("BITSIZE",
                              LibUtilities::CompressData::GetBitSizeStr());
        std::string base64string;
        ASSERTL0(Z_OK == CompressData::ZlibEncodeToBase64Str(fielddata[f],
                                                             base64string),
                 "Failed to compress field data.");

        elemTag->LinkEndChild(new TiXmlText(base64string));
    }
    doc.SaveFile(filename);

    m_comm->Block();

    // all data has been written
    if (m_comm->TreatAsRankZero())
    {
        tm1 = m_comm->Wtime();
        std::cout << " (" << tm1 - tm0 << "s, XML)" << std::endl;
    }
}

WriteMultiFldFileIDs()

void Nektar::LibUtilities::FieldIOXml::WriteMultiFldFileIDs	(	const std::string &	outFile,
		const std::vector< std::string >	fileNames,
		std::vector< std::vector< unsigned int > > &	elementList,
		const FieldMetaDataMap &	fieldmetadatamap = NullFieldMetaDataMap
	)

Write out a file containing element ID to partition mapping.

This function writes out an XML file (usually called Info.xml) that contains the element IDs that are contained within each partition, as well as the field metadata map.

Parameters

outFile	Output multi-field file name.
fileNames	List of partition filenames.
elementList	Vector of element IDs that lie on each process.
fieldmetadatamap	Field metadata map that is written into outFile.

Definition at line 380 of file FieldIOXml.cpp.

References Nektar::LibUtilities::FieldIO::AddInfoTag(), ASSERTL0, and Nektar::ParseUtils::GenerateSeqString().

Referenced by Nektar::LibUtilities::PtsIO::SetUpFieldMetaData(), and SetUpFieldMetaData().

{
    TiXmlDocument doc;
    TiXmlDeclaration *decl = new TiXmlDeclaration("1.0", "utf-8", "");
    doc.LinkEndChild(decl);

    ASSERTL0(fileNames.size() == elementList.size(),
             "Outfile names and list of elements ids does not match");

    TiXmlElement *root = new TiXmlElement("NEKTAR");
    doc.LinkEndChild(root);

    AddInfoTag(XmlTagWriterSharedPtr(new XmlTagWriter(root)), fieldmetadatamap);

    for (int t = 0; t < fileNames.size(); ++t)
    {
        if (elementList[t].size())
        {
            TiXmlElement *elemIDs = new TiXmlElement("Partition");
            root->LinkEndChild(elemIDs);

            elemIDs->SetAttribute("FileName", fileNames[t]);

            std::string IDstr = ParseUtils::GenerateSeqString(elementList[t]);

            elemIDs->LinkEndChild(new TiXmlText(IDstr));
        }
    }

    doc.SaveFile(outFile);
}

Member Data Documentation

className

std::string Nektar::LibUtilities::FieldIOXml::className

static

Initial value:

= GetFieldIOFactory().RegisterCreatorFunction(
    "Xml", FieldIOXml::create, "XML-based output of field data.")

Name of class.

Definition at line 213 of file FieldIOXml.h.