Nektar++
Public Member Functions | Static Public Member Functions | Protected Member Functions | Protected Attributes | List of all members
Nektar::LibUtilities::FieldIO Class Referenceabstract

Class for operating on Nektar++ input/output files. More...

#include <FieldIO.h>

Inheritance diagram for Nektar::LibUtilities::FieldIO:
[legend]

Public Member Functions

 FieldIO (LibUtilities::CommSharedPtr pComm, bool sharedFilesystem)
 Constructor for FieldIO base class. More...
 
virtual ~FieldIO ()
 
void Write (const std::string &outFile, std::vector< FieldDefinitionsSharedPtr > &fielddefs, std::vector< std::vector< NekDouble > > &fielddata, const FieldMetaDataMap &fieldinfomap=NullFieldMetaDataMap, const bool backup=false)
 Write out the field information to the file outFile. More...
 
void Import (const std::string &infilename, std::vector< FieldDefinitionsSharedPtr > &fielddefs, std::vector< std::vector< NekDouble > > &fielddata=NullVectorNekDoubleVector, FieldMetaDataMap &fieldinfomap=NullFieldMetaDataMap, const Array< OneD, int > &ElementIDs=NullInt1DArray)
 Read field information from the file infilename. More...
 
DataSourceSharedPtr ImportFieldMetaData (const std::string &filename, FieldMetaDataMap &fieldmetadatamap)
 Import the metadata from a field file. More...
 
const std::string & GetClassName () const
 
std::string GetFileEnding () const
 Helper function that determines default file extension. More...
 

Static Public Member Functions

static const std::string GetFileType (const std::string &filename, CommSharedPtr comm)
 Determine file type of given input file. More...
 
static std::shared_ptr< FieldIOCreateDefault (const LibUtilities::SessionReaderSharedPtr session)
 Returns an object for the default FieldIO method. More...
 
static std::shared_ptr< FieldIOCreateForFile (const LibUtilities::SessionReaderSharedPtr session, const std::string &filename)
 Construct a FieldIO object for a given input filename. More...
 
static void AddInfoTag (TagWriterSharedPtr root, const FieldMetaDataMap &fieldmetadatamap)
 Add provenance information to the field metadata map. More...
 

Protected Member Functions

int CheckFieldDefinition (const FieldDefinitionsSharedPtr &fielddefs)
 Check field definitions for correctness and return storage size. More...
 
std::vector< unsigned int > GetNumberOfCoeffsPerElement (const FieldDefinitionsSharedPtr &fielddefs)
 Compute number of data points needed to store expansion inside each element. More...
 
std::string SetUpOutput (const std::string outname, bool perRank, bool backup=false)
 Set up the filesystem ready for output. More...
 
virtual void v_Write (const std::string &outFile, std::vector< FieldDefinitionsSharedPtr > &fielddefs, std::vector< std::vector< NekDouble > > &fielddata, const FieldMetaDataMap &fieldinfomap, const bool backup=false)=0
 Write out the field information to the file outFile. More...
 
virtual void v_Import (const std::string &infilename, std::vector< FieldDefinitionsSharedPtr > &fielddefs, std::vector< std::vector< NekDouble > > &fielddata=NullVectorNekDoubleVector, FieldMetaDataMap &fieldinfomap=NullFieldMetaDataMap, const Array< OneD, int > &ElementIDs=NullInt1DArray)=0
 Read field information from the file infilename. More...
 
virtual DataSourceSharedPtr v_ImportFieldMetaData (const std::string &filename, FieldMetaDataMap &fieldmetadatamap)=0
 Import the metadata from a field file. More...
 
virtual std::string v_GetFileEnding () const
 
virtual const std::string & v_GetClassName () const =0
 

Protected Attributes

LibUtilities::CommSharedPtr m_comm
 Communicator to use when writing parallel format. More...
 
bool m_sharedFilesystem
 Boolean dictating whether we are on a shared filesystem. More...
 

Detailed Description

Class for operating on Nektar++ input/output files.

Nektar++ input/output of field data can be described as follows:

This base class represents the minimum functionality that subclasses need to implement in order to implement the above functionality. Each subclass is free to determine its own file structure and parallel behaviour.

Definition at line 226 of file FieldIO.h.

Constructor & Destructor Documentation

◆ FieldIO()

Nektar::LibUtilities::FieldIO::FieldIO ( LibUtilities::CommSharedPtr  pComm,
bool  sharedFilesystem 
)

Constructor for FieldIO base class.

Definition at line 320 of file FieldIO.cpp.

321 : m_comm(pComm), m_sharedFilesystem(sharedFilesystem)
322{
323}
bool m_sharedFilesystem
Boolean dictating whether we are on a shared filesystem.
Definition: FieldIO.h:282
LibUtilities::CommSharedPtr m_comm
Communicator to use when writing parallel format.
Definition: FieldIO.h:280

◆ ~FieldIO()

virtual Nektar::LibUtilities::FieldIO::~FieldIO ( )
inlinevirtual

Definition at line 232 of file FieldIO.h.

233 {
234 }

Member Function Documentation

◆ AddInfoTag()

void Nektar::LibUtilities::FieldIO::AddInfoTag ( TagWriterSharedPtr  root,
const FieldMetaDataMap fieldmetadatamap 
)
static

Add provenance information to the field metadata map.

This routine adds some basic provenance information to the field metadata to enable better tracking of version information:

  • Nektar++ version
  • Date and time of simulation
  • Hostname of the machine the simulation was performed on
  • git SHA1 and branch name, if Nektar++ was compiled from git and not e.g. a tarball.
Parameters
rootRoot tag, which is encapsulated using the TagWriter structure to enable multi-file format support.
fieldmetadatamapAny existing field metadata.

Definition at line 341 of file FieldIO.cpp.

343{
344 FieldMetaDataMap ProvenanceMap;
345
346 // Nektar++ release version from VERSION file
347 ProvenanceMap["NektarVersion"] = std::string(NEKTAR_VERSION);
348
349 // Date/time stamp
350 auto now = std::chrono::system_clock::now();
351 auto now_t = std::chrono::system_clock::to_time_t(now);
352 auto now_tm = *std::localtime(&now_t);
353 char buffer[128];
354 strftime(buffer, sizeof(buffer), "%d-%b-%Y %H:%M:%S", &now_tm);
355 ProvenanceMap["Timestamp"] = buffer;
356
357 // Hostname
358 boost::system::error_code ec;
359 ProvenanceMap["Hostname"] = ip::host_name(ec);
360
361 // Git information
362 // If built from a distributed package, do not include this
363 if (NekConstants::kGitSha1 != "GITDIR-NOTFOUND")
364 {
365 ProvenanceMap["GitSHA1"] = NekConstants::kGitSha1;
366 ProvenanceMap["GitBranch"] = NekConstants::kGitBranch;
367 }
368
369 TagWriterSharedPtr infoTag = root->AddChild("Metadata");
370
371 TagWriterSharedPtr provTag = infoTag->AddChild("Provenance");
372 for (auto &infoit : ProvenanceMap)
373 {
374 provTag->SetAttr(infoit.first, infoit.second);
375 }
376
377 //---------------------------------------------
378 // write field info section
379 if (fieldmetadatamap != NullFieldMetaDataMap)
380 {
381 for (auto &infoit : fieldmetadatamap)
382 {
383 infoTag->SetAttr(infoit.first, infoit.second);
384 }
385 }
386}
#define NEKTAR_VERSION
Definition: FieldIO.cpp:55
array buffer
Definition: GsLib.hpp:81
std::shared_ptr< TagWriter > TagWriterSharedPtr
Definition: FieldIO.h:78
std::map< std::string, std::string > FieldMetaDataMap
Definition: FieldIO.h:50
static FieldMetaDataMap NullFieldMetaDataMap
Definition: FieldIO.h:51
const std::string kGitBranch
Definition: GitRevision.h:46
const std::string kGitSha1
Definition: GitRevision.h:45

References Nektar::NekConstants::kGitBranch, Nektar::NekConstants::kGitSha1, NEKTAR_VERSION, and Nektar::LibUtilities::NullFieldMetaDataMap.

Referenced by Nektar::LibUtilities::FieldIOHdf5::v_Write(), Nektar::LibUtilities::FieldIOXml::v_Write(), Nektar::SpatialDomains::MeshGraphIOXml::v_WriteGeometry(), and Nektar::LibUtilities::FieldIOXml::WriteMultiFldFileIDs().

◆ CheckFieldDefinition()

int Nektar::LibUtilities::FieldIO::CheckFieldDefinition ( const FieldDefinitionsSharedPtr fielddefs)
protected

Check field definitions for correctness and return storage size.

Parameters
fielddefsField definitions to check.

Definition at line 689 of file FieldIO.cpp.

690{
691 // Return 0 if this is an empty parition
692 if (fielddefs->m_elementIDs.size() == 0)
693 {
694 return 0;
695 }
696
697 unsigned int numbasis = 0;
698
699 // Determine nummodes vector lists are correct length
700 switch (fielddefs->m_shapeType)
701 {
702 case eSegment:
703 numbasis = 1;
704 if (fielddefs->m_numHomogeneousDir)
705 {
706 numbasis += fielddefs->m_numHomogeneousDir;
707 }
708
709 break;
710 case eTriangle:
711 case eQuadrilateral:
712 if (fielddefs->m_numHomogeneousDir)
713 {
714 numbasis = 3;
715 }
716 else
717 {
718 numbasis = 2;
719 }
720 break;
721 case eTetrahedron:
722 case ePyramid:
723 case ePrism:
724 case eHexahedron:
725 numbasis = 3;
726 break;
727 default:
728 NEKERROR(ErrorUtil::efatal, "Unsupported shape type.");
729 break;
730 }
731
732 ASSERTL0(fielddefs->m_basis.size() == numbasis,
733 "Length of basis vector is incorrect");
734
735 if (fielddefs->m_uniOrder == true)
736 {
737 // Counter is updated by "GetNumberOfDataPoints()"
738 unsigned int cnt = 0;
739
740 const int datasize = GetNumberOfDataPoints(fielddefs, cnt);
741
742 return datasize * fielddefs->m_elementIDs.size();
743 }
744 else
745 {
746 // Counter is updated by "GetNumberOfDataPoints()"
747 unsigned int cnt = 0;
748
749 int datasize = 0;
750
751 for (int i = 0; i < fielddefs->m_elementIDs.size(); ++i)
752 {
753 datasize += GetNumberOfDataPoints(fielddefs, cnt);
754 }
755
756 return datasize;
757 }
758}
#define ASSERTL0(condition, msg)
Definition: ErrorUtil.hpp:208
#define NEKERROR(type, msg)
Assert Level 0 – Fundamental assert which is used whether in FULLDEBUG, DEBUG or OPT compilation mode...
Definition: ErrorUtil.hpp:202
int GetNumberOfDataPoints(const FieldDefinitionsSharedPtr &fielddefs, unsigned int &cnt)
Compute the number of values needed to store elemental expansion.
Definition: FieldIO.cpp:586

References ASSERTL0, Nektar::ErrorUtil::efatal, Nektar::LibUtilities::eHexahedron, Nektar::LibUtilities::ePrism, Nektar::LibUtilities::ePyramid, Nektar::LibUtilities::eQuadrilateral, Nektar::LibUtilities::eSegment, Nektar::LibUtilities::eTetrahedron, Nektar::LibUtilities::eTriangle, Nektar::LibUtilities::GetNumberOfDataPoints(), and NEKERROR.

Referenced by Nektar::LibUtilities::FieldIOXml::ImportFieldData(), Nektar::LibUtilities::FieldIOHdf5::v_Import(), Nektar::LibUtilities::FieldIOHdf5::v_Write(), and Nektar::LibUtilities::FieldIOXml::v_Write().

◆ CreateDefault()

FieldIOSharedPtr Nektar::LibUtilities::FieldIO::CreateDefault ( const LibUtilities::SessionReaderSharedPtr  session)
static

Returns an object for the default FieldIO method.

This function returns a FieldIO class as determined by the hard-coded default (XML), which can be overridden by changing the session reader SOLVERINFO variable FieldIOFormat.

Parameters
sessionSession reader
Returns
FieldIO object

Definition at line 194 of file FieldIO.cpp.

196{
197 std::string iofmt("Xml");
198 if (session->DefinesSolverInfo("IOFormat"))
199 {
200 iofmt = session->GetSolverInfo("IOFormat");
201 }
202
203 if (session->DefinesCmdLineArgument("io-format"))
204 {
205 iofmt = session->GetCmdLineArgument<std::string>("io-format");
206 }
207
208 return GetFieldIOFactory().CreateInstance(iofmt, session->GetComm(),
209 session->GetSharedFilesystem());
210}
tBaseSharedPtr CreateInstance(tKey idKey, tParam... args)
Create an instance of the class referred to by idKey.
FieldIOFactory & GetFieldIOFactory()
Returns the FieldIO factory.
Definition: FieldIO.cpp:69

References Nektar::LibUtilities::NekFactory< tKey, tBase, tParam >::CreateInstance(), and Nektar::LibUtilities::GetFieldIOFactory().

Referenced by Diffusion::Diffusion(), Nektar::VortexWaveInteraction::FileRelaxation(), Nektar::FilterBenchmark::FilterBenchmark(), Nektar::SolverUtils::FilterCheckpoint::FilterCheckpoint(), Nektar::FilterCheckpointCellModel::FilterCheckpointCellModel(), Nektar::FilterHilbertFFTPhase::FilterHilbertFFTPhase(), Nektar::SolverUtils::FilterModalEnergy::FilterModalEnergy(), Nektar::FilterOffsetPhase::FilterOffsetPhase(), Nektar::SolverUtils::FilterThresholdMax::FilterThresholdMax(), Nektar::SolverUtils::FilterThresholdMin::FilterThresholdMin(), Nektar::GlobalMapping::Mapping::Mapping(), and Nektar::SolverUtils::EquationSystem::v_InitObject().

◆ CreateForFile()

FieldIOSharedPtr Nektar::LibUtilities::FieldIO::CreateForFile ( const LibUtilities::SessionReaderSharedPtr  session,
const std::string &  filename 
)
static

Construct a FieldIO object for a given input filename.

This is a convenience function that takes an input filename and constructs the appropriate FieldIO subclass, using FieldIO::GetFileType.

Parameters
sessionSession reader
filenameInput filename
Returns
FieldIO class reader for filename.

Definition at line 223 of file FieldIO.cpp.

226{
227 const std::string iofmt =
228 FieldIO::GetFileType(filename, session->GetComm());
229 return GetFieldIOFactory().CreateInstance(iofmt, session->GetComm(),
230 session->GetSharedFilesystem());
231}
static const std::string GetFileType(const std::string &filename, CommSharedPtr comm)
Determine file type of given input file.
Definition: FieldIO.cpp:94

References Nektar::LibUtilities::NekFactory< tKey, tBase, tParam >::CreateInstance(), Nektar::LibUtilities::GetFieldIOFactory(), and GetFileType().

Referenced by Nektar::SolverUtils::UnsteadySystem::CheckForRestartTime(), Nektar::SolverUtils::SessionFunction::EvaluateFld(), Nektar::GlobalMapping::Mapping::EvaluateFunction(), Nektar::SolverUtils::EquationSystem::ImportFld(), Nektar::LinearisedAdvection::ImportFldBase(), Nektar::SolverUtils::FileFieldInterpolator::ImportFldBase(), Nektar::CellModel::LoadCellModel(), Nektar::SpatialDomains::MeshGraph::ReadExpansionInfo(), Nektar::SmoothedProfileMethod::ReadPhi(), Nektar::SolverUtils::ForcingMovingReferenceFrame::SetInitialConditions(), Nektar::SolverUtils::FilterBodyFittedVelocity::v_Initialise(), and Nektar::SolverUtils::FilterFieldConvert::v_Initialise().

◆ GetClassName()

const std::string & Nektar::LibUtilities::FieldIO::GetClassName ( ) const
inline

Definition at line 257 of file FieldIO.h.

258 {
259 return v_GetClassName();
260 }
virtual const std::string & v_GetClassName() const =0

References v_GetClassName().

◆ GetFileEnding()

std::string Nektar::LibUtilities::FieldIO::GetFileEnding ( ) const
inline

Helper function that determines default file extension.

Definition at line 273 of file FieldIO.h.

274 {
275 return v_GetFileEnding();
276 };
virtual std::string v_GetFileEnding() const
Definition: FieldIO.h:314

References v_GetFileEnding().

Referenced by Nektar::LibUtilities::PtsIO::Import(), Nektar::LibUtilities::FieldIOXml::SetUpFieldMetaData(), Nektar::LibUtilities::PtsIO::SetUpFieldMetaData(), and SetUpOutput().

◆ GetFileType()

const std::string Nektar::LibUtilities::FieldIO::GetFileType ( const std::string &  filename,
CommSharedPtr  comm 
)
static

Determine file type of given input file.

This method attempts to identify the file type of a given input file filename. It returns a string corresponding to GetFieldIOFactory() or throws an assertion if it cannot be identified.

Parameters
filenameInput filename
commCommunicator for parallel runs
Returns
FieldIO format of filename.

Definition at line 94 of file FieldIO.cpp.

96{
97 FieldIOType ioType = eXML;
98 int size = comm->GetSpaceComm()->GetSize();
99 bool root = comm->GetSpaceComm()->TreatAsRankZero();
100
101 if (size == 1 || root)
102 {
103 std::string datafilename;
104
105 // If input is a directory, check for root processor file.
106 if (fs::is_directory(filename))
107 {
108 fs::path fullpath = filename;
109 fs::path d = fullpath;
110 std::regex expr("P\\d{7}.fld");
111 std::smatch what;
112
113 bool found = false;
114 for (auto &f : fs::directory_iterator(d))
115 {
116 std::string filename = f.path().filename().string();
117 if (std::regex_match(filename, what, expr))
118 {
119 found = true;
120 fullpath = f.path();
121 break;
122 }
123 }
124
125 ASSERTL0(found, std::string("Failed to open a PXXXXXXX.fld file "
126 "in directory: " +
127 filename)
128 .c_str());
129
130 datafilename = PortablePath(fullpath);
131 }
132 else
133 {
134 datafilename = filename;
135 }
136
137 // Read first 8 bytes. If they correspond with magic bytes below it's an
138 // HDF5 file. XML is potentially a nightmare with all the different
139 // encodings so we'll just assume it's OK if it's not HDF.
140 const unsigned char magic[8] = {0x89, 0x48, 0x44, 0x46,
141 0x0d, 0x0a, 0x1a, 0x0a};
142
143 std::ifstream datafile(datafilename.c_str(), std::ios_base::binary);
144 ASSERTL0(datafile.good(), "Unable to open file: " + filename);
145
146 ioType = eHDF5;
147 for (unsigned i = 0; i < 8 && datafile.good(); ++i)
148 {
149 int byte = datafile.get();
150 if (byte != magic[i])
151 {
152 ioType = eXML;
153 break;
154 }
155 }
156 }
157
158 if (size > 1)
159 {
160 int code = (int)ioType;
161 comm->GetSpaceComm()->Bcast(code, 0);
162 ioType = (FieldIOType)code;
163 }
164
165 std::string iofmt;
166 if (ioType == eXML)
167 {
168 iofmt = "Xml";
169 }
170 else if (ioType == eHDF5)
171 {
172 iofmt = "Hdf5";
173 }
174 else
175 {
176 // Error
177 NEKERROR(ErrorUtil::efatal, "Unknown file format");
178 }
179
180 return iofmt;
181}
FieldIOType
Enumerator for auto-detection of FieldIO types.
Definition: FieldIO.cpp:77
static std::string PortablePath(const fs::path &path)
create portable path on different platforms for std::filesystem path.
Definition: Filesystem.hpp:66
std::vector< double > d(NPUPPER *NPUPPER)

References ASSERTL0, Nektar::UnitTests::d(), Nektar::ErrorUtil::efatal, Nektar::LibUtilities::eHDF5, Nektar::LibUtilities::eXML, NEKERROR, and Nektar::LibUtilities::PortablePath().

Referenced by CreateForFile(), Nektar::MultiRegions::ExpList::ExtractCoeffsFromFile(), Nektar::FieldUtils::Field::FieldIOForFile(), Nektar::LibUtilities::Import(), and Nektar::SpatialDomains::MeshPartition::ReadExpansions().

◆ GetNumberOfCoeffsPerElement()

std::vector< unsigned int > Nektar::LibUtilities::FieldIO::GetNumberOfCoeffsPerElement ( const FieldDefinitionsSharedPtr fielddefs)
protected

Compute number of data points needed to store expansion inside each element.

Parameters
fielddefsField definitions.

Definition at line 766 of file FieldIO.cpp.

768{
769 // Allocate vector with results
770 std::vector<unsigned int> coeffsPerElmt;
771
772 // Return empty vector if partition is empty
773 if (fielddefs->m_elementIDs.size() == 0)
774 {
775 return coeffsPerElmt;
776 }
777
778 // Add elements to vector
779 coeffsPerElmt.resize(fielddefs->m_elementIDs.size());
780
781 if (fielddefs->m_uniOrder == true)
782 {
783 // Counter is updated by "GetNumberOfDataPoints"
784 unsigned int cnt = 0;
785
786 const int datasize = GetNumberOfDataPoints(fielddefs, cnt);
787
788 for (int i = 0; i < fielddefs->m_elementIDs.size(); ++i)
789 {
790 coeffsPerElmt[i] = datasize;
791 }
792 }
793 else
794 {
795 // Counter is updated by "GetNumberOfDataPoints"
796 unsigned int cnt = 0;
797
798 for (int i = 0; i < fielddefs->m_elementIDs.size(); ++i)
799 {
800 coeffsPerElmt[i] = GetNumberOfDataPoints(fielddefs, cnt);
801 }
802 }
803
804 return coeffsPerElmt;
805}

References Nektar::LibUtilities::GetNumberOfDataPoints().

Referenced by Nektar::LibUtilities::FieldIOHdf5::v_Import().

◆ Import()

void Nektar::LibUtilities::FieldIO::Import ( const std::string &  infilename,
std::vector< FieldDefinitionsSharedPtr > &  fielddefs,
std::vector< std::vector< NekDouble > > &  fielddata = NullVectorNekDoubleVector,
FieldMetaDataMap fieldinfo = NullFieldMetaDataMap,
const Array< OneD, int > &  ElementIDs = NullInt1DArray 
)
inline

Read field information from the file infilename.

Parameters
infilenameInput filename (or directory if parallel format)
fielddefsOn return contains field definitions as read from the input.
fielddataOn return, contains binary field data that stores the input corresponding to fielddefs.
fieldinfoOn returnm, contains the associated field metadata map.
ElementIDsElement IDs that lie on this processor, which can be optionally supplied to avoid reading the entire file on each processor.

Definition at line 355 of file FieldIO.h.

360{
361 v_Import(infilename, fielddefs, fielddata, fieldinfo, ElementIDs);
362}
virtual void v_Import(const std::string &infilename, std::vector< FieldDefinitionsSharedPtr > &fielddefs, std::vector< std::vector< NekDouble > > &fielddata=NullVectorNekDoubleVector, FieldMetaDataMap &fieldinfomap=NullFieldMetaDataMap, const Array< OneD, int > &ElementIDs=NullInt1DArray)=0
Read field information from the file infilename.

References v_Import().

◆ ImportFieldMetaData()

DataSourceSharedPtr Nektar::LibUtilities::FieldIO::ImportFieldMetaData ( const std::string &  filename,
FieldMetaDataMap fieldmetadatamap 
)
inline

Import the metadata from a field file.

Parameters
filenameInput filename.
fieldmetadatamapOn return contains the field metadata map from filename.

Definition at line 371 of file FieldIO.h.

373{
374 return v_ImportFieldMetaData(filename, fieldmetadatamap);
375}
virtual DataSourceSharedPtr v_ImportFieldMetaData(const std::string &filename, FieldMetaDataMap &fieldmetadatamap)=0
Import the metadata from a field file.

References v_ImportFieldMetaData().

Referenced by Nektar::LibUtilities::PtsIO::Import(), and Nektar::LibUtilities::FieldIOXml::v_Import().

◆ SetUpOutput()

std::string Nektar::LibUtilities::FieldIO::SetUpOutput ( const std::string  outname,
bool  perRank,
bool  backup = false 
)
protected

Set up the filesystem ready for output.

This function sets up the output given an output filename outname. This will therefore remove any file or directory with the desired output filename and return the absolute path to the output.

If perRank is set, a new directory will be created to contain one file per process rank.

Parameters
outnameDesired output filename.
perRankTrue if one file-per-rank output is required.
Returns
Absolute path to resulting file.

Definition at line 403 of file FieldIO.cpp.

405{
406 ASSERTL0(!outname.empty(), "Empty path given to SetUpOutput()");
407
408 // Create a hash from the filename
409 std::size_t file_id = std::hash<std::string>{}(outname);
410
411 // Find the minimum and maximum hash for each process
412 std::size_t file_id_max{file_id};
413 std::size_t file_id_min{file_id};
414 m_comm->GetSpaceComm()->AllReduce(file_id_max, ReduceMax);
415 m_comm->GetSpaceComm()->AllReduce(file_id_min, ReduceMin);
416
417 // Check that each process has the same filename (hash)
418 ASSERTL0(file_id_min == file_id_max,
419 "All processes do not have the same filename.");
420
421 int nprocs = m_comm->GetSpaceComm()->GetSize();
422 bool root = m_comm->GetSpaceComm()->TreatAsRankZero();
423
424 // Path to output: will be directory if parallel, normal file if
425 // serial.
426 fs::path specPath(outname), fulloutname;
427
428 // in case we are rank 0 or not on a shared filesystem, check if the
429 // specPath already exists
430 if (backup && (root || !m_sharedFilesystem) && fs::exists(specPath))
431 {
432 // rename. foo/bar_123.chk -> foo/bar_123.bak0.chk and in case
433 // foo/bar_123.bak0.chk already exists, foo/bar_123.chk ->
434 // foo/bar_123.bak1.chk
435 fs::path bakPath = specPath;
436 int cnt = 0;
437 while (fs::exists(bakPath))
438 {
439 bakPath = specPath.parent_path();
440 bakPath += fs::path(bakPath.extension() == ".pit" ? "/" : "");
441 bakPath += specPath.stem();
442 bakPath += fs::path(".bak" + std::to_string(cnt++));
443 bakPath += specPath.extension();
444 }
445 std::cout << "renaming " << specPath << " -> " << bakPath << std::endl;
446 try
447 {
448 fs::rename(specPath, bakPath);
449 }
450 catch (fs::filesystem_error &e)
451 {
452 ASSERTL0(e.code() == fserrc::no_such_file_or_directory,
453 "Filesystem error: " + std::string(e.what()));
454 }
455 }
456
457 // wait until rank 0 has moved the old specPath and the changes
458 // have propagated through the filesystem
459 if (backup)
460 {
461 m_comm->GetSpaceComm()->Block();
462 int exists = 1;
463 while (exists && perRank)
464 {
465 exists = fs::exists(specPath);
466 m_comm->GetSpaceComm()->AllReduce(exists, ReduceMax);
467 }
468 }
469
470 if (nprocs == 1)
471 {
472 fulloutname = specPath;
473 }
474 else
475 {
476 // Guess the filename that might belong to this process.
477 boost::format pad("P%1$07d.%2$s");
478 pad % m_comm->GetSpaceComm()->GetRank() % GetFileEnding();
479
480 // Generate full path name
481 fs::path poutfile(pad.str());
482 fulloutname = specPath / poutfile;
483 }
484
485 // Remove any existing file which is in the way
486 if (m_comm->GetSpaceComm()->RemoveExistingFiles() && !backup)
487 {
489 {
490 // First, each process clears up its .fld file. This might or might
491 // not be there (we might have changed numbers of processors between
492 // runs, for example), but we can try anyway.
493 try
494 {
495 fs::remove_all(fulloutname);
496 }
497 catch (fs::filesystem_error &e)
498 {
499 ASSERTL0(e.code() == fserrc::no_such_file_or_directory,
500 "Filesystem error: " + std::string(e.what()));
501 }
502 }
503
504 m_comm->GetSpaceComm()->Block();
505
506 // Now get rank 0 processor to tidy everything else up.
507 if (root || !m_sharedFilesystem)
508 {
509 try
510 {
511 fs::remove_all(specPath);
512 }
513 catch (fs::filesystem_error &e)
514 {
515 ASSERTL0(e.code() == fserrc::no_such_file_or_directory,
516 "Filesystem error: " + std::string(e.what()));
517 }
518 }
519
520 // wait until rank 0 has removed specPath and the changes
521 // have propagated through the filesystem
522 m_comm->GetSpaceComm()->Block();
523 int exists = 1;
524 while (exists && perRank)
525 {
526 exists = fs::exists(specPath);
527 m_comm->GetSpaceComm()->AllReduce(exists, ReduceMax);
528 }
529 }
530
531 if (root)
532 {
533 std::cout << "Writing: \"" + specPath.string() +
534 (m_comm->IsParallelInTime() ? "\"\n" : "\"");
535 }
536
537 // serial processing just add ending.
538 if (nprocs == 1)
539 {
540 return LibUtilities::PortablePath(specPath);
541 }
542
543 // Create the destination directory
544 if (perRank)
545 {
546 try
547 {
548 if (root || !m_sharedFilesystem)
549 {
550 fs::create_directory(specPath);
551 }
552 }
553 catch (fs::filesystem_error &e)
554 {
556 "Filesystem error: " + std::string(e.what()));
557 }
558
559 m_comm->GetSpaceComm()->Block();
560
561 // Sit in a loop and make sure target directory has been created
562 int created = 0;
563 while (!created)
564 {
565 created = fs::is_directory(specPath);
566 m_comm->GetSpaceComm()->AllReduce(created, ReduceMin);
567 }
568 }
569 else
570 {
571 fulloutname = specPath;
572 }
573
574 // Return the full path to the partition for this process
575 return LibUtilities::PortablePath(fulloutname);
576}
std::string GetFileEnding() const
Helper function that determines default file extension.
Definition: FieldIO.h:273

References ASSERTL0, Nektar::ErrorUtil::efatal, CellMLToNektar.pycml::format, GetFileEnding(), m_comm, m_sharedFilesystem, NEKERROR, Nektar::LibUtilities::PortablePath(), Nektar::LibUtilities::ReduceMax, and Nektar::LibUtilities::ReduceMin.

Referenced by Nektar::LibUtilities::FieldIOHdf5::v_Write(), Nektar::LibUtilities::FieldIOXml::v_Write(), Nektar::LibUtilities::CsvIO::Write(), and Nektar::LibUtilities::PtsIO::Write().

◆ v_GetClassName()

virtual const std::string & Nektar::LibUtilities::FieldIO::v_GetClassName ( ) const
protectedpure virtual

◆ v_GetFileEnding()

virtual std::string Nektar::LibUtilities::FieldIO::v_GetFileEnding ( ) const
inlineprotectedvirtual

Reimplemented in Nektar::LibUtilities::CsvIO, and Nektar::LibUtilities::PtsIO.

Definition at line 314 of file FieldIO.h.

315 {
316 return "fld";
317 }

Referenced by GetFileEnding().

◆ v_Import()

virtual void Nektar::LibUtilities::FieldIO::v_Import ( const std::string &  infilename,
std::vector< FieldDefinitionsSharedPtr > &  fielddefs,
std::vector< std::vector< NekDouble > > &  fielddata = NullVectorNekDoubleVector,
FieldMetaDataMap fieldinfomap = NullFieldMetaDataMap,
const Array< OneD, int > &  ElementIDs = NullInt1DArray 
)
protectedpure virtual

Read field information from the file infilename.

Parameters
infilenameInput filename (or directory if parallel format)
fielddefsOn return contains field definitions as read from the input.
fielddataOn return, contains binary field data that stores the input corresponding to fielddefs.
fieldinfoOn returnm, contains the associated field metadata map.
ElementIDsElement IDs that lie on this processor, which can be optionally supplied to avoid reading the entire file on each processor.

Implemented in Nektar::LibUtilities::FieldIOHdf5, and Nektar::LibUtilities::FieldIOXml.

Referenced by Import().

◆ v_ImportFieldMetaData()

virtual DataSourceSharedPtr Nektar::LibUtilities::FieldIO::v_ImportFieldMetaData ( const std::string &  filename,
FieldMetaDataMap fieldmetadatamap 
)
protectedpure virtual

Import the metadata from a field file.

Parameters
filenameInput filename.
fieldmetadatamapOn return contains the field metadata map from filename.

Implemented in Nektar::LibUtilities::FieldIOHdf5, and Nektar::LibUtilities::FieldIOXml.

Referenced by ImportFieldMetaData().

◆ v_Write()

virtual void Nektar::LibUtilities::FieldIO::v_Write ( const std::string &  outFile,
std::vector< FieldDefinitionsSharedPtr > &  fielddefs,
std::vector< std::vector< NekDouble > > &  fielddata,
const FieldMetaDataMap fieldinfomap,
const bool  backup = false 
)
protectedpure virtual

Write out the field information to the file outFile.

Parameters
outFileOutput filename
fielddefsField definitions that define the output
fielddataBinary field data that stores the output corresponding to fielddefs.
fieldinfomapAssociated field metadata map.

Implemented in Nektar::LibUtilities::FieldIOHdf5, and Nektar::LibUtilities::FieldIOXml.

Referenced by Write().

◆ Write()

void Nektar::LibUtilities::FieldIO::Write ( const std::string &  outFile,
std::vector< FieldDefinitionsSharedPtr > &  fielddefs,
std::vector< std::vector< NekDouble > > &  fielddata,
const FieldMetaDataMap fieldinfomap = NullFieldMetaDataMap,
const bool  backup = false 
)
inline

Write out the field information to the file outFile.

Parameters
outFileOutput filename
fielddefsField definitions that define the output
fielddataBinary field data that stores the output corresponding to fielddefs.
fieldinfomapAssociated field metadata map.

Definition at line 333 of file FieldIO.h.

338{
339 v_Write(outFile, fielddefs, fielddata, fieldinfomap, backup);
340}
virtual void v_Write(const std::string &outFile, std::vector< FieldDefinitionsSharedPtr > &fielddefs, std::vector< std::vector< NekDouble > > &fielddata, const FieldMetaDataMap &fieldinfomap, const bool backup=false)=0
Write out the field information to the file outFile.

References v_Write().

Member Data Documentation

◆ m_comm

LibUtilities::CommSharedPtr Nektar::LibUtilities::FieldIO::m_comm
protected

◆ m_sharedFilesystem

bool Nektar::LibUtilities::FieldIO::m_sharedFilesystem
protected

Boolean dictating whether we are on a shared filesystem.

Definition at line 282 of file FieldIO.h.

Referenced by SetUpOutput().