Nektar++
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Pages
FieldIOHdf5.h
Go to the documentation of this file.
1 ///////////////////////////////////////////////////////////////////////////////
2 //
3 // File FieldIOHdf5.h
4 //
5 // For more information, please see: http://www.nektar.info
6 //
7 // The MIT License
8 //
9 // Copyright (c) 2006 Division of Applied Mathematics, Brown University (USA),
10 // Department of Aeronautics, Imperial College London (UK), and Scientific
11 // Computing and Imaging Institute, University of Utah (USA).
12 //
13 // License for the specific language governing rights and limitations under
14 // Permission is hereby granted, free of charge, to any person obtaining a
15 // copy of this software and associated documentation files (the "Software"),
16 // to deal in the Software without restriction, including without limitation
17 // the rights to use, copy, modify, merge, publish, distribute, sublicense,
18 // and/or sell copies of the Software, and to permit persons to whom the
19 // Software is furnished to do so, subject to the following conditions:
20 //
21 // The above copyright notice and this permission notice shall be included
22 // in all copies or substantial portions of the Software.
23 //
24 // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
25 // OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
26 // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
27 // THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
28 // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
29 // FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
30 // DEALINGS IN THE SOFTWARE.
31 //
32 // Description: Field IO to/from HDF5
33 //
34 ///////////////////////////////////////////////////////////////////////////////
35 
36 #ifndef NEKTAR_LIB_UTILITIES_BASIC_UTILS_FIELDIOHDF5_H
37 #define NEKTAR_LIB_UTILITIES_BASIC_UTILS_FIELDIOHDF5_H
38 
42 
43 namespace Nektar
44 {
45 namespace LibUtilities
46 {
47 
48 namespace H5
49 {
50 class Group;
51 typedef boost::shared_ptr<Group> GroupSharedPtr;
52 }
53 
54 /**
55  * @class Class encapsulating simple HDF5 data source using H5 reader utilities.
56  */
57 class H5DataSource : public DataSource
58 {
59 public:
60  /// Constructor based on filename.
61  H5DataSource(const std::string &fn, H5::PListSharedPtr parallelProps)
62  : doc(H5::File::Open(fn, H5F_ACC_RDONLY, parallelProps))
63  {
64  }
65 
66  /// Get H5::FileSharedPtr reference to file.
68  {
69  return doc;
70  }
71 
72  /// Get H5::FileSharedPtr reference to file.
73  const H5::FileSharedPtr Get() const
74  {
75  return doc;
76  }
77 
78  /// Static constructor for this data source.
80  const std::string &fn, H5::PListSharedPtr parallelProps)
81  {
82  return DataSourceSharedPtr(new H5DataSource(fn, parallelProps));
83  }
84 
85 private:
86  /// HDF5 document.
88 };
89 typedef boost::shared_ptr<H5DataSource> H5DataSourceSharedPtr;
90 
91 /**
92  * @class Simple class for writing hierarchical data using HDF5.
93  */
94 class H5TagWriter : public TagWriter
95 {
96 public:
97  /// Default constructor.
99 
100  /// Add a child node.
101  TagWriterSharedPtr AddChild(const std::string &name)
102  {
103  H5::GroupSharedPtr child = m_Group->CreateGroup(name);
104  return TagWriterSharedPtr(new H5TagWriter(child));
105  }
106 
107  /// Set an attribute key/value pair on this tag.
108  void SetAttr(const std::string &key, const std::string &val)
109  {
110  m_Group->SetAttribute(key, val);
111  }
112 
113 private:
114  /// HDF5 group for this tag.
116 };
117 typedef boost::shared_ptr<H5TagWriter> H5TagWriterSharedPtr;
118 
119 /**
120  * @class Class for operating on HDF5-based FLD files.
121  *
122  * This class implements a HDF5 reader/writer based on MPI/O that is designed to
123  * operate on a single file across all processors of a simulation. The
124  * definition follows vaguely similar lines to XML output but is stored somewhat
125  * differently to accommodate parallel reading and writing. At a basic level
126  * metadata is organised as follows:
127  *
128  * - Nektar++ data lies in the root `/NEKTAR` group.
129  * - The contents of a FieldDefinitions object is hashed to construct a unique
130  * identifier for each object, which becomes the name of a group within the
131  * root group. We then use the H5TagWriter to assign the field definitions
132  * to each group.
133  * - In a similar fashion, we create a `Metadata` group to contain field
134  * metadata that is written.
135  *
136  * We then define five data sets to contain field data:
137  *
138  * - The `DATA` dataset contains the double-precision modal coefficient data.
139  * - The `IDS` dataset contains the element IDs of the elements that are
140  * written out.
141  * - The `POLYORDERS` dataset is written if the field data contains variable
142  * polynomial order, and contains the (possibly hetergeneous) mode orders in
143  * each direction for each of the elements.
144  * - The `HOMOGENEOUSZIDS` dataset contains the IDs of z-planes for
145  * homogeneous simulations, if the data are homogeneous.
146  * - The `HOMOGENEOUSYIDS` dataset contains the IDs of y-planes for
147  * homogeneous simulations, if the data are homogeneous.
148  * - The `HOMOGENEOUSSIDS` dataset contains the strip IDs for
149  * homogeneous simulations, if the data are homogeneous and use strips.
150  *
151  * The ordering is defined according to the `DECOMPOSITION` dataset. A
152  * `decomposition' in this class is essentially a single field definition with
153  * its accompanying data. Data are written into each dataset by the order of
154  * each decomposition. Each decomposition contains the following seven integers
155  * that define it per field definition per processor:
156  *
157  * - Number of elements in this field definition (index #ELEM_DCMP_IDX).
158  * - Number of entries in the `DATA` array for this field definition
159  * (index #VAL_DCMP_IDX)
160  * - Number of entries in the `POLYORDERS` array for this field definition
161  * (index #ORDER_DCMP_IDX)
162  * - Number of entries in the `HOMOGENEOUSZIDS` array (index #HOMZ_DCMP_IDX).
163  * - Number of entries in the `HOMOGENEOUSYIDS` array (index #HOMY_DCMP_IDX).
164  * - Number of entries in the `HOMOGENEOUSSIDS` array (index #HOMS_DCMP_IDX).
165  * - Hash of the field definition, represented as a 32-bit integer, which
166  * describes the name of the attribute that contains the rest of the field
167  * definition information (e.g. field names, basis type, etc).
168  *
169  * The number of decompositions is therefore calculated as the field size
170  * divided by #MAX_DCMPS which allows us to calculate the offsets of the data
171  * for each field definition within the arrays.
172  */
173 class FieldIOHdf5 : public FieldIO
174 {
175 public:
176  static const unsigned int FORMAT_VERSION;
177 
178  static const unsigned int ELEM_DCMP_IDX;
179  static const unsigned int VAL_DCMP_IDX;
180  static const unsigned int ORDER_DCMP_IDX;
181  static const unsigned int HOMY_DCMP_IDX;
182  static const unsigned int HOMZ_DCMP_IDX;
183  static const unsigned int HOMS_DCMP_IDX;
184  static const unsigned int HASH_DCMP_IDX;
185  static const unsigned int MAX_DCMPS;
186 
187  static const unsigned int ELEM_CNT_IDX;
188  static const unsigned int VAL_CNT_IDX;
189  static const unsigned int ORDER_CNT_IDX;
190  static const unsigned int HOMY_CNT_IDX;
191  static const unsigned int HOMZ_CNT_IDX;
192  static const unsigned int HOMS_CNT_IDX;
193  static const unsigned int MAX_CNTS;
194 
195  static const unsigned int IDS_IDX_IDX;
196  static const unsigned int DATA_IDX_IDX;
197  static const unsigned int ORDER_IDX_IDX;
198  static const unsigned int HOMY_IDX_IDX;
199  static const unsigned int HOMZ_IDX_IDX;
200  static const unsigned int HOMS_IDX_IDX;
201  static const unsigned int MAX_IDXS;
202 
203  /// Creates an instance of this class
205  LibUtilities::CommSharedPtr pComm, bool sharedFilesystem)
206  {
208  sharedFilesystem);
209  }
210 
211  /// Name of class
212  LIB_UTILITIES_EXPORT static std::string className;
213 
216  bool sharedFilesystem);
217 
219  {
220  }
221 
222  /// Get class name
223  inline virtual const std::string &GetClassName() const
224  {
225  return className;
226  }
227 
228 private:
229  struct OffsetHelper {
230  OffsetHelper() : data(0), order(0), homy(0), homz(0), homs(0) {}
232  data(in.data), order(in.order), homy(in.homy), homz(in.homz),
233  homs(in.homs)
234  {
235  }
236 
237  uint64_t data, order, homy, homz, homs;
238  };
239 
240  LIB_UTILITIES_EXPORT virtual void v_Write(
241  const std::string &outFile,
242  std::vector<FieldDefinitionsSharedPtr> &fielddefs,
243  std::vector<std::vector<NekDouble> > &fielddata,
244  const FieldMetaDataMap &fieldinfomap = NullFieldMetaDataMap,
245  const bool backup = false);
246 
247  LIB_UTILITIES_EXPORT virtual void v_Import(
248  const std::string &infilename,
249  std::vector<FieldDefinitionsSharedPtr> &fielddefs,
250  std::vector<std::vector<NekDouble> > &fielddata =
252  FieldMetaDataMap &fieldinfomap = NullFieldMetaDataMap,
253  const Array<OneD, int> &ElementIDs = NullInt1DArray);
254 
256  const std::string &filename, FieldMetaDataMap &fieldmetadatamap);
257 
259  DataSourceSharedPtr dataSource, FieldMetaDataMap &fieldmetadatamap);
260 
262  H5::PListSharedPtr readPL,
263  H5::GroupSharedPtr root,
264  std::vector<uint64_t> &decomps,
265  uint64_t decomp,
266  OffsetHelper offset,
267  std::string group,
269 
271  H5::PListSharedPtr readPL,
272  H5::DataSetSharedPtr data_dset,
273  H5::DataSpaceSharedPtr data_fspace,
274  uint64_t data_i,
275  std::vector<uint64_t> &decomps,
276  uint64_t decomp,
277  const FieldDefinitionsSharedPtr fielddef,
278  std::vector<NekDouble> &fielddata);
279 };
280 }
281 }
282 #endif
boost::shared_ptr< H5DataSource > H5DataSourceSharedPtr
Definition: FieldIOHdf5.h:89
H5::GroupSharedPtr m_Group
HDF5 group for this tag.
Definition: FieldIOHdf5.h:115
const H5::FileSharedPtr Get() const
Get H5::FileSharedPtr reference to file.
Definition: FieldIOHdf5.h:73
boost::shared_ptr< Group > GroupSharedPtr
Definition: FieldIOHdf5.h:50
boost::shared_ptr< H5TagWriter > H5TagWriterSharedPtr
Definition: FieldIOHdf5.h:117
FieldIOHdf5(LibUtilities::CommSharedPtr pComm, bool sharedFilesystem)
Construct the FieldIO object for HDF5 output.
static const unsigned int HOMY_CNT_IDX
A helper for FieldIOHdf5::v_Write. Describes the position of the number of homogeneous y-planes in th...
Definition: FieldIOHdf5.h:190
static boost::shared_ptr< DataType > AllocateSharedPtr()
Allocate a shared pointer from the memory pool.
H5::FileSharedPtr doc
HDF5 document.
Definition: FieldIOHdf5.h:87
virtual void v_Write(const std::string &outFile, std::vector< FieldDefinitionsSharedPtr > &fielddefs, std::vector< std::vector< NekDouble > > &fielddata, const FieldMetaDataMap &fieldinfomap=NullFieldMetaDataMap, const bool backup=false)
Write a HDF5 file to outFile given the field definitions fielddefs, field data fielddata and metadata...
H5TagWriter(H5::GroupSharedPtr grp)
Default constructor.
Definition: FieldIOHdf5.h:98
static const unsigned int MAX_DCMPS
A helper for FieldIOHdf5::v_Write. Describes the maximum number of items in the decomposition per fie...
Definition: FieldIOHdf5.h:185
static const unsigned int HOMS_IDX_IDX
A helper for FieldIOHdf5::v_Write. Describes the position of the number of homogeneous strips within ...
Definition: FieldIOHdf5.h:200
boost::shared_ptr< FieldDefinitions > FieldDefinitionsSharedPtr
Definition: FieldIO.h:181
static const unsigned int ORDER_IDX_IDX
A helper for FieldIOHdf5::v_Write. Describes the position of the element order within the indexing se...
Definition: FieldIOHdf5.h:197
boost::shared_ptr< DataSet > DataSetSharedPtr
Definition: H5.h:100
void ImportHDF5FieldMetaData(DataSourceSharedPtr dataSource, FieldMetaDataMap &fieldmetadatamap)
Import field metadata from dataSource.
boost::shared_ptr< File > FileSharedPtr
Definition: H5.h:98
boost::shared_ptr< PList > PListSharedPtr
Definition: H5.h:102
std::map< std::string, std::string > FieldMetaDataMap
Definition: FieldIO.h:54
boost::shared_ptr< DataSource > DataSourceSharedPtr
Definition: FieldIO.h:81
static const unsigned int HASH_DCMP_IDX
The hash of the field definition information, which defines the name of the attribute containing the ...
Definition: FieldIOHdf5.h:184
Base class for writing hierarchical data (XML or HDF5).
Definition: FieldIO.h:60
static DataSourceSharedPtr create(const std::string &fn, H5::PListSharedPtr parallelProps)
Static constructor for this data source.
Definition: FieldIOHdf5.h:79
boost::shared_ptr< Comm > CommSharedPtr
Pointer to a Communicator object.
Definition: Comm.h:55
void ImportFieldDef(H5::PListSharedPtr readPL, H5::GroupSharedPtr root, std::vector< uint64_t > &decomps, uint64_t decomp, OffsetHelper offset, std::string group, FieldDefinitionsSharedPtr def)
Import field definitions from a HDF5 document.
static const unsigned int FORMAT_VERSION
Version of the Nektar++ HDF5 format, which is embedded into the main NEKTAR group as an attribute...
Definition: FieldIOHdf5.h:176
boost::shared_ptr< DataSpace > DataSpaceSharedPtr
Definition: H5.h:88
H5DataSource(const std::string &fn, H5::PListSharedPtr parallelProps)
Constructor based on filename.
Definition: FieldIOHdf5.h:61
static const unsigned int HOMY_IDX_IDX
A helper for FieldIOHdf5::v_Write. Describes the position of the number of y-planes within the indexi...
Definition: FieldIOHdf5.h:198
#define LIB_UTILITIES_EXPORT
static const unsigned int MAX_IDXS
A helper for FieldIOHdf5::v_Write. Describes the maximum number of items in the indexing set...
Definition: FieldIOHdf5.h:201
static std::vector< std::vector< NekDouble > > NullVectorNekDoubleVector
boost::shared_ptr< FieldIO > FieldIOSharedPtr
Definition: FieldIO.h:309
static const unsigned int HOMZ_CNT_IDX
A helper for FieldIOHdf5::v_Write. Describes the position of the number of homogeneous z-planes in th...
Definition: FieldIOHdf5.h:191
static const unsigned int VAL_DCMP_IDX
A helper for FieldIOHdf5::v_Write and FieldIOHdf5::v_Import. Describes the position of the number of ...
Definition: FieldIOHdf5.h:179
static const unsigned int IDS_IDX_IDX
A helper for FieldIOHdf5::v_Write. Describes the position of the element IDs within the indexing set...
Definition: FieldIOHdf5.h:195
void SetAttr(const std::string &key, const std::string &val)
Set an attribute key/value pair on this tag.
Definition: FieldIOHdf5.h:108
static std::string className
Name of class.
Definition: FieldIOHdf5.h:212
virtual void v_Import(const std::string &infilename, std::vector< FieldDefinitionsSharedPtr > &fielddefs, std::vector< std::vector< NekDouble > > &fielddata=NullVectorNekDoubleVector, FieldMetaDataMap &fieldinfomap=NullFieldMetaDataMap, const Array< OneD, int > &ElementIDs=NullInt1DArray)
Import a HDF5 format file.
static const unsigned int MAX_CNTS
A helper for FieldIOHdf5::v_Write. Describes the maximum number of items in the cnt array per field d...
Definition: FieldIOHdf5.h:193
static const unsigned int ELEM_CNT_IDX
A helper for FieldIOHdf5::v_Write. Describes the position of the number of elements in the cnt array...
Definition: FieldIOHdf5.h:187
static const unsigned int ORDER_CNT_IDX
A helper for FieldIOHdf5::v_Write. Describes the position of the number of order points in the cnt ar...
Definition: FieldIOHdf5.h:189
TagWriterSharedPtr AddChild(const std::string &name)
Add a child node.
Definition: FieldIOHdf5.h:101
static const unsigned int HOMY_DCMP_IDX
A helper for FieldIOHdf5::v_Write and FieldIOHdf5::v_Import. Describes the position of the number of ...
Definition: FieldIOHdf5.h:181
static const unsigned int DATA_IDX_IDX
A helper for FieldIOHdf5::v_Write. Describes the position of the data size within the indexing set...
Definition: FieldIOHdf5.h:196
static const unsigned int HOMS_CNT_IDX
A helper for FieldIOHdf5::v_Write. Describes the position of the number of homogeneous strips in the ...
Definition: FieldIOHdf5.h:192
static const unsigned int HOMZ_DCMP_IDX
A helper for FieldIOHdf5::v_Write and FieldIOHdf5::v_Import. Describes the position of the number of ...
Definition: FieldIOHdf5.h:182
virtual const std::string & GetClassName() const
Get class name.
Definition: FieldIOHdf5.h:223
static const unsigned int ELEM_DCMP_IDX
A helper for FieldIOHdf5::v_Write and FieldIOHdf5::v_Import. Describes the position of the number of ...
Definition: FieldIOHdf5.h:178
static const unsigned int HOMZ_IDX_IDX
A helper for FieldIOHdf5::v_Write. Describes the position of the number of z-planes within the indexi...
Definition: FieldIOHdf5.h:199
void ImportFieldData(H5::PListSharedPtr readPL, H5::DataSetSharedPtr data_dset, H5::DataSpaceSharedPtr data_fspace, uint64_t data_i, std::vector< uint64_t > &decomps, uint64_t decomp, const FieldDefinitionsSharedPtr fielddef, std::vector< NekDouble > &fielddata)
Import the field data from the HDF5 document.
Class for operating on Nektar++ input/output files.
Definition: FieldIO.h:224
H5::FileSharedPtr Get()
Get H5::FileSharedPtr reference to file.
Definition: FieldIOHdf5.h:67
boost::shared_ptr< TagWriter > TagWriterSharedPtr
Definition: FieldIO.h:71
static FieldIOSharedPtr create(LibUtilities::CommSharedPtr pComm, bool sharedFilesystem)
Creates an instance of this class.
Definition: FieldIOHdf5.h:204
static const unsigned int HOMS_DCMP_IDX
A helper for FieldIOHdf5::v_Write and FieldIOHdf5::v_Import. Describes the position of the number of ...
Definition: FieldIOHdf5.h:183
static Array< OneD, int > NullInt1DArray
static const unsigned int ORDER_DCMP_IDX
A helper for FieldIOHdf5::v_Write and FieldIOHdf5::v_Import. Describes the position of the number of ...
Definition: FieldIOHdf5.h:180
static FieldMetaDataMap NullFieldMetaDataMap
Definition: FieldIO.h:55
static const unsigned int VAL_CNT_IDX
A helper for FieldIOHdf5::v_Write. Describes the position of the number of data points in the cnt arr...
Definition: FieldIOHdf5.h:188
HDF5 Group wrapper.
Definition: H5.h:441
virtual DataSourceSharedPtr v_ImportFieldMetaData(const std::string &filename, FieldMetaDataMap &fieldmetadatamap)
Import field metadata from filename and return the data source which wraps filename.