HDF5Extractor - Revision history

Davidbrito: /* Input file */

2013-11-27T20:00:33Z

‎Input file

Davidbrito: /* Input file */

2013-11-27T02:47:14Z

‎Input file

Davidbrito: /* Introduction */

2013-11-27T02:46:34Z

‎Introduction

Davidbrito at 02:46, 27 November 2013

2013-11-27T02:46:11Z

Guillaume: 1 revision

2008-12-03T10:27:07Z

1 revision

192.168.20.177 at 12:18, 15 January 2008

2008-01-15T12:18:43Z

New page

The application '''HDF5Extractor''' extracts from an HDF5 file data to produce one or more smaller HDF5 files.

Running options for this application are specified in an [[HDF5Extractor#Input file|input file]] whose path is indicated in a [[nomfich.dat file]].

==Introduction==
The extraction operations which can be accomplished with HDF5Extractor are the following:

- extraction of data within a time window of the time span of the original HDF5 file;

- split the data in several HDF5 files (extraction by interval), each containing the data refering to a time interval;

- extraction of data within a 2D spatial window of the spatial span of the original HDF5 file;

- extraction of data from only some vertical layers when the original HDF5 file is 3D;

- extraction of data for only some results properties.

These operations can be ordered by the user in a isolated way or at the same time.

The extraction by interval is not accomplished if the requested interval is longer than the provided HDF5 time period or if the provided HDF5 file only has one time.

'''Typical use:'''

Operations of data compression on existing HDF5 files to remove useless data, namely to accomplish storage space economy or to prepare [[ConvertToHDF5#Glue files|glue]] operations.

'''Data input requirements:'''

An HDF5 file containing data to be extracted.

'''Ouput:'''

One HDF5 file with the extracted data, except in the case the operation of extraction of the data by time intervals is chosen.

When extraction by interval is performed it is produced one file per every time period of length equal to interval specified by the user, begining in the original HDF5 file start time, and is produced also an HDF5 file with the remainder of the spliting, so that all data from input file is extracted.

==Input file==
The name of the input file must be provided in the [[nomfich.dat file]] in use.
FILENAME : ... (path/name of HDF5 file with data to extract)

OUTPUTFILENAME : ... (path/name of output file (HDF5) with extracted data)

START_TIME : ... (start date for time window: yyyy mm dd hh mm ss)
END_TIME : ... (end date for time window: yyyy mm dd hh mm ss)

XY_WINDOW_OUTPUT : 0/1 (0 = extract all cells; 1 = extract only cells in spatial
window specified in XY_WINDOW_LIMITS; 0 = default)
XY_WINDOW_LIMITS : ... ... ... ... (window limits: ILB (i of lower-left cell), JLB
(j of lower-left cell), IUB (i of upper-right
cell), JUB (j of upper-right cell))

LAYERS_OUTPUT : 0/1 (0 = extract all layers, 1 = extract only layers specified in
LAYERS_MIN_MAX; 0 = default)
LAYERS_MIN_MAX : ... ... (KLB (lower layer), KUB (upper layer))

INTERVAL : 0/1 (0 = time extraction of period indicated in START_TIME
and END_TIME; 1 = time extraction of every time period
in file with length provided by DT_INTERVAL; 0 = default)
DT_INTERVAL : ... (time interval for extraction, in seconds)

CONVERT_V3_TO_V4 : 0/1 (0 = do not convert file format from MOHID V3 to MOHID V4;
1 = convert file format from MOHID V3 to MOHID V4; 0 = default)

(block for each property/parameter which is object of extraction, may be several)
<BeginParameter>
PROPERTY : ... (parameter name, according with MOHID V4)
HDF_GROUP : ... (complete path in HDF5 file to parameter data, according with
MOHID V4)

(if CONVERT_V3_TO_V4 : 1:)
PROPERTY_V3 : ... (parameter name in MOHID V3)
HDF_GROUP_V3 : ... (complete path in HDF5 file to parameter data in MOHID V3)
<EndParameter>

''Remarks:''

- in the spatial window specification i, j are the cell indentification indexes in the MOHID
framework;

- in layer specification "lower" means the layer closest to bottom, which has smaller layer
number (k), and "upper" means the layer closest to surface, which has a higher layer number
(k));

- extraction by interval option (INTERVAL : 1) produces one file per every time
period of length equal to interval specified by the user, begining in the file's start
time: e.g. if DT_INTERVAL : 21600 (6 hours interval) then a HDF5 file is produced with the
data of each 21600 seconds period and also with the remainder of the division, so that all
data from provided file is extracted;

- in case of extraction by interval the time window eventually defined by START_TIME and
END_TIME keywords is not considered;

- extraction by interval is not accomplished if interval is longer than the provided HDF5
time period (or if the provided HDF5 file only has one time);

- only time dependent parameters can be extracted, because extraction is meaningless
otherwise: for instance "Residual" and "Statistics" group parameters cannot be extracted;

- as parameters to specify in parameter block are meant result properties (but not only those
present in the "Result" group): "Grid" time dependent variables are not to be specified but
are automatically extracted;

- at least one parameter block must be present in input file for any extraction to be
accomplished.

==Sample==
OUTPUTFILENAME : compactedhydro.hdf5

FILENAME : Hydrodynamic_15.hdf5

START_TIME : 2005 2 28 0 0 0
END_TIME : 2005 2 28 18 22 0

XY_WINDOW_OUTPUT : 1
XY_WINDOW_LIMITS : 1 1 70 100

LAYERS_OUTPUT : 1
LAYERS_MIN_MAX : 10 11

INTERVAL : 1
DT_INTERVAL : 21600

CONVERT_V3_TO_V4 : 0

<BeginParameter>
PROPERTY : velocity modulus
HDF_GROUP : /Results/velocity modulus
<EndParameter>

<BeginParameter>
PROPERTY : velocity U
HDF_GROUP : /Results/velocity U
<EndParameter>

<BeginParameter>
PROPERTY : velocity V
HDF_GROUP : /Results/velocity V
<EndParameter>

<BeginParameter>
PROPERTY : water level
HDF_GROUP : /Results/water level
<EndParameter>

[[Category:Tools]]
[[Category:Hdf5]]

@@ Line 102: / Line 102: @@
   - extraction by skipping instants is done by selecting the first HDF instant and then
   selecting instants at a defined interval given by user in DT_SKIP_INSTANTS (e.g. selecting
-  daily results (86400) in hourly HDF.
+  daily results (86400s) in hourly HDF).
   - exctraction by skipping instants can be used in conjunction with extaction by interval

@@ Line 62: / Line 62: @@
   SKIP_INSTANTS            : 0/1 (0 = use all time instants from original HDF;
 = extract time instants at a defined frequency given by
-                                   DT_SKIP_INSTANTS (use lower frequency than data))
+                                   DT_SKIP_INSTANTS (use lower frequency than original))
   DT_SKIP_INSTANTS         : ... (time interval to extract instants, in seconds)

@@ Line 10: / Line 10: @@
 - split the data in several HDF5 files (extraction by interval), each containing the data refering to a time interval;
-- extraction of data using a time frequency lower than original (skipping instants)
+- extraction of data using a time frequency lower than original (skipping instants);
 - extraction of data within a 2D spatial window of the spatial span of the original HDF5 file;

@@ Line 9: / Line 9: @@
 - split the data in several HDF5 files (extraction by interval), each containing the data refering to a time interval;
 - extraction of data within a 2D spatial window of the spatial span of the original HDF5 file;
@@ Line 57: / Line 59: @@
                                   in file with length provided by DT_INTERVAL; 0 = default)
   DT_INTERVAL              : ... (time interval for extraction, in seconds)
   CONVERT_V3_TO_V4         : 0/1 (0 = do not convert file format from MOHID V3 to MOHID V4;
@@ Line 92: / Line 99: @@
   - extraction by interval is not accomplished if interval is longer than the provided HDF5
   time period (or if the provided HDF5 file only has one time);
   - only time dependent parameters can be extracted, because extraction is meaningless
@@ Line 104: / Line 118: @@
 ==Sample==
   OUTPUTFILENAME           : compactedhydro.hdf5
@@ Line 141: / Line 156: @@
   HDF_GROUP                : /Results/water level
   <EndParameter>
 [[Category:Tools]]
 [[Category:Hdf5]]

← Older revision	Revision as of 10:27, 3 December 2008
(No difference)