MohidWiki - User contributions [en]

Parallel processing

2010-06-08T12:33:48Z

AngelaCanas: /* Parallel processing via OpenMP */

The historical need in numerical models to reduce computational time became a priority to the Mohid development team as an operational hydrodynamic and water quality model to the Tagus Estuary, in Lisbon, Portugal, was implemented using the [[Mohid Water]] model full capabilities. Thus, parallel processing has been implemented in Mohid Water in 2003, by using [http://www.mcs.anl.gov/mpi/mpich/ MPICH], a free portable implementation of [http://www.mcs.anl.gov/mpi/ MPI], the standard for message-passing libraries.

Currently, and due to the use of the new Intel Fortran compiler both Mohid Water and Mohid Land have parallelization features using [http://www.openmp.org/ OpenMP].

== Parallel processing via MPI ==
The [[Mohid Water]] ability to run [[nested models]] was accomplished by creating a linked list of all the models and by attributing to each one a father-son identification, through which the models communicate. The first stage for introducing parallel processing in Mohid was to add the possibility of launching a process by each model to run, and then, using MPICH, establish communication between models. This enables each sub-model to run in a different processor (even if the processor belongs to a different computer, as long as it is in the same network) and in parallel, instead of running all in the same processor and each model having to wait for the others to perform their calculations.

Parallel processing as it is presently implemented in Mohid, could not be achieved without [[object oriented programming]] philosophy, as each model is an instance of [[class Model]] and no changes, exception made to the implementation of the MPI communications calls needed to be added. Using this feature, computational speed was improved (varying from application to application), as now the whole model will take the same time as the slowest model to run plus the time to communicate with the other processes. Here, the network communication speed plays an important role, as it can become limiting. However, the amount of information passing between models, depending of course on the memory allocated for each model, has not yet proven to be big enough to make a 100 Mbps network connection time limiting.

[[Domain Composition]] is an ongoing project, in a very early stage, aimed at decoupling a domain into several subdomain that communicate among them (2-way) via MPI.

Find here information on how to [[setup a MOHID simulation using MPI]] and on [[compiling Mohid with MPI]].

== Parallel processing via OpenMP ==
Parallel processing using OpenMP is currently being implemented in Mohid by defining directives '''to optimize loops'''. These directives are defined as comments in the code and therefore need special compilation options. See more on [[compiling Mohid with OpenMP]]. Without special compilation these instructions are not considered in normal Fortran compilation.

OpenMP parallel processing can be used in '''multi core processors''' present in the same computer. It cannot be used to parallel processing using processors located in several computers in a cluster.

Loops optimization is introduced in a first phase in loops referring to grid variables (grid indexes, k, j, i) located in the Modifier section of MOHID modules.
Modifier loops are possibly used several times or involve a large resource allocation in MOHID simulations, hence these are locations with larger potential resource gains involved in parallelization.

In case of loops with several looping variables, the parallelized variable is chosen according with cost involved in the loop through this variable. E.g. in a 3D loop (k, j, i loop variables) if j dimension is much larger than the k (number of layers) or i then parallel processing is introduced in j variable, since the resource costs and the savings achieved with the parallelization are larger in this loop than in the others.

'''''Basic OpenMP syntax:'''''

In this section is provided an introduction with the basic concepts in OpenMP programming. The OpenMP language reference manual (http://www.openmp.org) should be consulted for further details.

OpenMP processing is made by a set of threads: the '''Master''' thread and the '''Workers''' threads.

OpenMP parallelization is accomplished by defining '''Parallel regions''' and by creating the threads (one for each available core): the processing inside a parallel region is divided between the Workers and the Master, which then proceed in parallel (simultaneously), instead of the unique thread existent in unparallel processing.

Each thread can have a set of '''Private variables''', which are affected only by this thread. The choice of the private variables, which are explicitly defined in programming, is a central part of the OpenMP programming. Should be made private all the variables whose values are altered by each thread processing and that affect other threads processing.

An obvious choice of these private variables are the loop variables when these are used to alter positions in matrixes or vectors in each iteration of the loop.

However it should be noted that in private variables the values are undefined in enter and exit of the parallel region and that by default these variables have no storage association with the variables outside the parallel region (this default behavior can, however, be altered by specific OpenMP directives).

The instructions in OpenMP are provided by a set of '''Directives'''. These refer to several actions such as the definition of Parallel regions, Work sharing and Data attributes. Directives are specific for the underlying programming language being used, either C or Fortran.

In Fortran directives are case insensitive. The basic syntax in as following:

sentinel directive [clause [[,] clause] ...]

'''Sentinel''' is !$OMP in either fixed or free format. The continuation of directives (from one code line to another) are according with the underlying language: & in Fortran case.

'''Clauses''' are used to specify additional information for the directive. Important clauses are:

- PRIVATE (private variables list);

- NOWAIT: specify that threads will not syncronize (i.e. wait for each other) at the end of a specific construct (e.g. a DO loop) within a parallel region; this is specified at the end of a parallel construct;

A parallel region is typically specified as:

!$OMP PARALLEL PRIVATE(PrivateVariable1,...,PrivateVariableN)

Inside a parallel region the work is distributed by the threads through '''Work Sharing constructs'''.

For every processing made inside a parallel region must be assigned a Work Sharing construct. If this is not verified execution errors can occur.

An important Work Sharing construct is the DO loop, specified as:

!$OMP DO
... (do loop in Fortran)
!$OMP END DO

In this construct the iterations of the enclosed Fortran DO loop are distributed by the threads.

The way the work is distributed over the threads is managed by the SCHEDULE clause of the DO construct. An important option of SCHEDULE is DYNAMIC: provided a Chunk number of iterations each thread will process this fixed number of iterations. After finishing a Chunk each thread will began another available Chunk till all iterations are completed:

!$OMP DO SCHEDULE(DYNAMIC, CHUNK)
...
!$OMP END DO

The distribution of Chunk among the threads requires synchronization for each assignment. This causes a overhead that could be important. The DYNAMIC option is advisable when each iteration involves an amount of work which is not predictable. This could be the case when IF constructs are present inside the loop containg extra processing done only in specific cases. Also if the threads arrive to the DO loop at different times, e.g. if they come from a previous DO loop with a NOWAIT end clause.

When the amount of work required in each DO loop iteration is predictable and the same for all threads it is advisable to use the STATIC option of SCHEDULE. This is particularly useful for DO loops which are unique in a parallel region:

!$OMP DO SCHEDULE(STATIC, CHUNK)
...
!$OMP END DO

Important characteristics of the Work Sharing constructs are that they do not create new threads, must contemplate all the existing threads or none at all and there is no barrier on entry (any available thread is not required to wait for the others) but a barrier on exit exists. The barrier on exit can be removed by the referred NOWAIT clause.

Work Sharing directives can appear outside the lexical extent of a parallel region (sequential lines of code appearing inside the parallel region). If they are dynamically linked with a parallel region (e.g. they appear in processing a subroutine call) they are orphaned. If, however, they appear outside this dynamic connection with a parallel region (and also not in the lexical extent of the region) they are ignored and the enclosed code is performed by one thread only and no parallelization is processed.

Inside a Work Sharing construct can be defined a critical region, if it is conveninent that only one thread at a time processes a specific code. This can be used to avoid problems with input/output operations potentially occuring by several threads processing at same time (problems can occur because memory locations are being assessed at same time). This is commanded as following:

!$OMP CRITICAL [Name]
... (code to be processed sequentially)
!$OMP END CRITICAL [Name]

In this case, no barriers exist in the entry and exit and all threads process the enclosed code although one at a time.
If more than one critial region is defined in the code then every critical region should have a different name or the execution outcome could become undeterminated.
Caution should be given to the naming of critical sections: if these names do not conflict with variable names they do conflict with names of subroutines and common blocks.

Within a Work Sharing construct can also be specified that a portion of the code is processed only by one thread, e.g. to read information from a file common to all threads. This is done with the following syntax:

!$OMP SINGLE [clause ...]
... (code to be processed by only one thread)
!$OMP END SINGLE

When one wants this single thread to be the Master then following syntax is used:

!$OMP MASTER
... (code to be processed by only Master thread)
!$OMP END MASTER

As with CRITICAL, no barriers exist in SINGLE/MASTER in the entry and exit.

In these contructs the fact that no barrier exists at the exit may cause problems if the single thread processed code section is intended to be dealt with previously from the subsequent code. In this situation a barrier can be introduced at the end of the SINGLE/MASTER construct:

!$OMP BARRIER

Under this directive threads wait at the barrier point till all threads reach it.

An important rule about the BARRIER use is that this directive cannot be nested in a Work Sharing construct such a DO construct.

Sometimes some variables used by threads cannot be made private. One notorious case consists in variables performing sums of values obtained in iterations of a cycle, which when summing integers are sometimes called «counters». As these variables should accumulate values obtained in each iteration these cannot be made private. This would imply that a critical section should be introduced for the actualization of the variable which would make the code sequential. To avoid this there is an OpenMP clause called REDUCTION which when applied to a DO Work Sharing construct has the following syntax:

!$OMP DO SCHEDULE(DYNAMIC, CHUNK) REDUCTION(+ : Counter)

Where in this example «Counter» is the reduction variable.

The REDUCTION operation has as restriction that the reduction variable only is actualized once inside the construct and must be a shared variable.

In practice in the program execution a copy of the variable is made for each thread, which actualizes it as a local variable, and in the end of the construct the thread variables are added to the shared variable from which they originate. The knowledge of this process is important because in the case of the accumulation of real values this process can originate differences relative to the unparallel result due to rounded values.

'''''Parallelization overheads:'''''

Although parallelization involves potentially gains in computer resources use it also involves overheads which can sometimes be important.

The creation of the thread team (Workers) at the beginning of each parallel region is a source of such overheads. Because of this it is preferrable in each program/subrotine to be parallelized to create only one parallel region and then deal with the code that must be done only by one thread with OpenMP directives than to create several parallel regions.

Another source of overhead is the synchronization between threads in Work Sharing constructs.

These overheads cause that in some times the parallelization solution may not be performing better than the non parallelization solution, especially if the computational burden of the problem is low.

This is the case of DO constructs when each cycle iteration has a small computational burden (e.g. equaling one matrix's value to other matrix's value) and is to be expected in nested loops when the parallelized DO loop is inner.

Generally as the scale of the problem increases the overheads will be less significant and parallelization more advantageous.

In very small scale problems parallelization may provide more comsumption of computational resources than the non parallelization.

'''''Race condition:'''''

In some case execution of a parallel program may be suspended because different threads are competing for the actualization of some variable. This is called a race condition.

The programmer should take care to avoid this situation as its debugging is very difficult. Most commonly the execution will be suspended and the processors would become idle without any error message. When a vast part of the code is parallelized it is then almost impossible to find the cause of the error unless one reckons possible race condition situations.

A way to avoid this is to use critical regions or a REDUCTION clause.

'''''References:'''''

Chandra, Rohit, 2001, Parallel Programming in OpenMP, Academic Press.

[[Category:Programming]]

Parallel processing

2010-06-08T12:33:21Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2010-05-31T14:47:48Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2010-04-28T12:27:56Z

AngelaCanas: /* Parallel processing via OpenMP */

The historical need in numerical models to reduce computational time became a priority to the Mohid development team as an operational hydrodynamic and water quality model to the Tagus Estuary, in Lisbon, Portugal, was implemented using the [[Mohid Water]] model full capabilities. Thus, parallel processing has been implemented in Mohid Water in 2003, by using [http://www.mcs.anl.gov/mpi/mpich/ MPICH], a free portable implementation of [http://www.mcs.anl.gov/mpi/ MPI], the standard for message-passing libraries.

Currently, and due to the use of the new Intel Fortran compiler both Mohid Water and Mohid Land have parallelization features using [http://www.openmp.org/ OpenMP].

== Parallel processing via MPI ==
The [[Mohid Water]] ability to run [[nested models]] was accomplished by creating a linked list of all the models and by attributing to each one a father-son identification, through which the models communicate. The first stage for introducing parallel processing in Mohid was to add the possibility of launching a process by each model to run, and then, using MPICH, establish communication between models. This enables each sub-model to run in a different processor (even if the processor belongs to a different computer, as long as it is in the same network) and in parallel, instead of running all in the same processor and each model having to wait for the others to perform their calculations.

Parallel processing as it is presently implemented in Mohid, could not be achieved without [[object oriented programming]] philosophy, as each model is an instance of [[class Model]] and no changes, exception made to the implementation of the MPI communications calls needed to be added. Using this feature, computational speed was improved (varying from application to application), as now the whole model will take the same time as the slowest model to run plus the time to communicate with the other processes. Here, the network communication speed plays an important role, as it can become limiting. However, the amount of information passing between models, depending of course on the memory allocated for each model, has not yet proven to be big enough to make a 100 Mbps network connection time limiting.

[[Domain Composition]] is an ongoing project, in a very early stage, aimed at decoupling a domain into several subdomain that communicate among them (2-way) via MPI.

Find here information on how to [[setup a MOHID simulation using MPI]] and on [[compiling Mohid with MPI]].

== Parallel processing via OpenMP ==
Parallel processing using OpenMP is currently being implemented in Mohid by defining directives '''to optimize loops'''. These directives are defined as comments in the code and therefore need special compilation options. See more on [[compiling Mohid with OpenMP]]. Without special compilation these instructions are not considered in normal Fortran compilation.

OpenMP parallel processing can be used in '''multi core processors''' present in the same computer. It cannot be used to parallel processing using processors located in several computers in a cluster.

Loops optimization is introduced in a first phase in loops referring to grid variables (grid indexes, k, j, i) located in the Modifier section of MOHID modules.
Modifier loops are possibly used several times or involve a large resource allocation in MOHID simulations, hence these are locations with larger potential resource gains involved in parallelization.

In case of loops with several looping variables, the parallelized variable is chosen according with cost involved in the loop through this variable. E.g. in a 3D loop (k, j, i loop variables) if j dimension is much larger than the k (number of layers) or i then parallel processing is introduced in j variable, since the resource costs and the savings achieved with the parallelization are larger in this loop than in the others.

'''''Basic OpenMP syntax:'''''

In this section is provided an introduction with the basic concepts in OpenMP programming. The OpenMP language reference manual (http://www.openmp.org) should be consulted for further details.

OpenMP processing is made by a set of threads: the '''Master''' thread and the '''Workers''' threads.

OpenMP parallelization is accomplished by defining '''Parallel regions''' and by creating the threads (one for each available core): the processing inside a parallel region is divided between the Workers and the Master, which then proceed in parallel (simultaneously), instead of the unique thread existent in unparallel processing.

Each thread can have a set of '''Private variables''', which are affected only by this thread. The choice of the private variables, which are explicitly defined in programming, is a central part of the OpenMP programming. Should be made private all the variables whose values are altered by each thread processing and that affect other threads processing.

An obvious choice of these private variables are the loop variables when these are used to alter positions in matrixes or vectors in each iteration of the loop.

However it should be noted that in private variables the values are undefined in enter and exit of the parallel region and that by default these variables have no storage association with the variables outside the parallel region (this default behavior can, however, be altered by specific OpenMP directives).

The instructions in OpenMP are provided by a set of '''Directives'''. These refer to several actions such as the definition of Parallel regions, Work sharing and Data attributes. Directives are specific for the underlying programming language being used, either C or Fortran.

In Fortran directives are case insensitive. The basic syntax in as following:

sentinel directive [clause [[,] clause] ...]

'''Sentinel''' is !$OMP in either fixed or free format. The continuation of directives (from one code line to another) are according with the underlying language: & in Fortran case.

'''Clauses''' are used to specify additional information for the directive. Important clauses are:

- PRIVATE (private variables list);

- NOWAIT: specify that threads will not syncronize (i.e. wait for each other) at the end of a specific construct (e.g. a DO loop) within a parallel region; this is specified at the end of a parallel construct;

A parallel region is typically specified as:

!$OMP PARALLEL PRIVATE(PrivateVariable1,...,PrivateVariableN)

Inside a parallel region the work is distributed by the threads through '''Work Sharing constructs'''.

For every processing made inside a parallel region must be assigned a Work Sharing construct. If this is not verified execution errors can occur.

An important Work Sharing construct is the DO loop, specified as:

!$OMP DO
... (do loop in Fortran)
!$OMP END DO

In this construct the iterations of the enclosed Fortran DO loop are distributed by the threads.

The way the work is distributed over the threads is managed by the SCHEDULE clause of the DO construct. An important option of SCHEDULE is DYNAMIC: provided a Chunk number of iterations each thread will process this fixed number of iterations. After finishing a Chunk each thread will began another available Chunk till all iterations are completed:

!$OMP DO SCHEDULE(DYNAMIC, CHUNK)
...
!$OMP END DO

The distribution of Chunk among the threads requires synchronization for each assignment. This causes a overhead that could be important. The DYNAMIC option is advisable when each iteration involves an amount of work which is not predictable. This could be the case when IF constructs are present inside the loop containg extra processing done only in specific cases. Also if the threads arrive to the DO loop at different times, e.g. if they come from a previous DO loop with a NOWAIT end clause.

When the amount of work required in each DO loop iteration is predictable and the same for all threads it is advisable to use the STATIC option of SCHEDULE. This is particularly useful for DO loops which are unique in a parallel region:

!$OMP DO SCHEDULE(STATIC, CHUNK)
...
!$OMP END DO

Important characteristics of the Work Sharing constructs are that they do not create new threads, must contemplate all the existing threads or none at all and there is no barrier on entry (any available thread is not required to wait for the others) but a barrier on exit exists. The barrier on exit can be removed by the referred NOWAIT clause.

Work Sharing directives can appear outside the lexical extent of a parallel region (sequential lines of code appearing inside the parallel region). If they are dynamically linked with a parallel region (e.g. they appear in processing a subroutine call) they are orphaned. If, however, they appear outside this dynamic connection with a parallel region (and also not in the lexical extent of the region) they are ignored and the enclosed code is performed by one thread only and no parallelization is processed.

Inside a Work Sharing construct can be defined a critical region, if it is conveninent that only one thread at a time processes a specific code. This can be used to avoid problems with input/output operations potentially occuring by several threads processing at same time (problems can occur because memory locations are being assessed at same time). This is commanded as following:

!$OMP CRITICAL [Name]
... (code to be processed sequentially)
!$OMP END CRITICAL [Name]

In this case, no barriers exist in the entry and exit and all threads process the enclosed code although one at a time.
If more than one critial region is defined in the code then every critical region should have a different name or the execution outcome could become undeterminated.
Caution should be given to the naming of critical sections: if these names do not conflict with variable names they do conflict with names of subroutines and common blocks.

Within a Work Sharing construct can also be specified that a portion of the code is processed only by one thread, e.g. to read information from a file common to all threads. This is done with the following syntax:

!$OMP SINGLE [clause ...]
... (code to be processed by only one thread)
!$OMP END SINGLE

When one wants this single thread to be the Master then following syntax is used:

!$OMP MASTER
... (code to be processed by only Master thread)
!$OMP END MASTER

As with CRITICAL, no barriers exist in SINGLE/MASTER in the entry and exit.

In these contructs the fact that no barrier exists at the exit may cause problems if the single thread processed code section is intended to be dealt with previously from the subsequent code. In this situation a barrier can be introduced at the end of the SINGLE/MASTER construct:

!$OMP BARRIER

Under this directive threads wait at the barrier point till all threads reach it.

An important rule about the BARRIER use is that this directive cannot be nested in a Work Sharing construct such a DO construct.

'''''Parallelization overheads:'''''

Although parallelization involves potentially gains in computer resources use it also involves overheads which can sometimes be important.

The creation of the thread team (Workers) at the beginning of each parallel region is a source of such overheads. Because of this it is preferrable in each program/subrotine to be parallelized to create only one parallel region and then deal with the code that must be done only by one thread with OpenMP directives than to create several parallel regions.

Another source of overhead is the synchronization between threads in Work Sharing constructs.

These overheads cause that in some times the parallelization solution may not be performing better than the non parallelization solution, especially if the computational burden of the problem is low.

This is the case of DO constructs when each cycle iteration has a small computational burden (e.g. equaling one matrix's value to other matrix's value) and is to be expected in nested loops when the parallelized DO loop is inner.

Generally as the scale of the problem increases the overheads will be less significant and parallelization more advantageous.

In very small scale problems parallelization may provide more comsumption of computational resources than the non parallelization.

'''''Race condition:'''''

In some case execution of a parallel program may be suspended because different threads are competing for the actualization of some variable. This is called a race condition.

The programmer should take care to avoid this situation as its debugging is very difficult. Most commonly the execution will be suspended and the processors would become idle without any error message. When a vast part of the code is parallelized it is then almost impossible to find the cause of the error unless one reckons possible race condition situations.

A way to avoid this is to use critical regions or a REDUCTION clause.

[[Category:Programming]]

Parallel processing

2010-04-27T18:06:58Z

AngelaCanas: /* Parallel processing via OpenMP */

The historical need in numerical models to reduce computational time became a priority to the Mohid development team as an operational hydrodynamic and water quality model to the Tagus Estuary, in Lisbon, Portugal, was implemented using the [[Mohid Water]] model full capabilities. Thus, parallel processing has been implemented in Mohid Water in 2003, by using [http://www.mcs.anl.gov/mpi/mpich/ MPICH], a free portable implementation of [http://www.mcs.anl.gov/mpi/ MPI], the standard for message-passing libraries.

Currently, and due to the use of the new Intel Fortran compiler both Mohid Water and Mohid Land have parallelization features using [http://www.openmp.org/ OpenMP].

== Parallel processing via MPI ==
The [[Mohid Water]] ability to run [[nested models]] was accomplished by creating a linked list of all the models and by attributing to each one a father-son identification, through which the models communicate. The first stage for introducing parallel processing in Mohid was to add the possibility of launching a process by each model to run, and then, using MPICH, establish communication between models. This enables each sub-model to run in a different processor (even if the processor belongs to a different computer, as long as it is in the same network) and in parallel, instead of running all in the same processor and each model having to wait for the others to perform their calculations.

Parallel processing as it is presently implemented in Mohid, could not be achieved without [[object oriented programming]] philosophy, as each model is an instance of [[class Model]] and no changes, exception made to the implementation of the MPI communications calls needed to be added. Using this feature, computational speed was improved (varying from application to application), as now the whole model will take the same time as the slowest model to run plus the time to communicate with the other processes. Here, the network communication speed plays an important role, as it can become limiting. However, the amount of information passing between models, depending of course on the memory allocated for each model, has not yet proven to be big enough to make a 100 Mbps network connection time limiting.

[[Domain Composition]] is an ongoing project, in a very early stage, aimed at decoupling a domain into several subdomain that communicate among them (2-way) via MPI.

Find here information on how to [[setup a MOHID simulation using MPI]] and on [[compiling Mohid with MPI]].

== Parallel processing via OpenMP ==
Parallel processing using OpenMP is currently being implemented in Mohid by defining directives '''to optimize loops'''. These directives are defined as comments in the code and therefore need special compilation options. See more on [[compiling Mohid with OpenMP]]. Without special compilation these instructions are not considered in normal Fortran compilation.

OpenMP parallel processing can be used in '''multi core processors''' present in the same computer. It cannot be used to parallel processing using processors located in several computers in a cluster.

Loops optimization is introduced in a first phase in loops referring to grid variables (grid indexes, k, j, i) located in the Modifier section of MOHID modules.
Modifier loops are possibly used several times or involve a large resource allocation in MOHID simulations, hence these are locations with larger potential resource gains involved in parallelization.

In case of loops with several looping variables, the parallelized variable is chosen according with cost involved in the loop through this variable. E.g. in a 3D loop (k, j, i loop variables) if j dimension is much larger than the k (number of layers) or i then parallel processing is introduced in j variable, since the resource costs and the savings achieved with the parallelization are larger in this loop than in the others.

'''''Basic OpenMP syntax:'''''

In this section is provided an introduction with the basic concepts in OpenMP programming. The OpenMP language reference manual (http://www.openmp.org) should be consulted for further details.

OpenMP processing is made by a set of threads: the '''Master''' thread and the '''Workers''' threads.

OpenMP parallelization is accomplished by defining '''Parallel regions''' and by creating the threads (one for each available core): the processing inside a parallel region is divided between the Workers and the Master, which then proceed in parallel (simultaneously), instead of the unique thread existent in unparallel processing.

Each thread can have a set of '''Private variables''', which are affected only by this thread. The choice of the private variables, which are explicitly defined in programming, is a central part of the OpenMP programming. Should be made private all the variables whose values are altered by each thread processing and that affect other threads processing.

An obvious choice of these private variables are the loop variables when these are used to alter positions in matrixes or vectors in each iteration of the loop.

However it should be noted that in private variables the values are undefined in enter and exit of the parallel region and that by default these variables have no storage association with the variables outside the parallel region (this default behavior can, however, be altered by specific OpenMP directives).

The instructions in OpenMP are provided by a set of '''Directives'''. These refer to several actions such as the definition of Parallel regions, Work sharing and Data attributes. Directives are specific for the underlying programming language being used, either C or Fortran.

In Fortran directives are case insensitive. The basic syntax in as following:

sentinel directive [clause [[,] clause] ...]

'''Sentinel''' is !$OMP in either fixed or free format. The continuation of directives (from one code line to another) are according with the underlying language: & in Fortran case.

'''Clauses''' are used to specify additional information for the directive. Important clauses are:

- PRIVATE (private variables list);

- NOWAIT: specify that threads will not syncronize (i.e. wait for each other) at the end of a specific construct (e.g. a DO loop) within a parallel region; this is specified at the end of a parallel construct;

A parallel region is typically specified as:

!$OMP PARALLEL PRIVATE(PrivateVariable1,...,PrivateVariableN)

Inside a parallel region the work is distributed by the threads through '''Work Sharing constructs'''.

For every processing made inside a parallel region must be assigned a Work Sharing construct. If this is not verified execution errors can occur.

An important Work Sharing construct is the DO loop, specified as:

!$OMP DO
... (do loop in Fortran)
!$OMP END DO

In this construct the iterations of the enclosed Fortran DO loop are distributed by the threads.

The way the work is distributed over the threads is managed by the SCHEDULE clause of the DO construct. An important option of SCHEDULE is DYNAMIC: provided a Chunk number of iterations each thread will process this fixed number of iterations. After finishing a Chunk each thread will began another available Chunk till all iterations are completed:

!$OMP DO SCHEDULE(DYNAMIC, CHUNK)
...
!$OMP END DO

The distribution of Chunk among the threads requires synchronization for each assignment. This causes a overhead that could be important. The DYNAMIC option is advisable when each iteration involves an amount of work which is not predictable. This could be the case when IF constructs are present inside the loop containg extra processing done only in specific cases. Also if the threads arrive to the DO loop at different times, e.g. if they come from a previous DO loop with a NOWAIT end clause.

When the amount of work required in each DO loop iteration is predictable and the same for all threads it is advisable to use the STATIC option of SCHEDULE. This is particularly useful for DO loops which are unique in a parallel region:

!$OMP DO SCHEDULE(STATIC, CHUNK)
...
!$OMP END DO

Important characteristics of the Work Sharing constructs are that they do not create new threads, must contemplate all the existing threads or none at all and there is no barrier on entry (any available thread is not required to wait for the others) but a barrier on exit exists. The barrier on exit can be removed by the referred NOWAIT clause.

Work Sharing directives can appear outside the lexical extent of a parallel region (sequential lines of code appearing inside the parallel region). If they are dynamically linked with a parallel region (e.g. they appear in processing a subroutine call) they are orphaned. If, however, they appear outside this dynamic connection with a parallel region (and also not in the lexical extent of the region) they are ignored and the enclosed code is performed by one thread only and no parallelization is processed.

Inside a Work Sharing construct can be defined a critical region, if it is conveninent that only one thread at a time processes a specific code. This can be used to avoid problems with input/output operations potentially occuring by several threads processing at same time (problems can occur because memory locations are being assessed at same time). This is commanded as following:

!$OMP CRITICAL [Name]
... (code to be processed sequentially)
!$OMP END CRITICAL [Name]

In this case, no barriers exist in the entry and exit and all threads process the enclosed code although one at a time.
If more than one critial region is defined in the code then every critical region should have a different name or the execution outcome could become undeterminated.
Caution should be given to the naming of critical sections: if these names do not conflict with variable names they do conflict with names of subroutines and common blocks.

Within a Work Sharing construct can also be specified that a portion of the code is processed only by one thread, e.g. to read information from a file common to all threads. This is done with the following syntax:

!$OMP SINGLE [clause ...]
... (code to be processed by only one thread)
!$OMP END SINGLE

When one wants this single thread to be the Master then following syntax is used:

!$OMP MASTER
... (code to be processed by only Master thread)
!$OMP END MASTER

As with CRITICAL, no barriers exist in SINGLE/MASTER in the entry and exit.

In these contructs the fact that no barrier exists at the exit may cause problems if the single thread processed code section is intended to be dealt with previously from the subsequent code. In this situation a barrier can be introduced at the end of the SINGLE/MASTER construct:

!$OMP BARRIER

Under this directive threads wait at the barrier point till all threads reach it.

'''''Parallelization overheads:'''''

Although parallelization involves potentially gains in computer resources use it also involves overheads which can sometimes be important.

The creation of the thread team (Workers) at the beginning of each parallel region is a source of such overheads. Because of this it is preferrable in each program/subrotine to be parallelized to create only one parallel region and then deal with the code that must be done only by one thread with OpenMP directives than to create several parallel regions.

Another source of overhead is the synchronization between threads in Work Sharing constructs.

These overheads cause that in some times the parallelization solution may not be performing better than the non parallelization solution, especially if the computational burden of the problem is low.

This is the case of DO constructs when each cycle iteration has a small computational burden (e.g. equaling one matrix's value to other matrix's value) and is to be expected in nested loops when the parallelized DO loop is inner.

Generally as the scale of the problem increases the overheads will be less significant and parallelization more advantageous.

In very small scale problems parallelization may provide more comsumption of computational resources than the non parallelization.

'''''Race condition:'''''

In some case execution of a parallel program may be suspended because different threads are competing for the actualization of some variable. This is called a race condition.

The programmer should take care to avoid this situation as its debugging is very difficult. Most commonly the execution will be suspended and the processors would become idle without any error message. When a vast part of the code is parallelized it is then almost impossible to find the cause of the error unless one reckons possible race condition situations.

A way to avoid this is to use critical regions or a REDUCTION clause.

[[Category:Programming]]

Parallel processing

2010-04-15T11:42:41Z

AngelaCanas: /* Parallel processing via OpenMP */

The historical need in numerical models to reduce computational time became a priority to the Mohid development team as an operational hydrodynamic and water quality model to the Tagus Estuary, in Lisbon, Portugal, was implemented using the [[Mohid Water]] model full capabilities. Thus, parallel processing has been implemented in Mohid Water in 2003, by using [http://www.mcs.anl.gov/mpi/mpich/ MPICH], a free portable implementation of [http://www.mcs.anl.gov/mpi/ MPI], the standard for message-passing libraries.

Currently, and due to the use of the new Intel Fortran compiler both Mohid Water and Mohid Land have parallelization features using [http://www.openmp.org/ OpenMP].

== Parallel processing via MPI ==
The [[Mohid Water]] ability to run [[nested models]] was accomplished by creating a linked list of all the models and by attributing to each one a father-son identification, through which the models communicate. The first stage for introducing parallel processing in Mohid was to add the possibility of launching a process by each model to run, and then, using MPICH, establish communication between models. This enables each sub-model to run in a different processor (even if the processor belongs to a different computer, as long as it is in the same network) and in parallel, instead of running all in the same processor and each model having to wait for the others to perform their calculations.

Parallel processing as it is presently implemented in Mohid, could not be achieved without [[object oriented programming]] philosophy, as each model is an instance of [[class Model]] and no changes, exception made to the implementation of the MPI communications calls needed to be added. Using this feature, computational speed was improved (varying from application to application), as now the whole model will take the same time as the slowest model to run plus the time to communicate with the other processes. Here, the network communication speed plays an important role, as it can become limiting. However, the amount of information passing between models, depending of course on the memory allocated for each model, has not yet proven to be big enough to make a 100 Mbps network connection time limiting.

[[Domain Composition]] is an ongoing project, in a very early stage, aimed at decoupling a domain into several subdomain that communicate among them (2-way) via MPI.

Find here information on how to [[setup a MOHID simulation using MPI]] and on [[compiling Mohid with MPI]].

== Parallel processing via OpenMP ==
Parallel processing using OpenMP is currently being implemented in Mohid by defining directives '''to optimize loops'''. These directives are defined as comments in the code and therefore need special compilation options. See more on [[compiling Mohid with OpenMP]]. Without special compilation these instructions are not considered in normal Fortran compilation.

OpenMP parallel processing can be used in '''multi core processors''' present in the same computer. It cannot be used to parallel processing using processors located in several computers in a cluster.

Loops optimization is introduced in a first phase in loops referring to grid variables (grid indexes, k, j, i) located in the Modifier section of MOHID modules.
Modifier loops are possibly used several times or involve a large resource allocation in MOHID simulations, hence these are locations with larger potential resource gains involved in parallelization.

In case of loops with several looping variables, the parallelized variable is chosen according with cost involved in the loop through this variable. E.g. in a 3D loop (k, j, i loop variables) if j dimension is much larger than the k (number of layers) or i then parallel processing is introduced in j variable, since the resource costs and the savings achieved with the parallelization are larger in this loop than in the others.

'''''Basic OpenMP syntax:'''''

In this section is provided an introduction with the basic concepts in OpenMP programming. The OpenMP language reference manual (http://www.openmp.org) should be consulted for further details.

OpenMP processing is made by a set of threads: the '''Master''' thread and the '''Workers''' threads.

OpenMP parallelization is accomplished by defining '''Parallel regions''' and by creating the threads (one for each available core): the processing inside a parallel region is divided between the Workers and the Master, which then proceed in parallel (simultaneously), instead of the unique thread existent in unparallel processing.

Each thread can have a set of '''Private variables''', which are affected only by this thread. The choice of the private variables, which are explicitly defined in programming, is a central part of the OpenMP programming. Should be made private all the variables whose values are altered by each thread processing and that affect other threads processing.

An obvious choice of these private variables are the loop variables when these are used to alter positions in matrixes or vectors in each iteration of the loop.

However it should be noted that in private variables the values are undefined in enter and exit of the parallel region and that by default these variables have no storage association with the variables outside the parallel region (this default behavior can, however, be altered by specific OpenMP directives).

The instructions in OpenMP are provided by a set of '''Directives'''. These refer to several actions such as the definition of Parallel regions, Work sharing and Data attributes. Directives are specific for the underlying programming language being used, either C or Fortran.

In Fortran directives are case insensitive. The basic syntax in as following:

sentinel directive [clause [[,] clause] ...]

'''Sentinel''' is !$OMP in either fixed or free format. The continuation of directives (from one code line to another) are according with the underlying language: & in Fortran case.

'''Clauses''' are used to specify additional information for the directive. Important clauses are:

- PRIVATE (private variables list);

- NOWAIT: specify that threads will not syncronize (i.e. wait for each other) at the end of a specific construct (e.g. a DO loop) within a parallel region; this is specified at the end of a parallel construct;

A parallel region is typically specified as:

!$OMP PARALLEL PRIVATE(PrivateVariable1,...,PrivateVariableN)

Inside a parallel region the work is distributed by the threads through '''Work Sharing constructs'''.

For every processing made inside a parallel region must be assigned a Work Sharing construct. If this is not verified execution errors can occur.

An important Work Sharing construct is the DO loop, specified as:

!$OMP DO
... (do loop in Fortran)
!$OMP END DO

In this construct the iterations of the enclosed Fortran DO loop are distributed by the threads.

The way the work is distributed over the threads is managed by the SCHEDULE clause of the DO construct. An important option of SCHEDULE is DYNAMIC: provided a Chunk number of iterations each thread will process this fixed number of iterations. After finishing a Chunk each thread will began another available Chunk till all iterations are completed:

!$OMP DO SCHEDULE(DYNAMIC, CHUNK)
...
!$OMP END DO

The distribution of Chunk among the threads requires synchronization for each assignment. This causes a overhead that could be important. The DYNAMIC option is advisable when each iteration involves an amount of work which is not predictable. This could be the case when IF constructs are present inside the loop containg extra processing done only in specific cases. Also if the threads arrive to the DO loop at different times, e.g. if they come from a previous DO loop with a NOWAIT end clause.

When the amount of work required in each DO loop iteration is predictable and the same for all threads it is advisable to use the STATIC option of SCHEDULE. This is particularly useful for DO loops which are unique in a parallel region:

!$OMP DO SCHEDULE(STATIC, CHUNK)
...
!$OMP END DO

Important characteristics of the Work Sharing constructs are that they do not create new threads, must contemplate all the existing threads or none at all and there is no barrier on entry (any available thread is not required to wait for the others) but a barrier on exit exists. The barrier on exit can be removed by the referred NOWAIT clause.

Work Sharing directives can appear outside the lexical extent of a parallel region (sequential lines of code appearing inside the parallel region). If they are dynamically linked with a parallel region (e.g. they appear in processing a subroutine call) they are orphaned. If, however, they appear outside this dynamic connection with a parallel region (and also not in the lexical extent of the region) they are ignored and the enclosed code is performed by one thread only and no parallelization is processed.

Inside a Work Sharing construct can be defined a critical region, if it is conveninent that only one thread at a time processes a specific code. This can be used to avoid problems with input/output operations potentially occuring by several threads processing at same time (problems can occur because memory locations are being assessed at same time). This is commanded as following:

!$OMP CRITICAL [Name]
... (code to be processed sequentially)
!$OMP END CRITICAL [Name]

In this case, no barriers exist in the entry and exit and all threads process the enclosed code although one at a time.
If more than one critial region is defined in the code then every critical region should have a different name or the execution outcome could become undeterminated.

Within a Work Sharing construct can also be specified that a portion of the code is processed only by one thread, e.g. to read information from a file common to all threads. This is done with the following syntax:

!$OMP SINGLE [clause ...]
... (code to be processed by only one thread)
!$OMP END SINGLE

When one wants this single thread to be the Master then following syntax is used:

!$OMP MASTER
... (code to be processed by only Master thread)
!$OMP END MASTER

As with CRITICAL, no barriers exist in SINGLE/MASTER in the entry and exit.

In these contructs the fact that no barrier exists at the exit may cause problems if the single thread processed code section is intended to be dealt with previously from the subsequent code. In this situation a barrier can be introduced at the end of the SINGLE/MASTER construct:

!$OMP BARRIER

Under this directive threads wait at the barrier point till all threads reach it.

'''''Parallelization overheads:'''''

Although parallelization involves potentially gains in computer resources use it also involves overheads which can sometimes be important.

The creation of the thread team (Workers) at the beginning of each parallel region is a source of such overheads. Because of this it is preferrable in each program/subrotine to be parallelized to create only one parallel region and then deal with the code that must be done only by one thread with OpenMP directives than to create several parallel regions.

Another source of overhead is the synchronization between threads in Work Sharing constructs.

These overheads cause that in some times the parallelization solution may not be performing better than the non parallelization solution, especially if the computational burden of the problem is low.

This is the case of DO constructs when each cycle iteration has a small computational burden (e.g. equaling one matrix's value to other matrix's value) and is to be expected in nested loops when the parallelized DO loop is inner.

Generally as the scale of the problem increases the overheads will be less significant and parallelization more advantageous.

In very small scale problems parallelization may provide more comsumption of computational resources than the non parallelization.

'''''Race condition:'''''

In some case execution of a parallel program may be suspended because different threads are competing for the actualization of some variable. This is called a race condition.

The programmer should take care to avoid this situation as its debugging is very difficult. Most commonly the execution will be suspended and the processors would become idle without any error message. When a vast part of the code is parallelized it is then almost impossible to find the cause of the error unless one reckons possible race condition situations.

A way to avoid this is to use critical regions or a REDUCTION clause.

[[Category:Programming]]

Parallel processing

2010-04-14T16:27:30Z

AngelaCanas: /* Parallel processing via OpenMP */

The historical need in numerical models to reduce computational time became a priority to the Mohid development team as an operational hydrodynamic and water quality model to the Tagus Estuary, in Lisbon, Portugal, was implemented using the [[Mohid Water]] model full capabilities. Thus, parallel processing has been implemented in Mohid Water in 2003, by using [http://www.mcs.anl.gov/mpi/mpich/ MPICH], a free portable implementation of [http://www.mcs.anl.gov/mpi/ MPI], the standard for message-passing libraries.

Currently, and due to the use of the new Intel Fortran compiler both Mohid Water and Mohid Land have parallelization features using [http://www.openmp.org/ OpenMP].

== Parallel processing via MPI ==
The [[Mohid Water]] ability to run [[nested models]] was accomplished by creating a linked list of all the models and by attributing to each one a father-son identification, through which the models communicate. The first stage for introducing parallel processing in Mohid was to add the possibility of launching a process by each model to run, and then, using MPICH, establish communication between models. This enables each sub-model to run in a different processor (even if the processor belongs to a different computer, as long as it is in the same network) and in parallel, instead of running all in the same processor and each model having to wait for the others to perform their calculations.

Parallel processing as it is presently implemented in Mohid, could not be achieved without [[object oriented programming]] philosophy, as each model is an instance of [[class Model]] and no changes, exception made to the implementation of the MPI communications calls needed to be added. Using this feature, computational speed was improved (varying from application to application), as now the whole model will take the same time as the slowest model to run plus the time to communicate with the other processes. Here, the network communication speed plays an important role, as it can become limiting. However, the amount of information passing between models, depending of course on the memory allocated for each model, has not yet proven to be big enough to make a 100 Mbps network connection time limiting.

[[Domain Composition]] is an ongoing project, in a very early stage, aimed at decoupling a domain into several subdomain that communicate among them (2-way) via MPI.

Find here information on how to [[setup a MOHID simulation using MPI]] and on [[compiling Mohid with MPI]].

== Parallel processing via OpenMP ==
Parallel processing using OpenMP is currently being implemented in Mohid by defining directives '''to optimize loops'''. These directives are defined as comments in the code and therefore need special compilation options. See more on [[compiling Mohid with OpenMP]]. Without special compilation these instructions are not considered in normal Fortran compilation.

OpenMP parallel processing can be used in '''multi core processors''' present in the same computer. It cannot be used to parallel processing using processors located in several computers in a cluster.

Loops optimization is introduced in a first phase in loops referring to grid variables (grid indexes, k, j, i) located in the Modifier section of MOHID modules.
Modifier loops are possibly used several times or involve a large resource allocation in MOHID simulations, hence these are locations with larger potential resource gains involved in parallelization.

In case of loops with several looping variables, the parallelized variable is chosen according with cost involved in the loop through this variable. E.g. in a 3D loop (k, j, i loop variables) if j dimension is much larger than the k (number of layers) or i then parallel processing is introduced in j variable, since the resource costs and the savings achieved with the parallelization are larger in this loop than in the others.

'''''Basic OpenMP syntax:'''''

In this section is provided an introduction with the basic concepts in OpenMP programming. The OpenMP language reference manual (http://www.openmp.org) should be consulted for further details.

OpenMP processing is made by a set of threads: the '''Master''' thread and the '''Workers''' threads.

OpenMP parallelization is accomplished by defining '''Parallel regions''' and by creating the threads (one for each available core): the processing inside a parallel region is divided between the Workers and the Master, which then proceed in parallel (simultaneously), instead of the unique thread existent in unparallel processing.

Each thread can have a set of '''Private variables''', which are affected only by this thread. The choice of the private variables, which are explicitly defined in programming, is a central part of the OpenMP programming. Should be made private all the variables whose values are altered by each thread processing and that affect other threads processing.

An obvious choice of these private variables are the loop variables when these are used to alter positions in matrixes or vectors in each iteration of the loop.

However it should be noted that in private variables the values are undefined in enter and exit of the parallel region and that by default these variables have no storage association with the variables outside the parallel region (this default behavior can, however, be altered by specific OpenMP directives).

The instructions in OpenMP are provided by a set of '''Directives'''. These refer to several actions such as the definition of Parallel regions, Work sharing and Data attributes. Directives are specific for the underlying programming language being used, either C or Fortran.

In Fortran directives are case insensitive. The basic syntax in as following:

sentinel directive [clause [[,] clause] ...]

'''Sentinel''' is !$OMP in either fixed or free format. The continuation of directives (from one code line to another) are according with the underlying language: & in Fortran case.

'''Clauses''' are used to specify additional information for the directive. Important clauses are:

- PRIVATE (private variables list);

- NOWAIT: specify that threads will not syncronize (i.e. wait for each other) at the end of a specific construct (e.g. a DO loop) within a parallel region; this is specified at the end of a parallel construct;

A parallel region is typically specified as:

!$OMP PARALLEL PRIVATE(PrivateVariable1,...,PrivateVariableN)

Inside a parallel region the work is distributed by the threads through '''Work Sharing constructs'''.

For every processing made inside a parallel region must be assigned a Work Sharing construct. If this is not verified execution errors can occur.

An important Work Sharing construct is the DO loop, specified as:

!$OMP DO
... (do loop in Fortran)
!$OMP END DO

In this construct the iterations of the enclosed Fortran DO loop are distributed by the threads.

The way the work is distributed over the threads is managed by the SCHEDULE clause of the DO construct. An important option of SCHEDULE is DYNAMIC: provided a Chunk number of iterations each thread will process this fixed number of iterations. After finishing a Chunk each thread will began another available Chunk till all iterations are completed:

!$OMP DO SCHEDULE(DYNAMIC, CHUNK)
...
!$OMP END DO

The distribution of Chunk among the threads requires synchronization for each assignment. This causes a overhead that could be important. The DYNAMIC option is advisable when each iteration involves an amount of work which is not predictable. This could be the case when IF constructs are present inside the loop containg extra processing done only in specific cases. Also if the threads arrive to the DO loop at different times, e.g. if they come from a previous DO loop with a NOWAIT end clause.

When the amount of work required in each DO loop iteration is predictable and the same for all threads it is advisable to use the STATIC option of SCHEDULE. This is particularly useful for DO loops which are unique in a parallel region:

!$OMP DO SCHEDULE(STATIC, CHUNK)
...
!$OMP END DO

Important characteristics of the Work Sharing constructs are that they do not create new threads, must contemplate all the existing threads or none at all and there is no barrier on entry (any available thread is not required to wait for the others) but a barrier on exit exists. The barrier on exit can be removed by the referred NOWAIT clause.

Work Sharing directives can appear outside the lexical extent of a parallel region (sequential lines of code appearing inside the parallel region). If they are dynamically linked with a parallel region (e.g. they appear in processing a subroutine call) they are orphaned. If, however, they appear outside this dynamic connection with a parallel region (and also not in the lexical extent of the region) they are ignored and the enclosed code is performed by one thread only and no parallelization is processed.

Inside a Work Sharing construct can be defined a critical region, if it is conveninent that only one thread at a time processes a specific code. This can be used to avoid problems with input/output operations potentially occuring by several threads processing at same time (problems can occur because memory locations are being assessed at same time). This is commanded as following:

!$OMP CRITICAL [Name]
... (code to be processed sequentially)
!$OMP END CRITICAL [Name]

In this case, no barriers exist in the entry and exit and all threads process the enclosed code although one at a time.
If more than one critial region is defined in the code then every critical region should have a different name or the execution outcome could become undeterminated.

Within a Work Sharing construct can also be specified that a portion of the code is processed only by one thread, e.g. to read information from a file common to all threads. This is done with the following syntax:

!$OMP SINGLE [clause ...]
... (code to be processed by only one thread)
!$OMP END SINGLE

When one wants this single thread to be the Master then following syntax is used:

!$OMP MASTER
... (code to be processed by only Master thread)
!$OMP END MASTER

As with CRITICAL, no barriers exist in SINGLE/MASTER in the entry and exit.

In these contructs the fact that no barrier exists at the exit may cause problems if the single thread processed code section is intended to be dealt with previously from the subsequent code. In this situation a barrier can be introduced at the end of the SINGLE/MASTER construct:

!$OMP BARRIER

Under this directive threads wait at the barrier point till all threads reach it.

'''''Parallelization overheads:'''''

Although parallelization involves potentially gains in computer resources use it also involves overheads which can sometimes be important.

The creation of the thread team (Workers) at the beginning of each parallel region is a source of such overheads. Because of this it is preferrable in each program/subrotine to be parallelized to create only one parallel region and then deal with the code that must be done only by one thread with OpenMP directives than to create several parallel regions.

Another source of overhead is the synchronization between threads in Work Sharing constructs.

These overheads cause that in some times the parallelization solution may not be performing better than the non parallelization solution, especially if the computational burden of the problem is low.

This is the case of DO constructs when each cycle iteration has a small computational burden (e.g. equaling one matrix's value to other matrix's value) and is to be expected in nested loops when the parallelized DO loop is inner.

Generally as the scale of the problem increases the overheads will be less significant and parallelization more advantageous.

In very small scale problems parallelization may provide more comsumption of computational resources than the non parallelization.

'''''Race condition:'''''

In some case execution of a parallel program may be suspended because different threads are competing for the actualization of some variable. This is called a race condition.

The programmer should take care to avoid this situation as its debugging is very difficult. A way to avoid this is to use critical regions or a REDUCTION clause.

It is also convenient to use barriers to make the threads available inside parallel regions when they are required. This may be required when processing inside a parallel region is restringed in some part only to some threads, e.g., through the !$OMP MASTER directive.

[[Category:Programming]]

Parallel processing

2010-04-14T15:58:58Z

AngelaCanas: /* Parallel processing via OpenMP */

The historical need in numerical models to reduce computational time became a priority to the Mohid development team as an operational hydrodynamic and water quality model to the Tagus Estuary, in Lisbon, Portugal, was implemented using the [[Mohid Water]] model full capabilities. Thus, parallel processing has been implemented in Mohid Water in 2003, by using [http://www.mcs.anl.gov/mpi/mpich/ MPICH], a free portable implementation of [http://www.mcs.anl.gov/mpi/ MPI], the standard for message-passing libraries.

Currently, and due to the use of the new Intel Fortran compiler both Mohid Water and Mohid Land have parallelization features using [http://www.openmp.org/ OpenMP].

== Parallel processing via MPI ==
The [[Mohid Water]] ability to run [[nested models]] was accomplished by creating a linked list of all the models and by attributing to each one a father-son identification, through which the models communicate. The first stage for introducing parallel processing in Mohid was to add the possibility of launching a process by each model to run, and then, using MPICH, establish communication between models. This enables each sub-model to run in a different processor (even if the processor belongs to a different computer, as long as it is in the same network) and in parallel, instead of running all in the same processor and each model having to wait for the others to perform their calculations.

Parallel processing as it is presently implemented in Mohid, could not be achieved without [[object oriented programming]] philosophy, as each model is an instance of [[class Model]] and no changes, exception made to the implementation of the MPI communications calls needed to be added. Using this feature, computational speed was improved (varying from application to application), as now the whole model will take the same time as the slowest model to run plus the time to communicate with the other processes. Here, the network communication speed plays an important role, as it can become limiting. However, the amount of information passing between models, depending of course on the memory allocated for each model, has not yet proven to be big enough to make a 100 Mbps network connection time limiting.

[[Domain Composition]] is an ongoing project, in a very early stage, aimed at decoupling a domain into several subdomain that communicate among them (2-way) via MPI.

Find here information on how to [[setup a MOHID simulation using MPI]] and on [[compiling Mohid with MPI]].

== Parallel processing via OpenMP ==
Parallel processing using OpenMP is currently being implemented in Mohid by defining directives '''to optimize loops'''. These directives are defined as comments in the code and therefore need special compilation options. See more on [[compiling Mohid with OpenMP]]. Without special compilation these instructions are not considered in normal Fortran compilation.

OpenMP parallel processing can be used in '''multi core processors''' present in the same computer. It cannot be used to parallel processing using processors located in several computers in a cluster.

Loops optimization is introduced in a first phase in loops referring to grid variables (grid indexes, k, j, i) located in the Modifier section of MOHID modules.
Modifier loops are possibly used several times or involve a large resource allocation in MOHID simulations, hence these are locations with larger potential resource gains involved in parallelization.

In case of loops with several looping variables, the parallelized variable is chosen according with cost involved in the loop through this variable. E.g. in a 3D loop (k, j, i loop variables) if j dimension is much larger than the k (number of layers) or i then parallel processing is introduced in j variable, since the resource costs and the savings achieved with the parallelization are larger in this loop than in the others.

'''''Basic OpenMP syntax:'''''

In this section is provided an introduction with the basic concepts in OpenMP programming. The OpenMP language reference manual (http://www.openmp.org) should be consulted for further details.

OpenMP processing is made by a set of threads: the '''Master''' thread and the '''Workers''' threads.

OpenMP parallelization is accomplished by defining '''Parallel regions''' and by creating the threads (one for each available core): the processing inside a parallel region is divided between the Workers and the Master, which then proceed in parallel (simultaneously), instead of the unique thread existent in unparallel processing.

Each thread can have a set of '''Private variables''', which are affected only by this thread. The choice of the private variables, which are explicitly defined in programming, is a central part of the OpenMP programming. Should be made private all the variables whose values are altered by each thread processing and that affect other threads processing.

An obvious choice of these private variables are the loop variables when these are used to alter positions in matrixes or vectors in each iteration of the loop.

However it should be noted that in private variables the values are undefined in enter and exit of the parallel region and that by default these variables have no storage association with the variables outside the parallel region (this default behavior can, however, be altered by specific OpenMP directives).

The instructions in OpenMP are provided by a set of '''Directives'''. These refer to several actions such as the definition of Parallel regions, Work sharing and Data attributes. Directives are specific for the underlying programming language being used, either C or Fortran.

In Fortran directives are case insensitive. The basic syntax in as following:

sentinel directive [clause [[,] clause] ...]

'''Sentinel''' is !$OMP in either fixed or free format. The continuation of directives (from one code line to another) are according with the underlying language: & in Fortran case.

'''Clauses''' are used to specify additional information for the directive. Important clauses are:

- PRIVATE (private variables list);

- NOWAIT: specify that threads will not syncronize (i.e. wait for each other) at the end of a specific construct (e.g. a DO loop) within a parallel region; this is specified at the end of a parallel construct;

A parallel region is typically specified as:

!$OMP PARALLEL PRIVATE(PrivateVariable1,...,PrivateVariableN)

Inside a parallel region the work is distributed by the threads through '''Work Sharing constructs'''.

For every processing made inside a parallel region must be assigned a Work Sharing construct. If this is not verified execution errors can occur.

An important Work Sharing construct is the DO loop, specified as:

!$OMP DO
... (do loop in Fortran)
!$OMP END DO

In this construct the iterations of the enclosed Fortran DO loop are distributed by the threads.

The way the work is distributed over the threads is managed by the SCHEDULE clause of the DO construct. An important option of SCHEDULE is DYNAMIC: provided a Chunk number of iterations each thread will process this fixed number of iterations. After finishing a Chunk each thread will began another available Chunk till all iterations are completed:

!$OMP DO SCHEDULE(DYNAMIC, CHUNK)
...
!$OMP END DO

The distribution of Chunk among the threads requires synchronization for each assignment. This causes a overhead that could be important. The DYNAMIC option is advisable when each iteration involves an amount of work which is not predictable. This could be the case when IF constructs are present inside the loop containg extra processing done only in specific cases. Also if the threads arrive to the DO loop at different times, e.g. if they come from a previous DO loop with a NOWAIT end clause.

When the amount of work required in each DO loop iteration is predictable and the same for all threads it is advisable to use the STATIC option of SCHEDULE. This is particularly useful for DO loops which are unique in a parallel region:

!$OMP DO SCHEDULE(STATIC, CHUNK)
...
!$OMP END DO

Important characteristics of the Work Sharing constructs are that they do not create new threads, must contemplate all the existing threads or none at all and there is no barrier on entry (any available thread is not required to wait for the others) but a barrier on exit exists. The barrier on exit can be removed by the referred NOWAIT clause.

Work Sharing directives can appear outside the lexical extent of a parallel region (sequential lines of code appearing inside the parallel region). If they are dynamically linked with a parallel region (e.g. they appear in processing a subroutine call) they are orphaned. If, however, they appear outside this dynamic connection with a parallel region (and also not in the lexical extent of the region) they are ignored and the enclosed code is performed by one thread only and no parallelization is processed.

Inside a Work Sharing construct can be defined a critical region, if it is conveninent that only one thread at a time processes a specific code. This can be used to avoid problems with input/output operations potentially occuring by several threads processing at same time (problems can occur because memory locations are being assessed at same time). This is commanded as following:

!$OMP CRITICAL [Name]
... (code to be processed sequentially)
!$OMP END CRITICAL [Name]

In this case, no barriers exist in the entry and exit and all threads process the enclosed code although one at a time.
If more than one critial region is defined in the code then every critical region should have a different name or the execution outcome could become undeterminated.

Within a Work Sharing construct can also be specified that a portion of the code is processed only by one thread, e.g. to read information from a file common to all threads. This is done with the following syntax:

!$OMP SINGLE [clause ...]
... (code to be processed by only one thread)
!$OMP END SINGLE

When one wants this single thread to be the Master then following syntax is used:

!$OMP MASTER
... (code to be processed by only Master thread)
!$OMP END MASTER

As with CRITICAL, no barriers exist in SINGLE/MASTER in the entry and exit.

In these contructs the fact that no barrier exists at the exit may cause problems if the single thread processed code section is intended to be dealt with previously from the subsequent code. In this situation a barrier can be introduced at the end of the SINGLE/MASTER construct:

!$OMP BARRIER

Under this directive threads wait at the barrier point till all threads reach it.

'''''Parallelization overheads:'''''

Although parallelization involves potentially gains in computer resources use it also involves overheads which can sometimes be important.

The creation of the thread team (Workers) at the beginning of each parallel region is a source of such overheads. Because of this it is preferrable in each program/subrotine to be parallelized to create only one parallel region and then deal with the code that must be done only by one thread with OpenMP directives than to create several parallel regions.

Another source of overhead is the synchronization between threads in Work Sharing constructs.

These overheads cause that in some times the parallelization solution may not be performing better than the non parallelization solution, especially if the computational burden of the problem is low.

This is the case of DO constructs when each cycle iteration has a small computational burden (e.g. equaling one matrix's value to other matrix's value) and is to be expected in nested loops when the parallelized DO loop is inner.

Generally as the scale of the problem increases the overheads will be less significant and parallelization more advantageous.

In very small scale problems parallelization may provide more comsumption of computational resources than the non parallelization.

'''''Race condition:'''''

In some case execution of a parallel program may be suspended because different threads are competing for the actualization of some variable. This is called a race condition.

The programmer should take care to avoid this situation as its debugging is very difficult. A way to avoid this is to use critical regions or a REDUCTION clause.

[[Category:Programming]]

Parallel processing

2010-04-14T15:34:19Z

AngelaCanas: /* Parallel processing via OpenMP */

The historical need in numerical models to reduce computational time became a priority to the Mohid development team as an operational hydrodynamic and water quality model to the Tagus Estuary, in Lisbon, Portugal, was implemented using the [[Mohid Water]] model full capabilities. Thus, parallel processing has been implemented in Mohid Water in 2003, by using [http://www.mcs.anl.gov/mpi/mpich/ MPICH], a free portable implementation of [http://www.mcs.anl.gov/mpi/ MPI], the standard for message-passing libraries.

Currently, and due to the use of the new Intel Fortran compiler both Mohid Water and Mohid Land have parallelization features using [http://www.openmp.org/ OpenMP].

== Parallel processing via MPI ==
The [[Mohid Water]] ability to run [[nested models]] was accomplished by creating a linked list of all the models and by attributing to each one a father-son identification, through which the models communicate. The first stage for introducing parallel processing in Mohid was to add the possibility of launching a process by each model to run, and then, using MPICH, establish communication between models. This enables each sub-model to run in a different processor (even if the processor belongs to a different computer, as long as it is in the same network) and in parallel, instead of running all in the same processor and each model having to wait for the others to perform their calculations.

Parallel processing as it is presently implemented in Mohid, could not be achieved without [[object oriented programming]] philosophy, as each model is an instance of [[class Model]] and no changes, exception made to the implementation of the MPI communications calls needed to be added. Using this feature, computational speed was improved (varying from application to application), as now the whole model will take the same time as the slowest model to run plus the time to communicate with the other processes. Here, the network communication speed plays an important role, as it can become limiting. However, the amount of information passing between models, depending of course on the memory allocated for each model, has not yet proven to be big enough to make a 100 Mbps network connection time limiting.

[[Domain Composition]] is an ongoing project, in a very early stage, aimed at decoupling a domain into several subdomain that communicate among them (2-way) via MPI.

Find here information on how to [[setup a MOHID simulation using MPI]] and on [[compiling Mohid with MPI]].

== Parallel processing via OpenMP ==
Parallel processing using OpenMP is currently being implemented in Mohid by defining directives '''to optimize loops'''. These directives are defined as comments in the code and therefore need special compilation options. See more on [[compiling Mohid with OpenMP]]. Without special compilation these instructions are not considered in normal Fortran compilation.

OpenMP parallel processing can be used in '''multi core processors''' present in the same computer. It cannot be used to parallel processing using processors located in several computers in a cluster.

Loops optimization is introduced in a first phase in loops referring to grid variables (grid indexes, k, j, i) located in the Modifier section of MOHID modules.
Modifier loops are possibly used several times or involve a large resource allocation in MOHID simulations, hence these are locations with larger potential resource gains involved in parallelization.

In case of loops with several looping variables, the parallelized variable is chosen according with cost involved in the loop through this variable. E.g. in a 3D loop (k, j, i loop variables) if j dimension is much larger than the k (number of layers) or i then parallel processing is introduced in j variable, since the resource costs and the savings achieved with the parallelization are larger in this loop than in the others.

'''''Basic OpenMP syntax:'''''

In this section is provided an introduction with the basic concepts in OpenMP programming. The OpenMP language reference manual (http://www.openmp.org) should be consulted for further details.

OpenMP processing is made by a set of threads: the '''Master''' thread and the '''Workers''' threads.

OpenMP parallelization is accomplished by defining '''Parallel regions''' and by creating the threads (one for each available core): the processing inside a parallel region is divided between the Workers and the Master, which then proceed in parallel (simultaneously), instead of the unique thread existent in unparallel processing.

Each thread can have a set of '''Private variables''', which are affected only by this thread. The choice of the private variables, which are explicitly defined in programming, is a central part of the OpenMP programming. Should be made private all the variables whose values are altered by each thread processing and that affect other threads processing.

An obvious choice of these private variables are the loop variables when these are used to alter positions in matrixes or vectors in each iteration of the loop.

However it should be noted that in private variables the values are undefined in enter and exit of the parallel region and that by default these variables have no storage association with the variables outside the parallel region (this default behavior can, however, be altered by specific OpenMP directives).

The instructions in OpenMP are provided by a set of '''Directives'''. These refer to several actions such as the definition of Parallel regions, Work sharing and Data attributes. Directives are specific for the underlying programming language being used, either C or Fortran.

In Fortran directives are case insensitive. The basic syntax in as following:

sentinel directive [clause [[,] clause] ...]

'''Sentinel''' is !$OMP in either fixed or free format. The continuation of directives (from one code line to another) are according with the underlying language: & in Fortran case.

'''Clauses''' are used to specify additional information for the directive. Important clauses are:

- PRIVATE (private variables list);

- NOWAIT: specify that threads will not syncronize (i.e. wait for each other) at the end of a specific construct (e.g. a DO loop) within a parallel region; this is specified at the end of a parallel construct;

A parallel region is typically specified as:

!$OMP PARALLEL PRIVATE(PrivateVariable1,...,PrivateVariableN)

Inside a parallel region the work is distributed by the threads through '''Work Sharing constructs'''.

For every processing made inside a parallel region must be assigned a Work Sharing construct. If this is not verified execution errors can occur.

An important Work Sharing construct is the DO loop, specified as:

!$OMP DO
... (do loop in Fortran)
!$OMP END DO

In this construct the iterations of the enclosed Fortran DO loop are distributed by the threads.

The way the work is distributed over the threads is managed by the SCHEDULE clause of the DO construct. An important option of SCHEDULE is DYNAMIC: provided a Chunk number of iterations each thread will process this fixed number of iterations. After finishing a Chunk each thread will began another available Chunk till all iterations are completed:

!$OMP DO SCHEDULE(DYNAMIC, CHUNK)
...
!$OMP END DO

The distribution of Chunk among the threads requires synchronization for each assignment. This causes a overhead that could be important. The DYNAMIC option is advisable when each iteration involves an amount of work which is not predictable. This could be the case when IF constructs are present inside the loop containg extra processing done only in specific cases. Also if the threads arrive to the DO loop at different times, e.g. if they come from a previous DO loop with a NOWAIT end clause.

When the amount of work required in each DO loop iteration is predictable and the same for all threads it is advisable to use the STATIC option of SCHEDULE. This is particularly useful for DO loops which are unique in a parallel region:

!$OMP DO SCHEDULE(STATIC, CHUNK)
...
!$OMP END DO

Important characteristics of the Work Sharing constructs are that they do not create new threads, must contemplate all the existing threads or none at all and there is no barrier on entry (any available thread is not required to wait for the others) but a barrier on exit exists. The barrier on exit can be removed by the referred NOWAIT clause.

Work Sharing directives can appear outside the lexical extent of a parallel region (sequential lines of code appearing inside the parallel region). If they are dynamically linked with a parallel region (e.g. they appear in processing a subroutine call) they are orphaned. If, however, they appear outside this dynamic connection with a parallel region (and also not in the lexical extent of the region) they are ignored and the enclosed code is performed by one thread only and no parallelization is processed.

Inside a Work Sharing construct can be defined a critical region, if it is conveninent that only one thread at a time processes a specific code. This can be used to avoid problems with input/output operations potentially occuring by several threads processing at same time (problems can occur because memory locations are being assessed at same time). This is commanded as following:

!$OMP CRITICAL [Name]
... (code to be processed sequentially)
!$OMP END CRITICAL [Name]

In this case, no barriers exist in the entry and exit and all threads process the enclosed code although one at a time.

Within a Work Sharing construct can also be specified that a portion of the code is processed only by one thread, e.g. to read information from a file common to all threads. This is done with the following syntax:

!$OMP SINGLE [clause ...]
... (code to be processed by only one thread)
!$OMP END SINGLE

When one wants this single thread to be the Master then following syntax is used:

!$OMP MASTER
... (code to be processed by only Master thread)
!$OMP END MASTER

As with CRITICAL, no barriers exist in SINGLE/MASTER in the entry and exit.

In these contructs the fact that no barrier exists at the exit may cause problems if the single thread processed code section is intended to be dealt with previously from the subsequent code. In this situation a barrier can be introduced at the end of the SINGLE/MASTER construct:

!$OMP BARRIER

Under this directive threads wait at the barrier point till all threads reach it.

'''''Parallelization overheads:'''''

Although parallelization involves potentially gains in computer resources use it also involves overheads which can sometimes be important.

The creation of the thread team (Workers) at the beginning of each parallel region is a source of such overheads. Because of this it is preferrable in each program/subrotine to be parallelized to create only one parallel region and then deal with the code that must be done only by one thread with OpenMP directives than to create several parallel regions.

Another source of overhead is the synchronization between threads in Work Sharing constructs.

These overheads cause that in some times the parallelization solution may not be performing better than the non parallelization solution, especially if the computational burden of the problem is low.

This is the case of DO constructs when each cycle iteration has a small computational burden (e.g. equaling one matrix's value to other matrix's value) and is to be expected in nested loops when the parallelized DO loop is inner.

Generally as the scale of the problem increases the overheads will be less significant and parallelization more advantageous.

In very small scale problems parallelization may provide more comsumption of computational resources than the non parallelization.

'''''Race condition:'''''

In some case execution of a parallel program

[[Category:Programming]]

Parallel processing

2010-04-14T15:24:41Z

AngelaCanas: /* Parallel processing via OpenMP */

The historical need in numerical models to reduce computational time became a priority to the Mohid development team as an operational hydrodynamic and water quality model to the Tagus Estuary, in Lisbon, Portugal, was implemented using the [[Mohid Water]] model full capabilities. Thus, parallel processing has been implemented in Mohid Water in 2003, by using [http://www.mcs.anl.gov/mpi/mpich/ MPICH], a free portable implementation of [http://www.mcs.anl.gov/mpi/ MPI], the standard for message-passing libraries.

Currently, and due to the use of the new Intel Fortran compiler both Mohid Water and Mohid Land have parallelization features using [http://www.openmp.org/ OpenMP].

== Parallel processing via MPI ==
The [[Mohid Water]] ability to run [[nested models]] was accomplished by creating a linked list of all the models and by attributing to each one a father-son identification, through which the models communicate. The first stage for introducing parallel processing in Mohid was to add the possibility of launching a process by each model to run, and then, using MPICH, establish communication between models. This enables each sub-model to run in a different processor (even if the processor belongs to a different computer, as long as it is in the same network) and in parallel, instead of running all in the same processor and each model having to wait for the others to perform their calculations.

Parallel processing as it is presently implemented in Mohid, could not be achieved without [[object oriented programming]] philosophy, as each model is an instance of [[class Model]] and no changes, exception made to the implementation of the MPI communications calls needed to be added. Using this feature, computational speed was improved (varying from application to application), as now the whole model will take the same time as the slowest model to run plus the time to communicate with the other processes. Here, the network communication speed plays an important role, as it can become limiting. However, the amount of information passing between models, depending of course on the memory allocated for each model, has not yet proven to be big enough to make a 100 Mbps network connection time limiting.

[[Domain Composition]] is an ongoing project, in a very early stage, aimed at decoupling a domain into several subdomain that communicate among them (2-way) via MPI.

Find here information on how to [[setup a MOHID simulation using MPI]] and on [[compiling Mohid with MPI]].

== Parallel processing via OpenMP ==
Parallel processing using OpenMP is currently being implemented in Mohid by defining directives '''to optimize loops'''. These directives are defined as comments in the code and therefore need special compilation options. See more on [[compiling Mohid with OpenMP]]. Without special compilation these instructions are not considered in normal Fortran compilation.

OpenMP parallel processing can be used in '''multi core processors''' present in the same computer. It cannot be used to parallel processing using processors located in several computers in a cluster.

Loops optimization is introduced in a first phase in loops referring to grid variables (grid indexes, k, j, i) located in the Modifier section of MOHID modules.
Modifier loops are possibly used several times or involve a large resource allocation in MOHID simulations, hence these are locations with larger potential resource gains involved in parallelization.

In case of loops with several looping variables, the parallelized variable is chosen according with cost involved in the loop through this variable. E.g. in a 3D loop (k, j, i loop variables) if j dimension is much larger than the k (number of layers) or i then parallel processing is introduced in j variable, since the resource costs and the savings achieved with the parallelization are larger in this loop than in the others.

'''''Basic OpenMP syntax:'''''

In this section is provided an introduction with the basic concepts in OpenMP programming. The OpenMP language reference manual (http://www.openmp.org) should be consulted for further details.

OpenMP processing is made by a set of threads: the '''Master''' thread and the '''Workers''' threads.

OpenMP parallelization is accomplished by defining '''Parallel regions''' and by creating the threads (one for each available core): the processing inside a parallel region is divided between the Workers and the Master, which then proceed in parallel (simultaneously), instead of the unique thread existent in unparallel processing.

Each thread can have a set of '''Private variables''', which are affected only by this thread. The choice of the private variables, which are explicitly defined in programming, is a central part of the OpenMP programming. Should be made private all the variables whose values are altered by each thread processing and that affect other threads processing.

An obvious choice of these private variables are the loop variables when these are used to alter positions in matrixes or vectors in each iteration of the loop.

However it should be noted that in private variables the values are undefined in enter and exit of the parallel region and that by default these variables have no storage association with the variables outside the parallel region (this default behavior can, however, be altered by specific OpenMP directives).

The instructions in OpenMP are provided by a set of '''Directives'''. These refer to several actions such as the definition of Parallel regions, Work sharing and Data attributes. Directives are specific for the underlying programming language being used, either C or Fortran.

In Fortran directives are case insensitive. The basic syntax in as following:

sentinel directive [clause [[,] clause] ...]

'''Sentinel''' is !$OMP in either fixed or free format. The continuation of directives (from one code line to another) are according with the underlying language: & in Fortran case.

'''Clauses''' are used to specify additional information for the directive. Important clauses are:

- PRIVATE (private variables list);

- NOWAIT: specify that threads will not syncronize (i.e. wait for each other) at the end of a specific construct (e.g. a DO loop) within a parallel region; this is specified at the end of a parallel construct;

A parallel region is typically specified as:

!$OMP PARALLEL PRIVATE(PrivateVariable1,...,PrivateVariableN)

Inside a parallel region the work is distributed by the threads through '''Work Sharing constructs'''.

For every processing made inside a parallel region must be assigned a Work Sharing construct. If this is not verified execution errors can occur.

An important Work Sharing construct is the DO loop, specified as:

!$OMP DO
... (do loop in Fortran)
!$OMP END DO

In this construct the iterations of the enclosed Fortran DO loop are distributed by the threads.

The way the work is distributed over the threads is managed by the SCHEDULE clause of the DO construct. An important option of SCHEDULE is DYNAMIC: provided a Chunk number of iterations each thread will process this fixed number of iterations. After finishing a Chunk each thread will began another available Chunk till all iterations are completed:

!$OMP DO SCHEDULE(DYNAMIC, CHUNK)
...
!$OMP END DO

The distribution of Chunk among the threads requires synchronization for each assignment. This causes a overhead that could be important. The DYNAMIC option is advisable when each iteration involves an amount of work which is not predictable. This could be the case when IF constructs are present inside the loop containg extra processing done only in specific cases. Also if the threads arrive to the DO loop at different times, e.g. if they come from a previous DO loop with a NOWAIT end clause.

When the amount of work required in each DO loop iteration is predictable and the same for all threads it is advisable to use the STATIC option of SCHEDULE. This is particularly useful for DO loops which are unique in a parallel region:

!$OMP DO SCHEDULE(STATIC, CHUNK)
...
!$OMP END DO

Important characteristics of the Work Sharing constructs are that they do not create new threads, must contemplate all the existing threads or none at all and there is no barrier on entry (any available thread is not required to wait for the others) but a barrier on exit exists. The barrier on exit can be removed by the referred NOWAIT clause.

Work Sharing directives can appear outside the lexical extent of a parallel region (sequential lines of code appearing inside the parallel region). If they are dynamically linked with a parallel region (e.g. they appear in processing a subroutine call) they are orphaned. If, however, they appear outside this dynamic connection with a parallel region (and also not in the lexical extent of the region) they are ignored and the enclosed code is performed by one thread only and no parallelization is processed.

Inside a Work Sharing construct can be defined a critical region, if it is conveninent that only one thread at a time processes a specific code. This can be used to avoid problems with input/output operations potentially occuring by several threads processing at same time (problems can occur because memory locations are being assessed at same time). This is commanded as following:

!$OMP CRITICAL [Name]
... (code to be processed sequentially)
!$OMP END CRITICAL [Name]

In this case, no barriers exist in the entry and exit and all threads process the enclosed code although one at a time.

Within a Work Sharing construct can also be specified that a portion of the code is processed only by one thread, e.g. to read information from a file common to all threads. This is done with the following syntax:

!$OMP SINGLE [clause ...]
... (code to be processed by only one thread)
!$OMP END SINGLE

When one wants this single thread to be the Master then following syntax is used:

!$OMP MASTER
... (code to be processed by only Master thread)
!$OMP END MASTER

As with CRITICAL, no barriers exist in SINGLE/MASTER in the entry and exit.

In these contructs the fact that no barrier exists at the exit may cause problems if the single thread processed code section is intended to be dealt with previously from the subsequent code. In this situation a barrier can be introduced at the end of the SINGLE/MASTER construct:

!$OMP BARRIER

Under this directive threads wait at the barrier point till all threads reach it.

'''''Parallelization overheads:'''''

Although parallelization involves potentially gains in computer resources use it also involves overheads which can sometimes be important.

The creation of the thread team (Workers) at the beginning of each parallel region is a source of such overheads. Because of this it is preferrable in each program/subrotine to be parallelized to create only one parallel region and then deal with the code that must be done only by one thread with OpenMP directives than to create several parallel regions.

Another source of overhead is the synchronization between threads in Work Sharing constructs.

These overheads cause that in some times the parallelization solution may not be performing better than the non parallelization solution, especially if the computational burden of the problem is low.

This is the case of DO constructs when each cycle iteration has a small computational burden (e.g. equaling one matrix's value to other matrix's value) and is to be expected in nested loops when the parallelized DO loop is inner.

Generally as the scale of the problem increases the overheads will be less significant and parallelization more advantageous.

In very small scale problems parallelization may provide more comsumption of computational resources than the non parallelization.

'''''Race condition:'''''

[[Category:Programming]]

Parallel processing

2009-06-02T16:51:11Z

AngelaCanas: /* Parallel processing via OpenMP */

The historical need in numerical models to reduce computational time became a priority to the Mohid development team as an operational hydrodynamic and water quality model to the Tagus Estuary, in Lisbon, Portugal, was implemented using the [[Mohid Water]] model full capabilities. Thus, parallel processing has been implemented in Mohid Water in 2003, by using [http://www.mcs.anl.gov/mpi/mpich/ MPICH], a free portable implementation of [http://www.mcs.anl.gov/mpi/ MPI], the standard for message-passing libraries.

Currently, and due to the use of the new Intel Fortran compiler both Mohid Water and Mohid Land have parallelization features using [http://www.openmp.org/ OpenMP].

== Parallel processing via MPI ==
The [[Mohid Water]] ability to run [[nested models]] was accomplished by creating a linked list of all the models and by attributing to each one a father-son identification, through which the models communicate. The first stage for introducing parallel processing in Mohid was to add the possibility of launching a process by each model to run, and then, using MPICH, establish communication between models. This enables each sub-model to run in a different processor (even if the processor belongs to a different computer, as long as it is in the same network) and in parallel, instead of running all in the same processor and each model having to wait for the others to perform their calculations.

Parallel processing as it is presently implemented in Mohid, could not be achieved without [[object oriented programming]] philosophy, as each model is an instance of [[class Model]] and no changes, exception made to the implementation of the MPI communications calls needed to be added. Using this feature, computational speed was improved (varying from application to application), as now the whole model will take the same time as the slowest model to run plus the time to communicate with the other processes. Here, the network communication speed plays an important role, as it can become limiting. However, the amount of information passing between models, depending of course on the memory allocated for each model, has not yet proven to be big enough to make a 100 Mbps network connection time limiting.

[[Domain Composition]] is an ongoing project, in a very early stage, aimed at decoupling a domain into several subdomain that communicate among them (2-way) via MPI.

Find here information on how to [[setup a MOHID simulation using MPI]] and on [[compiling Mohid with MPI]].

== Parallel processing via OpenMP ==
Parallel processing using OpenMP is currently being implemented in Mohid by defining directives '''to optimize loops'''. These directives are defined as comments in the code and therefore need special compilation options. See more on [[compiling Mohid with OpenMP]]. Without special compilation these instructions are not considered in normal Fortran compilation.

OpenMP parallel processing can be used in '''multi core processors''' present in the same computer. It cannot be used to parallel processing using processors located in several computers in a cluster.

Loops optimization is introduced in a first phase in loops referring to grid variables (grid indexes, k, j, i) located in the Modifier section of MOHID modules.
Modifier loops are possibly used several times or involve a large resource allocation in MOHID simulations, hence these are locations with larger potential resource gains involved in parallelization.

In case of loops with several looping variables, the parallelized variable is chosen according with cost involved in the loop through this variable. E.g. in a 3D loop (k, j, i loop variables) if j dimension is much larger than the k (number of layers) or i then parallel processing is introduced in j variable, since the resource costs and the savings achieved with the parallelization are larger in this loop than in the others.

'''''Basic OpenMP syntax:'''''

In this section is provided an introduction with the basic concepts in OpenMP programming. The OpenMP language reference manual (http://www.openmp.org) should be consulted for further details.

OpenMP processing is made by a set of threads: the '''Master''' thread and the '''Workers''' threads.

OpenMP parallelization is accomplished by defining '''Parallel regions''' and by creating the threads (one for each available core): the processing inside a parallel region is divided between the Workers and the Master, which then proceed in parallel (simultaneously), instead of the unique thread existent in unparallel processing.

Each thread can have a set of '''Private variables''', which are affected only by this thread. The choice of the private variables, which are explicitly defined in programming, is a central part of the OpenMP programming. Should be made private all the variables whose values are altered by each thread processing and that affect other threads processing.

An obvious choice of these private variables are the loop variables when these are used to alter positions in matrixes or vectors in each iteration of the loop.

However it should be noted that in private variables the values are undefined in enter and exit of the parallel region and that by default these variables have no storage association with the variables outside the parallel region (this default behavior can, however, be altered by specific OpenMP directives).

The instructions in OpenMP are provided by a set of '''Directives'''. These refer to several actions such as the definition of Parallel regions, Work sharing and Data attributes. Directives are specific for the underlying programming language being used, either C or Fortran.

In Fortran directives are case insensitive. The basic syntax in as following:

sentinel directive [clause [[,] clause] ...]

'''Sentinel''' is !$OMP in either fixed or free format. The continuation of directives (from one code line to another) are according with the underlying language: & in Fortran case.

'''Clauses''' are used to specify additional information for the directive. Important clauses are:

- PRIVATE (private variables list);

- NOWAIT: specify that threads will not syncronize (i.e. wait for each other) at the end of a specific construct (e.g. a DO loop) within a parallel region; this is specified at the end of a parallel construct;

A parallel region is typically specified as:

!$OMP PARALLEL PRIVATE(PrivateVariable1,...,PrivateVariableN)

Inside a parallel region the work is distributed by the threads through '''Work Sharing constructs'''.

For every processing made inside a parallel region must be assigned a Work Sharing construct. If this is not verified execution errors can occur.

An important Work Sharing construct is the DO loop, specified as:

!$OMP DO
... (do loop in Fortran)
!$OMP END DO

In this construct the iterations of the enclosed Fortran DO loop are distributed by the threads.

The way the work is distributed over the threads is managed by the SCHEDULE clause of the DO construct. An important option of SCHEDULE is DYNAMIC: provided a Chunk number of iterations each thread will process this fixed number of iterations. After finishing a Chunk each thread will began another available Chunk till all iterations are completed:

!$OMP DO SCHEDULE(DYNAMIC, CHUNK)
...
!$OMP END DO

The distribution of Chunk among the threads requires synchronization for each assignment. This causes a overhead that could be important. The DYNAMIC option is advisable when each iteration involves an amount of work which is not predictable. This could be the case when IF constructs are present inside the loop containg extra processing done only in specific cases. Also if the threads arrive to the DO loop at different times, e.g. if they come from a previous DO loop with a NOWAIT end clause.

When the amount of work required in each DO loop iteration is predictable and the same for all threads it is advisable to use the STATIC option of SCHEDULE. This is particularly useful for DO loops which are unique in a parallel region:

!$OMP DO SCHEDULE(STATIC, CHUNK)
...
!$OMP END DO

Important characteristics of the Work Sharing constructs are that they do not create new threads, must contemplate all the existing threads or none at all and there is no barrier on entry (any available thread is not required to wait for the others) but a barrier on exit exists. The barrier on exit can be removed by the referred NOWAIT clause.

Work Sharing directives can appear outside the lexical extent of a parallel region (sequential lines of code appearing inside the parallel region). If they are dynamically linked with a parallel region (e.g. they appear in processing a subroutine call) they are orphaned. If, however, they appear outside this dynamic connection with a parallel region (and also not in the lexical extent of the region) they are ignored and the enclosed code is performed by one thread only and no parallelization is processed.

Inside a Work Sharing construct can be defined a critical region, if it is conveninent that only one thread at a time processes a specific code. This can be used to avoid problems with input/output operations potentially occuring by several threads processing at same time (problems can occur because memory locations are being assessed at same time). This is commanded as following:

!$OMP CRITICAL [Name]
... (code to be processed sequentially)
!$OMP END CRITICAL [Name]

In this case, no barriers exist in the entry and exit and all threads process the enclosed code although one at a time.

Within a Work Sharing construct can also be specified that a portion of the code is processed only by one thread, e.g. to read information from a file common to all threads. This is done with the following syntax:

!$OMP SINGLE [clause ...]
... (code to be processed by only one thread)
!$OMP END SINGLE

When one wants this single thread to be the Master then following syntax is used:

!$OMP MASTER
... (code to be processed by only Master thread)
!$OMP END MASTER

As with CRITICAL, no barriers exist in SINGLE/MASTER in the entry and exit.

In these contructs the fact that no barrier exists at the exit may cause problems if the single thread processed code section is intended to be dealt with previously from the subsequent code. In this situation a barrier can be introduced at the end of the SINGLE/MASTER construct:

!$OMP BARRIER

Under this directive threads wait at the barrier point till all threads reach it.

'''''Parallelization overheads:'''''

Although parallelization involves potentially gains in computer resources use it also involves overheads which can sometimes be important.

The creation of the thread team (Workers) at the beginning of each parallel region is a source of such overheads. Because of this it is preferrable in each program/subrotine to be parallelized to create only one parallel region and then deal with the code that must be done only by one thread with OpenMP directives than to create several parallel regions.

Another source of overhead is the synchronization between threads in Work Sharing constructs.

These overheads cause that in some times the parallelization solution may not be performing better than the non parallelization solution, especially if the computational burden of the problem is low.

This is the case of DO constructs when each cycle iteration has a small computational burden (e.g. equaling one matrix's value to other matrix's value) and is to be expected in nested loops when the parallelized DO loop is inner.

Generally as the scale of the problem increases the overheads will be less significant and parallelization more advantageous.

In very small scale problems parallelization may provide more comsumption of computational resources than the non parallelization.

[[Category:Programming]]

Parallel processing

2009-05-29T14:25:16Z

AngelaCanas: /* Parallel processing via OpenMP */

The historical need in numerical models to reduce computational time became a priority to the Mohid development team as an operational hydrodynamic and water quality model to the Tagus Estuary, in Lisbon, Portugal, was implemented using the [[Mohid Water]] model full capabilities. Thus, parallel processing has been implemented in Mohid Water in 2003, by using [http://www.mcs.anl.gov/mpi/mpich/ MPICH], a free portable implementation of [http://www.mcs.anl.gov/mpi/ MPI], the standard for message-passing libraries.

Currently, and due to the use of the new Intel Fortran compiler both Mohid Water and Mohid Land have parallelization features using [http://www.openmp.org/ OpenMP].

== Parallel processing via MPI ==
The [[Mohid Water]] ability to run [[nested models]] was accomplished by creating a linked list of all the models and by attributing to each one a father-son identification, through which the models communicate. The first stage for introducing parallel processing in Mohid was to add the possibility of launching a process by each model to run, and then, using MPICH, establish communication between models. This enables each sub-model to run in a different processor (even if the processor belongs to a different computer, as long as it is in the same network) and in parallel, instead of running all in the same processor and each model having to wait for the others to perform their calculations.

Parallel processing as it is presently implemented in Mohid, could not be achieved without [[object oriented programming]] philosophy, as each model is an instance of [[class Model]] and no changes, exception made to the implementation of the MPI communications calls needed to be added. Using this feature, computational speed was improved (varying from application to application), as now the whole model will take the same time as the slowest model to run plus the time to communicate with the other processes. Here, the network communication speed plays an important role, as it can become limiting. However, the amount of information passing between models, depending of course on the memory allocated for each model, has not yet proven to be big enough to make a 100 Mbps network connection time limiting.

[[Domain Composition]] is an ongoing project, in a very early stage, aimed at decoupling a domain into several subdomain that communicate among them (2-way) via MPI.

Find here information on how to [[setup a MOHID simulation using MPI]] and on [[compiling Mohid with MPI]].

== Parallel processing via OpenMP ==
Parallel processing using OpenMP is currently being implemented in Mohid by defining directives '''to optimize loops'''. These directives are defined as comments in the code and therefore need special compilation options. See more on [[compiling Mohid with OpenMP]]. Without special compilation these instructions are not considered in normal Fortran compilation.

OpenMP parallel processing can be used in '''multi core processors''' present in the same computer. It cannot be used to parallel processing using processors located in several computers in a cluster.

Loops optimization is introduced in a first phase in loops referring to grid variables (grid indexes, k, j, i) located in the Modifier section of MOHID modules.
Modifier loops are possibly used several times or involve a large resource allocation in MOHID simulations, hence these are locations with larger potential resource gains involved in parallelization.

In case of loops with several looping variables, the parallelized variable is chosen according with cost involved in the loop through this variable. E.g. in a 3D loop (k, j, i loop variables) if j dimension is much larger than the k (number of layers) or i then parallel processing is introduced in j variable, since the resource costs and the savings achieved with the parallelization are larger in this loop than in the others.

'''''Basic OpenMP syntax:'''''

In this section is provided an introduction with the basic concepts in OpenMP programming. The OpenMP language reference manual (http://www.openmp.org) should be consulted for further details.

OpenMP processing is made by a set of threads: the '''Master''' thread and the '''Workers''' threads.

OpenMP parallelization is accomplished by defining '''Parallel regions''' and by creating the threads (one for each available core): the processing inside a parallel region is divided between the Workers and the Master, which then proceed in parallel (simultaneously), instead of the unique thread existent in unparallel processing.

Each thread can have a set of '''Private variables''', which are affected only by this thread. The choice of the private variables, which are explicitly defined in programming, is a central part of the OpenMP programming. Should be made private all the variables whose values are altered by each thread processing and that affect other threads processing.

An obvious choice of these private variables are the loop variables when these are used to alter positions in matrixes or vectors in each iteration of the loop.

However it should be noted that in private variables the values are undefined in enter and exit of the parallel region and that by default these variables have no storage association with the variables outside the parallel region (this default behavior can, however, be altered by specific OpenMP directives).

The instructions in OpenMP are provided by a set of '''Directives'''. These refer to several actions such as the definition of Parallel regions, Work sharing and Data attributes. Directives are specific for the underlying programming language being used, either C or Fortran.

In Fortran directives are case insensitive. The basic syntax in as following:

sentinel directive [clause [[,] clause] ...]

'''Sentinel''' is !$OMP in either fixed or free format. The continuation of directives (from one code line to another) are according with the underlying language: & in Fortran case.

'''Clauses''' are used to specify additional information for the directive. Important clauses are:

- PRIVATE (private variables list);

- NOWAIT: specify that threads will not syncronize (i.e. wait for each other) at the end of a specific construct (e.g. a DO loop) within a parallel region; this is specified at the end of a parallel construct;

A parallel region is typically specified as:

!$OMP PARALLEL PRIVATE(PrivateVariable1,...,PrivateVariableN)

Inside a parallel region the work is distributed by the threads through '''Work Sharing constructs'''.

For every processing made inside a parallel region must be assigned a Work Sharing construct. If this is not verified execution errors can occur.

An important Work Sharing construct is the DO loop, specified as:

!$OMP DO
... (do loop in Fortran)
!$OMP END DO

In this construct the iterations of the enclosed Fortran DO loop are distributed by the threads.

The way the work is distributed over the threads is managed by the SCHEDULE clause of the DO construct. An important option of SCHEDULE is DYNAMIC: provided a Chunk number of iterations each thread will process this fixed number of iterations. After finishing a Chunk each thread will began another available Chunk till all iterations are completed:

!$OMP DO SCHEDULE(DYNAMIC, CHUNK)
...
!$OMP END DO

The distribution of Chunk among the threads requires synchronization for each assignment. This causes a overhead that could be important. The DYNAMIC option is advisable when each iteration involves an amount of work which is not predictable. This could be the case when IF constructs are present inside the loop containg extra processing done only in specific cases. Also if the threads arrive to the DO loop at different times, e.g. if they come from a previous DO loop with a NOWAIT end clause.

When the amount of work required in each DO loop iteration is predictable and the same for all threads it is advisable to use the STATIC option of SCHEDULE. This is particularly useful for DO loops which are unique in a parallel region:

!$OMP DO SCHEDULE(STATIC, CHUNK)
...
!$OMP END DO

Important characteristics of the Work Sharing constructs are that they do not create new threads, must contemplate all the existing threads or none at all and there is no barrier on entry (any available thread is not required to wait for the others) but a barrier on exit exists. The barrier on exit can be removed by the referred NOWAIT clause.

Work Sharing directives can appear outside the lexical extent of a parallel region (sequential lines of code appearing inside the parallel region). If they are dynamically linked with a parallel region (e.g. they appear in processing a subroutine call) they are orphaned. If, however, they appear outside this dynamic connection with a parallel region (and also not in the lexical extent of the region) they are ignored and the enclosed code is performed by one thread only and no parallelization is processed.

Inside a Work Sharing construct can be defined a critical region, if it is conveninent that only one thread at a time processes a specific code. This can be used to avoid problems with input/output operations potentially occuring by several threads processing at same time (problems can occur because memory locations are being assessed at same time). This is commanded as following:

!$OMP CRITICAL [Name]
... (code to be processed sequentially)
!$OMP END CRITICAL [Name]

In this case, no barriers exist in the entry and exit and all threads process the enclosed code although one at a time.

Within a Work Sharing construct can also be specified that a portion of the code is processed only by one thread, e.g. to read information from a file common to all threads. This is done with the following syntax:

!$OMP SINGLE [clause ...]
... (code to be processed by only one thread)
!$OMP END SINGLE

When one wants this single thread to be the Master then following syntax is used:

!$OMP MASTER
... (code to be processed by only Master thread)
!$OMP END MASTER

As with CRITICAL, no barriers exist in SINGLE/MASTER in the entry and exit.

In these contructs the fact that no barrier exists at the exit may cause problems if the single thread processed code section is intended to be dealt with previously from the subsequent code. In this situation a barrier can be introduced at the end of the SINGLE/MASTER construct:

!$OMP BARRIER

Under this directive threads wait at the barrier point till all threads reach it.

'''''Parallelization overheads:'''''

Although parallelization involves potentially gains in computer resources use it also involves overheads which can sometimes be important.

The creation of the thread team (Workers) at the beginning of each parallel region is a source of such overheads. Because of this it is preferrable in each program/subrotine to be parallelized to create only one parallel region and then deal with the code that must be done only by one thread with OpenMP directives than to create several parallel regions.

Another source of overhead is the synchronization between threads in Work Sharing constructs.

These overheads cause that in some times the parallelization solution may not be performing better than the non parallelization solution, especially if the computational burden of the problem is low. Generally as the scale of the problem increases the overheads will be less significant and parallelization more advantageous.

[[Category:Programming]]

Parallel processing

2009-05-29T14:05:56Z

AngelaCanas: /* Parallel processing via OpenMP */

The historical need in numerical models to reduce computational time became a priority to the Mohid development team as an operational hydrodynamic and water quality model to the Tagus Estuary, in Lisbon, Portugal, was implemented using the [[Mohid Water]] model full capabilities. Thus, parallel processing has been implemented in Mohid Water in 2003, by using [http://www.mcs.anl.gov/mpi/mpich/ MPICH], a free portable implementation of [http://www.mcs.anl.gov/mpi/ MPI], the standard for message-passing libraries.

Currently, and due to the use of the new Intel Fortran compiler both Mohid Water and Mohid Land have parallelization features using [http://www.openmp.org/ OpenMP].

== Parallel processing via MPI ==
The [[Mohid Water]] ability to run [[nested models]] was accomplished by creating a linked list of all the models and by attributing to each one a father-son identification, through which the models communicate. The first stage for introducing parallel processing in Mohid was to add the possibility of launching a process by each model to run, and then, using MPICH, establish communication between models. This enables each sub-model to run in a different processor (even if the processor belongs to a different computer, as long as it is in the same network) and in parallel, instead of running all in the same processor and each model having to wait for the others to perform their calculations.

Parallel processing as it is presently implemented in Mohid, could not be achieved without [[object oriented programming]] philosophy, as each model is an instance of [[class Model]] and no changes, exception made to the implementation of the MPI communications calls needed to be added. Using this feature, computational speed was improved (varying from application to application), as now the whole model will take the same time as the slowest model to run plus the time to communicate with the other processes. Here, the network communication speed plays an important role, as it can become limiting. However, the amount of information passing between models, depending of course on the memory allocated for each model, has not yet proven to be big enough to make a 100 Mbps network connection time limiting.

[[Domain Composition]] is an ongoing project, in a very early stage, aimed at decoupling a domain into several subdomain that communicate among them (2-way) via MPI.

Find here information on how to [[setup a MOHID simulation using MPI]] and on [[compiling Mohid with MPI]].

== Parallel processing via OpenMP ==
Parallel processing using OpenMP is currently being implemented in Mohid by defining directives '''to optimize loops'''. These directives are defined as comments in the code and therefore need special compilation options. See more on [[compiling Mohid with OpenMP]]. Without special compilation these instructions are not considered in normal Fortran compilation.

OpenMP parallel processing can be used in '''multi core processors''' present in the same computer. It cannot be used to parallel processing using processors located in several computers in a cluster.

Loops optimization is introduced in a first phase in loops referring to grid variables (grid indexes, k, j, i) located in the Modifier section of MOHID modules.
Modifier loops are possibly used several times or involve a large resource allocation in MOHID simulations, hence these are locations with larger potential resource gains involved in parallelization.

In case of loops with several looping variables, the parallelized variable is chosen according with cost involved in the loop through this variable. E.g. in a 3D loop (k, j, i loop variables) if j dimension is much larger than the k (number of layers) or i then parallel processing is introduced in j variable, since the resource costs and the savings achieved with the parallelization are larger in this loop than in the others.

'''''Basic OpenMP syntax:'''''

In this section is provided an introduction with the basic concepts in OpenMP programming. The OpenMP language reference manual (http://www.openmp.org) should be consulted for further details.

OpenMP processing is made by a set of threads: the '''Master''' thread and the '''Workers''' threads.

OpenMP parallelization is accomplished by defining '''Parallel regions''' and by creating the threads (one for each available core): the processing inside a parallel region is divided between the Workers and the Master, which then proceed in parallel (simultaneously), instead of the unique thread existent in unparallel processing.

Each thread can have a set of '''Private variables''', which are affected only by this thread. The choice of the private variables, which are explicitly defined in programming, is a central part of the OpenMP programming. Should be made private all the variables whose values are altered by each thread processing and that affect other threads processing.

An obvious choice of these private variables are the loop variables when these are used to alter positions in matrixes or vectors in each iteration of the loop.

However it should be noted that in private variables the values are undefined in enter and exit of the parallel region and that by default these variables have no storage association with the variables outside the parallel region (this default behavior can, however, be altered by specific OpenMP directives).

The instructions in OpenMP are provided by a set of '''Directives'''. These refer to several actions such as the definition of Parallel regions, Work sharing and Data attributes. Directives are specific for the underlying programming language being used, either C or Fortran.

In Fortran directives are case insensitive. The basic syntax in as following:

sentinel directive [clause [[,] clause] ...]

'''Sentinel''' is !$OMP in either fixed or free format. The continuation of directives (from one code line to another) are according with the underlying language: & in Fortran case.

'''Clauses''' are used to specify additional information for the directive. Important clauses are:

- PRIVATE (private variables list);

- NOWAIT: specify that threads will not syncronize (i.e. wait for each other) at the end of a specific construct (e.g. a DO loop) within a parallel region; this is specified at the end of a parallel construct;

A parallel region is typically specified as:

!$OMP PARALLEL PRIVATE(PrivateVariable1,...,PrivateVariableN)

Inside a parallel region the work is distributed by the threads through '''Work Sharing constructs'''.

For every processing made inside a parallel region must be assigned a Work Sharing construct. If this is not verified execution errors can occur.

An important Work Sharing construct is the DO loop, specified as:

!$OMP DO
... (do loop in Fortran)
!$OMP END DO

In this construct the iterations of the enclosed Fortran DO loop are distributed by the threads.

The way the work is distributed over the threads is managed by the SCHEDULE clause of the DO construct. An important option of SCHEDULE is DYNAMIC: provided a Chunk number of iterations each thread will process this fixed number of iterations. After finishing a Chunk each thread will began another available Chunk till all iterations are completed:

!$OMP DO SCHEDULE(DYNAMIC, CHUNK)
...
!$OMP END DO

The distribution of Chunk among the threads requires synchronization for each assignment. This causes a overhead that could be important. The DYNAMIC option is advisable when each iteration involves an amount of work which is not predictable. This could be the case when IF constructs are present inside the loop containg extra processing done only in specific cases. Also if the threads arrive to the DO loop at different times, e.g. if they come from a previous DO loop with a NOWAIT end clause.

When the amount of work required in each DO loop iteration is predictable and the same for all threads it is advisable to use the STATIC option of SCHEDULE. This is particularly useful for DO loops which are unique in a parallel region:

!$OMP DO SCHEDULE(STATIC, CHUNK)
...
!$OMP END DO

Important characteristics of the Work Sharing constructs are that they do not create new threads, must contemplate all the existing threads or none at all and there is no barrier on entry (any available thread is not required to wait for the others) but a barrier on exit exists. The barrier on exit can be removed by the referred NOWAIT clause.

Work Sharing directives can appear outside the lexical extent of a parallel region (sequential lines of code appearing inside the parallel region). If they are dynamically linked with a parallel region (e.g. they appear in processing a subroutine call) they are orphaned. If, however, they appear outside this dynamic connection with a parallel region (and also not in the lexical extent of the region) they are ignored and the enclosed code is performed by one thread only and no parallelization is processed.

Inside a Work Sharing construct can be defined a critical region, if it is conveninent that only one thread at a time processes a specific code. This can be used to avoid problems with input/output operations potentially occuring by several threads processing at same time (problems can occur because memory locations are being assessed at same time). This is commanded as following:

!$OMP CRITICAL [Name]
... (code to be processed sequentially)
!$OMP END CRITICAL [Name]

In this case, no barriers exist in the entry and exit and all threads process the enclosed code although one at a time.

Within a Work Sharing construct can also be specified that a portion of the code is processed only by one thread, e.g. to read information from a file common to all threads. This is done with the following syntax:

!$OMP SINGLE [clause ...]
... (code to be processed by only one thread)
!$OMP END SINGLE

When one wants this single thread to be the Master then following syntax is used:

!$OMP MASTER
... (code to be processed by only Master thread)
!$OMP END MASTER

As with CRITICAL, no barriers exist in SINGLE/MASTER in the entry and exit.

In these contructs the fact that no barrier exists at the exit may cause problems if the single thread processed code section is intended to be dealt with previously from the subsequent code. In this situation a barrier can be introduced at the end of the SINGLE/MASTER construct:

!$OMP BARRIER

Under this directive threads wait at the barrier point till all threads reach it.

'''''Parallelization overheads:'''''

[[Category:Programming]]

Parallel processing

2009-05-22T13:55:35Z

AngelaCanas: /* Parallel processing via OpenMP */

The historical need in numerical models to reduce computational time became a priority to the Mohid development team as an operational hydrodynamic and water quality model to the Tagus Estuary, in Lisbon, Portugal, was implemented using the [[Mohid Water]] model full capabilities. Thus, parallel processing has been implemented in Mohid Water in 2003, by using [http://www.mcs.anl.gov/mpi/mpich/ MPICH], a free portable implementation of [http://www.mcs.anl.gov/mpi/ MPI], the standard for message-passing libraries.

Currently, and due to the use of the new Intel Fortran compiler both Mohid Water and Mohid Land have parallelization features using [http://www.openmp.org/ OpenMP].

== Parallel processing via MPI ==
The [[Mohid Water]] ability to run [[nested models]] was accomplished by creating a linked list of all the models and by attributing to each one a father-son identification, through which the models communicate. The first stage for introducing parallel processing in Mohid was to add the possibility of launching a process by each model to run, and then, using MPICH, establish communication between models. This enables each sub-model to run in a different processor (even if the processor belongs to a different computer, as long as it is in the same network) and in parallel, instead of running all in the same processor and each model having to wait for the others to perform their calculations.

Parallel processing as it is presently implemented in Mohid, could not be achieved without [[object oriented programming]] philosophy, as each model is an instance of [[class Model]] and no changes, exception made to the implementation of the MPI communications calls needed to be added. Using this feature, computational speed was improved (varying from application to application), as now the whole model will take the same time as the slowest model to run plus the time to communicate with the other processes. Here, the network communication speed plays an important role, as it can become limiting. However, the amount of information passing between models, depending of course on the memory allocated for each model, has not yet proven to be big enough to make a 100 Mbps network connection time limiting.

[[Domain Composition]] is an ongoing project, in a very early stage, aimed at decoupling a domain into several subdomain that communicate among them (2-way) via MPI.

Find here information on how to [[setup a MOHID simulation using MPI]] and on [[compiling Mohid with MPI]].

== Parallel processing via OpenMP ==
Parallel processing using OpenMP is currently being implemented in Mohid by defining directives '''to optimize loops'''. These directives are defined as comments in the code and therefore need special compilation options. See more on [[compiling Mohid with OpenMP]]. Without special compilation these instructions are not considered in normal Fortran compilation.

OpenMP parallel processing can be used in '''multi core processors''' present in the same computer. It cannot be used to parallel processing using processors located in several computers in a cluster.

Loops optimization is introduced in a first phase in loops referring to grid variables (grid indexes, k, j, i) located in the Modifier section of MOHID modules.
Modifier loops are possibly used several times or involve a large resource allocation in MOHID simulations, hence these are locations with larger potential resource gains involved in parallelization.

In case of loops with several looping variables, the parallelized variable is chosen according with cost involved in the loop through this variable. E.g. in a 3D loop (k, j, i loop variables) if j dimension is much larger than the k (number of layers) or i then parallel processing is introduced in j variable, since the resource costs and the savings achieved with the parallelization are larger in this loop than in the others.

'''''Basic OpenMP syntax:'''''

In this section is provided an introduction with the basic concepts in OpenMP programming. The OpenMP language reference manual (http://www.openmp.org) should be consulted for further details.

OpenMP processing is made by a set of threads: the '''Master''' thread and the '''Workers''' threads.

OpenMP parallelization is accomplished by defining '''Parallel regions''' and by creating the threads (one for each available core): the processing inside a parallel region is divided between the Workers and the Master, which then proceed in parallel (simultaneously), instead of the unique thread existent in unparallel processing.

Each thread can have a set of '''Private variables''', which are affected only by this thread. The choice of the private variables, which are explicitly defined in programming, is a central part of the OpenMP programming. Should be made private all the variables whose values are altered by each thread processing and that affect other threads processing.

An obvious choice of these private variables are the loop variables when these are used to alter positions in matrixes or vectors in each iteration of the loop.

However it should be noted that in private variables the values are undefined in enter and exit of the parallel region and that by default these variables have no storage association with the variables outside the parallel region (this default behavior can, however, be altered by specific OpenMP directives).

The instructions in OpenMP are provided by a set of '''Directives'''. These refer to several actions such as the definition of Parallel regions, Work sharing and Data attributes. Directives are specific for the underlying programming language being used, either C or Fortran.

In Fortran directives are case insensitive. The basic syntax in as following:

sentinel directive [clause [[,] clause] ...]

'''Sentinel''' is !$OMP in either fixed or free format. The continuation of directives (from one code line to another) are according with the underlying language: & in Fortran case.

'''Clauses''' are used to specify additional information for the directive. Important clauses are:

- PRIVATE (private variables list);

- NOWAIT: specify that threads will not syncronize (i.e. wait for each other) at the end of a specific construct (e.g. a DO loop) within a parallel region; this is specified at the end of a parallel construct;

A parallel region is typically specified as:

!$OMP PARALLEL PRIVATE(PrivateVariable1,...,PrivateVariableN)

Inside a parallel region the work is distributed by the threads through '''Work Sharing constructs'''.

For every processing made inside a parallel region must be assigned a Work Sharing construct. If this is not verified execution errors can occur.

An important Work Sharing construct is the DO loop, specified as:

!$OMP DO
... (do loop in Fortran)
!$OMP END DO

In this construct the iterations of the enclosed Fortran DO loop are distributed by the threads.

The way the work is distributed over the threads is managed by the SCHEDULE clause of the DO construct. An important option of SCHEDULE is DYNAMIC: provided a Chunk number of iterations each thread will process this fixed number of iterations. After finishing a Chunk each thread will began another available Chunk till all iterations are completed:

!$OMP DO SCHEDULE(DYNAMIC, CHUNK)
...
!$OMP END DO

The distribution of Chunk among the threads requires synchronization for each assignment. This causes a overhead that could be important. The DYNAMIC option is advisable when each iteration involves an amount of work which is not predictable. This could be the case when IF constructs are present inside the loop containg extra processing done only in specific cases. Also if the threads arrive to the DO loop at different times, e.g. if they come from a previous DO loop with a NOWAIT end clause.

When the amount of work required in each DO loop iteration is predictable and the same for all threads it is advisable to use the STATIC option of SCHEDULE. This is particularly useful for DO loops which are unique in a parallel region:

!$OMP DO SCHEDULE(STATIC, CHUNK)
...
!$OMP END DO

Important characteristics of the Work Sharing constructs are that they do not create new threads, must contemplate all the existing threads or none at all and there is no barrier on entry (any available thread is not required to wait for the others) but a barrier on exit exists. The barrier on exit can be removed by the referred NOWAIT clause.

Work Sharing directives can appear outside the lexical extent of a parallel region (sequential lines of code appearing inside the parallel region). If they are dynamically linked with a parallel region (e.g. they appear in processing a subroutine call) they are orphaned. If, however, they appear outside this dynamic connection with a parallel region (and also not in the lexical extent of the region) they are ignored and the enclosed code is performed by one thread only and no parallelization is processed.

Inside a Work Sharing construct can be defined a critical region, if it is conveninent that only one thread at a time processes a specific code. This can be used to avoid problems with input/output operations potentially occuring by several threads processing at same time (problems can occur because memory locations are being assessed at same time). This is commanded as following:

!$OMP CRITICAL [Name]
... (code to be processed sequentially)
!$OMP END CRITICAL [Name]

In this case, no barriers exist in the entry and exit and all threads process the enclosed code although one at a time.

Within a Work Sharing construct can also be specified that a portion of the code is processed only by one thread, e.g. to read information from a file common to all threads. This is done with the following syntax:

!$OMP SINGLE [clause ...]
... (code to be processed by only one thread)
!$OMP END SINGLE

When one wants this single thread to be the Master then following syntax is used:

!$OMP MASTER
... (code to be processed by only Master thread)
!$OMP END MASTER

As with CRITICAL, no barriers exist in SINGLE/MASTER in the entry and exit.

In these contructs the fact that no barrier exists at the exit may cause problems if the single thread processed code section is intended to be dealt with previously from the subsequent code. In this situation a barrier can be introduced at the end of the SINGLE/MASTER construct:

!$OMP BARRIER

Under this directive threads wait at the barrier point till all threads reach it.

[[Category:Programming]]

Parallel processing

2009-05-18T12:46:57Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2009-05-18T12:43:40Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2009-05-15T20:15:21Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2009-05-15T20:03:56Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2009-05-15T18:28:19Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2009-05-15T18:11:32Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2009-05-12T13:16:51Z

AngelaCanas: /* Parallel processing via OpenMP */

ConvertToHDF5

2009-05-12T12:57:56Z

AngelaCanas: /* CONVERT MERCATOR FORMAT */

The '''ConvertToHDF5''' is an application which allows the making of several operations, called '''actions''', involving HDF5 files: conversion of data in other formats (e.g. NETCDF) to HDF5, grid interpolation, concatenation of several files.

Running options for this application are specified by the user in a input file named [[ConvertToHDF5#Input file (ConvertToHDF5Action.dat)|'''ConvertToHDF5Action.dat''']]. Several actions can be specified in the same input file, being processed sequentially by the ConvertToHDF5 application.

==Introduction==
The operations involving HDF5 files performed by ConvertToHDF5, specified individually by an action, can be organized in [[#file management|file management]], [[#grid interpolation|grid interpolation]] and [[#format conversion|format conversion]]. These types and the respective actions are detailed in the next sections.

The input file specification for each action can be found bellow in the [[#Input file (ConvertToHDF5Action.dat)|Input file (ConvertToHDF5Action.dat)]] section.

===File management===

====Glue files====
This action consists in joining or glue in a single HDF5 file two or more HDF5 files having the same HDF5 data groups and referring to time periods which come in sequence. Both sets of 2D and 3D HDF5 files can be glued.

'''Typical use:'''

Glue MOHID Water results files from several runs produced in continuous running of the model, for storage space economy reasons. Can be used to join data from other origins (e.g. results of meteorological models) as long as the HDF5 format is the one supported by MOHID Water.

'''Data input requirements:'''

HDF5 files to be glued. "Grid" and "Results" data groups should be equal in all these files.

'''Output:'''

HDF5 file with glued "Results" data. "Residual" and "Statistics" HDF5 data groups are not copied to the output file since they are time period specific (different values potentially occour in each file). General statistics can be calculated for the glued HDF5 file data using tool [[HDF5Statistics]].

'''ConvertToHDF5 action:''' [[#GLUES HDF5 FILES|GLUES HDF5 FILES]].

===Grid interpolation===

====Interpolate files====
This action performs the conversion of one HDF5 file data existing in one 2D or 3D spatial grid to another 2D or 3D spatial grid, creating a new HDF5 file. The interpolation is performed only for the data located a time window specified by the user.

The HDF5 file containing data to be interpolated is called the '''father file'''.

In case of 3D interpolation the application conducts first the horizontal grid interpolation
(keeping father geometry) and only after it conducts the vertical interpolation (from father geometry to new geometry).

Several types of 2D interpolation are available for use: bilinear, spline 2D and triangulation.
For vertical interpolation (used in 3D interpolation) can be supplied several polinomial degrees for interpolation.

'''Typical use:'''

Obtain an HDF5 file with data for forcing or providing initial conditions for a MOHID Water model, e.g. a meteorological forcing file.

'''Data input requirements:'''

For 2D/3D interpolation:

- father HDF5 file;

- father horizontal data grid, in a grid data file in the format supported by MOHID;

- new horizontal data grid, in a grid data file in the format supported by MOHID;

For 3D interpolation also needed:

- father vertical geometry, in a geometry file in the format supported by MOHID;

- new vertical geometry, in a geometry file in the format supported by MOHID;

- auxiliary horizontal data grid, in a grid data file in the format supported by MOHID; this file is used for horizontal grid interpolation in 3D interpolation operations.

'''Ouput:'''

HDF5 file with interpolated data. In case of 3D interpolation also produced an auxiliary HDF5 file with the result of the horizontal grid interpolation, which can be inspected to check if this operation is well performed.

'''ConvertToHDF5 action:''' [[#INTERPOLATE GRIDS|INTERPOLATE GRIDS]].

====Patch files====
This action consists in performing an interpolation of HDF5 data between grids, as in action [[#Interpolate files|Interpolate files]], but considering more than one HDF5 file as containing data to be interpolated to the new grid and a priority scale. The interpolation is performed only for the data located in the time window specified by the user. The present version of this action operates only on 2D data.

Each HDF5 file containing data to be interpolated is called a '''father file''' and has an user-attributed '''priority level''' to be respected in the interpolation process: for each new grid cell the ConvertToHDF5 application will look for data first on the Level 1 father file and only in the case this data is inexistent will it look for data in Level 2 file, proceeding in looking for higher level files if no data is found subsequentely.

'''Typical use:'''

To obtain an HDF5 file with data from several HDF5 files each containing data with different spatial resolution and only for a specific part of the new grid. This is, for instance, the case when one is preparing a best resolution meteorological HDF5 file for forcing MOHID Water from several meteorological model domains, having different spatial resolution and span, since the best resolution data is not available for all new grid cells.

'''Data input requirements:'''

The new horizontal data grid, in a grid data file in the format supported by MOHID, and for each father file:

- level of priority: 1 = maximum priority, priority decreases with increasing level value;

- data grid, in the form of a grid data file in the format supported by MOHID.

'''Ouput:'''

HDF5 file with patched data.

'''ConvertToHDF5 action:''' [[#PATCH HDF5 FILES|PATCH HDF5 FILES]].

===Format conversion===

====Meteorological model data====
Mohid does not simulate explicitly the atmosphere, but needs information about atmospheric properties in time and space. This requires that atmospheric properties are supplied to MOHID Water in supported formats. These formats can be derived from meteorological data in HDF5 format. Because the results of meteorological models are accessed in different formats conversion is required.

The formats currently convertible to HDF5 in ConvertToHDF5 include the MM5 and the ERA40. These are succintly detailed in the next sections.

=====''ERA40''=====
This format refers to the European Centre for Medium-Range Weather Forecasts (ECMWF) 40 years re-analysises results, acessed by site http://data.ecmwf.int/data/d/era40_daily/. This data is available for several meteorological variables with maximum 6 hour periodicity for days in the period from 1957-09-01 to 2002-08-31.

ERA40 data files are supplied by ECMWF in a NetCDF format and with an user-costumized time window, periodicity (time step range from 6 hours to a day) and meteorological properties set. The ERA40 meteorological properties which are recognized by MOHID are presented bellow together with the correspondent MOHID name:

---ERA40 NAME--- ---MOHID NAME---
sshf sensible heat
slhf latent heat
msl atmospheric pressure
tcc cloud cover
p10u wind velocity X
p10v wind velocity Y
p2t air temperature
ewss wind stress X
nsss wind stress Y

The standard ConvertToHDF5 action is to convert to HDF5 the data referring to all MOHID Water recognized property available in the ERA40 file, producing an individual HDF5 file for each property. The name of each HDF5 file generated includes the ERA40 meteorological property identificator correspondent to the data contained.

Alternatively, ConvertToHDF5 can copy to a single ASCII file the heading information concerning each meteorological variable considered in the original ERA40 file.

'''Typical use:'''

Obtain an HDF5 file with data suitable for being used for forcing MOHID Water models.

'''Data input requirements:'''

ERA40 NetCDF file.

'''Output:'''

One HDF5 file for each meteorological property contained in the original NetCDF file.

'''ConvertToHDF5 action:''' [[#CONVERT ERA40 FORMAT|CONVERT ERA40 FORMAT]].

=====''MM5''=====
This format relates to the Fifth-Generation NCAR / Penn State Mesoscale Model (MM5) output files format. Almost every atmospheric property needed by MOHID Water is present in MM5 output files, enabling to run prediction simulations with MOHID Water when access to MM5 prevision files is available.

The ConvertToHDF5 action converts MM5 results files from the original format to HDF5 format, allowing the easy use of these results in the MOHID framework. Conversion is only performed for the MM5 properties and the time window specified by the user.

Besides the conversion, the application can calculate some properties not contained in
the MM5 files using the available information: these are windstress, relative humidity and precipitation.

For conversion to be completed it is required the horizontal grid information of MM5 results which is available in special TERRAIN files.

'''Typical use:'''

Produce HDF5 meteorological data usable for forcing MOHID Water models.

'''Data input requirements:'''

MM5 results file to convert and MM5 TERRAIN file. The TERRAIN file supplies the MM5 results grid information.

'''Ouput:'''

An HDF5 file with MM5 results and a grid data file in MOHID format with the MM5 grid information.
This last file can be used to interpolate the MM5 data from the original grid to a new grid (see [[#Interpolate files|Interpolate files]]), for instance to produce an HDF5 file suitable for forcing MOHID Water models.

'''Compilation:'''

Caution! The ConvertToHDF5 executable must be compiled with the [[Big-endian little-endian|Big-Endian]] option set (see compatibility in the project's settings).

'''ConvertToHDF5 action:''' [[#CONVERT MM5 FORMAT|CONVERT MM5 FORMAT]].

=====''Aladin''=====
This format relates to Aladin meteorological model results. Some of the atmospheric property needed by MOHID Water is present in Aladin output files, enabling to run prediction simulations with MOHID Water when access to Aladin prevision files is available.

The ConvertToHDF5 action converts Aladin results files from the original format to HDF5 format, allowing the easy use of these results in the MOHID framework. Conversion is only performed for the MM5 properties and the time window specified by the user.

'''Typical use:'''

Produce HDF5 meteorological data usable for forcing MOHID Water models.

'''Data input requirements:'''

Aladin netcdf results file to convert.

'''Ouput:'''

An HDF5 file with Aladin results and a grid data file in MOHID format with the Aladin grid pseudo-information: a fake orography is created of 100 m depth.
This last file can be used to interpolate the Aladin data from the original grid to a new grid (see [[#Interpolate files|Interpolate files]]), for instance to produce an HDF5 file suitable for forcing MOHID Water models.

'''Compilation:'''

Caution! The ConvertToHDF5 executable must be compiled with the [[Big-endian little-endian|Big-Endian]] option set (see compatibility in the project's settings).

'''ConvertToHDF5 action:''' [[#CONVERT ALADIN FORMAT|CONVERT ALADIN FORMAT]].

====Ocean model data====
Ocean model data, available in diverse formats, can be used by MOHID Water to specify boundary (open ocean boundary and surface), initial conditions or for validation. These uses require that the model data is in HDF5 format and conversion is therefore needed.

Currently the large scale ocean models formats convertible into HDF5 by ConvertToHDF5 includes MERCATOR.

=====''MERCATOR''=====
MERCATOR data files are supplied in a NetCDF format and with an user-costumized spatial window and periodicity. Water level and water properties (temperature and salinity) data is available in type T files, velocity component u data is available in type U files and velocity component v data is available in type V files. The type of data of a specific MERCATOR file is generally indicated in the file name.

The standard ConvertToHDF5 action is to convert to HDF5 the data referring to temperature, salinity, water level, component u of velocity and component v of velocity.

'''Typical use:'''

Obtain HDF5 MERCATOR data usable for forcing or validation of MOHID Water models.

'''Data input requirements:'''

NetCDF MERCATOR results data files and NetCDF MERCATOR grid data files. It should be provided one grid data file of each type: T, U and V. These are generally provided by the MERCATOR services together with the results files.

'''Ouput:'''

One HDF5 file containing all properties contained in the recognized set of properties (temperature, salinity, water level, velocity u and velocity v) and the correspondent grid data and geometry files, containing respectively the horizontal grid and the vertical discretization of the HDF5 file. The grid data and geometry files can be used afterwards to interpolate the MERCATOR data to another grid and geometry (see [[#Interpolate files|Interpolate files]]).

'''ConvertToHDF5 action:''' [[#CONVERT MERCATOR FORMAT|CONVERT MERCATOR FORMAT]].

====Climatological data====
Climatological data can be used in MOHID Water to specify boundary (open ocean boundary and surface), initial conditions or for validation, in case more realistic data (measurements or model) data is unavailable. This data is generally supplied by producers in formats not readly usable by MOHID Water which justifies the existence of a conversion tool.

Two climatological data format conversions are implemented in ConvertToHDF5: Levitus ocean data and Hellerman Rosenstein meteorological data.

=====''Levitus''=====
The Levitus climatology provides results for water temperature and salinity.
The ConvertToHDF5 action converts the climatological data for the properties and spatial window requested by the user.
Typically, it requires 3 steps to complete the task:

- convert levitus format

- extrapolate the data to the whole levitus domain(required to avoid uncoincidental coastlines)

- interpolate with the model grid(bathymetry)

'''Typical use:'''

Obtain climatological data in HDF5 format to use as boundary forcing and/or initial condition specification in MOHID Water models.

'''Data input requirements:'''

Levitus climatological data files, one per property and per time period (e.g a month).

'''Ouput:'''

HDF5 file with Levitus climatological data, grid data file with the horizontal
grid of the data and a geometry file with vertical discretization of the data (MOHID formats).
The grid data and the geometry files can be used to interpolate the climatological data from the original grid to a new grid (see [[#Interpolate files|Interpolate files]]).

'''ConvertToHDF5 action:''' [[#CONVERT LEVITUS FORMAT|CONVERT LEVITUS FORMAT]].

=====''Hellerman Rosenstein''=====
This is a meteorological climatology providing wind stress. There is a file per wind stress component. Since the data refer to surface values it is a 2D field.

The ConvertToHDF5 action converts the climatological data for the properties and spatial window provided by the user.

'''Typical use:'''

Obtain climatological data in HDF5 format to use as meteorological forcing in MOHID Water models.

'''Data input requirements:'''

Hellerman Rosenstein climatological data ASCII files, one per wind stress component.

'''Ouput:'''

HDF5 file with Hellerman Rosenstein climatological data and grid data file with the horizontal
grid of the climatological data. This grid data file can be used to interpolate the climatological data from the original horizontal grid to a new grid (see [[#Interpolate files|Interpolate files]]).

'''ConvertToHDF5 action:''' [[#CONVERT HELLERMAN ROSENSTEIN ASCII|CONVERT HELLERMAN ROSENSTEIN ASCII]].

=====''World Ocean Atlas 2005''=====
The World Ocean Atlas (WOA) 2005 climatology provides results for water temperature, salinity and several water quality and biology properties.

Description, Action and Input Files are described in a separate page: [[ConvertToHDF5 WOA2005]].

==Input file (ConvertToHDF5Action.dat)==
===General structure===
<begin_file> (block containing instructions for running a specific action)
ACTION : ... (intended action)
... (action specific instructions)
<end_file>

<begin_file>
ACTION : ...
...
<end_file>

===GLUES HDF5 FILES===
<begin_file>
ACTION : GLUES HDF5 FILES

3D_FILE : 0/1 (0 = 2D file, 1 = 3D file)

TIME_GROUP : ... (Default="Time". Other option: "SurfaceTime".)

BASE_GROUP : ... (Default="Results". Other options: "Residual", "SurfaceResults".)

OUTPUTFILENAME : ... (path/name of HDF5 file to be created)

(block of HDF5 data files)
<<begin_list>>
... (path/name of HDF5 file with data to be included in glue, one per line, at least two files)
...
<<end_list>>
<end_file>

===INTERPOLATE GRIDS===
<begin_file>
ACTION : INTERPOLATE GRIDS

TYPE_OF_INTERPOLATION : ... (type of horizontal interpolation: 1 = Bilinear, 2 = Spline2D,
3 = Triangulation)

INTERPOLATION_WINDOW : ... ... ... ... (2D spatial window to consider for interpolation:
Xmin Ymin Xmax Ymax; default = all domain)

START : ... (start date for output file: yyyy mm dd hh mm ss)
END : ... (end date for output file: yyyy mm dd hh mm ss)

INTERPOLATION3D : 0/1 (0 = 2D interpolation, 1 = 3D interpolation)

FATHER_FILENAME : ... (path/name of input HDF5 file with data to be interpolated)
FATHER_GRID_FILENAME : ... (path/name of input grid data file with horizontal
discretization of input HDF5 file)

OUTPUTFILENAME : ... (path/name of output HDF5 file to be created)
NEW_GRID_FILENAME : ... (path/name of input grid data file with horizontal
discretization intended for output HDF5 file)

EXTRAPOLATE_2D : 0/1/2/3/4/5 (2D extrapolation: 0=no extrapolation, 1=medium
triangulation, 2=high triangulation,
3=nearest neighbour, 4=nearest cell,
5=constant value)

EXTRAPOLATE_VALUE : ... (name of the value to extrapolate to when EXTRAPOLATE_2D is
set to constant value (5))

DO_NOT_BELIEVE_MAP : 0/1 (0=consider input HDF5 file map, 1=do not consider input HDF5
file map)

BASE_GROUP : ... (name of base group of HDF5 variables containing data to be
interpolated; default is "/Results")

(if INTERPOLATION3D : 1 also required:)
FATHER_GEOMETRY : ... (path/name of file (MOHID format) with vertical discretization
of input HDF5 file)
NEW_GEOMETRY : ... (path/name of file (MOHID format) with vertical discretization
intended for output HDF5 file)
POLI_DEGREE : 1/... (degree of vertical interpolation: 1=linear, ...)

AUX_GRID_FILENAME : ... (path/name of input grid data file with horizontal
discretization intended for auxiliar output HDF5 file;
default is file provided in NEW_GRID_FILENAME)

AUX_OUTPUTFILENAME : ... (path/name of auxiliar output HDF5 file to contain result
of horizontal grid interpolation)
<end_file>

''Remarks:''

- the file indicated in AUX_GRID_FILENAME can be different from the one indicated in
NEW_GRID_FILENAME in terms of bathymetry, while the horizontal grid should be, commonly, the
same: this altered bathymetry can be used to extend the water column in the original data so
that the process of vertical interpolation is done easily;

- in case of INTERPOLATION3D : 1, ConvertToHDF5 can generate new versions of bathymetry which
are consistent with the geometry definition (extension is '.new'); there are possibly three
bathymetry changes referring to father grid, new grid and aux grid (the same bathymetry is
not altered twice); although initially new and aux grid are the same they can result
different because of bathymetry changes;

- in case the new geometry is 2D and father geometry is 3D then POLI_DEGREE : 1
(linear interpolation) should be used;

- EXTRAPOLATE_2D : 1/2/3/4/5 should be considered if it is expected that the coast line is not
coincidental in the father and new grids, to avoid lack of data in the interpolation
process; extrapolation is performed for all cells even the land cells;

- in case of DO_NOT_BELIEVE_MAP : 1 the application generates a map based on cells where
interpolation results are available; this causes that if EXTRAPOLATE_2D : 1/2/3/4/5 is used
the AUX_GRID_FILENAME should not have land cells in order for the new map to be concurrent
with the result of extrapolation and avoid errors generation, specially if INTERPOLATION3D :
1 is considered.

===PATCH HDF5 FILES===
<begin_file>
ACTION : PATCH HDF5 FILES

TYPE_OF_INTERPOLATION : ... (type of interpolation: 3 = Triangulation, default and only
one implemented)

START : ... (start date for output file: yyyy mm dd hh mm ss)
END : ... (end date for output file: yyyy mm dd hh mm ss)

(block for each father HDF5 file, should be at least two)
<<begin_father>>
LEVEL : ... (integer priority level: 1 = highest, increase for lower
priority)
FATHER_FILENAME : ... (path/name of input HDF5 file with data to be interpolated)
FATHER_GRID_FILENAME : ... (path/name of input grid data file with horizontal
discretization of input HDF5 file)
<<end_father>>

OUTPUTFILENAME : ... (path/name of output HDF5 file to be created)
NEW_GRID_FILENAME : ... (path/name of input grid data file with horizontal
discretization intended for output HDF5 file)
<end_file>

===CONVERT ERA40 FORMAT===
<begin_file>
ACTION : CONVERT ERA40 FORMAT

FILENAME : ... (path/name of ERA40 NetCDF file)
OUTPUTFILENAME : ... (path/name of HDF5 file to be created)
(root of name for all files produced)

CONVERT_TO_ASCII : 0/1 (1 = convert variable heading info for ASCII file; 0 = default)
CONVERT_TO_HDF5 : 0/1 (1 = convert to HDF5 file; 0 = default)
GRIDTO180 : 0/1 (1 = convert grid from [0 360] to [-180 180], 0 = default)

XX_VARIABLE : ... (name of longitude variable in the input file: usual name
is "longitude")
YY_VARIABLE : ... (name of longitude variable in the input file: usual name
is "latitude")
TIME_VARIABLE : ... (name of time variable in the input file: usual name is
"time")
<end_file>

''Remarks:''

- either CONVERT_TO_ASCII : 1 or CONVERT_TO_HDF5 : 1 must be chosen for any action to be
performed by ConvertToHDF5;

- when CONVERT_TO_HDF5 : 1 an HDF5 file is produced for every variable contained in the
original ERA40 file; the name of each file is composed of the name indicated on FILENAME
concatenated with the ERA40 variable identifier;

- to the XX_VARIABLE, YY_VARIABLE and TIME_VARIABLE keywords should generally be
specified "longitude", "latitude" and "time", respectively; the option to
include as keywords was made only to make the application robust to future variable name
changes.

===CONVERT MM5 FORMAT===
<begin_file>
ACTION : CONVERT MM5 FORMAT

FILENAME : ... (path/name of MM5 file)
TERRAIN_FILENAME : ... (path/name of MM5 TERRAIN file)

OUTPUTFILENAME : ... (path/name of HDF5 file to be created)
OUTPUT_GRID_FILENAME : ... (path/name of grid data file with horizontal grid of MM5 data
to be created)

COMPUTE_WINDSTRESS : 0/1 (1 = compute and write wind stress field; 0 = default)
COMPUTE_RELATIVE_HUMIDITY : 0/1 (1 = compute and write relative humidity field; 0 = default)
COMPUTE_PRECIPITATION : 0/1 (1 = compute and write precipitation field; 0 = default)
COMPUTE_WINDMODULUS : 0/1 (1 = compute wind modulus; 0 = default)

WRITE_XYZ : 0/1 (1 = write xyz center grid cells; 0 = default)
WRITE_TERRAIN : 0/1 (1 = write MM5 TERRAIN fields; 0 = default)

START : ... (start date for output file: yyyy mm dd hh mm ss)
END : ... (end date for output file: yyyy mm dd hh mm ss)

(block of MM5 properties to convert)
<<BeginFields>>
... (name of MM5 property to convert do HDF5 format, one per line)
...
<<EndFields>>

<end_file>

''Remarks:''

- the name of each MM5 property to convert in <<BeginFields>>...<<EndFields>> block must
conform to the MOHID designation specified in code of ModuleGlobalData; the correspondence is
the following (see [[Module_InterfaceWaterAir]] for a more detailed explanation).

---MM5 NAME--- ---MOHID NAME---
T2 air temperature
PSTARCRS atmospheric pressure
U10 wind velocity X
V10 wind velocity Y
UST wind shear velocity
LHFLUX latent heat
SWDOWN sensible heat
SWDOWN solar radiation
LWDOWN infrared radiation
SWOUT top outgoing shortwave radiation
LWOUT top outgoing longwave radiation
SOIL T 1 soil temperature layer 1
SOIL T 1 soil temperature layer 2
SOIL T 1 soil temperature layer 3
SOIL T 1 soil temperature layer 4
SOIL T 1 soil temperature layer 5
SOIL T 1 soil temperature layer 6
Q2 2-meter mixing ratio
TSEASFC sea water temperature
PBL HGT PBL height
PBL REGIME PBL regime
RAIN CON accumulated convective precipitation (cm)
RAIN NON accumulated non-convective precipitation (cm)
GROUND T ground temperature
RES TEMP infinite reservoir slab temperature
U wind velocity X_3D
V wind velocity Y_3D
W wind velocity Z_3D
T air temperature_3D
PP atmospheric pressure_3D
Q mixing ratio_3D
CLW cloud water mixing ratio_3D
RNW rain water mixing ratio_3D
ICE cloud ice mixing ratio_3D
SNOW snow mixing ratio_3D
RAD TEND atmospheric radiation tendency_3D

===CONVERT ALADIN FORMAT===
<begin_file>
ACTION : CONVERT ALADIN FORMAT

OUTPUTFILENAME : aladin.hdf5
OUTPUT_GRID_FILENAME : aladin_griddata.dat

!Put here the name of any netcdf file for grid-data generation's sake.
INPUT_GRID_FILENAME : D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKCLOUD_OPASYMP_19723_20088.nc

<<begin_input_files>>
(path to aladin netcdf file)\ALADIN_BULKIR_OPASYMP_19723_20088.nc
...
<<end_input_files>>

<end_file>

''Remarks:''

- the name of each Aladin property to convert in <<begin_input_files>>...<<end_input_files>> block must conform to the following variables

---ALADIN NAME--- ---MOHID NAME---
soclotot CloudCover_
sohumrel RelativeHumidity_
sofluxir NonSolarFlux_
sosspres AtmosphericPressure_
sosolarf SolarRadiation_
sotemair AirTemperature_
sowinmod WindModulus_
sowaprec Precipitation_
sozotaux WindStressX_
sometauy WindStressY_
sowindu10 WindVelocityX_
sowindv10 WindVelocityY_

===CONVERT MERCATOR FORMAT===
<begin_file>
ACTION : CONVERT MERCATOR FORMAT

READ_OPTION : 1/2/3/4 (version of MERCATOR files)

OUTPUTFILENAME : ... (path/name of HDF5 file to be created)
OUTPUT_GRID_FILENAME : ... (path/name of grid data with horizontal discretization to be
created)
OUTPUT_GEOMETRY_FILENAME : ... (path/name of geometry file with vertical discretization to be
created)

(if READ_OPTION : 1:)
BASE_BULLETIN : ...
DATES_FILE : ...
NUM_DATES : ...

(if READ_OPTION : 2/3:)
INPUT_GRID_FILENAME : ... (path/name of file with horizontal discretization of water
properties and water level data)
(if READ_OPTION : 2:)
INPUT_GRID_FILENAME_U : ... (path/name of file with horizontal discretization of velocity
component U data)
INPUT_GRID_FILENAME_V : ... (path/name of file with horizontal discretization of velocity
component V data)

(if READ_OPTION : 3:)
INPUT_BATHY_FILENAME : ... (path/name of file with bathymetry)

(if READ_OPTION : 3/4:)
CALC_BAROTROPIC_VEL : 0/1 (1 = calculate barotropic velocity, 0 = not calculate;
default = 0)

(if CALC_BAROTROPIC_VEL : 1 and READ_OPTION : 3:)
INPUT_MESH_ZGRID_FILENAME : ... (path/name of file with information about layers ticknesses)

(block of MERCATOR data files)
<<begin_input_files>>
... (path/name of MERCATOR NetCDF data file, one per line, can be several)
...
<<<end_input_files>>>

<end_file>

===CONVERT LEVITUS FORMAT===
<begin_file>
ACTION : CONVERT LEVITUS FORMAT

OUTPUTFILENAME : ... (path/name of HDF5 file to be created)
OUTPUT_GRID_FILENAME : ... (path/name of grid data with horizontal discretization to be
created)
OUTPUT_GEOMETRY_FILENAME : ... (path/name of geometry file with vertical discretization to be
created)

PERIODICITY : ... (periodicity of Levitus data: "monthly"/"annual"; default is
"monthly")

SPATIAL_RESOLUTION : ... (spatial resolution (degrees) of horizontal Levitus grid)

FILL_VALUE : ... (real value identificator for missing data; default is
"-99.999900")

(definition of spatial window to be present in output HDF5 file)
LOWER_LEFT_CORNER : ... ... (longitude and latitude (degrees) of south west corner)
UPPER_RIGHT_CORNER : ... ... (longitude and latitude (degrees) of north east corner)

(block for each water property to be present in output HDF5 file, can be several)
<<beginfield>>
NAME : ... (name of property)
ANNUAL_FILE : ... (path/name of Levitus annual file)

(block of Levitus data files)
<<<begin_input_files>>>
... (path/name of Levitus data file (e.g. a monthly data file), one per line, can be several)
...
<<<end_input_files>>>

<<endfield>>

<end_file>

===CONVERT HELLERMAN ROSENSTEIN ASCII===
<begin_file>
ACTION : CONVERT HELLERMAN ROSENSTEIN ASCII

OUTPUTFILENAME : ... (path/name of HDF5 file to be created)
OUTPUT_GRID_FILENAME : ... (path/name of grid data with horizontal discretization to be
created)

PERIODICITY : ... (periodicity of Hellerman Rosenstein data: "monthly")

SPATIAL_RESOLUTION : ... (spatial resolution (degrees) of horizontal Hellerman
Rosenstein grid: default and only allowed value is "2.")

FILL_VALUE : ... (real value identificator for missing data; default is
"-99.999900")

(definition of spatial window to be present in output HDF5 file)
LOWER_LEFT_CORNER : ... ... (longitude and latitude (degrees) of south west corner)
UPPER_RIGHT_CORNER : ... ... (longitude and latitude (degrees) of north east corner)

(block for each Hellerman Rosenstein data file)
<<beginfield>>
NAME : ... (name of property: "wind stress X"/"wind stress Y")
FILE : ... (path/name Hellerman Rosenstein file)
<<endfield>>

<end_file>

==Samples==
All sample files are named ''ConvertToHDF5Action.dat''.

===Glue several MOHID(.hdf5) files===
<begin_file>
ACTION : GLUES HDF5 FILES

OUTPUTFILENAME : SurfaceHydro_OP.hdf5

<<begin_list>>
D:\Projectos\SurfaceHydrodynamic_21.hdf5
D:\Projectos\SurfaceHydrodynamic_22.hdf5
<<end_list>>
<end_file>

===Interpolate 2D MOHID(.hdf5) files to a new grid===
<begin_file>
ACTION : INTERPOLATE GRIDS

TYPE_OF_INTERPOLATION : 1
FATHER_FILENAME : D:\Projectos\MohidRun\test\res\Lagrangian_1.hdf5
OUTPUTFILENAME : OilSpillThickness_GridRegular.hdf5

START : 2006 6 21 17 22 30
END : 2006 6 22 17 22 0

FATHER_GRID_FILENAME : D:\Projectos\MohidRun\GeneralData\batim\Tagus.dat_A
NEW_GRID_FILENAME : TagusConstSpacing.dat

BASE_GROUP : /Results/Oil/Data_2D
<end_file>

===Interpolate 3D MOHID(.hdf5) files to a new grid===
<begin_file>
ACTION : INTERPOLATE GRIDS

TYPE_OF_INTERPOLATION : 1
FATHER_FILENAME : D:\Projectos\MohidRun\test\res\Lagrangian_1.hdf5
OUTPUTFILENAME : OilSpillThickness_GridRegular.hdf5

START : 2006 6 21 17 22 30
END : 2006 6 22 17 22 0

FATHER_GRID_FILENAME : D:\Projectos\MohidRun\GeneralData\batim\Tagus.dat_A
NEW_GRID_FILENAME : TagusConstSpacing.dat

BASE_GROUP : /Results/Oil/Data_2D

INTERPOLATION3D : 1
FATHER_GEOMETRY : D:\Projectos\MohidRun\test\data\Geometry_1.dat
NEW_GEOMETRY : TagusGeometry.dat
AUX_GRID_FILENAME : TagusConstSpacing.dat
AUX_OUTPUTFILENAME : Aux_GridRegular.hdf5
<end_file>

===Patch several MOHID(.hdf5) files to a new grid===
<begin_file>
ACTION : PATCH HDF5 FILES

TYPE_OF_INTERPOLATION : 3

START : 2005 2 28 13 0 0
END : 2005 3 1 13 0 0

<<begin_father>>
LEVEL : 3
FATHER_FILENAME : K:\MM5output\2005022812_2005030712\MM5OUT_D1.hdf5
FATHER_GRID_FILENAME : K:\MM5output\2005022812_2005030712\grid1.dat
<<end_father>>

<<begin_father>>
LEVEL : 2
FATHER_FILENAME : K:\MM5output\2005022812_2005030712\MM5OUT_D2.hdf5
FATHER_GRID_FILENAME : K:\MM5output\2005022812_2005030712\grid2.dat
<<end_father>>

<<begin_father>>
LEVEL : 1
FATHER_FILENAME : K:\MM5output\2005022812_2005030712\MM5OUT_D3.hdf5
FATHER_GRID_FILENAME : K:\MM5output\2005022812_2005030712\grid3.dat
<<end_father>>

OUTPUTFILENAME : MM5Forcing.hdf5
NEW_GRID_FILENAME : K:\Simula\GeneralData\Batim\CostaPortuguesa.dat
<end_file>

===Convert an ERA40 file to MOHID(.hdf5)===
<begin_file>
ACTION : CONVERT ERA40 FORMAT

FILENAME : D:\Aplica\ERA40\1971ERA1973.nc
OUTPUTFILENAME : D:\Aplica\ERA40\1971ERA1973T2

CONVERT_TO_ASCII : 0
CONVERT_TO_HDF5 : 1

XX_VARIABLE : longitude
YY_VARIABLE : latitude
TIME_VARIABLE : time
<end_file>

===Convert a MM5 file to MOHID(.hdf5)===
<begin_file>
ACTION : CONVERT MM5 FORMAT

FILENAME : K:\MM5output\DataCenter\Modelos\Meteo_IST\Fev2005\MMOUT_D3
TERRAIN_FILENAME : K:\MM5output\DataCenter\Modelos\Meteo_IST\Fev2005\TERRAIN_D3
OUTPUT_GRID_FILENAME : K:\MM5output\DataCenter\Modelos\Meteo_IST\Fev2005\grid3.dat
OUTPUTFILENAME : K:\MM5output\DataCenter\Modelos\Meteo_IST\Fev2005\MM5_D3.hdf5

COMPUTE_WINDSTRESS : 0
COMPUTE_RELATIVE_HUMIDITY : 1
WRITE_XYZ : 0

<<BeginFields>>
solar radiation
air temperature
wind velocity X
wind velocity Y
sensible heat
latent heat
atmospheric pressure
sea water temperature
<<EndFields>>
<end_file>

===Convert Mercator-Ocean(.nc) to MOHID(.hdf5)===
<begin_file>
ACTION : CONVERT MERCATOR FORMAT

OUTPUTFILENAME : Psy2v2r1v_R20060628/MercatorR20060628.hdf5
OUTPUT_GRID_FILENAME : Psy2v2r1v_R20060628/MercatorGridR20060628.dat
OUTPUT_GEOMETRY_FILENAME : Psy2v2r1v_R20060628/MercatorGeometryR20060628.dat

INPUT_GRID_FILENAME : GridFiles/ist_meteog-gridT.nc
INPUT_GRID_FILENAME_U : GridFiles/ist_meteog-gridU.nc
INPUT_GRID_FILENAME_V : GridFiles/ist_meteog-gridV.nc

<<begin_input_files>>
Psy2v2r1v_R20060628/ist_meteog-mercatorPsy2v2r1v_T_MEAN_ANA_20060621_R20060628.nc
Psy2v2r1v_R20060628/ist_meteog-mercatorPsy2v2r1v_T_MEAN_ANA_20060622_R20060628.nc
Psy2v2r1v_R20060628/ist_meteog-mercatorPsy2v2r1v_T_MEAN_ANA_20060623_R20060628.nc
<<end_input_files>>
<end_file>

===Convert Levitus format to MOHID(.hdf5) and interpolate grid===
==== Convert ====
First convert the Levitus ASCII format to a raw HDF5 format:
<begin_file>
ACTION : CONVERT LEVITUS FORMAT

OUTPUTFILENAME : Levitus.hdf5
OUTPUT_GRID_FILENAME : LevitusGrid.dat
OUTPUT_GEOMETRY_FILENAME : LevitusGeometry.dat

PERIODICITY : monthly
SPATIAL_RESOLUTION : 0.25
FILL_VALUE : -99.9999

LOWER_LEFT_CORNER : -16.0 31
UPPER_RIGHT_CORNER : 1. 40

<<beginfield>>
NAME : salinity
ANNUAL_FILE : DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s000hr.obj

<<<begin_input_files>>>
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s001
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s002
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s003
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s004
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s005
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s006
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s007
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s008
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s009
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s010
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s011
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s012
<<<end_input_files>>>
<<endfield>>

<<beginfield>>
NAME : temperature
ANNUAL_FILE : DataCenter\DadosBase\Ocean\Levitus\Data\Temp\t000hr.obj

<<<begin_input_files>>>
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t001
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t002
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t003
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t004
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t005
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t006
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t007
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t008
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t009
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t010
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t011
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t012
<<<end_input_files>>>
<<endfield>>
<end_file>

==== Extrapolate ====
Then extrapolate the data (still in the raw HDF5 format):
<begin_file>
ACTION : INTERPOLATE GRIDS

TYPE_OF_INTERPOLATION : 1

FATHER_FILENAME : Levitus.hdf5
FATHER_GRID_FILENAME : LevitusGrid.dat

OUTPUTFILENAME : LeviTusAllPointsWithData.hdf5
NEW_GRID_FILENAME : LevitusGrid.dat

START : -9999 1 1 0 0 0
END : -9999 12 1 0 0 0

INTERPOLATION3D : 1
FATHER_GEOMETRY : LevitusGeometry.dat
NEW_GEOMETRY : LevitusGeometry.dat
AUX_GRID_FILENAME : LevitusGrid.dat
AUX_OUTPUTFILENAME : AuxLeviTusAllPointsWithData.hdf5

POLI_DEGREE : 3
DO_NOT_BELIEVE_MAP : 1
EXTRAPOLATE_2D : 2
<end_file>

==== Interpolate ====
Finally, interpolate to the final grid and geometry (same as the [[#Interpolate 3D MOHID(.hdf5) files to a new grid| Interpolate 3D sample]]):
<begin_file>
ACTION : INTERPOLATE GRIDS

TYPE_OF_INTERPOLATION : 1
FATHER_FILENAME : LeviTusAllPointsWithData.hdf5
OUTPUTFILENAME : CadizMonthlyLevitus.hdf5
FATHER_GRID_FILENAME : LevitusGrid.dat
NEW_GRID_FILENAME : Algarve0.02SigmaSmooth_V3_CartMoreLayers.dat

START : -9999 1 1 0 0 0
END : -9999 12 1 0 0 0

INTERPOLATION3D : 1
FATHER_GEOMETRY : LevitusGeometry.dat
NEW_GEOMETRY : Geometry_1.dat
AUX_OUTPUTFILENAME : AuxCadizMonthlyLevitus.hdf5
AUX_GRID_FILENAME : Aux12km.dat

POLI_DEGREE : 3
DO_NOT_BELIEVE_MAP : 1
<end_file>

Note that the programme may construct a new bathymetry twice. Use this bathymetry only on the AUX_GRID_FILENAME keyword.

===Convert Hellerman Rosenstein ASCII format to MOHID(.hdf5) ===
<begin_file>
ACTION : CONVERT HELLERMAN ROSENSTEIN ASCII

OUTPUTFILENAME : ClimatologicWindStress.hdf5
OUTPUT_GRID_FILENAME : ClimatologicWindStressGrid.dat

PERIODICITY : monthly
SPATIAL_RESOLUTION : 2.
FILL_VALUE : -99.9999

LOWER_LEFT_CORNER : -180 -90
UPPER_RIGHT_CORNER : 180 90

<<beginfield>>
NAME : wind stress X
FILE : D:\Aplica\Dados\Hellerman_Rosenstein\TAUXX.DAT
<<endfield>>

<<beginfield>>
NAME : wind stress Y
FILE : D:\Aplica\Dados\Hellerman_Rosenstein\TAUYY.DAT
<<endfield>>
<end_file>

===Convert ALADIN(.nc) format to MOHID(.hdf5)===
<begin_file>
ACTION : CONVERT ALADIN FORMAT

OUTPUTFILENAME : aladin.hdf5
OUTPUT_GRID_FILENAME : aladin_griddata.dat

!Put here the name of any netcdf file for grid-data generation's sake.
INPUT_GRID_FILENAME : D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKCLOUD_OPASYMP_19723_20088.nc

<<begin_input_files>>
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKIR_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKPRES_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKSOLAR_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKTAIR_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKWIND_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_FLUXPRE_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_STRESSU_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_STRESSV_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_U10_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_V10_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKCLOUD_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKHUMI_OPASYMP_19723_20088.nc
<<end_input_files>>

<end_file>

== OceanColor modules compilation ==
Compiling the [[ConvertToHDF5]] tool with the OceanColor modules is more complicated than one might expect. A solution is proposed here for a release version using the Compaq Visual Fortran 6.6c. The difficulties rise because C code is embedded with a fortran interface and also, extra libraries such as hdf4 are required.

=== Pre-requisites ===

This is a list of prerequisites to successfully compile the tool:
*Compaq Visual Fortran 6.5 with patch 6.6c,
*VS .NET 2003 (Vc7 in particular),
*Hdf5 libraries ('''hdf5.lib''' '''hdf5_fortran.lib''' '''hdf5_hl.lib'''),
*Netcdf libraries ('''netcdf.lib''' '''netcdf_.lib'''),
*Hdf4 libraries ('''hd421.lib''', '''hm421.lib'''),
*szlib, zlib and jpeg libraries ('''szlib.lib''', '''zlib.lib''' and '''libjpeg.lib'''),
*the fortran source files ('''ModuleConvertModisL2.F90 ModuleConvertModisL3.F90 ModuleConvertOceanColorL2.F90'''),
*the C source files and their fortran interface files ('''readL2scan.c readL2Seadas.c''' and '''cdata.f crossp.f fgeonav.f''').

=== CVF IDE configuration ===
# Configure everything as specified in [[Compiling with CVF]].
# Add the source files listed in the prerequisites above to the source files listing.
# Go to '''Tools-->Options...-->Directories'''. There, add the '''$DOTNET2K3/Vc7/bin''' to the '''Executable files''''; the '''$DOTNET2K3/Vc7/include''' and '''$DOTNET2K3/Vc7/PlatformSDK/include''' to the '''Include files'''; and finally, the '''$DOTNET2K3/Vc7/lib''', '''$DOTNET2K3/Vc7/PlatformSDK/lib''' and '''$DOTNET2K3/Vc7/PlatformSDK/bin''' to the '''Library files'''.
# Go to '''Projects-->Settings-->Release-->Link-->Input'''. There, add the following libraries: '''netcdf.lib netcdf_.lib hd421.lib hm421.lib libjpeg.lib'''. (Make sure the hdf5 libraries as well as the szlib and zlib libraries are already mentioned).

=== Troubleshoots ===
'''Q: I get unresolved external references during linkage, but I have all the libraries mentioned above included. What should I do?'''

A: Unresolved external references can come out for two reasons:
#you didn't specified all the libraries required or all the paths for the default libraries or,
#[http://en.wikipedia.org/wiki/Name_decoration name mangling] problems. Use the [[dumpbin]] utility to the libraries to checkout which language convention they are using. If that's the problem then you need to try to get new libraries with the correct naming convention.

That's it, you should now be able to build the [[ConvertToHdf5]] project successfully.

'''Q: I got a message saying the entry point _NF_PUT_ATT_REAL@28 could not be located in netcdf.dll'''

A: copy the file netcdf.dll to the exe folder

== References ==
*[http://www.hdfgroup.org/ HDF5 Homepage]
*[http://www.hdfgroup.org/ HDF4 Homepage]

== Links ==

*[[Module_Atmosphere]]
*[[Module_InterfaceWaterAir]]
*[[Coupling_Water-Atmosphere_User_Manual]]

[[Category:Tools]]
[[Category:Hdf5]]

ConvertToHDF5

2009-05-12T12:57:28Z

AngelaCanas: /* CONVERT MERCATOR FORMAT */

ConvertToHDF5

2009-05-12T12:36:18Z

AngelaCanas: /* CONVERT MERCATOR FORMAT */

The '''ConvertToHDF5''' is an application which allows the making of several operations, called '''actions''', involving HDF5 files: conversion of data in other formats (e.g. NETCDF) to HDF5, grid interpolation, concatenation of several files.

Running options for this application are specified by the user in a input file named [[ConvertToHDF5#Input file (ConvertToHDF5Action.dat)|'''ConvertToHDF5Action.dat''']]. Several actions can be specified in the same input file, being processed sequentially by the ConvertToHDF5 application.

==Introduction==
The operations involving HDF5 files performed by ConvertToHDF5, specified individually by an action, can be organized in [[#file management|file management]], [[#grid interpolation|grid interpolation]] and [[#format conversion|format conversion]]. These types and the respective actions are detailed in the next sections.

The input file specification for each action can be found bellow in the [[#Input file (ConvertToHDF5Action.dat)|Input file (ConvertToHDF5Action.dat)]] section.

===File management===

====Glue files====
This action consists in joining or glue in a single HDF5 file two or more HDF5 files having the same HDF5 data groups and referring to time periods which come in sequence. Both sets of 2D and 3D HDF5 files can be glued.

'''Typical use:'''

Glue MOHID Water results files from several runs produced in continuous running of the model, for storage space economy reasons. Can be used to join data from other origins (e.g. results of meteorological models) as long as the HDF5 format is the one supported by MOHID Water.

'''Data input requirements:'''

HDF5 files to be glued. "Grid" and "Results" data groups should be equal in all these files.

'''Output:'''

HDF5 file with glued "Results" data. "Residual" and "Statistics" HDF5 data groups are not copied to the output file since they are time period specific (different values potentially occour in each file). General statistics can be calculated for the glued HDF5 file data using tool [[HDF5Statistics]].

'''ConvertToHDF5 action:''' [[#GLUES HDF5 FILES|GLUES HDF5 FILES]].

===Grid interpolation===

====Interpolate files====
This action performs the conversion of one HDF5 file data existing in one 2D or 3D spatial grid to another 2D or 3D spatial grid, creating a new HDF5 file. The interpolation is performed only for the data located a time window specified by the user.

The HDF5 file containing data to be interpolated is called the '''father file'''.

In case of 3D interpolation the application conducts first the horizontal grid interpolation
(keeping father geometry) and only after it conducts the vertical interpolation (from father geometry to new geometry).

Several types of 2D interpolation are available for use: bilinear, spline 2D and triangulation.
For vertical interpolation (used in 3D interpolation) can be supplied several polinomial degrees for interpolation.

'''Typical use:'''

Obtain an HDF5 file with data for forcing or providing initial conditions for a MOHID Water model, e.g. a meteorological forcing file.

'''Data input requirements:'''

For 2D/3D interpolation:

- father HDF5 file;

- father horizontal data grid, in a grid data file in the format supported by MOHID;

- new horizontal data grid, in a grid data file in the format supported by MOHID;

For 3D interpolation also needed:

- father vertical geometry, in a geometry file in the format supported by MOHID;

- new vertical geometry, in a geometry file in the format supported by MOHID;

- auxiliary horizontal data grid, in a grid data file in the format supported by MOHID; this file is used for horizontal grid interpolation in 3D interpolation operations.

'''Ouput:'''

HDF5 file with interpolated data. In case of 3D interpolation also produced an auxiliary HDF5 file with the result of the horizontal grid interpolation, which can be inspected to check if this operation is well performed.

'''ConvertToHDF5 action:''' [[#INTERPOLATE GRIDS|INTERPOLATE GRIDS]].

====Patch files====
This action consists in performing an interpolation of HDF5 data between grids, as in action [[#Interpolate files|Interpolate files]], but considering more than one HDF5 file as containing data to be interpolated to the new grid and a priority scale. The interpolation is performed only for the data located in the time window specified by the user. The present version of this action operates only on 2D data.

Each HDF5 file containing data to be interpolated is called a '''father file''' and has an user-attributed '''priority level''' to be respected in the interpolation process: for each new grid cell the ConvertToHDF5 application will look for data first on the Level 1 father file and only in the case this data is inexistent will it look for data in Level 2 file, proceeding in looking for higher level files if no data is found subsequentely.

'''Typical use:'''

To obtain an HDF5 file with data from several HDF5 files each containing data with different spatial resolution and only for a specific part of the new grid. This is, for instance, the case when one is preparing a best resolution meteorological HDF5 file for forcing MOHID Water from several meteorological model domains, having different spatial resolution and span, since the best resolution data is not available for all new grid cells.

'''Data input requirements:'''

The new horizontal data grid, in a grid data file in the format supported by MOHID, and for each father file:

- level of priority: 1 = maximum priority, priority decreases with increasing level value;

- data grid, in the form of a grid data file in the format supported by MOHID.

'''Ouput:'''

HDF5 file with patched data.

'''ConvertToHDF5 action:''' [[#PATCH HDF5 FILES|PATCH HDF5 FILES]].

===Format conversion===

====Meteorological model data====
Mohid does not simulate explicitly the atmosphere, but needs information about atmospheric properties in time and space. This requires that atmospheric properties are supplied to MOHID Water in supported formats. These formats can be derived from meteorological data in HDF5 format. Because the results of meteorological models are accessed in different formats conversion is required.

The formats currently convertible to HDF5 in ConvertToHDF5 include the MM5 and the ERA40. These are succintly detailed in the next sections.

=====''ERA40''=====
This format refers to the European Centre for Medium-Range Weather Forecasts (ECMWF) 40 years re-analysises results, acessed by site http://data.ecmwf.int/data/d/era40_daily/. This data is available for several meteorological variables with maximum 6 hour periodicity for days in the period from 1957-09-01 to 2002-08-31.

ERA40 data files are supplied by ECMWF in a NetCDF format and with an user-costumized time window, periodicity (time step range from 6 hours to a day) and meteorological properties set. The ERA40 meteorological properties which are recognized by MOHID are presented bellow together with the correspondent MOHID name:

---ERA40 NAME--- ---MOHID NAME---
sshf sensible heat
slhf latent heat
msl atmospheric pressure
tcc cloud cover
p10u wind velocity X
p10v wind velocity Y
p2t air temperature
ewss wind stress X
nsss wind stress Y

The standard ConvertToHDF5 action is to convert to HDF5 the data referring to all MOHID Water recognized property available in the ERA40 file, producing an individual HDF5 file for each property. The name of each HDF5 file generated includes the ERA40 meteorological property identificator correspondent to the data contained.

Alternatively, ConvertToHDF5 can copy to a single ASCII file the heading information concerning each meteorological variable considered in the original ERA40 file.

'''Typical use:'''

Obtain an HDF5 file with data suitable for being used for forcing MOHID Water models.

'''Data input requirements:'''

ERA40 NetCDF file.

'''Output:'''

One HDF5 file for each meteorological property contained in the original NetCDF file.

'''ConvertToHDF5 action:''' [[#CONVERT ERA40 FORMAT|CONVERT ERA40 FORMAT]].

=====''MM5''=====
This format relates to the Fifth-Generation NCAR / Penn State Mesoscale Model (MM5) output files format. Almost every atmospheric property needed by MOHID Water is present in MM5 output files, enabling to run prediction simulations with MOHID Water when access to MM5 prevision files is available.

The ConvertToHDF5 action converts MM5 results files from the original format to HDF5 format, allowing the easy use of these results in the MOHID framework. Conversion is only performed for the MM5 properties and the time window specified by the user.

Besides the conversion, the application can calculate some properties not contained in
the MM5 files using the available information: these are windstress, relative humidity and precipitation.

For conversion to be completed it is required the horizontal grid information of MM5 results which is available in special TERRAIN files.

'''Typical use:'''

Produce HDF5 meteorological data usable for forcing MOHID Water models.

'''Data input requirements:'''

MM5 results file to convert and MM5 TERRAIN file. The TERRAIN file supplies the MM5 results grid information.

'''Ouput:'''

An HDF5 file with MM5 results and a grid data file in MOHID format with the MM5 grid information.
This last file can be used to interpolate the MM5 data from the original grid to a new grid (see [[#Interpolate files|Interpolate files]]), for instance to produce an HDF5 file suitable for forcing MOHID Water models.

'''Compilation:'''

Caution! The ConvertToHDF5 executable must be compiled with the [[Big-endian little-endian|Big-Endian]] option set (see compatibility in the project's settings).

'''ConvertToHDF5 action:''' [[#CONVERT MM5 FORMAT|CONVERT MM5 FORMAT]].

=====''Aladin''=====
This format relates to Aladin meteorological model results. Some of the atmospheric property needed by MOHID Water is present in Aladin output files, enabling to run prediction simulations with MOHID Water when access to Aladin prevision files is available.

The ConvertToHDF5 action converts Aladin results files from the original format to HDF5 format, allowing the easy use of these results in the MOHID framework. Conversion is only performed for the MM5 properties and the time window specified by the user.

'''Typical use:'''

Produce HDF5 meteorological data usable for forcing MOHID Water models.

'''Data input requirements:'''

Aladin netcdf results file to convert.

'''Ouput:'''

An HDF5 file with Aladin results and a grid data file in MOHID format with the Aladin grid pseudo-information: a fake orography is created of 100 m depth.
This last file can be used to interpolate the Aladin data from the original grid to a new grid (see [[#Interpolate files|Interpolate files]]), for instance to produce an HDF5 file suitable for forcing MOHID Water models.

'''Compilation:'''

Caution! The ConvertToHDF5 executable must be compiled with the [[Big-endian little-endian|Big-Endian]] option set (see compatibility in the project's settings).

'''ConvertToHDF5 action:''' [[#CONVERT ALADIN FORMAT|CONVERT ALADIN FORMAT]].

====Ocean model data====
Ocean model data, available in diverse formats, can be used by MOHID Water to specify boundary (open ocean boundary and surface), initial conditions or for validation. These uses require that the model data is in HDF5 format and conversion is therefore needed.

Currently the large scale ocean models formats convertible into HDF5 by ConvertToHDF5 includes MERCATOR.

=====''MERCATOR''=====
MERCATOR data files are supplied in a NetCDF format and with an user-costumized spatial window and periodicity. Water level and water properties (temperature and salinity) data is available in type T files, velocity component u data is available in type U files and velocity component v data is available in type V files. The type of data of a specific MERCATOR file is generally indicated in the file name.

The standard ConvertToHDF5 action is to convert to HDF5 the data referring to temperature, salinity, water level, component u of velocity and component v of velocity.

'''Typical use:'''

Obtain HDF5 MERCATOR data usable for forcing or validation of MOHID Water models.

'''Data input requirements:'''

NetCDF MERCATOR results data files and NetCDF MERCATOR grid data files. It should be provided one grid data file of each type: T, U and V. These are generally provided by the MERCATOR services together with the results files.

'''Ouput:'''

One HDF5 file containing all properties contained in the recognized set of properties (temperature, salinity, water level, velocity u and velocity v) and the correspondent grid data and geometry files, containing respectively the horizontal grid and the vertical discretization of the HDF5 file. The grid data and geometry files can be used afterwards to interpolate the MERCATOR data to another grid and geometry (see [[#Interpolate files|Interpolate files]]).

'''ConvertToHDF5 action:''' [[#CONVERT MERCATOR FORMAT|CONVERT MERCATOR FORMAT]].

====Climatological data====
Climatological data can be used in MOHID Water to specify boundary (open ocean boundary and surface), initial conditions or for validation, in case more realistic data (measurements or model) data is unavailable. This data is generally supplied by producers in formats not readly usable by MOHID Water which justifies the existence of a conversion tool.

Two climatological data format conversions are implemented in ConvertToHDF5: Levitus ocean data and Hellerman Rosenstein meteorological data.

=====''Levitus''=====
The Levitus climatology provides results for water temperature and salinity.
The ConvertToHDF5 action converts the climatological data for the properties and spatial window requested by the user.
Typically, it requires 3 steps to complete the task:

- convert levitus format

- extrapolate the data to the whole levitus domain(required to avoid uncoincidental coastlines)

- interpolate with the model grid(bathymetry)

'''Typical use:'''

Obtain climatological data in HDF5 format to use as boundary forcing and/or initial condition specification in MOHID Water models.

'''Data input requirements:'''

Levitus climatological data files, one per property and per time period (e.g a month).

'''Ouput:'''

HDF5 file with Levitus climatological data, grid data file with the horizontal
grid of the data and a geometry file with vertical discretization of the data (MOHID formats).
The grid data and the geometry files can be used to interpolate the climatological data from the original grid to a new grid (see [[#Interpolate files|Interpolate files]]).

'''ConvertToHDF5 action:''' [[#CONVERT LEVITUS FORMAT|CONVERT LEVITUS FORMAT]].

=====''Hellerman Rosenstein''=====
This is a meteorological climatology providing wind stress. There is a file per wind stress component. Since the data refer to surface values it is a 2D field.

The ConvertToHDF5 action converts the climatological data for the properties and spatial window provided by the user.

'''Typical use:'''

Obtain climatological data in HDF5 format to use as meteorological forcing in MOHID Water models.

'''Data input requirements:'''

Hellerman Rosenstein climatological data ASCII files, one per wind stress component.

'''Ouput:'''

HDF5 file with Hellerman Rosenstein climatological data and grid data file with the horizontal
grid of the climatological data. This grid data file can be used to interpolate the climatological data from the original horizontal grid to a new grid (see [[#Interpolate files|Interpolate files]]).

'''ConvertToHDF5 action:''' [[#CONVERT HELLERMAN ROSENSTEIN ASCII|CONVERT HELLERMAN ROSENSTEIN ASCII]].

=====''World Ocean Atlas 2005''=====
The World Ocean Atlas (WOA) 2005 climatology provides results for water temperature, salinity and several water quality and biology properties.

Description, Action and Input Files are described in a separate page: [[ConvertToHDF5 WOA2005]].

==Input file (ConvertToHDF5Action.dat)==
===General structure===
<begin_file> (block containing instructions for running a specific action)
ACTION : ... (intended action)
... (action specific instructions)
<end_file>

<begin_file>
ACTION : ...
...
<end_file>

===GLUES HDF5 FILES===
<begin_file>
ACTION : GLUES HDF5 FILES

3D_FILE : 0/1 (0 = 2D file, 1 = 3D file)

TIME_GROUP : ... (Default="Time". Other option: "SurfaceTime".)

BASE_GROUP : ... (Default="Results". Other options: "Residual", "SurfaceResults".)

OUTPUTFILENAME : ... (path/name of HDF5 file to be created)

(block of HDF5 data files)
<<begin_list>>
... (path/name of HDF5 file with data to be included in glue, one per line, at least two files)
...
<<end_list>>
<end_file>

===INTERPOLATE GRIDS===
<begin_file>
ACTION : INTERPOLATE GRIDS

TYPE_OF_INTERPOLATION : ... (type of horizontal interpolation: 1 = Bilinear, 2 = Spline2D,
3 = Triangulation)

INTERPOLATION_WINDOW : ... ... ... ... (2D spatial window to consider for interpolation:
Xmin Ymin Xmax Ymax; default = all domain)

START : ... (start date for output file: yyyy mm dd hh mm ss)
END : ... (end date for output file: yyyy mm dd hh mm ss)

INTERPOLATION3D : 0/1 (0 = 2D interpolation, 1 = 3D interpolation)

FATHER_FILENAME : ... (path/name of input HDF5 file with data to be interpolated)
FATHER_GRID_FILENAME : ... (path/name of input grid data file with horizontal
discretization of input HDF5 file)

OUTPUTFILENAME : ... (path/name of output HDF5 file to be created)
NEW_GRID_FILENAME : ... (path/name of input grid data file with horizontal
discretization intended for output HDF5 file)

EXTRAPOLATE_2D : 0/1/2/3/4/5 (2D extrapolation: 0=no extrapolation, 1=medium
triangulation, 2=high triangulation,
3=nearest neighbour, 4=nearest cell,
5=constant value)

EXTRAPOLATE_VALUE : ... (name of the value to extrapolate to when EXTRAPOLATE_2D is
set to constant value (5))

DO_NOT_BELIEVE_MAP : 0/1 (0=consider input HDF5 file map, 1=do not consider input HDF5
file map)

BASE_GROUP : ... (name of base group of HDF5 variables containing data to be
interpolated; default is "/Results")

(if INTERPOLATION3D : 1 also required:)
FATHER_GEOMETRY : ... (path/name of file (MOHID format) with vertical discretization
of input HDF5 file)
NEW_GEOMETRY : ... (path/name of file (MOHID format) with vertical discretization
intended for output HDF5 file)
POLI_DEGREE : 1/... (degree of vertical interpolation: 1=linear, ...)

AUX_GRID_FILENAME : ... (path/name of input grid data file with horizontal
discretization intended for auxiliar output HDF5 file;
default is file provided in NEW_GRID_FILENAME)

AUX_OUTPUTFILENAME : ... (path/name of auxiliar output HDF5 file to contain result
of horizontal grid interpolation)
<end_file>

''Remarks:''

- the file indicated in AUX_GRID_FILENAME can be different from the one indicated in
NEW_GRID_FILENAME in terms of bathymetry, while the horizontal grid should be, commonly, the
same: this altered bathymetry can be used to extend the water column in the original data so
that the process of vertical interpolation is done easily;

- in case of INTERPOLATION3D : 1, ConvertToHDF5 can generate new versions of bathymetry which
are consistent with the geometry definition (extension is '.new'); there are possibly three
bathymetry changes referring to father grid, new grid and aux grid (the same bathymetry is
not altered twice); although initially new and aux grid are the same they can result
different because of bathymetry changes;

- in case the new geometry is 2D and father geometry is 3D then POLI_DEGREE : 1
(linear interpolation) should be used;

- EXTRAPOLATE_2D : 1/2/3/4/5 should be considered if it is expected that the coast line is not
coincidental in the father and new grids, to avoid lack of data in the interpolation
process; extrapolation is performed for all cells even the land cells;

- in case of DO_NOT_BELIEVE_MAP : 1 the application generates a map based on cells where
interpolation results are available; this causes that if EXTRAPOLATE_2D : 1/2/3/4/5 is used
the AUX_GRID_FILENAME should not have land cells in order for the new map to be concurrent
with the result of extrapolation and avoid errors generation, specially if INTERPOLATION3D :
1 is considered.

===PATCH HDF5 FILES===
<begin_file>
ACTION : PATCH HDF5 FILES

TYPE_OF_INTERPOLATION : ... (type of interpolation: 3 = Triangulation, default and only
one implemented)

START : ... (start date for output file: yyyy mm dd hh mm ss)
END : ... (end date for output file: yyyy mm dd hh mm ss)

(block for each father HDF5 file, should be at least two)
<<begin_father>>
LEVEL : ... (integer priority level: 1 = highest, increase for lower
priority)
FATHER_FILENAME : ... (path/name of input HDF5 file with data to be interpolated)
FATHER_GRID_FILENAME : ... (path/name of input grid data file with horizontal
discretization of input HDF5 file)
<<end_father>>

OUTPUTFILENAME : ... (path/name of output HDF5 file to be created)
NEW_GRID_FILENAME : ... (path/name of input grid data file with horizontal
discretization intended for output HDF5 file)
<end_file>

===CONVERT ERA40 FORMAT===
<begin_file>
ACTION : CONVERT ERA40 FORMAT

FILENAME : ... (path/name of ERA40 NetCDF file)
OUTPUTFILENAME : ... (path/name of HDF5 file to be created)
(root of name for all files produced)

CONVERT_TO_ASCII : 0/1 (1 = convert variable heading info for ASCII file; 0 = default)
CONVERT_TO_HDF5 : 0/1 (1 = convert to HDF5 file; 0 = default)
GRIDTO180 : 0/1 (1 = convert grid from [0 360] to [-180 180], 0 = default)

XX_VARIABLE : ... (name of longitude variable in the input file: usual name
is "longitude")
YY_VARIABLE : ... (name of longitude variable in the input file: usual name
is "latitude")
TIME_VARIABLE : ... (name of time variable in the input file: usual name is
"time")
<end_file>

''Remarks:''

- either CONVERT_TO_ASCII : 1 or CONVERT_TO_HDF5 : 1 must be chosen for any action to be
performed by ConvertToHDF5;

- when CONVERT_TO_HDF5 : 1 an HDF5 file is produced for every variable contained in the
original ERA40 file; the name of each file is composed of the name indicated on FILENAME
concatenated with the ERA40 variable identifier;

- to the XX_VARIABLE, YY_VARIABLE and TIME_VARIABLE keywords should generally be
specified "longitude", "latitude" and "time", respectively; the option to
include as keywords was made only to make the application robust to future variable name
changes.

===CONVERT MM5 FORMAT===
<begin_file>
ACTION : CONVERT MM5 FORMAT

FILENAME : ... (path/name of MM5 file)
TERRAIN_FILENAME : ... (path/name of MM5 TERRAIN file)

OUTPUTFILENAME : ... (path/name of HDF5 file to be created)
OUTPUT_GRID_FILENAME : ... (path/name of grid data file with horizontal grid of MM5 data
to be created)

COMPUTE_WINDSTRESS : 0/1 (1 = compute and write wind stress field; 0 = default)
COMPUTE_RELATIVE_HUMIDITY : 0/1 (1 = compute and write relative humidity field; 0 = default)
COMPUTE_PRECIPITATION : 0/1 (1 = compute and write precipitation field; 0 = default)
COMPUTE_WINDMODULUS : 0/1 (1 = compute wind modulus; 0 = default)

WRITE_XYZ : 0/1 (1 = write xyz center grid cells; 0 = default)
WRITE_TERRAIN : 0/1 (1 = write MM5 TERRAIN fields; 0 = default)

START : ... (start date for output file: yyyy mm dd hh mm ss)
END : ... (end date for output file: yyyy mm dd hh mm ss)

(block of MM5 properties to convert)
<<BeginFields>>
... (name of MM5 property to convert do HDF5 format, one per line)
...
<<EndFields>>

<end_file>

''Remarks:''

- the name of each MM5 property to convert in <<BeginFields>>...<<EndFields>> block must
conform to the MOHID designation specified in code of ModuleGlobalData; the correspondence is
the following (see [[Module_InterfaceWaterAir]] for a more detailed explanation).

---MM5 NAME--- ---MOHID NAME---
T2 air temperature
PSTARCRS atmospheric pressure
U10 wind velocity X
V10 wind velocity Y
UST wind shear velocity
LHFLUX latent heat
SWDOWN sensible heat
SWDOWN solar radiation
LWDOWN infrared radiation
SWOUT top outgoing shortwave radiation
LWOUT top outgoing longwave radiation
SOIL T 1 soil temperature layer 1
SOIL T 1 soil temperature layer 2
SOIL T 1 soil temperature layer 3
SOIL T 1 soil temperature layer 4
SOIL T 1 soil temperature layer 5
SOIL T 1 soil temperature layer 6
Q2 2-meter mixing ratio
TSEASFC sea water temperature
PBL HGT PBL height
PBL REGIME PBL regime
RAIN CON accumulated convective precipitation (cm)
RAIN NON accumulated non-convective precipitation (cm)
GROUND T ground temperature
RES TEMP infinite reservoir slab temperature
U wind velocity X_3D
V wind velocity Y_3D
W wind velocity Z_3D
T air temperature_3D
PP atmospheric pressure_3D
Q mixing ratio_3D
CLW cloud water mixing ratio_3D
RNW rain water mixing ratio_3D
ICE cloud ice mixing ratio_3D
SNOW snow mixing ratio_3D
RAD TEND atmospheric radiation tendency_3D

===CONVERT ALADIN FORMAT===
<begin_file>
ACTION : CONVERT ALADIN FORMAT

OUTPUTFILENAME : aladin.hdf5
OUTPUT_GRID_FILENAME : aladin_griddata.dat

!Put here the name of any netcdf file for grid-data generation's sake.
INPUT_GRID_FILENAME : D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKCLOUD_OPASYMP_19723_20088.nc

<<begin_input_files>>
(path to aladin netcdf file)\ALADIN_BULKIR_OPASYMP_19723_20088.nc
...
<<end_input_files>>

<end_file>

''Remarks:''

- the name of each Aladin property to convert in <<begin_input_files>>...<<end_input_files>> block must conform to the following variables

---ALADIN NAME--- ---MOHID NAME---
soclotot CloudCover_
sohumrel RelativeHumidity_
sofluxir NonSolarFlux_
sosspres AtmosphericPressure_
sosolarf SolarRadiation_
sotemair AirTemperature_
sowinmod WindModulus_
sowaprec Precipitation_
sozotaux WindStressX_
sometauy WindStressY_
sowindu10 WindVelocityX_
sowindv10 WindVelocityY_

===CONVERT MERCATOR FORMAT===
<begin_file>
ACTION : CONVERT MERCATOR FORMAT

READ_OPTION : 1/2/3/4 (version of MERCATOR files)

OUTPUTFILENAME : ... (path/name of HDF5 file to be created)
OUTPUT_GRID_FILENAME : ... (path/name of grid data with horizontal discretization to be
created)
OUTPUT_GEOMETRY_FILENAME : ... (path/name of geometry file with vertical discretization to be
created)

INPUT_GRID_FILENAME : ... (path/name of file with horizontal discretization of water
properties and water level data)
INPUT_GRID_FILENAME_U : ... (path/name of file with horizontal discretization of velocity
component U data)
INPUT_GRID_FILENAME_V : ... (path/name of file with horizontal discretization of velocity
component V data)

(block of MERCATOR data files)
<<begin_input_files>>
... (path/name of MERCATOR NetCDF data file, one per line, can be several)
...
<<<end_input_files>>>

<end_file>

===CONVERT LEVITUS FORMAT===
<begin_file>
ACTION : CONVERT LEVITUS FORMAT

OUTPUTFILENAME : ... (path/name of HDF5 file to be created)
OUTPUT_GRID_FILENAME : ... (path/name of grid data with horizontal discretization to be
created)
OUTPUT_GEOMETRY_FILENAME : ... (path/name of geometry file with vertical discretization to be
created)

PERIODICITY : ... (periodicity of Levitus data: "monthly"/"annual"; default is
"monthly")

SPATIAL_RESOLUTION : ... (spatial resolution (degrees) of horizontal Levitus grid)

FILL_VALUE : ... (real value identificator for missing data; default is
"-99.999900")

(definition of spatial window to be present in output HDF5 file)
LOWER_LEFT_CORNER : ... ... (longitude and latitude (degrees) of south west corner)
UPPER_RIGHT_CORNER : ... ... (longitude and latitude (degrees) of north east corner)

(block for each water property to be present in output HDF5 file, can be several)
<<beginfield>>
NAME : ... (name of property)
ANNUAL_FILE : ... (path/name of Levitus annual file)

(block of Levitus data files)
<<<begin_input_files>>>
... (path/name of Levitus data file (e.g. a monthly data file), one per line, can be several)
...
<<<end_input_files>>>

<<endfield>>

<end_file>

===CONVERT HELLERMAN ROSENSTEIN ASCII===
<begin_file>
ACTION : CONVERT HELLERMAN ROSENSTEIN ASCII

OUTPUTFILENAME : ... (path/name of HDF5 file to be created)
OUTPUT_GRID_FILENAME : ... (path/name of grid data with horizontal discretization to be
created)

PERIODICITY : ... (periodicity of Hellerman Rosenstein data: "monthly")

SPATIAL_RESOLUTION : ... (spatial resolution (degrees) of horizontal Hellerman
Rosenstein grid: default and only allowed value is "2.")

FILL_VALUE : ... (real value identificator for missing data; default is
"-99.999900")

(definition of spatial window to be present in output HDF5 file)
LOWER_LEFT_CORNER : ... ... (longitude and latitude (degrees) of south west corner)
UPPER_RIGHT_CORNER : ... ... (longitude and latitude (degrees) of north east corner)

(block for each Hellerman Rosenstein data file)
<<beginfield>>
NAME : ... (name of property: "wind stress X"/"wind stress Y")
FILE : ... (path/name Hellerman Rosenstein file)
<<endfield>>

<end_file>

==Samples==
All sample files are named ''ConvertToHDF5Action.dat''.

===Glue several MOHID(.hdf5) files===
<begin_file>
ACTION : GLUES HDF5 FILES

OUTPUTFILENAME : SurfaceHydro_OP.hdf5

<<begin_list>>
D:\Projectos\SurfaceHydrodynamic_21.hdf5
D:\Projectos\SurfaceHydrodynamic_22.hdf5
<<end_list>>
<end_file>

===Interpolate 2D MOHID(.hdf5) files to a new grid===
<begin_file>
ACTION : INTERPOLATE GRIDS

TYPE_OF_INTERPOLATION : 1
FATHER_FILENAME : D:\Projectos\MohidRun\test\res\Lagrangian_1.hdf5
OUTPUTFILENAME : OilSpillThickness_GridRegular.hdf5

START : 2006 6 21 17 22 30
END : 2006 6 22 17 22 0

FATHER_GRID_FILENAME : D:\Projectos\MohidRun\GeneralData\batim\Tagus.dat_A
NEW_GRID_FILENAME : TagusConstSpacing.dat

BASE_GROUP : /Results/Oil/Data_2D
<end_file>

===Interpolate 3D MOHID(.hdf5) files to a new grid===
<begin_file>
ACTION : INTERPOLATE GRIDS

TYPE_OF_INTERPOLATION : 1
FATHER_FILENAME : D:\Projectos\MohidRun\test\res\Lagrangian_1.hdf5
OUTPUTFILENAME : OilSpillThickness_GridRegular.hdf5

START : 2006 6 21 17 22 30
END : 2006 6 22 17 22 0

FATHER_GRID_FILENAME : D:\Projectos\MohidRun\GeneralData\batim\Tagus.dat_A
NEW_GRID_FILENAME : TagusConstSpacing.dat

BASE_GROUP : /Results/Oil/Data_2D

INTERPOLATION3D : 1
FATHER_GEOMETRY : D:\Projectos\MohidRun\test\data\Geometry_1.dat
NEW_GEOMETRY : TagusGeometry.dat
AUX_GRID_FILENAME : TagusConstSpacing.dat
AUX_OUTPUTFILENAME : Aux_GridRegular.hdf5
<end_file>

===Patch several MOHID(.hdf5) files to a new grid===
<begin_file>
ACTION : PATCH HDF5 FILES

TYPE_OF_INTERPOLATION : 3

START : 2005 2 28 13 0 0
END : 2005 3 1 13 0 0

<<begin_father>>
LEVEL : 3
FATHER_FILENAME : K:\MM5output\2005022812_2005030712\MM5OUT_D1.hdf5
FATHER_GRID_FILENAME : K:\MM5output\2005022812_2005030712\grid1.dat
<<end_father>>

<<begin_father>>
LEVEL : 2
FATHER_FILENAME : K:\MM5output\2005022812_2005030712\MM5OUT_D2.hdf5
FATHER_GRID_FILENAME : K:\MM5output\2005022812_2005030712\grid2.dat
<<end_father>>

<<begin_father>>
LEVEL : 1
FATHER_FILENAME : K:\MM5output\2005022812_2005030712\MM5OUT_D3.hdf5
FATHER_GRID_FILENAME : K:\MM5output\2005022812_2005030712\grid3.dat
<<end_father>>

OUTPUTFILENAME : MM5Forcing.hdf5
NEW_GRID_FILENAME : K:\Simula\GeneralData\Batim\CostaPortuguesa.dat
<end_file>

===Convert an ERA40 file to MOHID(.hdf5)===
<begin_file>
ACTION : CONVERT ERA40 FORMAT

FILENAME : D:\Aplica\ERA40\1971ERA1973.nc
OUTPUTFILENAME : D:\Aplica\ERA40\1971ERA1973T2

CONVERT_TO_ASCII : 0
CONVERT_TO_HDF5 : 1

XX_VARIABLE : longitude
YY_VARIABLE : latitude
TIME_VARIABLE : time
<end_file>

===Convert a MM5 file to MOHID(.hdf5)===
<begin_file>
ACTION : CONVERT MM5 FORMAT

FILENAME : K:\MM5output\DataCenter\Modelos\Meteo_IST\Fev2005\MMOUT_D3
TERRAIN_FILENAME : K:\MM5output\DataCenter\Modelos\Meteo_IST\Fev2005\TERRAIN_D3
OUTPUT_GRID_FILENAME : K:\MM5output\DataCenter\Modelos\Meteo_IST\Fev2005\grid3.dat
OUTPUTFILENAME : K:\MM5output\DataCenter\Modelos\Meteo_IST\Fev2005\MM5_D3.hdf5

COMPUTE_WINDSTRESS : 0
COMPUTE_RELATIVE_HUMIDITY : 1
WRITE_XYZ : 0

<<BeginFields>>
solar radiation
air temperature
wind velocity X
wind velocity Y
sensible heat
latent heat
atmospheric pressure
sea water temperature
<<EndFields>>
<end_file>

===Convert Mercator-Ocean(.nc) to MOHID(.hdf5)===
<begin_file>
ACTION : CONVERT MERCATOR FORMAT

OUTPUTFILENAME : Psy2v2r1v_R20060628/MercatorR20060628.hdf5
OUTPUT_GRID_FILENAME : Psy2v2r1v_R20060628/MercatorGridR20060628.dat
OUTPUT_GEOMETRY_FILENAME : Psy2v2r1v_R20060628/MercatorGeometryR20060628.dat

INPUT_GRID_FILENAME : GridFiles/ist_meteog-gridT.nc
INPUT_GRID_FILENAME_U : GridFiles/ist_meteog-gridU.nc
INPUT_GRID_FILENAME_V : GridFiles/ist_meteog-gridV.nc

<<begin_input_files>>
Psy2v2r1v_R20060628/ist_meteog-mercatorPsy2v2r1v_T_MEAN_ANA_20060621_R20060628.nc
Psy2v2r1v_R20060628/ist_meteog-mercatorPsy2v2r1v_T_MEAN_ANA_20060622_R20060628.nc
Psy2v2r1v_R20060628/ist_meteog-mercatorPsy2v2r1v_T_MEAN_ANA_20060623_R20060628.nc
<<end_input_files>>
<end_file>

===Convert Levitus format to MOHID(.hdf5) and interpolate grid===
==== Convert ====
First convert the Levitus ASCII format to a raw HDF5 format:
<begin_file>
ACTION : CONVERT LEVITUS FORMAT

OUTPUTFILENAME : Levitus.hdf5
OUTPUT_GRID_FILENAME : LevitusGrid.dat
OUTPUT_GEOMETRY_FILENAME : LevitusGeometry.dat

PERIODICITY : monthly
SPATIAL_RESOLUTION : 0.25
FILL_VALUE : -99.9999

LOWER_LEFT_CORNER : -16.0 31
UPPER_RIGHT_CORNER : 1. 40

<<beginfield>>
NAME : salinity
ANNUAL_FILE : DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s000hr.obj

<<<begin_input_files>>>
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s001
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s002
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s003
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s004
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s005
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s006
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s007
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s008
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s009
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s010
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s011
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s012
<<<end_input_files>>>
<<endfield>>

<<beginfield>>
NAME : temperature
ANNUAL_FILE : DataCenter\DadosBase\Ocean\Levitus\Data\Temp\t000hr.obj

<<<begin_input_files>>>
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t001
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t002
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t003
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t004
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t005
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t006
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t007
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t008
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t009
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t010
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t011
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t012
<<<end_input_files>>>
<<endfield>>
<end_file>

==== Extrapolate ====
Then extrapolate the data (still in the raw HDF5 format):
<begin_file>
ACTION : INTERPOLATE GRIDS

TYPE_OF_INTERPOLATION : 1

FATHER_FILENAME : Levitus.hdf5
FATHER_GRID_FILENAME : LevitusGrid.dat

OUTPUTFILENAME : LeviTusAllPointsWithData.hdf5
NEW_GRID_FILENAME : LevitusGrid.dat

START : -9999 1 1 0 0 0
END : -9999 12 1 0 0 0

INTERPOLATION3D : 1
FATHER_GEOMETRY : LevitusGeometry.dat
NEW_GEOMETRY : LevitusGeometry.dat
AUX_GRID_FILENAME : LevitusGrid.dat
AUX_OUTPUTFILENAME : AuxLeviTusAllPointsWithData.hdf5

POLI_DEGREE : 3
DO_NOT_BELIEVE_MAP : 1
EXTRAPOLATE_2D : 2
<end_file>

==== Interpolate ====
Finally, interpolate to the final grid and geometry (same as the [[#Interpolate 3D MOHID(.hdf5) files to a new grid| Interpolate 3D sample]]):
<begin_file>
ACTION : INTERPOLATE GRIDS

TYPE_OF_INTERPOLATION : 1
FATHER_FILENAME : LeviTusAllPointsWithData.hdf5
OUTPUTFILENAME : CadizMonthlyLevitus.hdf5
FATHER_GRID_FILENAME : LevitusGrid.dat
NEW_GRID_FILENAME : Algarve0.02SigmaSmooth_V3_CartMoreLayers.dat

START : -9999 1 1 0 0 0
END : -9999 12 1 0 0 0

INTERPOLATION3D : 1
FATHER_GEOMETRY : LevitusGeometry.dat
NEW_GEOMETRY : Geometry_1.dat
AUX_OUTPUTFILENAME : AuxCadizMonthlyLevitus.hdf5
AUX_GRID_FILENAME : Aux12km.dat

POLI_DEGREE : 3
DO_NOT_BELIEVE_MAP : 1
<end_file>

Note that the programme may construct a new bathymetry twice. Use this bathymetry only on the AUX_GRID_FILENAME keyword.

===Convert Hellerman Rosenstein ASCII format to MOHID(.hdf5) ===
<begin_file>
ACTION : CONVERT HELLERMAN ROSENSTEIN ASCII

OUTPUTFILENAME : ClimatologicWindStress.hdf5
OUTPUT_GRID_FILENAME : ClimatologicWindStressGrid.dat

PERIODICITY : monthly
SPATIAL_RESOLUTION : 2.
FILL_VALUE : -99.9999

LOWER_LEFT_CORNER : -180 -90
UPPER_RIGHT_CORNER : 180 90

<<beginfield>>
NAME : wind stress X
FILE : D:\Aplica\Dados\Hellerman_Rosenstein\TAUXX.DAT
<<endfield>>

<<beginfield>>
NAME : wind stress Y
FILE : D:\Aplica\Dados\Hellerman_Rosenstein\TAUYY.DAT
<<endfield>>
<end_file>

===Convert ALADIN(.nc) format to MOHID(.hdf5)===
<begin_file>
ACTION : CONVERT ALADIN FORMAT

OUTPUTFILENAME : aladin.hdf5
OUTPUT_GRID_FILENAME : aladin_griddata.dat

!Put here the name of any netcdf file for grid-data generation's sake.
INPUT_GRID_FILENAME : D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKCLOUD_OPASYMP_19723_20088.nc

<<begin_input_files>>
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKIR_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKPRES_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKSOLAR_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKTAIR_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKWIND_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_FLUXPRE_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_STRESSU_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_STRESSV_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_U10_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_V10_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKCLOUD_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKHUMI_OPASYMP_19723_20088.nc
<<end_input_files>>

<end_file>

== OceanColor modules compilation ==
Compiling the [[ConvertToHDF5]] tool with the OceanColor modules is more complicated than one might expect. A solution is proposed here for a release version using the Compaq Visual Fortran 6.6c. The difficulties rise because C code is embedded with a fortran interface and also, extra libraries such as hdf4 are required.

=== Pre-requisites ===

This is a list of prerequisites to successfully compile the tool:
*Compaq Visual Fortran 6.5 with patch 6.6c,
*VS .NET 2003 (Vc7 in particular),
*Hdf5 libraries ('''hdf5.lib''' '''hdf5_fortran.lib''' '''hdf5_hl.lib'''),
*Netcdf libraries ('''netcdf.lib''' '''netcdf_.lib'''),
*Hdf4 libraries ('''hd421.lib''', '''hm421.lib'''),
*szlib, zlib and jpeg libraries ('''szlib.lib''', '''zlib.lib''' and '''libjpeg.lib'''),
*the fortran source files ('''ModuleConvertModisL2.F90 ModuleConvertModisL3.F90 ModuleConvertOceanColorL2.F90'''),
*the C source files and their fortran interface files ('''readL2scan.c readL2Seadas.c''' and '''cdata.f crossp.f fgeonav.f''').

=== CVF IDE configuration ===
# Configure everything as specified in [[Compiling with CVF]].
# Add the source files listed in the prerequisites above to the source files listing.
# Go to '''Tools-->Options...-->Directories'''. There, add the '''$DOTNET2K3/Vc7/bin''' to the '''Executable files''''; the '''$DOTNET2K3/Vc7/include''' and '''$DOTNET2K3/Vc7/PlatformSDK/include''' to the '''Include files'''; and finally, the '''$DOTNET2K3/Vc7/lib''', '''$DOTNET2K3/Vc7/PlatformSDK/lib''' and '''$DOTNET2K3/Vc7/PlatformSDK/bin''' to the '''Library files'''.
# Go to '''Projects-->Settings-->Release-->Link-->Input'''. There, add the following libraries: '''netcdf.lib netcdf_.lib hd421.lib hm421.lib libjpeg.lib'''. (Make sure the hdf5 libraries as well as the szlib and zlib libraries are already mentioned).

=== Troubleshoots ===
'''Q: I get unresolved external references during linkage, but I have all the libraries mentioned above included. What should I do?'''

A: Unresolved external references can come out for two reasons:
#you didn't specified all the libraries required or all the paths for the default libraries or,
#[http://en.wikipedia.org/wiki/Name_decoration name mangling] problems. Use the [[dumpbin]] utility to the libraries to checkout which language convention they are using. If that's the problem then you need to try to get new libraries with the correct naming convention.

That's it, you should now be able to build the [[ConvertToHdf5]] project successfully.

'''Q: I got a message saying the entry point _NF_PUT_ATT_REAL@28 could not be located in netcdf.dll'''

A: copy the file netcdf.dll to the exe folder

== References ==
*[http://www.hdfgroup.org/ HDF5 Homepage]
*[http://www.hdfgroup.org/ HDF4 Homepage]

== Links ==

*[[Module_Atmosphere]]
*[[Module_InterfaceWaterAir]]
*[[Coupling_Water-Atmosphere_User_Manual]]

[[Category:Tools]]
[[Category:Hdf5]]

ConvertToHDF5

2009-05-12T12:30:45Z

AngelaCanas: /* CONVERT MM5 FORMAT */

The '''ConvertToHDF5''' is an application which allows the making of several operations, called '''actions''', involving HDF5 files: conversion of data in other formats (e.g. NETCDF) to HDF5, grid interpolation, concatenation of several files.

Running options for this application are specified by the user in a input file named [[ConvertToHDF5#Input file (ConvertToHDF5Action.dat)|'''ConvertToHDF5Action.dat''']]. Several actions can be specified in the same input file, being processed sequentially by the ConvertToHDF5 application.

==Introduction==
The operations involving HDF5 files performed by ConvertToHDF5, specified individually by an action, can be organized in [[#file management|file management]], [[#grid interpolation|grid interpolation]] and [[#format conversion|format conversion]]. These types and the respective actions are detailed in the next sections.

The input file specification for each action can be found bellow in the [[#Input file (ConvertToHDF5Action.dat)|Input file (ConvertToHDF5Action.dat)]] section.

===File management===

====Glue files====
This action consists in joining or glue in a single HDF5 file two or more HDF5 files having the same HDF5 data groups and referring to time periods which come in sequence. Both sets of 2D and 3D HDF5 files can be glued.

'''Typical use:'''

Glue MOHID Water results files from several runs produced in continuous running of the model, for storage space economy reasons. Can be used to join data from other origins (e.g. results of meteorological models) as long as the HDF5 format is the one supported by MOHID Water.

'''Data input requirements:'''

HDF5 files to be glued. "Grid" and "Results" data groups should be equal in all these files.

'''Output:'''

HDF5 file with glued "Results" data. "Residual" and "Statistics" HDF5 data groups are not copied to the output file since they are time period specific (different values potentially occour in each file). General statistics can be calculated for the glued HDF5 file data using tool [[HDF5Statistics]].

'''ConvertToHDF5 action:''' [[#GLUES HDF5 FILES|GLUES HDF5 FILES]].

===Grid interpolation===

====Interpolate files====
This action performs the conversion of one HDF5 file data existing in one 2D or 3D spatial grid to another 2D or 3D spatial grid, creating a new HDF5 file. The interpolation is performed only for the data located a time window specified by the user.

The HDF5 file containing data to be interpolated is called the '''father file'''.

In case of 3D interpolation the application conducts first the horizontal grid interpolation
(keeping father geometry) and only after it conducts the vertical interpolation (from father geometry to new geometry).

Several types of 2D interpolation are available for use: bilinear, spline 2D and triangulation.
For vertical interpolation (used in 3D interpolation) can be supplied several polinomial degrees for interpolation.

'''Typical use:'''

Obtain an HDF5 file with data for forcing or providing initial conditions for a MOHID Water model, e.g. a meteorological forcing file.

'''Data input requirements:'''

For 2D/3D interpolation:

- father HDF5 file;

- father horizontal data grid, in a grid data file in the format supported by MOHID;

- new horizontal data grid, in a grid data file in the format supported by MOHID;

For 3D interpolation also needed:

- father vertical geometry, in a geometry file in the format supported by MOHID;

- new vertical geometry, in a geometry file in the format supported by MOHID;

- auxiliary horizontal data grid, in a grid data file in the format supported by MOHID; this file is used for horizontal grid interpolation in 3D interpolation operations.

'''Ouput:'''

HDF5 file with interpolated data. In case of 3D interpolation also produced an auxiliary HDF5 file with the result of the horizontal grid interpolation, which can be inspected to check if this operation is well performed.

'''ConvertToHDF5 action:''' [[#INTERPOLATE GRIDS|INTERPOLATE GRIDS]].

====Patch files====
This action consists in performing an interpolation of HDF5 data between grids, as in action [[#Interpolate files|Interpolate files]], but considering more than one HDF5 file as containing data to be interpolated to the new grid and a priority scale. The interpolation is performed only for the data located in the time window specified by the user. The present version of this action operates only on 2D data.

Each HDF5 file containing data to be interpolated is called a '''father file''' and has an user-attributed '''priority level''' to be respected in the interpolation process: for each new grid cell the ConvertToHDF5 application will look for data first on the Level 1 father file and only in the case this data is inexistent will it look for data in Level 2 file, proceeding in looking for higher level files if no data is found subsequentely.

'''Typical use:'''

To obtain an HDF5 file with data from several HDF5 files each containing data with different spatial resolution and only for a specific part of the new grid. This is, for instance, the case when one is preparing a best resolution meteorological HDF5 file for forcing MOHID Water from several meteorological model domains, having different spatial resolution and span, since the best resolution data is not available for all new grid cells.

'''Data input requirements:'''

The new horizontal data grid, in a grid data file in the format supported by MOHID, and for each father file:

- level of priority: 1 = maximum priority, priority decreases with increasing level value;

- data grid, in the form of a grid data file in the format supported by MOHID.

'''Ouput:'''

HDF5 file with patched data.

'''ConvertToHDF5 action:''' [[#PATCH HDF5 FILES|PATCH HDF5 FILES]].

===Format conversion===

====Meteorological model data====
Mohid does not simulate explicitly the atmosphere, but needs information about atmospheric properties in time and space. This requires that atmospheric properties are supplied to MOHID Water in supported formats. These formats can be derived from meteorological data in HDF5 format. Because the results of meteorological models are accessed in different formats conversion is required.

The formats currently convertible to HDF5 in ConvertToHDF5 include the MM5 and the ERA40. These are succintly detailed in the next sections.

=====''ERA40''=====
This format refers to the European Centre for Medium-Range Weather Forecasts (ECMWF) 40 years re-analysises results, acessed by site http://data.ecmwf.int/data/d/era40_daily/. This data is available for several meteorological variables with maximum 6 hour periodicity for days in the period from 1957-09-01 to 2002-08-31.

ERA40 data files are supplied by ECMWF in a NetCDF format and with an user-costumized time window, periodicity (time step range from 6 hours to a day) and meteorological properties set. The ERA40 meteorological properties which are recognized by MOHID are presented bellow together with the correspondent MOHID name:

---ERA40 NAME--- ---MOHID NAME---
sshf sensible heat
slhf latent heat
msl atmospheric pressure
tcc cloud cover
p10u wind velocity X
p10v wind velocity Y
p2t air temperature
ewss wind stress X
nsss wind stress Y

The standard ConvertToHDF5 action is to convert to HDF5 the data referring to all MOHID Water recognized property available in the ERA40 file, producing an individual HDF5 file for each property. The name of each HDF5 file generated includes the ERA40 meteorological property identificator correspondent to the data contained.

Alternatively, ConvertToHDF5 can copy to a single ASCII file the heading information concerning each meteorological variable considered in the original ERA40 file.

'''Typical use:'''

Obtain an HDF5 file with data suitable for being used for forcing MOHID Water models.

'''Data input requirements:'''

ERA40 NetCDF file.

'''Output:'''

One HDF5 file for each meteorological property contained in the original NetCDF file.

'''ConvertToHDF5 action:''' [[#CONVERT ERA40 FORMAT|CONVERT ERA40 FORMAT]].

=====''MM5''=====
This format relates to the Fifth-Generation NCAR / Penn State Mesoscale Model (MM5) output files format. Almost every atmospheric property needed by MOHID Water is present in MM5 output files, enabling to run prediction simulations with MOHID Water when access to MM5 prevision files is available.

The ConvertToHDF5 action converts MM5 results files from the original format to HDF5 format, allowing the easy use of these results in the MOHID framework. Conversion is only performed for the MM5 properties and the time window specified by the user.

Besides the conversion, the application can calculate some properties not contained in
the MM5 files using the available information: these are windstress, relative humidity and precipitation.

For conversion to be completed it is required the horizontal grid information of MM5 results which is available in special TERRAIN files.

'''Typical use:'''

Produce HDF5 meteorological data usable for forcing MOHID Water models.

'''Data input requirements:'''

MM5 results file to convert and MM5 TERRAIN file. The TERRAIN file supplies the MM5 results grid information.

'''Ouput:'''

An HDF5 file with MM5 results and a grid data file in MOHID format with the MM5 grid information.
This last file can be used to interpolate the MM5 data from the original grid to a new grid (see [[#Interpolate files|Interpolate files]]), for instance to produce an HDF5 file suitable for forcing MOHID Water models.

'''Compilation:'''

Caution! The ConvertToHDF5 executable must be compiled with the [[Big-endian little-endian|Big-Endian]] option set (see compatibility in the project's settings).

'''ConvertToHDF5 action:''' [[#CONVERT MM5 FORMAT|CONVERT MM5 FORMAT]].

=====''Aladin''=====
This format relates to Aladin meteorological model results. Some of the atmospheric property needed by MOHID Water is present in Aladin output files, enabling to run prediction simulations with MOHID Water when access to Aladin prevision files is available.

The ConvertToHDF5 action converts Aladin results files from the original format to HDF5 format, allowing the easy use of these results in the MOHID framework. Conversion is only performed for the MM5 properties and the time window specified by the user.

'''Typical use:'''

Produce HDF5 meteorological data usable for forcing MOHID Water models.

'''Data input requirements:'''

Aladin netcdf results file to convert.

'''Ouput:'''

An HDF5 file with Aladin results and a grid data file in MOHID format with the Aladin grid pseudo-information: a fake orography is created of 100 m depth.
This last file can be used to interpolate the Aladin data from the original grid to a new grid (see [[#Interpolate files|Interpolate files]]), for instance to produce an HDF5 file suitable for forcing MOHID Water models.

'''Compilation:'''

Caution! The ConvertToHDF5 executable must be compiled with the [[Big-endian little-endian|Big-Endian]] option set (see compatibility in the project's settings).

'''ConvertToHDF5 action:''' [[#CONVERT ALADIN FORMAT|CONVERT ALADIN FORMAT]].

====Ocean model data====
Ocean model data, available in diverse formats, can be used by MOHID Water to specify boundary (open ocean boundary and surface), initial conditions or for validation. These uses require that the model data is in HDF5 format and conversion is therefore needed.

Currently the large scale ocean models formats convertible into HDF5 by ConvertToHDF5 includes MERCATOR.

=====''MERCATOR''=====
MERCATOR data files are supplied in a NetCDF format and with an user-costumized spatial window and periodicity. Water level and water properties (temperature and salinity) data is available in type T files, velocity component u data is available in type U files and velocity component v data is available in type V files. The type of data of a specific MERCATOR file is generally indicated in the file name.

The standard ConvertToHDF5 action is to convert to HDF5 the data referring to temperature, salinity, water level, component u of velocity and component v of velocity.

'''Typical use:'''

Obtain HDF5 MERCATOR data usable for forcing or validation of MOHID Water models.

'''Data input requirements:'''

NetCDF MERCATOR results data files and NetCDF MERCATOR grid data files. It should be provided one grid data file of each type: T, U and V. These are generally provided by the MERCATOR services together with the results files.

'''Ouput:'''

One HDF5 file containing all properties contained in the recognized set of properties (temperature, salinity, water level, velocity u and velocity v) and the correspondent grid data and geometry files, containing respectively the horizontal grid and the vertical discretization of the HDF5 file. The grid data and geometry files can be used afterwards to interpolate the MERCATOR data to another grid and geometry (see [[#Interpolate files|Interpolate files]]).

'''ConvertToHDF5 action:''' [[#CONVERT MERCATOR FORMAT|CONVERT MERCATOR FORMAT]].

====Climatological data====
Climatological data can be used in MOHID Water to specify boundary (open ocean boundary and surface), initial conditions or for validation, in case more realistic data (measurements or model) data is unavailable. This data is generally supplied by producers in formats not readly usable by MOHID Water which justifies the existence of a conversion tool.

Two climatological data format conversions are implemented in ConvertToHDF5: Levitus ocean data and Hellerman Rosenstein meteorological data.

=====''Levitus''=====
The Levitus climatology provides results for water temperature and salinity.
The ConvertToHDF5 action converts the climatological data for the properties and spatial window requested by the user.
Typically, it requires 3 steps to complete the task:

- convert levitus format

- extrapolate the data to the whole levitus domain(required to avoid uncoincidental coastlines)

- interpolate with the model grid(bathymetry)

'''Typical use:'''

Obtain climatological data in HDF5 format to use as boundary forcing and/or initial condition specification in MOHID Water models.

'''Data input requirements:'''

Levitus climatological data files, one per property and per time period (e.g a month).

'''Ouput:'''

HDF5 file with Levitus climatological data, grid data file with the horizontal
grid of the data and a geometry file with vertical discretization of the data (MOHID formats).
The grid data and the geometry files can be used to interpolate the climatological data from the original grid to a new grid (see [[#Interpolate files|Interpolate files]]).

'''ConvertToHDF5 action:''' [[#CONVERT LEVITUS FORMAT|CONVERT LEVITUS FORMAT]].

=====''Hellerman Rosenstein''=====
This is a meteorological climatology providing wind stress. There is a file per wind stress component. Since the data refer to surface values it is a 2D field.

The ConvertToHDF5 action converts the climatological data for the properties and spatial window provided by the user.

'''Typical use:'''

Obtain climatological data in HDF5 format to use as meteorological forcing in MOHID Water models.

'''Data input requirements:'''

Hellerman Rosenstein climatological data ASCII files, one per wind stress component.

'''Ouput:'''

HDF5 file with Hellerman Rosenstein climatological data and grid data file with the horizontal
grid of the climatological data. This grid data file can be used to interpolate the climatological data from the original horizontal grid to a new grid (see [[#Interpolate files|Interpolate files]]).

'''ConvertToHDF5 action:''' [[#CONVERT HELLERMAN ROSENSTEIN ASCII|CONVERT HELLERMAN ROSENSTEIN ASCII]].

=====''World Ocean Atlas 2005''=====
The World Ocean Atlas (WOA) 2005 climatology provides results for water temperature, salinity and several water quality and biology properties.

Description, Action and Input Files are described in a separate page: [[ConvertToHDF5 WOA2005]].

==Input file (ConvertToHDF5Action.dat)==
===General structure===
<begin_file> (block containing instructions for running a specific action)
ACTION : ... (intended action)
... (action specific instructions)
<end_file>

<begin_file>
ACTION : ...
...
<end_file>

===GLUES HDF5 FILES===
<begin_file>
ACTION : GLUES HDF5 FILES

3D_FILE : 0/1 (0 = 2D file, 1 = 3D file)

TIME_GROUP : ... (Default="Time". Other option: "SurfaceTime".)

BASE_GROUP : ... (Default="Results". Other options: "Residual", "SurfaceResults".)

OUTPUTFILENAME : ... (path/name of HDF5 file to be created)

(block of HDF5 data files)
<<begin_list>>
... (path/name of HDF5 file with data to be included in glue, one per line, at least two files)
...
<<end_list>>
<end_file>

===INTERPOLATE GRIDS===
<begin_file>
ACTION : INTERPOLATE GRIDS

TYPE_OF_INTERPOLATION : ... (type of horizontal interpolation: 1 = Bilinear, 2 = Spline2D,
3 = Triangulation)

INTERPOLATION_WINDOW : ... ... ... ... (2D spatial window to consider for interpolation:
Xmin Ymin Xmax Ymax; default = all domain)

START : ... (start date for output file: yyyy mm dd hh mm ss)
END : ... (end date for output file: yyyy mm dd hh mm ss)

INTERPOLATION3D : 0/1 (0 = 2D interpolation, 1 = 3D interpolation)

FATHER_FILENAME : ... (path/name of input HDF5 file with data to be interpolated)
FATHER_GRID_FILENAME : ... (path/name of input grid data file with horizontal
discretization of input HDF5 file)

OUTPUTFILENAME : ... (path/name of output HDF5 file to be created)
NEW_GRID_FILENAME : ... (path/name of input grid data file with horizontal
discretization intended for output HDF5 file)

EXTRAPOLATE_2D : 0/1/2/3/4/5 (2D extrapolation: 0=no extrapolation, 1=medium
triangulation, 2=high triangulation,
3=nearest neighbour, 4=nearest cell,
5=constant value)

EXTRAPOLATE_VALUE : ... (name of the value to extrapolate to when EXTRAPOLATE_2D is
set to constant value (5))

DO_NOT_BELIEVE_MAP : 0/1 (0=consider input HDF5 file map, 1=do not consider input HDF5
file map)

BASE_GROUP : ... (name of base group of HDF5 variables containing data to be
interpolated; default is "/Results")

(if INTERPOLATION3D : 1 also required:)
FATHER_GEOMETRY : ... (path/name of file (MOHID format) with vertical discretization
of input HDF5 file)
NEW_GEOMETRY : ... (path/name of file (MOHID format) with vertical discretization
intended for output HDF5 file)
POLI_DEGREE : 1/... (degree of vertical interpolation: 1=linear, ...)

AUX_GRID_FILENAME : ... (path/name of input grid data file with horizontal
discretization intended for auxiliar output HDF5 file;
default is file provided in NEW_GRID_FILENAME)

AUX_OUTPUTFILENAME : ... (path/name of auxiliar output HDF5 file to contain result
of horizontal grid interpolation)
<end_file>

''Remarks:''

- the file indicated in AUX_GRID_FILENAME can be different from the one indicated in
NEW_GRID_FILENAME in terms of bathymetry, while the horizontal grid should be, commonly, the
same: this altered bathymetry can be used to extend the water column in the original data so
that the process of vertical interpolation is done easily;

- in case of INTERPOLATION3D : 1, ConvertToHDF5 can generate new versions of bathymetry which
are consistent with the geometry definition (extension is '.new'); there are possibly three
bathymetry changes referring to father grid, new grid and aux grid (the same bathymetry is
not altered twice); although initially new and aux grid are the same they can result
different because of bathymetry changes;

- in case the new geometry is 2D and father geometry is 3D then POLI_DEGREE : 1
(linear interpolation) should be used;

- EXTRAPOLATE_2D : 1/2/3/4/5 should be considered if it is expected that the coast line is not
coincidental in the father and new grids, to avoid lack of data in the interpolation
process; extrapolation is performed for all cells even the land cells;

- in case of DO_NOT_BELIEVE_MAP : 1 the application generates a map based on cells where
interpolation results are available; this causes that if EXTRAPOLATE_2D : 1/2/3/4/5 is used
the AUX_GRID_FILENAME should not have land cells in order for the new map to be concurrent
with the result of extrapolation and avoid errors generation, specially if INTERPOLATION3D :
1 is considered.

===PATCH HDF5 FILES===
<begin_file>
ACTION : PATCH HDF5 FILES

TYPE_OF_INTERPOLATION : ... (type of interpolation: 3 = Triangulation, default and only
one implemented)

START : ... (start date for output file: yyyy mm dd hh mm ss)
END : ... (end date for output file: yyyy mm dd hh mm ss)

(block for each father HDF5 file, should be at least two)
<<begin_father>>
LEVEL : ... (integer priority level: 1 = highest, increase for lower
priority)
FATHER_FILENAME : ... (path/name of input HDF5 file with data to be interpolated)
FATHER_GRID_FILENAME : ... (path/name of input grid data file with horizontal
discretization of input HDF5 file)
<<end_father>>

OUTPUTFILENAME : ... (path/name of output HDF5 file to be created)
NEW_GRID_FILENAME : ... (path/name of input grid data file with horizontal
discretization intended for output HDF5 file)
<end_file>

===CONVERT ERA40 FORMAT===
<begin_file>
ACTION : CONVERT ERA40 FORMAT

FILENAME : ... (path/name of ERA40 NetCDF file)
OUTPUTFILENAME : ... (path/name of HDF5 file to be created)
(root of name for all files produced)

CONVERT_TO_ASCII : 0/1 (1 = convert variable heading info for ASCII file; 0 = default)
CONVERT_TO_HDF5 : 0/1 (1 = convert to HDF5 file; 0 = default)
GRIDTO180 : 0/1 (1 = convert grid from [0 360] to [-180 180], 0 = default)

XX_VARIABLE : ... (name of longitude variable in the input file: usual name
is "longitude")
YY_VARIABLE : ... (name of longitude variable in the input file: usual name
is "latitude")
TIME_VARIABLE : ... (name of time variable in the input file: usual name is
"time")
<end_file>

''Remarks:''

- either CONVERT_TO_ASCII : 1 or CONVERT_TO_HDF5 : 1 must be chosen for any action to be
performed by ConvertToHDF5;

- when CONVERT_TO_HDF5 : 1 an HDF5 file is produced for every variable contained in the
original ERA40 file; the name of each file is composed of the name indicated on FILENAME
concatenated with the ERA40 variable identifier;

- to the XX_VARIABLE, YY_VARIABLE and TIME_VARIABLE keywords should generally be
specified "longitude", "latitude" and "time", respectively; the option to
include as keywords was made only to make the application robust to future variable name
changes.

===CONVERT MM5 FORMAT===
<begin_file>
ACTION : CONVERT MM5 FORMAT

FILENAME : ... (path/name of MM5 file)
TERRAIN_FILENAME : ... (path/name of MM5 TERRAIN file)

OUTPUTFILENAME : ... (path/name of HDF5 file to be created)
OUTPUT_GRID_FILENAME : ... (path/name of grid data file with horizontal grid of MM5 data
to be created)

COMPUTE_WINDSTRESS : 0/1 (1 = compute and write wind stress field; 0 = default)
COMPUTE_RELATIVE_HUMIDITY : 0/1 (1 = compute and write relative humidity field; 0 = default)
COMPUTE_PRECIPITATION : 0/1 (1 = compute and write precipitation field; 0 = default)
COMPUTE_WINDMODULUS : 0/1 (1 = compute wind modulus; 0 = default)

WRITE_XYZ : 0/1 (1 = write xyz center grid cells; 0 = default)
WRITE_TERRAIN : 0/1 (1 = write MM5 TERRAIN fields; 0 = default)

START : ... (start date for output file: yyyy mm dd hh mm ss)
END : ... (end date for output file: yyyy mm dd hh mm ss)

(block of MM5 properties to convert)
<<BeginFields>>
... (name of MM5 property to convert do HDF5 format, one per line)
...
<<EndFields>>

<end_file>

''Remarks:''

- the name of each MM5 property to convert in <<BeginFields>>...<<EndFields>> block must
conform to the MOHID designation specified in code of ModuleGlobalData; the correspondence is
the following (see [[Module_InterfaceWaterAir]] for a more detailed explanation).

---MM5 NAME--- ---MOHID NAME---
T2 air temperature
PSTARCRS atmospheric pressure
U10 wind velocity X
V10 wind velocity Y
UST wind shear velocity
LHFLUX latent heat
SWDOWN sensible heat
SWDOWN solar radiation
LWDOWN infrared radiation
SWOUT top outgoing shortwave radiation
LWOUT top outgoing longwave radiation
SOIL T 1 soil temperature layer 1
SOIL T 1 soil temperature layer 2
SOIL T 1 soil temperature layer 3
SOIL T 1 soil temperature layer 4
SOIL T 1 soil temperature layer 5
SOIL T 1 soil temperature layer 6
Q2 2-meter mixing ratio
TSEASFC sea water temperature
PBL HGT PBL height
PBL REGIME PBL regime
RAIN CON accumulated convective precipitation (cm)
RAIN NON accumulated non-convective precipitation (cm)
GROUND T ground temperature
RES TEMP infinite reservoir slab temperature
U wind velocity X_3D
V wind velocity Y_3D
W wind velocity Z_3D
T air temperature_3D
PP atmospheric pressure_3D
Q mixing ratio_3D
CLW cloud water mixing ratio_3D
RNW rain water mixing ratio_3D
ICE cloud ice mixing ratio_3D
SNOW snow mixing ratio_3D
RAD TEND atmospheric radiation tendency_3D

===CONVERT ALADIN FORMAT===
<begin_file>
ACTION : CONVERT ALADIN FORMAT

OUTPUTFILENAME : aladin.hdf5
OUTPUT_GRID_FILENAME : aladin_griddata.dat

!Put here the name of any netcdf file for grid-data generation's sake.
INPUT_GRID_FILENAME : D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKCLOUD_OPASYMP_19723_20088.nc

<<begin_input_files>>
(path to aladin netcdf file)\ALADIN_BULKIR_OPASYMP_19723_20088.nc
...
<<end_input_files>>

<end_file>

''Remarks:''

- the name of each Aladin property to convert in <<begin_input_files>>...<<end_input_files>> block must conform to the following variables

---ALADIN NAME--- ---MOHID NAME---
soclotot CloudCover_
sohumrel RelativeHumidity_
sofluxir NonSolarFlux_
sosspres AtmosphericPressure_
sosolarf SolarRadiation_
sotemair AirTemperature_
sowinmod WindModulus_
sowaprec Precipitation_
sozotaux WindStressX_
sometauy WindStressY_
sowindu10 WindVelocityX_
sowindv10 WindVelocityY_

===CONVERT MERCATOR FORMAT===
<begin_file>
ACTION : CONVERT MERCATOR FORMAT

OUTPUTFILENAME : ... (path/name of HDF5 file to be created)
OUTPUT_GRID_FILENAME : ... (path/name of grid data with horizontal discretization to be
created)
OUTPUT_GEOMETRY_FILENAME : ... (path/name of geometry file with vertical discretization to be
created)

INPUT_GRID_FILENAME : ... (path/name of file with horizontal discretization of water
properties and water level data)
INPUT_GRID_FILENAME_U : ... (path/name of file with horizontal discretization of velocity
component U data)
INPUT_GRID_FILENAME_V : ... (path/name of file with horizontal discretization of velocity
component V data)

(block of MERCATOR data files)
<<begin_input_files>>
... (path/name of MERCATOR NetCDF data file, one per line, can be several)
...
<<<end_input_files>>>

<end_file>

===CONVERT LEVITUS FORMAT===
<begin_file>
ACTION : CONVERT LEVITUS FORMAT

OUTPUTFILENAME : ... (path/name of HDF5 file to be created)
OUTPUT_GRID_FILENAME : ... (path/name of grid data with horizontal discretization to be
created)
OUTPUT_GEOMETRY_FILENAME : ... (path/name of geometry file with vertical discretization to be
created)

PERIODICITY : ... (periodicity of Levitus data: "monthly"/"annual"; default is
"monthly")

SPATIAL_RESOLUTION : ... (spatial resolution (degrees) of horizontal Levitus grid)

FILL_VALUE : ... (real value identificator for missing data; default is
"-99.999900")

(definition of spatial window to be present in output HDF5 file)
LOWER_LEFT_CORNER : ... ... (longitude and latitude (degrees) of south west corner)
UPPER_RIGHT_CORNER : ... ... (longitude and latitude (degrees) of north east corner)

(block for each water property to be present in output HDF5 file, can be several)
<<beginfield>>
NAME : ... (name of property)
ANNUAL_FILE : ... (path/name of Levitus annual file)

(block of Levitus data files)
<<<begin_input_files>>>
... (path/name of Levitus data file (e.g. a monthly data file), one per line, can be several)
...
<<<end_input_files>>>

<<endfield>>

<end_file>

===CONVERT HELLERMAN ROSENSTEIN ASCII===
<begin_file>
ACTION : CONVERT HELLERMAN ROSENSTEIN ASCII

OUTPUTFILENAME : ... (path/name of HDF5 file to be created)
OUTPUT_GRID_FILENAME : ... (path/name of grid data with horizontal discretization to be
created)

PERIODICITY : ... (periodicity of Hellerman Rosenstein data: "monthly")

SPATIAL_RESOLUTION : ... (spatial resolution (degrees) of horizontal Hellerman
Rosenstein grid: default and only allowed value is "2.")

FILL_VALUE : ... (real value identificator for missing data; default is
"-99.999900")

(definition of spatial window to be present in output HDF5 file)
LOWER_LEFT_CORNER : ... ... (longitude and latitude (degrees) of south west corner)
UPPER_RIGHT_CORNER : ... ... (longitude and latitude (degrees) of north east corner)

(block for each Hellerman Rosenstein data file)
<<beginfield>>
NAME : ... (name of property: "wind stress X"/"wind stress Y")
FILE : ... (path/name Hellerman Rosenstein file)
<<endfield>>

<end_file>

==Samples==
All sample files are named ''ConvertToHDF5Action.dat''.

===Glue several MOHID(.hdf5) files===
<begin_file>
ACTION : GLUES HDF5 FILES

OUTPUTFILENAME : SurfaceHydro_OP.hdf5

<<begin_list>>
D:\Projectos\SurfaceHydrodynamic_21.hdf5
D:\Projectos\SurfaceHydrodynamic_22.hdf5
<<end_list>>
<end_file>

===Interpolate 2D MOHID(.hdf5) files to a new grid===
<begin_file>
ACTION : INTERPOLATE GRIDS

TYPE_OF_INTERPOLATION : 1
FATHER_FILENAME : D:\Projectos\MohidRun\test\res\Lagrangian_1.hdf5
OUTPUTFILENAME : OilSpillThickness_GridRegular.hdf5

START : 2006 6 21 17 22 30
END : 2006 6 22 17 22 0

FATHER_GRID_FILENAME : D:\Projectos\MohidRun\GeneralData\batim\Tagus.dat_A
NEW_GRID_FILENAME : TagusConstSpacing.dat

BASE_GROUP : /Results/Oil/Data_2D
<end_file>

===Interpolate 3D MOHID(.hdf5) files to a new grid===
<begin_file>
ACTION : INTERPOLATE GRIDS

TYPE_OF_INTERPOLATION : 1
FATHER_FILENAME : D:\Projectos\MohidRun\test\res\Lagrangian_1.hdf5
OUTPUTFILENAME : OilSpillThickness_GridRegular.hdf5

START : 2006 6 21 17 22 30
END : 2006 6 22 17 22 0

FATHER_GRID_FILENAME : D:\Projectos\MohidRun\GeneralData\batim\Tagus.dat_A
NEW_GRID_FILENAME : TagusConstSpacing.dat

BASE_GROUP : /Results/Oil/Data_2D

INTERPOLATION3D : 1
FATHER_GEOMETRY : D:\Projectos\MohidRun\test\data\Geometry_1.dat
NEW_GEOMETRY : TagusGeometry.dat
AUX_GRID_FILENAME : TagusConstSpacing.dat
AUX_OUTPUTFILENAME : Aux_GridRegular.hdf5
<end_file>

===Patch several MOHID(.hdf5) files to a new grid===
<begin_file>
ACTION : PATCH HDF5 FILES

TYPE_OF_INTERPOLATION : 3

START : 2005 2 28 13 0 0
END : 2005 3 1 13 0 0

<<begin_father>>
LEVEL : 3
FATHER_FILENAME : K:\MM5output\2005022812_2005030712\MM5OUT_D1.hdf5
FATHER_GRID_FILENAME : K:\MM5output\2005022812_2005030712\grid1.dat
<<end_father>>

<<begin_father>>
LEVEL : 2
FATHER_FILENAME : K:\MM5output\2005022812_2005030712\MM5OUT_D2.hdf5
FATHER_GRID_FILENAME : K:\MM5output\2005022812_2005030712\grid2.dat
<<end_father>>

<<begin_father>>
LEVEL : 1
FATHER_FILENAME : K:\MM5output\2005022812_2005030712\MM5OUT_D3.hdf5
FATHER_GRID_FILENAME : K:\MM5output\2005022812_2005030712\grid3.dat
<<end_father>>

OUTPUTFILENAME : MM5Forcing.hdf5
NEW_GRID_FILENAME : K:\Simula\GeneralData\Batim\CostaPortuguesa.dat
<end_file>

===Convert an ERA40 file to MOHID(.hdf5)===
<begin_file>
ACTION : CONVERT ERA40 FORMAT

FILENAME : D:\Aplica\ERA40\1971ERA1973.nc
OUTPUTFILENAME : D:\Aplica\ERA40\1971ERA1973T2

CONVERT_TO_ASCII : 0
CONVERT_TO_HDF5 : 1

XX_VARIABLE : longitude
YY_VARIABLE : latitude
TIME_VARIABLE : time
<end_file>

===Convert a MM5 file to MOHID(.hdf5)===
<begin_file>
ACTION : CONVERT MM5 FORMAT

FILENAME : K:\MM5output\DataCenter\Modelos\Meteo_IST\Fev2005\MMOUT_D3
TERRAIN_FILENAME : K:\MM5output\DataCenter\Modelos\Meteo_IST\Fev2005\TERRAIN_D3
OUTPUT_GRID_FILENAME : K:\MM5output\DataCenter\Modelos\Meteo_IST\Fev2005\grid3.dat
OUTPUTFILENAME : K:\MM5output\DataCenter\Modelos\Meteo_IST\Fev2005\MM5_D3.hdf5

COMPUTE_WINDSTRESS : 0
COMPUTE_RELATIVE_HUMIDITY : 1
WRITE_XYZ : 0

<<BeginFields>>
solar radiation
air temperature
wind velocity X
wind velocity Y
sensible heat
latent heat
atmospheric pressure
sea water temperature
<<EndFields>>
<end_file>

===Convert Mercator-Ocean(.nc) to MOHID(.hdf5)===
<begin_file>
ACTION : CONVERT MERCATOR FORMAT

OUTPUTFILENAME : Psy2v2r1v_R20060628/MercatorR20060628.hdf5
OUTPUT_GRID_FILENAME : Psy2v2r1v_R20060628/MercatorGridR20060628.dat
OUTPUT_GEOMETRY_FILENAME : Psy2v2r1v_R20060628/MercatorGeometryR20060628.dat

INPUT_GRID_FILENAME : GridFiles/ist_meteog-gridT.nc
INPUT_GRID_FILENAME_U : GridFiles/ist_meteog-gridU.nc
INPUT_GRID_FILENAME_V : GridFiles/ist_meteog-gridV.nc

<<begin_input_files>>
Psy2v2r1v_R20060628/ist_meteog-mercatorPsy2v2r1v_T_MEAN_ANA_20060621_R20060628.nc
Psy2v2r1v_R20060628/ist_meteog-mercatorPsy2v2r1v_T_MEAN_ANA_20060622_R20060628.nc
Psy2v2r1v_R20060628/ist_meteog-mercatorPsy2v2r1v_T_MEAN_ANA_20060623_R20060628.nc
<<end_input_files>>
<end_file>

===Convert Levitus format to MOHID(.hdf5) and interpolate grid===
==== Convert ====
First convert the Levitus ASCII format to a raw HDF5 format:
<begin_file>
ACTION : CONVERT LEVITUS FORMAT

OUTPUTFILENAME : Levitus.hdf5
OUTPUT_GRID_FILENAME : LevitusGrid.dat
OUTPUT_GEOMETRY_FILENAME : LevitusGeometry.dat

PERIODICITY : monthly
SPATIAL_RESOLUTION : 0.25
FILL_VALUE : -99.9999

LOWER_LEFT_CORNER : -16.0 31
UPPER_RIGHT_CORNER : 1. 40

<<beginfield>>
NAME : salinity
ANNUAL_FILE : DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s000hr.obj

<<<begin_input_files>>>
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s001
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s002
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s003
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s004
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s005
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s006
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s007
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s008
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s009
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s010
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s011
DataCenter\DadosBase\Ocean\Levitus\Data\Salinity\s012
<<<end_input_files>>>
<<endfield>>

<<beginfield>>
NAME : temperature
ANNUAL_FILE : DataCenter\DadosBase\Ocean\Levitus\Data\Temp\t000hr.obj

<<<begin_input_files>>>
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t001
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t002
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t003
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t004
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t005
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t006
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t007
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t008
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t009
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t010
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t011
DataCenter\DadosBase\Ocean\Levitus\Data\Temperature\t012
<<<end_input_files>>>
<<endfield>>
<end_file>

==== Extrapolate ====
Then extrapolate the data (still in the raw HDF5 format):
<begin_file>
ACTION : INTERPOLATE GRIDS

TYPE_OF_INTERPOLATION : 1

FATHER_FILENAME : Levitus.hdf5
FATHER_GRID_FILENAME : LevitusGrid.dat

OUTPUTFILENAME : LeviTusAllPointsWithData.hdf5
NEW_GRID_FILENAME : LevitusGrid.dat

START : -9999 1 1 0 0 0
END : -9999 12 1 0 0 0

INTERPOLATION3D : 1
FATHER_GEOMETRY : LevitusGeometry.dat
NEW_GEOMETRY : LevitusGeometry.dat
AUX_GRID_FILENAME : LevitusGrid.dat
AUX_OUTPUTFILENAME : AuxLeviTusAllPointsWithData.hdf5

POLI_DEGREE : 3
DO_NOT_BELIEVE_MAP : 1
EXTRAPOLATE_2D : 2
<end_file>

==== Interpolate ====
Finally, interpolate to the final grid and geometry (same as the [[#Interpolate 3D MOHID(.hdf5) files to a new grid| Interpolate 3D sample]]):
<begin_file>
ACTION : INTERPOLATE GRIDS

TYPE_OF_INTERPOLATION : 1
FATHER_FILENAME : LeviTusAllPointsWithData.hdf5
OUTPUTFILENAME : CadizMonthlyLevitus.hdf5
FATHER_GRID_FILENAME : LevitusGrid.dat
NEW_GRID_FILENAME : Algarve0.02SigmaSmooth_V3_CartMoreLayers.dat

START : -9999 1 1 0 0 0
END : -9999 12 1 0 0 0

INTERPOLATION3D : 1
FATHER_GEOMETRY : LevitusGeometry.dat
NEW_GEOMETRY : Geometry_1.dat
AUX_OUTPUTFILENAME : AuxCadizMonthlyLevitus.hdf5
AUX_GRID_FILENAME : Aux12km.dat

POLI_DEGREE : 3
DO_NOT_BELIEVE_MAP : 1
<end_file>

Note that the programme may construct a new bathymetry twice. Use this bathymetry only on the AUX_GRID_FILENAME keyword.

===Convert Hellerman Rosenstein ASCII format to MOHID(.hdf5) ===
<begin_file>
ACTION : CONVERT HELLERMAN ROSENSTEIN ASCII

OUTPUTFILENAME : ClimatologicWindStress.hdf5
OUTPUT_GRID_FILENAME : ClimatologicWindStressGrid.dat

PERIODICITY : monthly
SPATIAL_RESOLUTION : 2.
FILL_VALUE : -99.9999

LOWER_LEFT_CORNER : -180 -90
UPPER_RIGHT_CORNER : 180 90

<<beginfield>>
NAME : wind stress X
FILE : D:\Aplica\Dados\Hellerman_Rosenstein\TAUXX.DAT
<<endfield>>

<<beginfield>>
NAME : wind stress Y
FILE : D:\Aplica\Dados\Hellerman_Rosenstein\TAUYY.DAT
<<endfield>>
<end_file>

===Convert ALADIN(.nc) format to MOHID(.hdf5)===
<begin_file>
ACTION : CONVERT ALADIN FORMAT

OUTPUTFILENAME : aladin.hdf5
OUTPUT_GRID_FILENAME : aladin_griddata.dat

!Put here the name of any netcdf file for grid-data generation's sake.
INPUT_GRID_FILENAME : D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKCLOUD_OPASYMP_19723_20088.nc

<<begin_input_files>>
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKIR_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKPRES_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKSOLAR_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKTAIR_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKWIND_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_FLUXPRE_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_STRESSU_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_STRESSV_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_U10_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_V10_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKCLOUD_OPASYMP_19723_20088.nc
D:\Aplica\BiscayAplica\FORCAGES\METEO\ALADIN_BULKHUMI_OPASYMP_19723_20088.nc
<<end_input_files>>

<end_file>

== OceanColor modules compilation ==
Compiling the [[ConvertToHDF5]] tool with the OceanColor modules is more complicated than one might expect. A solution is proposed here for a release version using the Compaq Visual Fortran 6.6c. The difficulties rise because C code is embedded with a fortran interface and also, extra libraries such as hdf4 are required.

=== Pre-requisites ===

This is a list of prerequisites to successfully compile the tool:
*Compaq Visual Fortran 6.5 with patch 6.6c,
*VS .NET 2003 (Vc7 in particular),
*Hdf5 libraries ('''hdf5.lib''' '''hdf5_fortran.lib''' '''hdf5_hl.lib'''),
*Netcdf libraries ('''netcdf.lib''' '''netcdf_.lib'''),
*Hdf4 libraries ('''hd421.lib''', '''hm421.lib'''),
*szlib, zlib and jpeg libraries ('''szlib.lib''', '''zlib.lib''' and '''libjpeg.lib'''),
*the fortran source files ('''ModuleConvertModisL2.F90 ModuleConvertModisL3.F90 ModuleConvertOceanColorL2.F90'''),
*the C source files and their fortran interface files ('''readL2scan.c readL2Seadas.c''' and '''cdata.f crossp.f fgeonav.f''').

=== CVF IDE configuration ===
# Configure everything as specified in [[Compiling with CVF]].
# Add the source files listed in the prerequisites above to the source files listing.
# Go to '''Tools-->Options...-->Directories'''. There, add the '''$DOTNET2K3/Vc7/bin''' to the '''Executable files''''; the '''$DOTNET2K3/Vc7/include''' and '''$DOTNET2K3/Vc7/PlatformSDK/include''' to the '''Include files'''; and finally, the '''$DOTNET2K3/Vc7/lib''', '''$DOTNET2K3/Vc7/PlatformSDK/lib''' and '''$DOTNET2K3/Vc7/PlatformSDK/bin''' to the '''Library files'''.
# Go to '''Projects-->Settings-->Release-->Link-->Input'''. There, add the following libraries: '''netcdf.lib netcdf_.lib hd421.lib hm421.lib libjpeg.lib'''. (Make sure the hdf5 libraries as well as the szlib and zlib libraries are already mentioned).

=== Troubleshoots ===
'''Q: I get unresolved external references during linkage, but I have all the libraries mentioned above included. What should I do?'''

A: Unresolved external references can come out for two reasons:
#you didn't specified all the libraries required or all the paths for the default libraries or,
#[http://en.wikipedia.org/wiki/Name_decoration name mangling] problems. Use the [[dumpbin]] utility to the libraries to checkout which language convention they are using. If that's the problem then you need to try to get new libraries with the correct naming convention.

That's it, you should now be able to build the [[ConvertToHdf5]] project successfully.

'''Q: I got a message saying the entry point _NF_PUT_ATT_REAL@28 could not be located in netcdf.dll'''

A: copy the file netcdf.dll to the exe folder

== References ==
*[http://www.hdfgroup.org/ HDF5 Homepage]
*[http://www.hdfgroup.org/ HDF4 Homepage]

== Links ==

*[[Module_Atmosphere]]
*[[Module_InterfaceWaterAir]]
*[[Coupling_Water-Atmosphere_User_Manual]]

[[Category:Tools]]
[[Category:Hdf5]]

Parallel processing

2009-05-11T19:39:58Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2009-05-11T18:43:44Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2009-05-11T18:33:18Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2009-05-11T18:05:04Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2009-05-11T18:00:51Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2009-05-11T16:55:29Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2009-05-11T14:20:07Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2009-05-11T14:19:47Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2009-05-11T14:19:11Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2009-05-11T14:18:38Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2009-05-11T14:10:57Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2009-05-11T14:01:46Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2009-05-11T13:50:45Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2009-05-11T13:30:37Z

AngelaCanas: /* Parallel processing via OpenMP */

Parallel processing

2009-05-11T13:08:16Z

AngelaCanas: /* Parallel processing via OpenMP */