The MACE (Multi-block Adaptive Computational Engine) infrastructure support multi-block grids where multiple distributed and adaptive grid blocks with heterogeneous discretization are coupled together with lower dimensional (also distributed and adaptive) mortar grids. MACE is being extensively used to support subsurface modeling and oil reservoir simulations.
PAGH (Parallel Adaptive Grid Hierarchy), developed by Erik Schnetter working with ASC (Astrophysics Simulation Collabortory) project members, Evans, Goodale and Parashar, is a GrACE-based driver routine designed and developed for the ASC. It provides an interface for memory management and I/O for the grid functions used in a simulation. On parallel machines, the driver also manages necessary communication between the individual processing nodes.
PAGH replaces the standard Cactus driver PUGH (Parallel Uniform Grid Hierarchy) which is limited to uniform structured grids. PAGH extends the driver to distributed adaptive grid hierarchies. It also provides facilities for adaptive mesh refinement (AMR) using the Berger-Oliger AMR formulation, where the refined regions are rectangular boxes that have to be completely inside a coarser box. PAGH restricts the refined regions to have an integer refinement factor and to be aligned with respect to the coarser regions. The number of refinement levels and the number of refined regions per level are not restricted. The location and the size of the refined regions can be changed at runtime. Based on a user-defined truncation error estimator, refined regions automatically track regions of high error and adapt to the computational needs. To facilitate the truncation error estimation, PAGH can provide a shadow hierarchy, i.e. a twin to the computational domain with a lower resolution. The truncation error can be estimated by comparing the results obtained on both domains. This capability uses the support for a Shadow hierarchy provided by GrACE. PAGH parameterizes all technical aspects of the refinement procedure and allows them to be changed by the user. These aspects include certain space/time tradeoffs, and the prolongation and restriction stencils used to transport data between the refinement levels. Reasonable defaults are provided for most applications.
PAGH is suited for all 3D applications that need to selectively enhance the spatial resolution to meet their numerical requirements within the bounds given by the available computational resources.