CouPE — Coupled Physics Environment

A glass-box multi-physics framework

CouPE (Coupled Physics Environment) is a flexible, coupled, multi-physics framework library that aims to solve tightly coupled physical phenomena with accurate solution procedures using loosely coupled software interfaces. CouPE leverages existing software libraries for handling the scalable linear algebra solvers and discrete mesh data structures that serve as a “data backplane” to link physics modules through their spatial domains via interfaces to PETSc [1] and Mesh- Oriented database (MOAB) [2] respectively. The plugin architecture employed within CouPE enables quick prototyping of different coupling strategies (Operator-Split, Picard, n-Richardson, Jacobian-Free Newton-Krylov variants) which rely on minimally intrusive wrappers to individual physics modules that have been written using MPI-based paradigm in C/C++/Fortran languages.

The CouPE library was created as part of a greater effort to bring together and solve coupled multi-physics problems in nuclear engineering analysis. While the original design was written to specifically meet the needs of the problem at hand, the motivation for a more general coupler grew. The design of CouPE is being generalized to make it Physics agnostic so that arbitrary modules can be dependent on each other’s solution fields through a uniform interface. We hope this will be an evolutionary framework to encompass multi-physics and multi-scale problems in a general setting without sacrificing accuracy or efficiency.

Background

High fidelity computer simulations of coupled multi-physics problems require solving large systems of stiff, nonlinear, coupled equations. Some examples include nuclear engineering, plasma physics, subsurface modeling, fluid-structure interaction, climatology and almost all non-trivial physical phenomena. Often, the physical length and time scales are widely varying O(10² – 10¹⁰). The solution to such systems entail efficient, stable, highly accurate, consistent methods to reduce simulation errors to user specified requirements.
Typically, there are two approaches to assembling a coupled-physics simulation capability for tackling such complex systems. In one approach, pieces of existing single-physics codes are assembled into an overall coupled simulation code. This has been referred to as a “small-f” or “bottom-up” framework approach [3]. The other approach is to use an integrated, coupled physics modeling framework, with new code pieces for each relevant physics area developed inside that framework. This is sometimes referred to as a “large-F” or “top-down” Framework approach, with several implementations available[5, 6]. One advantage to the former approach is that it clearly takes advantage of the several man-years already invested in existing physics modeling codes, while a possible disadvantage is that it may preclude fully-coupled solution methods that are the strength of large-F Frameworks.
CouPE is motivated to serve as a bridge between the small-F and large-F frameworks by providing a mechanism to both couple existing codes with simplified wrappers to a MOAB mesh/solution transfer capability and to create a physics module with FEM-based discretizations that can be directly assembled on a MOAB mesh natively. Such a unified formulation could yield a framework necessary to consistently address errors from approximate numerical methods and other uncertainty sources in simulations.

Coupling Schemes

Traditional solution methods involve variations of ‘Black-box’ coupling strategies with problem-specific approximations. These are often closely tied to a test scenario of interest during analysis and hence provide the most efficient solvers under certain constraints. The generality of such approaches to all problem state-space is limited and can often introduce approximation errors in the treatment of nonlinear terms due to nature of self-inconsistent state solution coupling that is inherent in these approximations. Such methods mathematically decompose PDEs into simpler operators to create specialized algorithms, which is referred to in literature as Operator-Splitting (OS). These OS schemes also involve explicit treatment of nonlinearities and hence are alternately called as explicit coupling.
CouPE enables scalable and extensible coupling of different physics components that are nonlinearly dependent on each other’s solution fields in complex relationships. Typically, multi-physics coupled codes for simulating different physics phenomena employ validated and verified, efficient mono-physics codes with MPI architecture to achieve the loose or explicit coupling with an OS methodology [6]. Such “divide and conquer” methods provide flexibility in the usage of standard industrial codes and avoid replicating man years of development and testing by following the bottom-up approach. These OS coupling schemes suffer when rigorous requirements on data exchange and consistency of the coupling terms are not met by sacrificing on either the accuracy or stability of the numerical solution.

Pros

• Individual physics implementation can use specialized solution algorithms for high efficiency.
• Verification/Validation of single physics codes are easier.

Pros

• Often the nonlinearities between physics are unresolved.
• Spatial and temporal errors are introduced due to inconsistent treatment of coupled terms.
• Explicit linearization introduces splitting errors with loss of temporal accuracy and unconditional stability

Schematics of OS schemes in a nuclear engineering simulation (two physics: Neutronics and Thermal-Hydraulics)

As an alternative, it is possible to perform multiple iterations between the physics components such that the coupled nonlinear terms are resolved completely at each step before marching forward in time. Such iterative schemes are consistent and can reuse the components utilized in traditional OS strategies. Using either variants of Picard or nonlinear-Richardson iterations, the solution convergence can be controlled and tightened based on user requirements.

Modules (proposed)

1. Physics – The Physics wrapper that describes a physics. Uses spatio-temporal discretization and provides access to underlying discrete systems that can be solved for obtained solution fields of interest.
2. CoupledSolver – The Core solver object that orchestrates the coupled physics simulation
   1. PhysicsManager – Responsible for storing and invoking various collections of physics that are related (coupled)
   2. LooseSolver – Solver variants utilizing Operator-Split schemes or Picard linearizations
   3. TightSolver – If the residual functional for Physics object is available, some form of nonlinear-Richardson iteration or Newton-Krylov variants or a combination thereof can be used.
3. SpatialIntegrator – Generic tools to accurately compute different types of basis functions, quadrature rules on a given element type of user-specified order will be available through a lightweight Local Discretization library. Also provides MOAB based solution transfer mechanisms to perform point-location and spatial projection via the MBCoupler tool which is used by the CoupledSolver for transferring solution fields between physics that are nonlinearly coupled. Optionally, interfaces to some standard FEM libraries like libMesh, Deal.II and Feel++ will be provided in CouPE with appropriate MOAB mesh wrappers to transparently enable physics coupling with existing codes.
4. TemporalIntegrator – Tools to perform high-order temporal integration using various Runge-Kutta schemes; Options include IMEX-type, DIRK, SDIRK, FIRK (Radau, Lobatto) schemes.

Schematic

References

[1] Satish Balay, William D. Gropp, Lois Curfman McInnes, and Barry F. Smith. Efficient management of parallelism in object oriented numerical software libraries. In E. Arge, A. M. Bruaset, and H. P. Langtangen, editors, Modern Software Tools in Scientific Computing, pages 163–202. Birkhäuser Press, 1997.
[2] Timothy J. Tautges, Hong-Jun Kim, Alvaro Caceres, and Rajeev Jain. Coupled Multi-Physics simulation frameworks for reactor simulation: A Bottom-Up approach. In International Conference on Mathematics and Computational Methods Applied to Nuclear Science and Engineering (M&C 2011), Rio de Janiero, Brazil, May 2011. American Nuclear Society.
[3] A. Siegel, T. Tautges, A. Caceres, D. Kaushik, P. Fischer, G. Palmiotti, M. Smith, and J. Ragusa. Software design of SHARP. In Joint International Topical Meeting on Mathematics and Computations and Supercomputing in Nuclear Applications, M and C + SNA 2007. American Nuclear Society, April 2007. Conference Proceedings.
[4] James R Stewart and H.Carter Edwards. A framework approach for developing parallel adaptive multiphysics applications. Finite Elements in Analysis and Design, 40(12):1599–1617, July 2004.
[5] Derek Gaston, Chris Newman, Glen Hansen, and Damien Lebrun-Grandi. MOOSE: a parallel computational framework for coupled systems of nonlinear equations. Nuclear Engineering and Design, 239(10):1768–1778, October 2009.
[6] V.S. Mahadevan, High resolution numerical methods for coupled non-linear multi-physics simulations with applications in reactor analysis, PhD Dissertation, Texas A&M University, 2010.