The Accord programming framework: Enabling the development and management of autonomic applications

Motivation

The underlying computational and communication environments, for example the Grids and pervasive environment, become large, heterogeneous, dynamic and uncertain. These characteristics result in application development, configuration and management complexities that break current programming models and frameworks based on passive components and static compositions. This has led researchers to consider alternative programming paradigms and management techniques that are based on strategies used by biological systems to deal with complexity, dynamism, heterogeneity and uncertainty. The approach, referred to as autonomic computing, aims at realizing computing systems and applications capable of managing themselves with minimum human interference.

The Accord programming framework (a sub project in AutoMate) builds on the separation of composition aspect (organization, communication and coordination) from computation aspect underlying the component-based programming models/frameworks, and extends it to enable the computational behaviors of components as well as their composition to be automatically managed at runtime using high-level rules. These rules can be used to automatically calibrate computational behaviors of individual components and establish/delete/modify interaction relationships between components at runtime, and as a result applications can at runtime adapt to the changing environment. These context-aware and self-managed applications can fully exploit the heterogeneity, dynamism and uncertainty nature of the Grid environments.

Contributions

• The separation of functional features from non-functional features: An autonomic component separately exposes its functional features and non-functional features such as resource requirements, security policies etc. The separation enables that different features are used to achieve computational or performance objectives at compile time and at runtime.
  
• The separation of composition (organization, communication, and coordination) from computation: This separation enables the composition of components to be described at compile time, but enforced and calibrated at runtime according to current environment situation.
  
• The separation of rules from the rule enforcement mechanism: Descriptive high-level rules are used to compose components into an application and manage it at runtime. The separation enables these rules to be flexibly added/deleted/modified at runtime, and interpreted/executed by the rule enforcement mechanism, provided that the rules conform to the semantics and syntax understood by the enforcement mechanism.
• Locality: Obtaining and maintaining global knowledge is extremely expensive or even impossible in a large-scale, dynamic, heterogeneous and uncertain environment, for example the Grid environment. Therefore, the application objectives are decomposed into sub objectives (in terms of rules) and these rules are injected into individual components. Individual components will independently make their decisions based on the rules and their direct contextual environment. A top-down decomposition strategy and a runtime negotiation model are used to ensure that local decisions and behaviors taken by individual components achieve the global objectives.
  

Approach Overview

The autonomic component is the fundamental unit for composition, deployment, and execution in Accord programming framework. An autonomic component exposes its functionalities through functional port, exposes its sensors and actuators for external monitoring and steering through control port, and performs rule operations via operational port. An autonomic component contains a component manager, which is responsible for analyzing and executing the high-level rules for runtime managing its computational behaviors and interaction relationships with other autonomic components in an application context.

The autonomic components can be dynamically composed together. We view dynamic composition as runtime replacement of components, addition or deletion of components, and modification of interaction relationships between components during the execution of the applications.

The application workflow is decomposed into communication rules and coordination rules, which are injected to corresponding component manager embedded within individual autonomic components. The rules are evaluated and executed by the component managers to automatically establish interaction relationship between components in a distributed and parallel manner. Therefore, dynamical addition/deletion of components can be achieved via addition/deletion of related interaction rules from/to the components involved in the specific interaction, while components which are not involved will not even notice the change. Similarly, modification of interactions between the components can be achieved through modifying corresponding communication or coordination rules.

To ensure rule correctness, we design a rule template for each communication paradigm (e.g., RPC/RMI, publish/subscribe) and coordination pattern (e.g., conditional branch, loop, sequence, parallel execution). A rule template is instantiated via filling in parameters, such as the names of interaction parties and the exchanging data.

An agent framework is created at runtime to enable the decomposition of application workflows, instantiation and injection of rules into individual components, local evaluation and execution of rules in a distributed manner to achieve global objectives. The agent framework consists of component managers and composition agent(s).

• The composition agent(s) interacts with users to receive rule operations, generates and injects rules into component managers, and coordinates these component managers. The composition agent(s) might be a hierarchy of agents for the sake of scalability.
• The component managers evaluate and execute local rules according to current contextual environment to actually establish / change communication channels, coordination relationships with other components, and calibrate the components' runtime computational behaviors. The component manager listens to incoming requests, invoking functional interfaces of the computational component, and sending the outputs to destination components using the specific communication paradigms based on rules. While the internal computational components are passive, and only expose functional interfaces for invocation, and sensors/actuators for monitoring and controlling. 

Implementation and applications

Accord has been implemented on DOE CCA Ccaffeine framework to enable self-managed scientific applications that can adapt to the changes in the execution environment and maintain acceptable performance. The Ccaffeine framework was extended with (1) component manger that monitors and manages the behaviors of individual components, and (2) composition manager that manages, adapts and optimizes the execution of an application at runtime. A three-phase rule execution model to enable consistent and efficient rule execution for distributed/parallel scientific applications. Performance data collection is enabled by TAU library.

Accord has been used to enable the Hydrodynamics Shock Simulation with self-management capabilities. This simulation simulates the interaction of a hydrodynamic shock with a density-stratified interface. At runtime, the self-managed simulation can switch between different algorithms to process data for less cache misses, and adapt to the current latency and available bandwidth in the environment by using different communication mechanisms. Accord has also been used to allow CH4 ignition simulation to select backward difference formulas at runtime as the simulation processes to a different data domain. This capability results in an average 11:33% computational saving.

Continuing research

Accord is being improved and extended with more capabilities, for example heuristic rules replaced by Model-based Online Control. For more information, please visit http://www.caip.rutgers.edu/~virajb/researchhighlights.html.

Author: Hua Liu