The Adele configuration manager

Reviewer: Armin B. Cremers

Configuration management (CM) is the discipline of organizing and controlling evolving systems. For hardware systems, CM has been an established field since the 1950s. It was initially developed in the aerospace industry as an approach for guaranteeing reproducibility of spacecraft. The problem was that spacecraft underwent numerous, inadequately documented engineering changes during development. Moreover, the prototypes embodying the changes were usually expended during flight test. Consequently, neither accurate plans nor prototypes were available for replicating successful designs. Configuration management was thus born out of the need to track what designers and engineers developed during the course of a project. Software Configuration Management (SCM) is the specialization of CM for software systems. Software systems, too, undergo numerous, usually inadequately documented changes. Though software prototypes are not necessarily expended, they nevertheless are an insufficient basis for further development, because of severe information loss during development and maintenance. To prevent information loss, traditional CM mandates strict procedures for identification of parts and assemblies, for control and auditing of changes, and for status accounting. SCM goes beyond these procedures in several ways. First, since all software items are under computer control, many of the identification and control tasks can be automated and perfected. Furthermore, since software development tools run under computer control, they can be integrated directly with automatic SCM procedures. We are now beginning to see the results of this integration in terms of automated SCM systems that take away much of the tedium of traditional configuration management procedures, provide better and faster services, and reduce errors. As CAD/CAM and robotics bring manufacturing more and more under computer control, similar improvements can be expected in this area as well (From the Preface). Leblang Leblang's paper gives a competent survey of configuration management challenges, presents state-of-the-art solutions, and discusses their strengths and weaknesses. Not surprisingly, Leblang advertises the configuration management tool produced by his company. More than half of the paper treats his own product in great detail. Many aspects of configuration management are discussed solely on the example of this product without mentioning how they relate to other tools. Also, the choice of evaluation criteria applied to other systems appears to be somewhat biased. The discussion often loses the thread and becomes absorbed in lengthy presentations of Leblang's own system. This is regrettable, since the essence of the paper offers a good introductory overview of configuration management issues. SCM is introduced as “the art of keeping track of what has changed and how things are combined…to produce complete software systems. The SCM is responsible for the `who, what, when, how, and why' of every source code change and every software system build.” To this end, SCM encompasses version control, environment management, build management, and process control. The paper compares and contrasts various approaches to CM on the basis of how they address these four categories. CM systems are a repository for sources, binaries, and other project objects. Most systems are built on the notion of a vault, a central repository from which sources must be extracted and deposited in a private work area for access by development tools. Usually, all sources are extracted from the vault, creating many copies of each file. Concurrent updates of different copies eventually result in a variety of problems, including lost changes. The fundamental problem inherent in vaults is that the software development tools cannot directly access the version controlled objects. An attempt to solve this problem is to provide an application programming interface (API) to the repository, declaring it to be a standard and requiring all software development tools to use the declared interface. The drawback of this standard repository approach is that all tools must be changed repeatedly to incorporate an interface that will quickly become outmoded. The most advanced solution is the use of a virtual file system, which intercepts native operating system I/O calls and redirects them to the repository. The repository appears to be a standard directory tree, integrated into the regular filesystem. A version control system keeps track of all changes to every file and supports parallel development by enabling easy creation of variants (branching) and the later reintegration of the variants (merging), without a slow approval process. A significant problem of version control is coordination when a change spans multiple objects; the branch created on one object is not easily associated with the branches on other objects created as part of the same activity. One approach is to use symbolic names for branches, giving related versions the same branch name. Another popular approach is change sets, which group together related changes of different objects. Branches and change sets can be beneficially integrated. Branches are used to organize broad development lines, whereas change sets are used to record which changes went together for a specific change request. Parallel development efforts are usually merged back into a main development line. An automatic merge facility can help with the mechanics of merging source code changes. A merge tool that knows the common ancestor of the versions being merged can generally merge with little or no human interaction. The price of automatic merging is occasional inappropriate merge decisions (in about 1 percent of the cases), but most of them are easily detected because they result in compiler syntax errors. Automatic mergers can also support nonstandard merge cases. Examples are ladder merges, which repeatedly merge the same long-lived branch back into the main line; additive merges, which propagate just one specific change from one development line to another development line; and subtractive merges, which remove changes made in a range of versions. A key aspect of any configuration management system is the ability to define a workspace that contains the right set of source versions (environment management). There are a variety of ways to establish a workspace. Common models are unconstrained, hierarchical, change sets, and rule-based environments. In unconstrained environments, users extract sources from a vault and put them into private areas of their own choosing. It is difficult to have a predictable and repeatable process in an unconstrained environment. In hierarchical models, the workspace is defined to be a copy of a parent workspace plus all local changes. Hierarchical models do not provide the level of flexibility needed in a large development effort. There always must be a common parent, which must be irreversibly modified in order to integrate changes. Also, a change in one subenvironment cannot be easily propagated to a sibling environment without updating the parent and thus affecting all siblings. A hierarchy is insufficient for creating a new configuration in which features from only selected siblings are combined. Change sets are a valuable tool for grouping related changes. They work best when changes are organized into development lines that form an evolving base for changes that will be released together. In practice, commercial software vendors organize changes so that one bug fix builds on earlier bug fixes. At some point, all of these so-called patch-bundles for different product components are combined, and a maintenance release is issued. The highest flexibility is provided by rule-based selection of the object versions that are part of the current work environment. Much of the power of this approach is lost if rules are evaluated prematurely. Only dynamic rule-based models, in which rule evaluation is deferred until objects are really accessed, guarantee that the contents of an environment are always up to date. The cost of runtime evaluation of rules and creation of readable documents from compressed storage can be minimized by storing the loaded documents in a network-wide on-disk cache (with a reported hit rate of over 99 percent). Ultimately, source code must be built into derived objects: binaries, executable images, and other objects that constitute a final product. Build management is concerned with management of the derivation process, of derived objects, and of all the objects that are required as input for rebuilding a given derived object. Derived objects must be built from the correct source versions using the correct tools and options. Ideally, tool versions, build options, and environment variables would play a role in the rebuild logic. After a derived object is produced, it must be possible to determine a reliable bill of materials describing all of the constituent parts of that object. The bill of materials should be absolutely correct and detailed enough to easily reconstruct the complete source environment in effect at the time the object was built. Many systems use programmer-specified dependency information. But in practice, manual maintenance of complete and correct dependency information is an unwanted and unwieldy task. An elegant solution to gathering a bill of materials without burdening the programmer is enabled by systems based on virtual file systems. They can automatically collect the desired information by monitoring the file I/O done by the compiler and other tools during a build. A software development process consists of everything involved in the conception, creation, deployment, and maintenance of a software product. Besides configuration management, problem tracking, activity definition, and scheduling, automated testing, metrics monitoring, and other tasks have their unique roles. A class of tools called workflow managers attempts to automate business processes with the goal of reducing the costs and increasing the overall quality of products and services. The key concept in workflow tools is action-response: there are a number of tasks, assigned to various people, that must be accomplished. When an action is taken on one of the tasks, a response is automatically generated. Various state-transition rules govern which actions can be taken. The effect of a dedicated workflow management system can also be achieved by less sophisticated tools if they provide the basic action-response mechanism, for example via so-called triggers. However, building the desired software development process policy on low-level mechanisms requires more programming and maintenance effort. Kaiser, Popovich, and Ben-Shaul There are generally two foci of interest in software process modeling languages: One approach stresses constraints on process steps (like prerequisites and consequences), and the other expresses the topology of control flow and synchronization. In this paper, the shortcomings of every approach are first properly discussed by means of the ISPW7 benchmark. The authors then advocate combining both viewpoints in a single formalism and present their own bilevel implementation. The two building blocks of their solution are briefly described: Activity Structures Language (ASL) and their own rule-based MARVEL Strategy Language. In the integration of these two formalisms, the authors introduce a notion of parameter variable bindings into ASL. By this technique, they handle object propagation in a sequence of activities, and decide whether bindings should be retained in the next iteration or be renewable. ASL definitions are translated to MSL classes: additional conditions and effects are inserted into the rules to enforce the specified topology. The paper does not provide a gentle introduction to the main features of the bilevel approach. It is, rather, a tour through MARVEL and ASL. The purpose of the bilevel approach is made clear, but the solution is only sketched and presupposes familiarity with the MARVEL system, especially when the connection between ASL activities, the object structure, and the rules is discussed. The bilevel approach seems convincing, but the authors correctly point out that experience in industrial settings will be needed. Mahler Mahler discusses variants in software configuration management. He first demands a clear separation of variant management and version control mechanisms. In this context, versioning often only deals with the support of concurrently modifying a software component with the intention of later merging the different versions into a new revision. Variants, in contrast, represent components of the software product where different variants are never merged. Simple yet practically important examples are variants targeted for specific platforms. Nevertheless, a widely accepted definition of the term “variant” that is both general enough and facilitates meaningful technical support has yet to be found. Section 2 introduces the two main techniques used for the representation of software variants, namely variant segregation and single-source variation. Segregation implies a separate copy of a component for each variant, while single-source variation uses only one physical representation and extracts the desired variants upon request, usually by conditional compilation. The pros and cons of these techniques with respect to redundancy and consistency are discussed. The remainder of section 2 deals with advanced issues of variant management; the most interesting are mechanisms for consistent aggregation of variants as well as multiple variance, namely, variation of products or components in more than one dimension. In section 3, Mahler analyzes currently available tools and their approaches to solving the problems identified in the previous section. Furthermore, he investigates the tools' ability to support the build of product variants as well as the authoring of component variants. The section provides a concise overview of supported concepts as well as implementation techniques. In particular, language-supported tools are compared to system modeling approaches. System modeling uses an abstract description of the product's components and their relationships in order to select the appropriate compiler invocations during a build request. Language support, in contrast, enables a configuration management system to extract the system model automatically. However, language support lacks the needed flexibility with respect to flexible combinations of tools for the software production process. The last section introduces the author's own approach to variant management as implemented in “shape,” the build driver of the software configuration management system ShapeTools. Shape relies on a system description file similar to a makefile, but, basically by a dynamic evaluation of description file variables, it is able to assemble product components (and their variants, respectively) according to a context derived during the build process itself. The author sees this technique as a meta-mechanism independent of the employed variant representation techniques or other variant management support. A higher-level tool can use the logical variant concept of shape in order to provide project-specific variant management concepts. The chapter is both an introduction of variant management problems and a thorough discussion of some advanced issues in this area. Mahler clearly explains the closely intertwined aspects of variant management support and discusses approaches found in the literature in as much depth as space limitations allow. The shape approach to some of the identified problems, however, is on the level of makefile definitions, and can be considered a rather low-level mechanism. On the other hand, it is not Mahler's intent to present shape as a general solution for variant management problems. Rather, the possible use of shape as a variant controller within a higher-level tool is proposed. Estublier and Casallas This chapter presents the Adele software configuration manager—a commercial system successfully used in software development for more than ten years—and discusses some general problems of versioning. The main contribution of the authors is an attempt to convey the experience accumulated in three relatively independent computer science communities: software engineering, computer-aided design, and database management. Each of these communities has to deal with versioning; however, each of them has evolved practically isolated from the others. The Adele data model is object-oriented. Some current extensions to the original data model are introduced, among which are relationships for the representation of complex objects and queries for navigation in the object network. The proposed extensions enable distinguishing between different types of versioning, for example temporal, logical, and dynamic versioning. The configuration model, initially supposed to generate configurations automatically (that proved to be too restrictive in practice), is also extended in order to open it up to users. The authors discuss the problem of communication between the workspace and the object repository and between multiple workspaces. The specificity of the software engineering environment is that the workspace must be accessible for different software tools (editors, compilers, and so on), which, as a rule, are inherently external to the version manager: they are intended to operate on the standard file system and to compete for the workspace resources independently. The authors give a short overview of different approaches to workspace control and discuss pros and cons. It is argued that a configuration manager should provide a universal mechanism that would enable a controlled execution of diverse processes, such as performing configuration consistency checks. The process management technique proposed in the paper is based on the trigger (Event-Condition-Action, or ECA rules) formalism. This powerful but low-level mechanism gives Adele users the means to customize their own control strategies. The authors present an example of such high-level process support built on the basis of the ECA rules. The chapter finally summarizes the experience gained with the Adele configuration manager and outlines directions for future research. Beware—there are numerous typos and errors in the paper, leading to difficulties in understanding. Fowler, Korn, and Rao The design and implementation of the Multiple Dimensional File System ( n -DFS) is presented. It is a logical file system, which is layered between the operating system and the user's applications. Its purpose is to tailor the underlying traditional UNIX-like file system to meet the needs of configuration management in software development environments. To achieve this goal, several new services like naming, file, and monitoring services have been added by using a modified system library. This library is linked by applications and runs in their address space. Services are provided for viewpathing, versioning, event notification, visual process management, and tree replication including parrot service. The architecture of n -DFS is generic and extendable and provides, for example, a per-process name space and a mechanism to attach services to the name space. To measure the overhead of the logical layer and the costs of the viewpathing and replication services, two benchmarks have been performed. The comparison of n -DFS with related projects shows the main difference, namely the use of a modified system library. This way of extending the underlying file system impairs neither the syntax nor the semantics of any system call, nor does it require any changes to the kernel or any application. The usefulness of the provided services is made clear by using examples to illustrate the different features. A deeper background in operating systems is only needed in the context of the architecture and implementation of n -DFS. Any knowledge of configuration management software is useful for a better understanding of the comparison of n -DFS, because no reference values are given for the measured performance results. The whole project was inspired by the predecessor version 3DFS and two other related systems. So the idea of overloading file system semantics to improve software reusability and customer acceptability is not new, but its implementation by using a modified library is stated to be unique. The only problem with this concept is that the underlying file system must provide dynamic linking of shared libraries. Otherwise, the applications have to be re-linked with n -DFS's library. For programs with static linking and no source code available for rebuilding , the usefulness of n -DFS is restricted. This means that the chosen implementation method may cause some drawbacks. Conclusion The papers collected in this book provide a good introduction to the state of the art in software configuration management. The field has evolved from software version control to a comprehensive subfield of modern software technology spanning a wide spectrum of modeling tools and techniques. Tichy has done a good job in selecting subjects of practical benefit and scientific promise, and has provided a concise and useful overview of the field in his preface. Both the references appearing in every chapter and the index are of help to those readers who wish to take the book as a starting point for their own explorations.