The Datacycle architecture

Reviewer: William Campbell McGee

In contrast to conventional DBMS architectures, in which the database is held in a central location to which applications send requests for data, the Datacycle architecture employs the radical notion of sending the entire database cyclically to the application sites and allowing applications to extract data of interest as it goes by. The technology that makes this approach feasible is high-bandwidth fiber optic communication facilities for broadcasting the database, and special VLSI hardware at the application sites for extracting data. A sense of the performance possible with this approach is given in the 1987 paper [1] that first described the architecture, which states that a database of more than four gigabytes can be transmitted in less than one second over a single fiber. The approach is not unlike the operation of early information retrieval systems, in which batches of requests were processed periodically with a single pass of the data. The difference now is that batches are processed in under a second rather than in minutes, hours, or days. This paper describes a prototype implementation of the Datacycle architecture and its use in the advanced telephone directory service application for which it was initially conceived. While the Datacycle architecture appears well suited to information filtering applications (for example, it provides an efficient database trigger facility that could be used to alert subscribers to the appearance of relevant information in the database), the paper says nothing specific about information filtering. The paper consists of an overview section, which introduces the key elements of the architecture (a centralized storage pump and update manager and distributed access managers) and describes the application interface (SQL); a section on query processing, giving more detail on the access managers and the VLSI hardware required for efficient data selection; and a section on transaction processing, where the mechanism used for serializing transactions (optimistic concurrency control) is discussed in detail. The paper concludes with a discussion of the telephone directory service application, and some of the steps needed to upgrade the prototype to production status. One of the main strengths cited for the architecture is its scalability: database capacity is increased by adding storage pumps (all multiplexed onto a single optical fiber), and processing capability is increased by adding access managers. The number of read-only transactions that can execute concurrently is not limited by processing resources. Update transaction throughput may be limited by the centralized update manager and by inter-transaction update interference requiring transaction backout, however. It is interesting how many “hard” database problems go away with this approach. This system has no need for indexes and their attendant optimization problems; no need for periodic data reorganization to improve performance; simple methods for evaluating database triggers and global data consistency; efficient search on any attribute in a record; efficient processing of queries that require inspection of many records in the database (such as spatial and fuzzy queries); and freedom from traditional hotspots that limit concurrency (for example, summary data can be calculated on demand without performance penalty). So impressive are these gains that one must look carefully for problems that are not solved. One area cited by the paper is operations that are necessarily centralized and therefore potential bottlenecks or system availability exposures. Also, the authors note that the prototype is not well suited to selection based on unformatted data (such as text), although work is under way on hardware assists for this operation. They are careful to point out that the architecture is, in a sense, special purpose, although it appears to be much more broadly applicable than the authors admit. The paper is clearly written and sufficiently detailed to provide a good idea of the potential and problems in the Datacycle architecture.