CASEVision/ClearCase: SGI's Configuration Management System

This section describes Atria Software's ClearCase configuration management system. The ClearCase product is a powerful configuration management tool that provides a safe, orderly working environment for software engineers. In ClearCase environments, all file changes are recorded; separate versions of files can be made to enable concurrent work; and records of builds are kept for ready reproducibility and efficient rebuilds. This is accomplished through the following ClearCase features:

Versioned Object Bases

ClearCase uses a VOB for database management. A VOB protects the integrity of project data by ensuring that no files get inadvertently deleted or overwritten. Each VOB stores objects (that is, files, links, and directories) as well as records associated with objects, such as the description of a file checkout or the record of a particular software build. A VOB appears to standard UNIX commands as a UNIX file system; its contents can be processed using both standard UNIX tools and special ClearCase tools. Yet a VOB is more than simply a collection of directories and files; it provides historical and administrative information essential to the task of managing a development project.

A major feature of versioned object bases is version control. Version control tracks the evolution of source files, derived objects (built files), and directories by creating version trees. A version tree is a hierarchical structure representing the evolution of a single file. Figure 6 shows a typical version tree in ClearCase illustrating some of these features.

The main version of the file, typically the one used in releases, is considered to evolve along the main branch or trunk of the tree. When temporary versions of the file are needed, as in bug fixes, testing, or experimental development, they are branched off the main line to preserve the integrity of the main version. Users provide names for the branches of a tree. A branch begins as a duplicate of a version of the file; from there it can undergo numerous changes without disturbing the main version. There is no limit to the number or nesting levels of branches. Branches can be merged back into the other branches and/or can continue to evolve independently. Users can tag any version with a label; this is useful for special access.

FIGURE 5. Typical Version Tree for a File In Figure 5, the tree has a trunk labeled /main that shows the evolution of the main file from version 0 through version 4. Version 1 of the main line file was branched off to become version 0 of a branch called /bugfix. The use of labels is illustrated by the tags RLS1, RLS1.5, and RLS2, which represent three versions used in releases. Version 1 from the /bugfix branch was merged back into the /main branch to produce /main version 4. Merging of branches can be manual or automatic. Notice that /bugfix version 1 was also used to produce /bugfix version 2, continuing the branch.

Users can also define attributes (or meta-data) associated with objects in the VOB. These attributes provide additional information such as an object's QA status. Users have ready access to any version in the VOB and can perform queries to obtain version- related and attribute information for any object.

ClearCase Views

The ClearCase views concept tackles one of software engineering's toughest problems: ensuring safe concurrent development. Concurrent development typically requires that certain files be shared among users while other files must be shielded for isolated development. The ClearCase product achieves safe concurrent development through the view concept. A view defines the particular versions in a VOB that appear in a user's environment. Users define the name of the view and create the view configuration specification (or config spec, for short) which defines the versions of objects to be used in the environment.

A view establishes a mapping between the versions of the files stored in the VOB and the versions that appear in the user's working environment. Let's continue the previous example and assume that there are two views looking at the file named application.c: the "devel" view used by a software developer and the "maint" view used by a maintenance programmer. Figure 6 shows two views looking at the application.c version tree.

FIGURE 6. Two Views of a File Named application.c The filtering of versions in a view is performed on an element-by-element basis in the view configuration specification. The config spec is an ASCII text file containing configuration rules. A configuration rule takes the following form:

objectclass pattern versionselector

where objectclass is a keyword indicating a kind of object (currently limited to the element type); pattern is a filename or pathname possibly with wildcards; and versionselector indicates the version, by version number, version label, attribute value, or a condition defined in the configuration specification language.

Figure 7 demonstrates the effects of looking at the same file through two different views: "devel" and "maint".

FIGURE 7. Two Typical Views The sequence starts from the directory /usr/people/demo/demodir/bounce. First the "devel" view is opened. This is done through the command cleartool setview followed by the name of the view. Then a standard UNIX ls command is performed, followed by cleartool ls, which is the equivalent within the ClearCase environment. In a ClearCase view, cleartool ls indicates the version's location in its version tree. This information appears at the end of the response to cleartool ls. Note the difference between the versions of the app.c file appearing in each view.

The command clearcase catcs (which is short for concatenate config spec) lists the view configuration specification. Observe the differences between the specifications for "devel" and "maint".

ClearCase View Configuration Specification

Let's take a closer look at the config spec for the "maint" view. The rules are: 

% cleartool setview maint
% cleartool catcs
element * CHECKEDOUT			// config rule #1
element * main/bugfix/LATEST	        // config rule #2
element opt.c RLS2			// config rule #3
element * RLS1 -mkbranch bugfix	        // config rule #4
element * /main/LATEST			// config rule #5
The view configuration specification determines which versions of elements in a VOB appear in a user's working environment.

Config rule #1 in the "maint" config spec causes the currently checked out version of an element to appear, if there is one. If there are no checked out elements, then one of the subsequent config rules apply.

Config rule #2 causes the most recent version of an element on the /bugfix branch to be used. The keyword LATEST indicates the newest version on the designated branch. This is a useful technique for maintenance programmers.

Config rule #3 ensures that only the version of the element opt.c with the label RLS2 will be used.

Config rule #4 selects the version labeled RLS1 (if there is one) for elements selected by earlier rules. This rule also has the effect (through mkbranch) that whenever a version selected by this rule is checked out, a /bugfix branch is created for that element. Combined with config rule#2, this rule preserves the integrity of the main (or prior) branch version, ensuring that only versions on the /bugfix branch are checked out.

Config rule #5 is only reached by elements with neither a bugfix branch nor a label RLS1. For these elements, the latest version on the main branch is used. This generally includes most directory elements, since development directories do not generally change from release to release and are usually not labeled.

A configuration specification can be changed by any text editor. The edcs command supplies the current config spec to $EDITOR.

ClearCase Element Description

Special ClearCase commands can show additional info as in Figure 8. An SQL like query mechanism supports extensive database queries on the VOB.

FIGURE 8. Typical Version Description

ClearCase Version History

ClearCase commands can show historical information. This is especially important when merging complicated changes.


% cleartool lshistory c_frontend/src/*.c
Jul 15	chu	opt.c	created version /bugfix/2
Jul 15	luke	parse.c	created version /main/11 (RLS2)
Jul 12	chu	opt.c	created version /main/4 (RLS2)
Jul 05	luke	opt.c	created version /bugfix/1 (RLS1.5)
Jun 30	solo	parse.c	created version /main/10
Jun 27	solo	opt.c	created version /main/3
May 11	chu	opt.c	created version /main/2

Checkin/Checkout

New versions are created by checking out a branch of an element, editing it, building with it, and then checking it in. Both "reserved" and "non-reserved" checkouts are supported. "Reserved" guarantees you can checkin later. "Unreserved" lets "first-back" win and others merge. The following shows how the ClearCase software retains checkout history.

% cleartool lscheckout -r c_frontend
Jul 17	solo	src/lex.c	checkedout	/main/4 (reserved)
Jul 16	luke	doc/rlsnotes	checkedout	/main/27 (unreserved)
Jul 15	chu	src/parse.c	checkedout	/bugfix/11 (reserved)

Checkout causes a writable copy to appear in-place. No parallel trees or search paths are needed. In most source management systems, elaborate scripts must be built and maintained to support the copying and linking of cleartext copies for the engineering workareas. This method of in-place work allows build scripts to access the correct file versions transparently, eliminating the need for the maintenance of the scripts. 

% cleartool checkout c_frontend/src/lex.c
% ls -l c_frontend/src/lex.c
-rw-rw----	solo	r_d	12421	Jun 30		c_frontend/src/lex.c

Build Management with clearmake

The clearmake tool is an upward-compatible version of make. It reads standard makefiles (target description files) and supports most of the makefile features on the SGI platform. Thus, users do not have to change their makefiles in order to migrate into the ClearCase environment. The clearmake tool audits the build process, recording all the files that go into the build, the versions of the transforming tools (that is, compilers, linkers, etc.), the build rules, and the environment variables set at the time of the build. This audit is called a build configuration record (or configuration record). For each target file it builds, clearmake creates a configuration record and stores it in a VOB. When asked to rebuild the same target, clearmake examines the previous configuration records and will reuse an existing instance of the target (built by the same user or by any other ClearCase user) to avoid unnecessary build time.

The clearmake tool also has a parallel build feature that allows large, hierarchical builds to be spread across a local area network, producing a significant savings in build time.

Building with ClearCase / Configuration Records

Clearmake uses Makefiles, but no header file dependencies are needed. This is because the ClearCase tool audits all files that are read and/or written to during a build. Thus, all source versions that are accessed (Actor.c) (see Figure 9), tools that are used (/bin/cc), objects produced (Actor.o), and sub-objects accessed, if any (utils.a) are all captured by the audit mechanism. The audit plus translate rule and options are all recorded in a Configuration Record and attached to the new derived objects.

FIGURE 9. Typical Build Configuration Record

Comparing Configuration Records

Higher level objects record sub-objects used, and build configuration records can be compared. This solves the "who did it" problem, so that engineers don't have to waste time figuring out who changed a header file. Figure 10 shows a typical comparison:

FIGURE 10. Comparison of Build Configuration Records

Sharing Binaries

In many development groups, libraries are built and rebuilt, causing numerous unnecessary builds.

In the ClearCase environment, when the user asks to make a target, clearmake attempts to avoid building by finding an existing derived object that can be reused. Derived objects are initially written to view-private storage, but their configuration record is registered globally with a VOB. The user's views as well as global derived objects are checked. When shared by several views, derived objects are promoted to VOB storage.

In order to be reused, the existing derived object's config record must satisfy the user's current config spec's requests. 

% clearmake myproj
building target opt.o ...
found parse.o built 18-Jul by luke
building myprog

Binaries (particularly archives) can be shared by checking them in. This freezes the object and is a prelude to releasing it. 

% cleartool checkin backend.a
% cleartool mklabel RL2 backend.a