This chapter is a guide to using and modifying Coda source code at CMU-SCS. It is mainly intended to help new members of the project come up to speed. But it may also help sites outside CMU that receive a Coda distribution to devise procedures appropriate to their environments.
All files relating to Coda, with the sole exception of RCS files (see Section
XXX), are stored in /coda/project/coda.
Let's call this $CODA.
The directories under $CODA correspond to different releases of Coda, such as
alpha, beta, and so on. A release is a complete set of mutually consistent
sources, libraries and binaries. "Mutually consistent" means, for example, that the
Venus and server from a release will work with each other. It also means that you can
freely use the libraries and include files to compile new binaries in that
release. @ux[There are no guarantees of any kind across releases.]
Before a new instance of a release is made, careful thought must be given as to how current users of that release will be affected. Will all such users have to be upgraded in one fell swoop? Or can they switch over at their own leisure? As the deployed Coda system grows, upgrading everyone in one fell swoop gets harder. After all, some of the users may be in hiding with their disconnected laptops!
By convention, the omega release corresponds to the version of Coda in production
use. Most users Venii, as well as the servers on production machines,
are from this release. Hence the value of /usr/coda/SYMLINK on those users workstations
is $CODA/omega.
The alpha release of software corresponds to a lightly tested version of the system.
Individual developers have done a significant amount of testing of alpha software, but
it hasnt yet been stressed heavily, nor has it been tested in real use. Hence alpha
server code should not be run on servers that hold real user data. They should only be
run on the alpha servers (also known as "test" servers). Similarly for other alpha
software. When you are doing code development, you
usually pick up include files and libraries from the alpha release. Thus, the
private versions of software you build is, at best, of alpha quality.
The beta release corresponds to a version of the system that is expected to be released
soon for production use.
A small number of users, typically members of the Coda project, depend upon
this release. Actual use by such users is part of the testing of the beta
release. Eventually, when we have enough confidence in the stability and correctness of
the beta release, we promote it omega.
beta is really a symlink to a volume mount point beta-<unique>,
where <unique> is an identifier made up by the makebeta script described below.
Upgrading the beta release consists of creating a new volume, mounting it
at beta-<unique>, populating it using the steps described in Section
XXX,
and finally changing the beta symlink to point to beta-<unique>.
The old beta volume is preserved for a while, until is clear
that the new release hasnt triggered any serious problems. It can then be purged
to reclaim space, or recycled for a future beta release.
omega is also a symlink to a volume mount point beta-<unique>.
After a beta release has been running robustly for a while, it is upgraded
to omega by merely changing the value of the omega symlink. No recompilation
is involved.
When a beta-<unique> release is created, the makebeta script creates an RCS branch
named beta-<unique> for every file in the release.
Normally no development or changes are done along this branch.
However, it provides a way to introduce an emergency fix
in a beta or omega release if the need ever arises.
Lets illustrate this with a specific example. Suppose I want to upgrade the module
rpc2 and build a new Venus to use the upgrade. I begin by creating two
private directories, say /coda/usr/satya/src/rpc2 and /coda/usr/satya/src/venus,
and populating them with source files from the alpha release of these two modules.
Section
XXX tells you how this step actually gets done. Now I proceed to modify files in
rpc2 and to compile and test the module using standalone test programs.
Then I modify files in venus to use the upgrade and then build a Venus.
When building Venus, I must make sure that the version of rpc2 used is the one I just built.
Section
XXX tells you how to do this. Now I have a Venus that I can test.
As bugs are found in the new Venus, I iterate the above procedure.
When I am confident that my changes to rpc2 and venus are right, I update the alpha
release of these two modules.
The process of updating the alpha release is known as installation.
Installation is the point at which work done by a Coda project member
becomes visible to others in the project. Prior to this point, all work is done in
that users own private directory.
Installation always occurs at the granularity of entire modules. In other words, one never
installs an individual library, include file, etc. Rather the entire source code for a module and
all relevant files compiled from it are installed together. Modified files are automatically
checked into RCS as part of the installation procedure.
At some future time, these changes along with many others that were installed
will make it into a beta release. Section
XXX shows you how
this is done. After some use as beta, it will be promoted to omega.
Each release has the structure shown in Figure XXX. Note that this layout is identical for all releases.
Underlined names are symlinks to machine-specific directories. On a 386
machine for example, bin is a symlink to i386_mach/bin.
Although only two machine-specific directories (pmax_mach and
i386_mach) are shown, there can be many more.
The source tree for the release is in src. A copy of the header
files from src
is in include. Both src and include are machine-independent. For each
supported machine type, there is a directory containing binaries (bin) and libraries (lib).
The source tree in a release is fully self-contained. In other words, if you started out with empty
include, lib, and bin directories, you could completely populate them by compiling
the source code in src. The only exception to this are the files in include-special,
lib-special, src-special, and bin-special. These directories contain a very small number of files
that have to be copied in by hand. The sources for these files are not in src.
The src directory in each release is organized as in Figure
XXX. The MAKECODA
script simplifies the compilation of the entire release. The Makeconf file defines in
a single spot many key variables and paths used by the makefiles in individual modules. Those
makefiles inherit these definitions automatically, when the CMU-SCS make is used.
The SOURCEME file contains a minimal set of environment definitions. By sourcing this
file before you compile anything, you can be sure that you arent obtaining binaries, libraries
etc. from non-standard places. This is especially important if your .login or .cshrc
files define elaborate PATH, CPATH, LPATH variables.
Underlined names are symlinks to RCS directories. For example,
RCSLINK is a symlink to /afs/cs/project/coda/rcs;
auth2/RCS is a symlink to ../RCSLINK/auth2; and so on.
Indirecting via RCSLINK makes it simple to relocate the RCS
directories without changing lots of individual symlinks. Such relocation
might happen, for example, when Coda sources are used outside CMU.
The structure of a typical module is shown in Figure XXX.
In this particular module, rpc2, the underlined name RCS indicates a
symlink to ../RCSLINK/rpc2. Indirection via RCSLINK makes
relocating the RCS directories a simple matter.
Notice that only the source files are located in this directory; there are no object
files. The CMU-SCS make facility puts all compilation targets elsewhere, thus
allowing the source directory to be readonly. This has two advantages. First, the
source directories are uncluttered. Second, it simplifies building Coda for multiple
machine types, since the target directories can be different for each machine type.
Also notice the presence of multiple makefiles in this module. Although this is not
characteristic of all Coda modules, it is typical of some. The true dependencies are
capture in Makefile.real, and the others, like Makefile.coda and Makefile.misc
invoke Makefile.real after defining environment variables appropriately. This simplifies
the use of these modules outside Coda. If you are compiling by hand, you have to say
"make -f Makefile.coda <target>", rather than just "make <target>".
Most Coda modules rely on files installed by other Coda modules. It is therefore important to install modules in the correct order. Otherwise you could get yourself into real trouble. If you are compiling from scratch, the out-of-order installations will simply fail. But if you are modifying an existing release, you could end up with mysterious bugs because obsolete versions of header files and libraries may be used.
You are @ux[strongly] urged to use the script MAKECODA, described in Section
XXX.
The correct precedences of modules are wired into the script, so you dont have to
deal with them.
For the curious, the correct order of compilation of modules is given below. In principle, modules of the same precedence (i.e. in the same set) can be compiled in parallel. But I havent actually tried that yet.
scripts Miscellaneous scripts, including alphaci on which intalls of all other modules rely mlwp Lightweight process package dir Directory package used by Venus and server sys Miscellaneous routines sunrpc Sun Microsystems public domain XDR code and interface for device driver/venus interaction igmp Internet multicast support for old RFC
(dummied out currently; someone should fix these to use the new RFC) util Utility routines rpc2 RPC package camstuff Header files that allow runtime choice of RVM or VM for persistence on servers blurb Program to adjust copyrights pdbstuff Protection database management rp2gen Stub generator for RPC2 comm Communication layer above RPC2 for connection management (not yet in use) libal Access list package. vicedep Header files and RPC2 interface definition files put here to break circular dependencies fail Network failure and variable speed emulator auth2 Authentication server login Implements clog, cunlog, ctokens, etc. cfs The VFS driver; most of this code is linked into the Mach kernel vv Version vector routines mond Coda usage data collector resolve Library used by repair (should get integrated into repair) vol Volume package used by server res VM-based directory resolution algorithms repair Repair tool venus Cache manager volutil Volume utilities vtools Miscellaneous Venus tools rvmres RVM-based directory resolution algorithms vice Server code update The daemon which updates server databases norton A Coda server, RVM debugger asr Application Specific Resolver package egasr ASR examples@end)annote
To compile Coda "out of the box" you need the following compilation tools:
make The makefiles in the Coda sources exploit many of the enhancements
made by CMU to Unix make. Conditional macro expansion, the ability to place targets
in a different place from the sources, and the inheritance of variable definitions from
Makeconf are the three major features we rely on.long to represent enum.
The RT C compilers default behavior of using char to represent small-valued enums will
cause insidious problems in RPC2 communication. On CMU-SCS machines this is done by defining the
environment variable PLAINC to be /usr/misc/.hc/bin/hc2. The MAKECODA script does this
automatically for you, but you must remember to do this manually if you arent using it. (Some .logins have the variable ccC defined to hc; you must delete this definition
also).It should be possible to modify the code to use other versions of C++
(such as g++), or to use standard Unix make. But we havent
tried this, and dont plan to.
Before you can compile Coda, you need to populate the {include,lib,bin,src}-special directories.
These contain files that are (a) needed but arent in the Coda sources or (b) standard Mach header files, with slight modifications. The MAKECODA
script contains an up-to-date list of what these files should be. Here is
a list that was current at the time of writing this document:
RVM is a lightweight transactional package that is used on servers and clients. It is a package that is independent of Coda. The current set of files from this package are:
include-special/{rvm.h,rvm_lwp.h,rvm_statistics.h,
rvm_segment.h,rds.h}
lib-special/{librvm.a,librvmlwp.a,libseg.a,librds.a}
bin-special/{rdsinit,rvmutl}
src-special/{Makeconf,Makefile,READ_ME,plumber,rds,
rvm,seg}
These files pertain to a file-tracing
facility, dfstrace, used at CMU. To function as a
trace-driven simulator, Venus requires the following files:
include-special/tracelib.h
lib-special/libtrace.a
If you don't intend to use Venus as a simulator, you could construct a dummy libtrace.a with
empty routines to avoid unresolved references. You do need tracelib.h though.
malloc The Coda makefiles allow you to build versions of venus and codasrv
that use a special malloc to help detect memory leaks. The following files are needed
for this:
include-special/newplumb.h
lib-special/{libplumber.a,libnewplumb.a}
If you don't plan to build the plumbing facilities, you can just create zero-length files with
these names to keep the MAKECODA script happy.
The changes in these files had to be made
because of compilation errors
from C++, or (as in the case of assert.h) to define different behavior for standard macros:
include-special/{assert.h,cthreads.h,setjmp.h,sys/inode.h,
i386/fpreg.h,i386_mach/endian.h}
Once you have taken care of the prerequisites and special files, you can compile a release.
The simplest way to do this is to run the MAKECODA script in src.
This script takes one required and two optional arguments.
The required argument is OBJECTDIR, which is the pathname of
the directory where the object files should be placed. It is sensible to specify a different
directory for each machine type, and the @@sys facility of AFS and Coda lets you do this.
So, for example, to compile the beta release, I would do the following:
cd /coda/project/coda/beta/src
./MAKECODA OBJECTDIR=/coda/usr/satya/OBJS/@@sys
The MAKECODA script will first check to make sure that all necessary
special files are present. If any are missing it will prompt you. It then
goes through the Coda modules in the correct order and does a make install on each. The usual checkin to RCS that is done by
alphaci as part of make install (see Section
XXX) is supressed.
If all goes well, everything in Coda will be compiled, and the bin, lib,
and include directores will be populated. You will have to repeat this once
for each machine type.
Sometimes, you will run into a problem part-way through MAKECODA. After you
have fixed the problem, youd probably like to continue where you left off rather
than redoing everything from the beginning. Hence MAKECODA lets you specify
the name of the module to start from. Here is an example:
./MAKECODA OBJECTDIR=/tmp/@@sys FIRSTMODULE=vol
Finally, the pathname of the root of the release you are compiling is specified by
the variable ROOT in MAKECODA. You can change this by editing MAKECODA,
and this is in fact what the makebeta script does for you when you create a new release.
But you can also override it on the command line thus:
cd /tmp
MAKECODA OBJECTDIR=/tmp/@@sys ROOT=/coda/project/coda/alpha
Once the alpha release has been built, you can start code
development. We have already seen the general procedure in Section
XXX. Lets look at an example in more detail.
Suppose I am working on the module vtools and need to change
the files cmon.c and codacon.c. Heres what I would do:
# Create a scratch directory for my work
mkdir /coda/usr/satya/src
cd /coda/usr/satya/src
# Set up links to RCS directory, and root of release
ln -s /afs/cs/project/coda/rcs/coda-1.0 RCSLINK
ln -s /coda/project/coda/alpha/src SRCROOT
ln -s SRCROOT/Makeconf
# Create directory for this module
mkdir vtools
cd vtools
ln -s ../RCSLINK/vtools RCS
# Lock and checkout the file (s) to be modified
rcsco -l cmon.c codacon.c
# Now edit cmon.c and codacon.c
# Then compile this module
# Source the standard environment file to make sure your
# compilation environment (PATH, LPATH, CPATH, etc.) is set up right
source ../SRCROOT/SOURCEME
make OBJECTDIR=/coda/usr/satya/OBJS/@@sys cmon codacon
# Test cmon & codacon, then iterate on edit/debug cycle above
# Now you are ready to install your changes.
# You must first create a file called RCSMSG, and enter text
# in it that will become the RCS log message for the checkin.
# It will also be posted to the changelog bboard, so that
# others in the group will know of your installation
echo "Fixes to annoying bugs ... blah blah blah ..." > RCSMSG
make OBJECTDIR=/coda/usr/satya/OBJS/@@sys install
# You are now done!
# The install step automatically released the write locks on cmon.c
# and codacon.c
# Repeat the install step for each of the other supported platforms.
Notice that many other files may be need for compilation, but you dont have to check them out. This is because the CMU-SCS make knows to check out any needed files automatically into the compilation target area.
The automatic checkin is done by a script called alphaci that is invoked
as the last step of make install. alphaci will give you an error if
the RCSMSG file is missing; it moves it to RCSMSG.old once it has
checked in files. alphaci is smart enough to discover new files that
arent mentioned in RCS, and prompts you to ask if you want them checked in
too. Often you may not want this, because the files in question were just
test files created for debugging. alphaci assumes that only writable
files should be checked in; it prompts you about what to do with files that
are not writable.
The RCSMSG file is also used by alphaci for posting on the changelog bboard (cmu.cs.proj.coda.changelog at CMU). You should follow the posts on this
bboard closely, so that you are aware of changes to Coda modules by other
project members. Here are some typical posts from the bboard:
03-Feb-93 18:56 M Satya Installed scripts for IBMRT
Posted by alphaci from STRAUSS.CODA.CS.CMU.EDU
No files checked into RCS
03-Feb-93 18:50 M Satya Installed scripts for I386
Posted by alphaci from WEBER.CODA.CS.CMU.EDU
No files checked into RCS
03-Feb-93 18:22 M Satya Installed scripts for PMAX
Posted by alphaci from MOZART.CODA.CS.CMU.EDU
Files checked into RCS: Makefile makebeta restartserver restore.sh
RCS message follows:
Created new script, makebeta, to make a clone of the entire alpha
source tree, and to create branches in RCS for all files.
Notice how the installations for machine types I386 and IBMRT caused no RCS
files to be checked in. This is because the installation for the first machine
type, PMAX, did the checkin.
As we have seen earlier, there are 3 important releases of software:
alpha, beta, and omega. Servers are also classified
by this scheme. At the time of writing this document, there were
3 omega servers (rossini, puccini, and scarlatti),
3 beta servers (grieg, haydn, and wagner), and
4 alpha servers (schumann, gershwin, mahler, and vivaldi).
The omega servers hold real data, so software let loose on them should
have been tested very well. Having to reinitialize and to restore many gigabytes on
the omega servers because of storage corruption is not an experience
you are likely to forget! The beta servers hold some real data, but
the number of users depending on that data is restricted to a few Coda project
members. Further, that data is of a kind that can be easily reconstructed.
The alpha servers are used by project members for testing.
Obviously, only one person can test their software on a server at a time, so you
should coordinate use of the servers with the other members.
When the alpha release has diverged substantially from beta,
and is relatively stable, we will decide to make a new beta release.
One member of the Coda project will serve as release coordinator for the promotion.
The first step in this process is for the release coordinator to post on
cmu.cs.proj.coda.general, asking project members to checkin changes and to drop
all RCS locks on the main line of development by a certain deadline.
As soon as possible you should release all mainline locks. It is ok to
hold on to locks on private branches. Once the deadline expires, the release
coordinator will feel free to break mainline locks.
In the second step, the release coordinator runs the makebeta script to
create a clone of the current state of alpha. makebeta first
synthesizes a unique identifier of the form date_xxxxx where xxxxx is
the number of seconds since midnight. The name of the new release is then
beta-<unique identifier>; for example, beta-3Feb1993_43696.
This name is used to tag a new RCS branch for every file in the release,
so that emergency fixes are possible long after the alpha release has
diverged from this beta release.
The makebeta script works as follows:
makebeta checks to make
sure the directory is empty before proceeding, to avoid accidental clobbers.
For brevity, we refer to this directory as $BETA.makebeta looks at /coda/project/coda/alpha/src ($ALPHA/src, for short)
and determines the names of the modules in it.$ALPHA modules in $BETA. In making this copy,
it actually checks out the files (unlocked) from RCS, and compares them with $ALPHA.
This is a sanity check, to ensure that RCS and $ALPHA are in sync. One side effect
of this comparison is to detect RCS locks.$ALPHA, makebeta pauses and asks the user to go take a look at whats wrong.
It then asks the user whether to proceed.makebeta creates a new RCS branch, named after this release, and re-checks
out files from the branch so that $Header$ in those files is correctly set.MAKECODA and Makeconf at the top level of $BETA
are edited by makebeta so that all references to $ALPHA are changed to $BETA.makebeta copies over all the special directories ({lib,bin,src,include}-special).The above procedure is quite slow, since many RCS interactions are involved. It usually takes
a few hours. If all goes well, makebeta requires little babysitting.
Note that the script does no locking. In other words, the $ALPHA tree better remain
frozen for the entire duration of the promotion.
In the third step, the release coordinator cds into $BETA/src, and runs MAKECODA
once for each machine type. Note that he doesnt have to specify anything other than OBJECTDIR,
since makebeta has already set ROOT correctly in the MAKECODA script. This step
guarantees that the binaries in $BETA were indeed compiled from the sources in $BETA.
In the final step, the release coordinator makes the symlink beta point to $BETA.
This is the "commit" step that blesses the newly-cloned release as the beta release.
After the beta release has been stressed for some time,
it will be promoted to omega. This is a much simpler procedure than the
alpha to beta promotion. The release coordinator merely has to make the
symlink omega point to the beta-xxxx where xxxx is the unique
identity of the release being promoted. This "commit" step blesses what was
hitherto beta as omega
Since Coda is distributed outside CMU, it is important that
every source file contain a copyright notice. The blurb program
simplifies adding and changing copyright notices. blurb expects
to find the copyright notice is at the very beginning of a file, so make
sure you dont move it when modifying a file.
The only modules without Coda copyright notices are sunrpc,
which is public-domain code from Sun, and cfs, which is mostly
Mach kernel code.
Some files in Coda are derived from the 1986 version of AFS-2.0. Since AFS-2.0 is owned by IBM, these files have an IBM copyright notice in addition to the Coda copyright notice.
When you modify a file or add a new file, you should pay attention to the copyright notices. Here are some simple rules to follow:
A simple way to think of this is that some files in Coda are "tainted" (i.e., they are derived from AFS-2.0). Tainted files can infect other files if enough of their innards are copied. Existing untainted files, and new files, stay untainted forever. In general, do your best to keep the number of tainted files to a minimum. Note that this discipline regarding copyrights is not intended to be onerous or constraining --- just common sense and a little self-discipline.
1. Branch overview
One common problem in managing large software projects comes
to play when there are several people working on the same sets
of files. RCS helps by providing per-file locking to guarantee
that no two users modify the same file at the same time. If
several users are modifying the same line of development,
however, this locking does nothing to guarantee that one users
changes in one file wont interfere with another users changes
in another file.
To combat this problem, RCS provides the notion of a "branch".
A branch is a separate line of development carried out in parallel
to the main line of development. Version found on branches have
more than two digits in their version numbers. All digits make up
the version number of an element on a branch. For example, version
1.2.1.1 is an element in the first branch off of version 1.2.
As branch numbers are hard to deal with, you are recommended to
assign symbolic names to branches.
2. How to use branch
For example, a typical command to create and name a branch would be
rcsci -b -nKUDO_STATISTICS foo.c
where KUDO_STATISTICS is a symbolic name of branch
* This command will create a branch named KUDO_STATISTICS
off the last version on the mainline of foo.c
If you would like to specify the version number of which the first
branch element is off, a command would be
rcsci -b -r1.2 -nKUDO_STATISTICS foo.c
where 1.2 is the version number of which the first
branch element is off
The name is assigned to the branch (not the branched element), so
the subsequent commands such as
rcsco -l -rKUDO_STATISTICS foo.c
will refer to the latest element in that line of development.
3. What I checked in using branch
I checked in the following files using branch with the symbolic name
"KUDO_STATISTICS".
vicedep/mond.rpc2
venus/ fso.h
fso0.c
fso1.c
hdb.c
sighand.c
venus.c
venus.private.h
venusresov.h
venusutil.c
venusvol.c
venusvol.h
venusvm.c
venusvm.h
vol_vsr.c
vproc.c
The purpose of the modifications is to collect session statistics
and send them to the mond data collector. Currently, venii built
with these files are running on Brahms and faust, safely (I think so).
<TO BE COMPLETED>