+
Written by Scott,
+
+sharrison@users.sourceforge.net,
+January 17, 2001
+
Last updated, August 14, 2002
+
+- -+ +
- +
Using CVS
+These instructions assume that you are using a Linux or UNIX based +terminal. ++
- +
+Using CVS: Logging in and out (cvs login; cvs logout)
++In order to log into CVS, CVS needs to be part of your system environment. +You can do this by: +
++ +export CVSROOT=:pserver:USERNAME@install.lon-capa.org:/home/cvs + +
++To actually login, you will need to execute the following command: +
++ +cvs login + +
++You are then prompted for a password. +If you do not have a password, or the password is not working, you +should contact helen@lon-capa.org. +
++The first time you use CVS, you need to CHECKOUT the repository. +Generally speaking, you need to checkout loncapa only once +per machine. +To check-out the repository, use the checkout command. +(Otherwise, just enter your CVS directory, cd loncapa.) +
++ +cvs checkout loncapa
+
+cd loncapa + +After completing work with the CVS repository, +you can log out: +
++ +cvs logout + +
+- +
+Using CVS: Updating files (cvs update -d)
++After entering your CVS source tree (cd loncapa), +you should frequently update the software changes that +other people have made. This is done with the update command. +
++ + +cvs update -d + + +
++The cvs update command creates output +as it updates your CVS source tree. Common flags are +'U' and 'P'; they indicate that a file in your +loncapa directory is now updated with +changes made by another programmer. +
++ +`U FILE'
++The file was brought up to date in your loncapa. ++
'U' is done for: +
* any file that exists in the repository but not in your source, and +
* files that you have not changed but are not the most recent versions +available in the repository. +
The network behavior of 'U' is that the entire new file is uploaded +from the CVS server. ++`P FILE' +
++Like `U', but the CVS server sends a patch instead of an entire file. +++'U' and 'P' essentially accomplish the same thing, just in +different ways. +
++Usually, when you do not cvs commit your code changes, +the update command will tell you that you have modified +your file with the 'M' flag. +
++`M FILE' +
++ The file is modified in your working loncapa directory. + This is probably based on changes you made and have not yet + "cvs commit"-ed. +++Sometimes, it will occur that: +
++
+- you have modified a file and not yet committed it
+- someone else *has* modified a file and *has* committed it
++Generally speaking, this is your fault. It is your +responsibility to resolve conflicts. cvs update informs +you of a conflict with the 'C' flag. +
++`C FILE' +
++ A conflict was detected while trying to merge your changes to FILE + with changes from the source repository. +++You will need to open the file and examine it; CVS will have added in +markup tags like "<<<<<<" to tell you about the merging +conflicts. (Sometimes, CVS will intelligently merge in other changes and +give you the 'M' flag, but many times you will have to manually edit +the file as just described.) +
+- +
+Using CVS: Saving files (cvs commit)
++cvs commit works to submit changes to an existing +file within the repository. If a file does not currently exist, then you +will first need to cvs add it as described in the following +section. +
+Running the cvs commit command without additional arguments will +commit all of your changes within the current directory and subdirectories. ++cvs commit +
++A more precise approach to using cvs commit is to pass it specific +file names. (Usually you should do this.) +
++cvs commit FILENAME +
++Note that CVS typically invokes the +vi editor and solicits +comments about your latest changes to the software. Your comments should be +both short yet uniquely descriptive. For example: +
++
+- BAD - "made some changes and am drinking soda"
+- GOOD - "implemented syntax checking of perl scripts +with -c flag"
+- +
+Using CVS: Adding files (cvs add)
++cvs add FILENAME +
++Then you can run cvs commit FILENAME and this file will +become an "official" part of LON-CAPA. +
+- +
+Using CVS: Adding directories (cvs add/import)
++cvs add DIRECTORYNAME +
++There is no need to run cvs commit. Directories immediately +become part of the LON-CAPA CVS source tree by only using the cvs add +command. +
++You should not ordinarily need to use the cvs import command. +If misused, cvs import can lead to the loss of code within +the repository. +
+- +
+Using CVS: What to do when you're not sure about your files + (cvs update -d)
++Once in a while, multiple programmers may be working on the +same file. Most conflicts are avoidable if everybody regularly +commits their changes AND if everybody +regularly updates the CVS source tree they are working on. +
++If you are absent from programming for a few days, and +fail to run cvs update -d on your CVS source +repository, you have only yourself to blame if you find yourself writing +code in a file that is not up-to-date. +
+- +
+Viewing the software (make HTML)
++Commands +
++cd loncom/build
+
+rm -Rf HTML (or alternatively, "make clean")
+make HTML
+cd HTML
+(look at the index.html file with a web browser such as Netscape) ++General description of what happens +
++This is the actual make target code. +
++ +HTML: + install -d HTML + cp $(SOURCE)/doc/loncapafiles/*.gif HTML + cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \ + perl lpml_parse.pl html development default "$(SOURCE)" '$(TARGET)' \ + > HTML/index.html + +++What basically happens is that specially marked-up data in the LON-CAPA +cvs repository file doc/loncapafiles/loncapafiles.lpml is parsed +into a more viewable format by loncom/build/lpml_parse.pl. The +resulting file gives a very well organized view of all the files, directories, +links, ownerships, permissions, and brief documentation of what each +file does. +
+- +
Compiling the software (make build)
+Commands ++cd loncom/build +
+
make build ++General description of what happens +
++This is the actual make target code. +
++ +build: Makefile.build pod2html.sh pod2man.sh + echo -n "" > WARNINGS + make -f Makefile.build all + make warningnote + +Makefile.build: $(SOURCE)/doc/loncapafiles/loncapafiles.lpml lpml_parse.pl + cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \ + perl lpml_parse.pl build $(CATEGORY) $(DIST) "$(SOURCE)" "$(TARGET)" \ + > Makefile.build + +++loncom/build/lpml_parse.pl reads in all the build information out +of doc/loncapafiles/loncapafiles.lpml. A new Makefile named +loncom/build/Makefile.build is dynamically constructed. +This dynamically generated Makefile is then used to build and compile +all the software targets from source. This can take several minutes +(it depends on the speed of the machine you compile with). +
++Example +
++Here is information for one file tth.so provided in +doc/loncapafiles/loncapafiles.lpml. +
++<file> +<source>loncom/homework/caparesponse/capa.so</source> +<target dist='default'>usr/lib/perl5/site_perl/5.005/capa.so</target> +<target dist='redhat7 redhat7.1'>usr/lib/perl5/site_perl/5.6.0/capa.so</target> +<categoryname>system file</categoryname> +<description> +shared library file for dynamic loading and unloading +</description> +<build trigger='always run'> +loncom/homework/caparesponse/commands +</build> +<dependencies> +caparesponse.c; +caparesponse.pm; +README; +Makefile.PL; +capa.i; +commands +</dependencies> +</file> +++loncom/build/lpml_parse.pl sees the build tags and sets up +a dynamic file Makefile.build to run the command inside the +build tags. The files listed inside the dependencies tags +are included in the Makefile.build so as to determine whether +or not there is a need to compile. +
++Here is an example of a dynamically generated Makefile.build +that builds two LON-CAPA files (one of which is tth.so). +
++all: ../homework/caparesponse/capa.so ../modules/TexConvert/tthperl/tth.so + +../homework/caparesponse/capa.so: ../homework/caparesponse/caparesponse.c ../homework/caparesponse/caparesponse.pm alwaysrun + cd ../homework/caparesponse/; sh ./commands + +../modules/TexConvert/tthperl/tth.so: ../modules/TexConvert/tthperl/../tthdynamic/tthfunc.c ../modules/TexConvert/tthperl/../ttmdynamic/ttmfunc.c + cd ../modules/TexConvert/tthperl/; sh ./commands + +alwaysrun: ++- +
Adding/removing files from the LON-CAPA installation + (doc/loncapafiles/loncapafiles.html)
++To add and remove (and alter) +
++All that you have to do to alter the behavior of the installation is +edit a single file (doc/loncapafiles/loncapafiles.lpml). +Adding, removing, and altering files requires proper attention +to the syntax of file format of course. +
++File Format +
++The preceding "make build" documentation +gives an example of a file entry describing one particular file. +All data within loncapafiles.lpml is specified according +to markup tags. The format and syntax of loncapafiles.lpml +is currently best described by the HTML documentation code at the beginning of +loncapafiles.html (as well as, by example, seeing how various +information is coded). All in all, the syntax is quite simple. +
++Philosophy and notes (the thing nobody reads) +
++Packaging the software from CVS onto a machine file system requires many +things: +
++
+- documenting every component of the software,
+- handling CVS source to file system target information,
+- handling (according to a hierarchical scheme of grouping) file +ownership and permissions,
+- handling (according to a hierarchical scheme of grouping) directory +ownership and permissions,
+- handling symbolic links,
+- providing for multiple options of installation targets (e.g. RedHat versus +Debian),
+- providing for different file ownerships and permissions to apply +to the same file,
+- allowing system software documentation to be automatically generated +(see information on "make html"),
+- providing information in an easily adjustable form as new demands +are made on the software packaging system,
+- providing software package information (for RPM),
+- having information in a format that allows for troubleshooting +the current status of the machine file system,
+- allow for changes to the structure of the CVS repository,
+- and something that is simple enough for any one to immediately work with, +without having to learn any specifics (or hidden traps) of complicated +Makefile's or a new macro language (m4?).
++I looked into, and tried, different ways of accomplishing the above +including automake and recursive make. The automake system seemed quite +complicated (and needlessly so in terms of this project since, by and large, +it works to coordinate many different types of build/compilation parameters +whereas we are more concerned with installation parameters). The other +alternative, recursive make, +has significant deficiencies since not all the information +is kept in one place, and there are significant levels of dependency +between all the things that must be done to keep software packaging +up to date. A particularly convincing article I found when looking into +much of this was + +"Recursive Make Considered Harmful" by Peter Miller. Other complications +were that, at the time, it was unclear as to what categories +of software files we had, and whether or not the directory structure +of CVS would remain constant. With an ever-developing directory structure +to CVS, I preferred to organize the information on a per-file basis +as opposed to a per-directory basis. +Additionally, a standard big Makefile assumes certain "normalcy" to +the directory structure of different potential operating system directories +(RedHat vs. Debian). +
++If you take time to look at loncapafiles.lpml +(and perhaps run the make HTML command) +you will find that the organizing information according to the markup +syntax in loncapafiles.lpml is simple. Simple is good. +
++loncom/build/lpml_parse.pl is the script (invoked automatically +by the various targets in loncom/build/Makefile) that reads +doc/loncapafiles/loncapafiles.lpml. lpml_parse.pl +is capable of reading and returning different types of information +from loncapafiles.lpml depending on how lpml_parse.pl +is invoked. lpml_parse.pl has yet to have introduced new sources +of error, and has been tested in quite a number of ways. As with +any parser however, I remain paranoid. +
++Finally, some notes on the development. +lpml_parse.pl is very fast and styled after a state-based SAX-like +approach. I do eventually want to use a real XML/XSLT approach, however +I hesitate to make everyone everywhere install something like +XML::Xalan. +Also note that loncapafiles.lpml has a +DTD (loncom/build/lpml.dtd) against which it is valid. +I would also like to use more ENTITY's inside lpml.dtd but currently +the perl XML modules available at CPAN do not digest complex ENTITY's that +well. +
++The lpml_parse.pl-loncapafiles.lpml +combination has been highly efficient and error-free. +
+- +
+Configurable files versus non-configurable files
++Machine-specific information is the difference +
++The current list of configurable files for the LON-CAPA system is +/etc/httpd/conf/loncapa.conf, +/etc/ntp.conf, +/etc/krb.conf, +/etc/ntp/step-tickers, +/home/httpd/html/res/adm/includes/copyright.tab, +/home/httpd/html/res/adm/includes/un_keyword.tab, +/home/httpd/hosts.tab, and +/home/httpd/spare.tab. +
++All of these configurable files contain machine-specific information. +For instance, the overall LON-CAPA system relies on unique host IDs such +as msua3, s1, s2, msul1, and 103a1 (specified as a "PerlSetVar lonHostID" +field within /etc/httpd/conf/loncapa.conf). +Non-configurable files simply do NOT have machine-specific information. +
++The impact on updating software +
++What this means in terms of software updating is that: +
++
+- non-configurable files can be simply overwritten with newer versions +(without "anything" else to worry about),
+- and configurable files must follow these steps to be safely +overwritten: +
++
+- have their machine-specific information saved,
+- be overwritten, and then
+- have their machine-specific information restored.
+- +
+Updating the non-configurable files on your machine (make install)
+Commands ++cd loncom/build +
+
make install ++General description of what happens +
++This is the actual make target code. +
++ +install: TEST_hosts_tab Makefile.install Makefile + echo -n "" > WARNINGS + make -f Makefile.install SOURCE="$(SOURCE)" TARGET="$(TARGET)" \ + directories + make -f Makefile.install SOURCE="$(SOURCE)" TARGET="$(TARGET)" files + make -f Makefile.install SOURCE="$(SOURCE)" TARGET="$(TARGET)" links + make SOURCE="$(SOURCE)" TARGET="$(TARGET)" \ + NORESTORECONF="$(NORESTORECONF)" configinstall + make postinstall + make warningnote + echo "You can run 'make test' to see if your system is ready to go!" + +Makefile.install: $(SOURCE)/doc/loncapafiles/loncapafiles.lpml lpml_parse.pl + cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \ + perl lpml_parse.pl install $(CATEGORY) $(DIST) "$(SOURCE)" \ + "$(TARGET)" > Makefile.install + +++For safety reasons (so as to preserve a machine's configuration), +configuration files are NOT installed during this step. This means +that files such as /etc/httpd/conf/loncapa.conf, +/home/httpd/html/res/adm/includes/copyright.tab, and +/home/httpd/spare.tab are not overwritten, but remain as old, +non-updated copies. (To automatically update these files and save/restore +their encoded machine configuration, you must run "make configinstall"). +
+- +
Updating the configurable files on your machine (make configinstall)
+Commands +cd loncom/build -rm -Rf HTML -make HTML -cd HTML -(look at the index.html file with a web browser such as Netscape) -
+General description of what happens +
++This is the actual make target code. +
++ +configinstall: Makefile.configinstall + make -f Makefile.configinstall SOURCE="$(SOURCE)" TARGET="$(TARGET)" \ + configfiles + if (test "0" = $(NORESTORECONF)); then \ + perl loncaparestoreconfigurations suffix .lpmlnew; fi + +Makefile.configinstall: $(SOURCE)/doc/loncapafiles/loncapafiles.lpml lpml_parse.pl + cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \ + perl lpml_parse.pl configinstall $(CATEGORY) $(DIST) "$(SOURCE)" \ + "$(TARGET)" > Makefile.configinstall + ++
+Configuration files are installed during this step. This means +that files such as /etc/httpd/conf/loncapa.conf, +/home/httpd/html/res/adm/includes/copyright.tab, and +/home/httpd/spare.tab are overwritten. Before being overwritten, +a backup copy is made though. Information is read out of these +backup copies and restored to the new files by the +loncaparestoreconfigurations script. To ensure that +new file permissions and ownerships are installed, a final set of +chown and chmod commands are called for each of +the configuration files. +
++For the truly paranoid +
++If you are truly paranoid, you can just make the +Makefile.configinstall file and then save, copy, +and restore all the configuration values yourself. +loncaparestoreconfigurations is pretty smart though, has yet to +fail, and besides, when needed, backup copies are made. +
++LON-CAPA is currently installed through "intelligent tarballs". +What I am describing now is part of an earlier (and perhaps future) effort +involving RPMs. +
++Commands +
+
+cd loncom/build
+rm -Rf BinaryRoot (or alternatively, "make clean")
+make RPM
+(to subsequently install, you can type commands like
+"rpm -Uvh --force LON-CAPA-base-3.1-1.i386.rpm")
+
+Configuration files +
++Configuration files are automatically saved with the file suffix +".rpmsave". So /etc/httpd/conf/loncapa.conf is saved as +/etc/httpd/conf/loncapa.conf.rpmsave. +The loncaparestoreconfigurations script should work to restore +configurations in this case. However, please note that if you install an RPM +twice without restoring your configuration, you will overwrite the +".rpmsave" files. +
++General description of what happens +
++This is the actual make target code. +
++ +RPM: BinaryRoot base_rpm_file_list + cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \ + perl lpml_parse.pl make_rpm $(CATEGORY) $(DIST) $(SOURCE) $(TARGET) \ + > base_customizerpm.xml + cat base_rpm_file_list.txt | perl make_rpm.pl base 3.2 1 '' '' \ + BinaryRoot base_customizerpm.xml + +BinaryRoot: base_rpm_file_list + make TARGET='BinaryRoot' NORESTORECONF='1' install + +base_rpm_file_list: + cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \ + perl lpml_parse.pl rpm_file_list $(CATEGORY) $(DIST) $(SOURCE) \ + 'BinaryRoot' | sort > base_rpm_file_list.txt + ++
+A BinaryRoot directory is generated that reflects the locations, +ownerships, permissions, and contents for all the CVS source +files, compiled binaries, directories, and links as they should eventually +occur on the '/' filesystem location. +
++loncom/build/make_rpm.pl (also available at +CPAN) is robust (tested over the +span of months) and, unlike other automated RPM-builders, cleanly +builds new RPMs without any after-effect of temporary files left +on the system. The generated RPM is labeled in the format +LON-CAPA-base-(VERSION)-(RELEASE).i386. VERSION is specified inside the +Makefile. +
+