+
Written by Scott Harrison, January 17, 2001
+
Last updated, April 27, 2002
+
++ +General description of what happens ++ +
- +These instructions assume that you are using a Linux or UNIX based +terminal. +
Using CVS
+
+- +
+Using CVS: Logging in and out (cvs login; cvs logout)
++To log into CVS, CVS needs to be part of your system environment. +You can accomplish this by: + +
++export CVSROOT=:pserver:USERNAME@install.lon-capa.org:/home/cvs ++ ++To actually login, you will need to execute the following command: + +
++cvs login ++ +You will be prompted for a password. +If you do not have a password, or the password is not working, you +should contact helen@lon-capa.org ++If you have yet to check out the CVS repository, you can use the +checkout command. (Otherwise, just enter your +CVS repository, cd loncapa.) + +
++cvs checkout loncapa +cd loncapa ++ +After you have completed your work with the CVS repository, it +is recommended that you log out: + +
++cvs logout ++ +- +
+Using CVS: Updating files (cvs update -d)
++After entering your CVS source tree (cd loncapa), +you should frequently update the software changes that +programmers other than yourself have made. This is done +with the update command. + +
++cvs update -d ++ ++The cvs update command will give you output +as it updates your CVS source tree. Commonly you will +see 'U' and 'P' which indicate that a file has been updated with +changes another programmer has made. + +
++`U FILE' + The file was brought up to date with respect to the repository. + This is done for any file that exists in the repository but not in + your source, and for files that you haven't changed but are not + the most recent versions available in the repository. + +`P FILE' + Like `U', but the CVS server sends a patch instead of an entire + file. These two things accomplish the same thing. ++ ++Usually, when you do not cvs commit the code changes that you +make, the update command will tell you that you have modified +your file with the 'M' flag. + +
+`M FILE' + The file is modified in your working directory. This is probably + based on changes you made and have not yet "cvs commit"-ed. ++ +Sometimes, it will occur that: ++
+Generally speaking, this is your fault. It is your +responsibility to resolve conflicts. cvs update informs +you of a conflict with the 'C' flag. + +- you have modified a file and not yet committed it
+- someone else *has* modified a file and *has* committed it
++`C FILE' + A conflict was detected while trying to merge your changes to FILE + with changes from the source repository. ++ +You will need to open the file and examine it; CVS will have added in +markup tags like "<<<<<<" to tell you about the merging +conflicts. (Sometimes, CVS will intelligently merge in other changes and +give you the 'M' flag, but many times you will have to manually edit +the file as just described.) + +- +
+Using CVS: Saving files (cvs commit)
++cvs commit works to submit changes to an existing +file within the repository. If a file does not currently exist, then you +will first need to cvs add it as described in the following +section. +
+Running the cvs commit command without passing it an argument will +commit all changes that you have within the current directory and +subdirectories. + ++cvs commit ++ + ++A more precise approach to using cvs commit is to pass it specific +file names. (Usually you should do this.) + +
++cvs commit FILENAME ++ ++Note that CVS typically invokes the vi editor and solicits comments +about your latest changes to the software. Your comments should be +both short yet uniquely descriptive. For example: +
+
+ +- BAD - "made some changes and am drinking soda"
+- GOOD - "implemented syntax checking of perl scripts +with -c flag"
+- +
+Using CVS: Adding files (cvs add)
++ +
++cvs add FILENAME ++ ++Then you can run cvs commit FILENAME and this file will +become an "official" part of LON-CAPA. +
+- +
+Using CVS: Adding directories (cvs add/import)
++ +
++cvs add DIRECTORYNAME ++ ++There is no need to run cvs commit. Directories immediately +become part of the LON-CAPA CVS source tree by only using the cvs add +command. +
+- +
+Using CVS: What to do when you're not sure about your files + (cvs update -d)
++Every once in a while, multiple programmers may be working on the +same file. Most conflicts are avoidable if everybody regularly +commits their changes AND if everybody +regularly updates the CVS source tree they are working on. +
++In other words, if you are absent from programming for a few days, and +fail to run cvs update -d on your CVS source +repository, you have only yourself to blame if you find yourself writing +code in a file that is not up-to-date. +
+- +
+Commands + +Viewing the software (make HTML)
cd loncom/build rm -Rf HTML (or alternatively, "make clean") make HTML cd HTML (look at the index.html file with a web browser such as Netscape) -- -General description of what happens -+
This is the actual make target code.
-
-
+
+
+
What basically happens is that specially marked-up data in the LON-CAPA
-cvs repository file doc/loncapafiles.html is parsed into a more
-viewable format by loncom/build/parse.pl. The resulting
+cvs repository file doc/loncapafiles.lpml is parsed into a more
+viewable format by loncom/build/lpml_parse.pl. The resulting
file gives a very well organized view of all the files, directories,
links, ownerships, permissions, and brief documentation of what each
file does.
-
HTML:
- install -d HTML
- cp ../../doc/loncapafiles/*.gif HTML
- perl parse.pl ../../doc/loncapafiles/loncapafiles.html HTML > HTML/index.html
+ install -d HTML
+ cp $(SOURCE)/doc/loncapafiles/*.gif HTML
+ cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
+ perl lpml_parse.pl html development default "$(SOURCE)" '$(TARGET)' \
+ > HTML/index.html
-
-
+
+ +
cd loncom/build make build -- -General description of what happens -
+ +
+General description of what happens +
+
This is the actual make target code.
-
-
-
-build:
- perl parse.pl ../../doc/loncapafiles/loncapafiles.html build > Makefile.build
- make -f Makefile.build all
-
-
-
-loncom/build/parse.pl reads in all the build information out
-of doc/loncapafiles/loncapafiles.html. A new Makefile named
-loncom/build/Makefile.build is dynamically constructed.
+
+
+
+build: Makefile.build pod2html.sh pod2man.sh
+ echo -n "" > WARNINGS
+ make -f Makefile.build all
+ make warningnote
+
+Makefile.build: $(SOURCE)/doc/loncapafiles/loncapafiles.lpml lpml_parse.pl
+ cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
+ perl lpml_parse.pl build $(CATEGORY) $(DIST) "$(SOURCE)" "$(TARGET)" \
+ > Makefile.build
+
+
+
+loncom/build/lpml_parse.pl reads in all the build information out
+of doc/loncapafiles/loncapafiles.lpml. A new Makefile named
+loncom/build/Makefile.build is dynamically constructed.
This dynamically generated Makefile is then run to build/compile
all the software targets from source. This currently takes 10 minutes
(depends on the speed of the machine you compile with).
-
-Here is information for one file tth.so provided in
-doc/loncapafiles/loncapafiles.html.
-
-
-
-
-loncom/build/parse.pl sees the BUILD tags and sets up
-a dynamic file Makefile.build to run the command inside the
-BUILD tags (currently, DEPENDENCIES is not used for anything
-besides documentation).
-
<METAGROUP>
-
<LONCAPA TYPE=LOCATION DIST="redhat6.2" SOURCE="loncom/modules/TexConvert/tthperl/tth.so" TARGET="usr/lib/perl5/site_perl/5.005/tth.so" CATEGORY="system file">
-
<DESCRIPTION>
-
shared library file for dynamic loading and unloading of TeX-to-HTML functionality
-
</DESCRIPTION>
-
<BUILD>
-
loncom/modules/TexConvert/tthperl/commands
-
</BUILD>
-
<DEPENDENCIES>
-
../tthdynamic/tthfunc.c
-
../ttmdynamic/ttmfunc.c
-
</DEPENDENCIES>
-
-Here is an example of a dynamically generated Makefile.build
-that builds two LON-CAPA files (one of which is tth.so.
-
-
+
+Here is information for one file tth.so provided in
+doc/loncapafiles/loncapafiles.lpml.
+
+
+<file>
+<source>loncom/homework/caparesponse/capa.so</source>
+<target dist='default'>usr/lib/perl5/site_perl/5.005/capa.so</target>
+<target dist='redhat7 redhat7.1'>usr/lib/perl5/site_perl/5.6.0/capa.so</target>
+<categoryname>system file</categoryname>
+<description>
+shared library file for dynamic loading and unloading
+</description>
+<build trigger='always run'>
+loncom/homework/caparesponse/commands
+</build>
+<dependencies>
+caparesponse.c;
+caparesponse.pm;
+README;
+Makefile.PL;
+capa.i;
+commands
+</dependencies>
+</file>
+
+
+loncom/build/lpml_parse.pl sees the build tags and sets up
+a dynamic file Makefile.build to run the command inside the
+build tags. The files listed inside the dependencies tags
+are included in the Makefile.build so as to determine whether
+or not there is a need to compile.
+
+Here is an example of a dynamically generated Makefile.build
+that builds two LON-CAPA files (one of which is tth.so).
+
+
all: ../homework/caparesponse/capa.so ../modules/TexConvert/tthperl/tth.so
-../homework/caparesponse/capa.so: ../homework/caparesponse/caparesponse.c ../ho
-mework/caparesponse/caparesponse.pm alwaysrun
+../homework/caparesponse/capa.so: ../homework/caparesponse/caparesponse.c ../homework/caparesponse/caparesponse.pm alwaysrun
cd ../homework/caparesponse/; sh ./commands
-../modules/TexConvert/tthperl/tth.so: ../modules/TexConvert/tthperl/../tthdynam
-ic/tthfunc.c ../modules/TexConvert/tthperl/../ttmdynamic/ttmfunc.c
+../modules/TexConvert/tthperl/tth.so: ../modules/TexConvert/tthperl/../tthdynamic/tthfunc.c ../modules/TexConvert/tthperl/../ttmdynamic/ttmfunc.c
cd ../modules/TexConvert/tthperl/; sh ./commands
alwaysrun:
-
-
-
+All that you have to do to alter the behavior of the installation is +edit a single file (doc/loncapafiles/loncapafiles.lpml). +Adding, removing, and altering files requires proper attention +to the syntax of file format of course. +
+File Format ++The preceding "make build" documentation +gives an example of a file entry describing one particular file. +All data within loncapafiles.lpml is specified according +to markup tags. The format and syntax of loncapafiles.lpml +is currently best described by the HTML documentation code at the beginning of +loncapafiles.html (as well as, by example, seeing how various +information is coded). All in all, the syntax is quite simple. +
+Philosophy and notes (the thing nobody reads) ++Packaging the software from CVS onto a machine file system requires many +things: +
+I looked into, and tried, different ways of accomplishing the above +including automake and recursive make. The automake system seemed quite +complicated (and needlessly so in terms of this project since, by and large, +it works to coordinate many different types of build/compilation parameters +whereas we are more concerned with installation parameters). Recursive make +has significant deficiencies in the sense that not all the information +is kept in one place, and there are significant levels of dependency +between all the things that must be done to keep software packaging +up to date. A particularly convincing article I found when looking into +much of this was + +"Recursive Make Considered Harmful" by Peter Miller. Complicating +matters was, at the time, it was unclear as to what categories +of software files we had, and whether or not the directory structure +of CVS would remain constant. With an ever-developing directory structure +to CVS, I preferred to organize the information on a per-file basis +as opposed to a per-directory basis (although there is a successful +implementation of a standard big Makefile in loncom/Makefile). +Additionally, a standard big Makefile assumes certain "normalcy" to +the directory structure of different potential operating system directories +(RedHat vs. Debian). +
++If you take time to look at loncapafiles.lpml +(and perhaps run the make HTML command) +you will find that the organizing information according to the markup +syntax in loncapafiles.lpml is simple. Simple is good. +
++loncom/build/lpml_parse.pl is the script (invoked automatically +by the various targets in loncom/build/Makefile) that reads +doc/loncapafiles/loncapafiles.lpml. lpml_parse.pl +is capable of reading and returning different types of information +from loncapafiles.lpml depending on how lpml_parse.pl +is invoked. lpml_parse.pl has yet to have introduced new sources +of error, and has been tested in quite a number of ways. As with +any parser however, I remain paranoid. +
++lpml_parse.pl is very fast and styled after a state-based SAX-like +approach. Additionally, loncapafiles.lpml has a +DTD (loncom/build/lpml.dtd) against which it is valid. +I would like to use more ENTITY's inside lpml.dtd but currently +Perl XML modules available at CPAN do not digest complex ENTITY's that well. +
++The lpml_parse.pl-loncapafiles.lpml +combination has been working very efficiently and error-free. +
++Machine-specific information is the difference +
++The current list of configurable files for the LON-CAPA system is +/etc/httpd/access.conf, /etc/smb.conf, /etc/ntp.conf, /etc/krb.conf, +/etc/atalk/config, /etc/ntp/step-tickers, +/home/httpd/html/res/adm/includes/copyright.tab, +/home/httpd/html/res/adm/includes/un_keyword.tab, +/home/httpd/hosts.tab, and +/home/httpd/spare.tab. +
++All of these configurable files contain machine-specific information. +For instance, the LON-CAPA system relies on unique host IDs such +as msua3, s1, s2, msul1, and 103a1 (specified as a "PerlSetVar lonHostID" +field within /etc/httpd/access.conf). +Non-configurable files simply do NOT have machine-specific information. +The impact on updating software +
+What this means in terms of software updating is that +
+cd loncom/build +make install ++ +
+General description of what happens +
+
+This is the actual make target code.
+
+
+
+install: TEST_hosts_tab Makefile.install Makefile
+ echo -n "" > WARNINGS
+ make -f Makefile.install SOURCE="$(SOURCE)" TARGET="$(TARGET)" \
+ directories
+ make -f Makefile.install SOURCE="$(SOURCE)" TARGET="$(TARGET)" files
+ make -f Makefile.install SOURCE="$(SOURCE)" TARGET="$(TARGET)" links
+ make SOURCE="$(SOURCE)" TARGET="$(TARGET)" \
+ NORESTORECONF="$(NORESTORECONF)" configinstall
+ make postinstall
+ make warningnote
+ echo "You can run 'make test' to see if your system is ready to go!"
+
+Makefile.install: $(SOURCE)/doc/loncapafiles/loncapafiles.lpml lpml_parse.pl
+ cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
+ perl lpml_parse.pl install $(CATEGORY) $(DIST) "$(SOURCE)" \
+ "$(TARGET)" > Makefile.install
+
+
+
+For safety reasons (so as to not mess up a machine's configuration),
+configuration files are NOT installed during this step. This means
+that files such as /etc/httpd/access.conf, /etc/smb.conf, /etc/atalk/config,
+/home/httpd/html/res/adm/includes/copyright.tab, and
+/home/httpd/spare.tab are not overwritten, but remain as old, non-updated
+copies. (To automatically update these files and save/restore
+their encoded machine configuration, you must run "make configinstall").
+
+cd loncom/build +make configinstall ++ +General description of what happens +
+This is the actual make target code.
+
+
+
+configinstall: Makefile.configinstall
+ make -f Makefile.configinstall SOURCE="$(SOURCE)" TARGET="$(TARGET)" \
+ configfiles
+ if (test "0" = $(NORESTORECONF)); then \
+ perl loncaparestoreconfigurations suffix .lpmlnew; fi
+
+Makefile.configinstall: $(SOURCE)/doc/loncapafiles/loncapafiles.lpml lpml_parse.pl
+ cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
+ perl lpml_parse.pl configinstall $(CATEGORY) $(DIST) "$(SOURCE)" \
+ "$(TARGET)" > Makefile.configinstall
+
+
+
+Configuration files are installed during this step. This means
+that files such as /etc/httpd/access.conf, /etc/smb.conf, /etc/atalk/config,
+/home/httpd/html/res/adm/includes/copyright.tab, and
+/home/httpd/spare.tab are overwritten. Before being overwritten,
+a backup copy is made though. Information is read out of these
+backup copies and restored to the new files by the
+loncaparestoreconfigurations script. To ensure that
+new file permissions and ownerships are installed, a final set of
+chown and chmod commands are called for each of
+the configuration files.
+
+For the truly paranoid +
++If you are truly paranoid, you can just make the +Makefile.configinstall file and then save, copy, +and restore all the configuration values yourself. +loncaparestoreconfigurations is pretty smart though, has yet to +fail, and besides, when needed backup copies are made. +
+
+LON-CAPA is currently installed through "intelligent tarballs". This
+is part of an earlier (and perhaps future) effort involving RPMs.
+Commands
+
+
+cd loncom/build
+rm -Rf BinaryRoot (or alternatively, "make clean")
+make RPM
+ (to subsequently install, you can type commands like
+ "rpm -Uvh --force LON-CAPA-base-3.1-1.i386.rpm")
+
+
+
+Never never never never never manually install the +LON-CAPA-setup-3.1-1.i386.rpm. This RPM is meant to only be +installed by the CD installation process (it wipes out +the existing /etc/passwd file). +
++Configuration files +
++Configuration files are automatically saved with the file suffix +".rpmsave". So /etc/httpd/conf/access.conf is saved as +/etc/httpd/conf/access.conf.rpmsave. You can restore +the machine-specific configuration information by running +the /usr/sbin/loncaparestoreconfigurations. However, +a warning is important here. If you install an RPM twice +without restoring your configuration, you will overwrite the +".rpmsave" files. +
++General description of what happens +
+
+This is the actual make target code.
+
+
+
+RPM: BinaryRoot base_rpm_file_list
+ cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
+ perl lpml_parse.pl make_rpm $(CATEGORY) $(DIST) $(SOURCE) $(TARGET) \
+ > base_customizerpm.xml
+ cat base_rpm_file_list.txt | perl make_rpm.pl base 3.2 '' '' \
+ BinaryRoot base_customizerpm.xml
+
+BinaryRoot: base_rpm_file_list
+ make TARGET='BinaryRoot' NORESTORECONF='1' install
+
+base_rpm_file_list:
+ cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
+ perl lpml_parse.pl rpm_file_list $(CATEGORY) $(DIST) $(SOURCE) \
+ 'BinaryRoot' | sort > base_rpm_file_list.txt
+
+
+
+A BinaryRoot directory is generated that reflects the locations,
+ownerships, permissions, and contents for all the CVS source
+files, compiled binaries, directories, and links as they should eventually
+occur on the '/' filesystem location.
+
+loncom/build/make_rpm.pl is robust (tested over the +span of months) and, unlike other automated RPM-builders, cleanly +builds new RPMs without any after-effect of temporary files left +on the system. (On the negative side, there are a number of +LON-CAPA specific customizations inside make_rpm.pl which, for +the sake of reusability, should eventually be removed). Two new RPMs +are generated: LON-CAPA-base-3.1-1.i386 and LON-CAPA-setup-3.1-1.i386.rpm +(again, never manually install LON-CAPA-setup-3.1-1.i386.rpm). +
+