[BACK]Return to readme.html CVS log [TXT] [DIR] Up to [LON-CAPA] / loncom / build

Diff for /loncom/build/readme.html between versions 1.9 and 1.23

version 1.9, 2001/01/17 12:00:49 version 1.23, 2011/01/09 19:01:23
Line 1 Line 1
 <H1>LON-CAPA Software Developer Instructions</H1>  <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
   <!-- The LearningOnline Network with CAPA -->
   <!-- $Id$ -->
   <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   <head>
   <meta http-equiv="Content-Type" content="text/html; charset=utf-8"></meta>
   <title>LON-CAPA Software Developer Guide</title>
   </head>
   <body>
   <h1>LON-CAPA Software Developer Guide</h1>
   <p>
   Created: <i>January 17, 2001</i>
   <br />Last updated: <i>January 9, 2011</i>
   </p>
   <ol>
   <li><a href="#Using_CVS">Using CVS</a>
   <ul>
   <li><a href="#cvslog">Setting up cvs access</a></li>
   <li><a href="#cvsupdate">Updating files (cvs update -d)</a></li>
   <li><a href="#cvssave">Saving files (cvs commit)</a></li>
   <li><a href="#cvsadd">Adding files (cvs add)</a></li>
   <li><a href="#cvsadddir">Adding directories (cvs add/import)</a></li>
   <li><a href="#cvsnotsure">What to do when you're not sure about your files
   (cvs update)</a></li>
   </ul></li>
   <li><a href="#makeHTML">Viewing the software (make HTML)</a></li>
   <li><a href="#makebuild">Compiling the software (make build)</a></li>
   <li><a href="#loncapafiles">Adding/removing files from the LON-CAPA
   installation (doc/loncapafiles/loncapafiles.lpml)</a></li>
   <li><a href="#configversusnonconfig">Configurable files versus
   non-configurable files</a></li>
   <li><a href="#makeinstall">Updating the non-configurable files on your
   machine (make install)</a></li>
   <li><a href="#makeconfiginstall">Updating the configurable files on your
   machine (make configinstall)</a></li>
   <li><a href="#makeRPM">Building RPMs (make RPM)</a></li>
   </ol>
   
 <OL>  <ol>
 <LI><A HREF="#Using_CVS">Using CVS</A>  
 <UL>  <li><a name="Using_CVS" /><h2>Using CVS</h2><br />
 <LI><A HREF="#cvslog">Logging in and out (cvs login; cvs logout)</A>  These instructions assume that you are using a Linux or UNIX based
 <LI><A HREF="#cvsget">Getting files (cvs update -d)</A>  terminal.
 <LI><A HREF="#cvsupdate">Updating files (cvs update -d)</A>  <ul>
 <LI><A HREF="#cvssave">Saving files (cvs commit)</A>  <li><a name="cvslog" />
 <LI><A HREF="#cvsadd">Adding files (cvs add)</A>      <h3>Setting up cvs access</h3>
 <LI><A HREF="#cvsadddir">Adding directories (cvs add/import)</A>  <p>
 <LI><A HREF="#cvsnotsure">What to do when you're not sure about your files (cvs update)</A>  CVS needs to be part of your system environment in order to access the LON-CAPA CVS repository, 
 </UL>  You can do this by:
 <LI><A HREF="#makeHTML">Viewing the software (make HTML)</A>  </p>
 <LI><A HREF="#makebuild">Compiling the software (make build)</A>  <p>
 <LI><A HREF="#loncapafiles">Adding/removing files from the LON-CAPA installation (doc/loncapafiles/loncapafiles.html)</A>  <font color="#008800">
 <LI><A HREF="#configversusnonconfig">Configurable files versus non-configurable files</A>  <tt>export CVS_RSH=ssh</tt><br />
 <LI><A HREF="#makeinstall">Updating the non-configurable files on your machine (make install)</A>  <tt>export CVSROOT=:ext:USERNAME@source.lon-capa.org:/home/cvs</tt>
 <LI><A HREF="#makeconfiginstall">Updating the configurable files on your machine (make configinstall)</A>  </font>
 <LI><A HREF="#makeRPM">Building RPMs (make RPM)</A>  </p>
 </OL>  <br />
   To actually issue CVS commands you will need to: 
 <OL>  <br /><br />
 <A NAME="Using_CVS">  <ol>
 <LI><H2>Using CVS</H2>  <li>Have installed the CVS client, e.g., <font color="#008800"><tt>yum install cvs</tt></font></li>
 <UL>  <li>Have been issued a CVS account, (send a request to the <a href="mailto:helpdesk@loncapa.org">LON-CAPA helpdesk</a>)</li>
 <LI><A NAME="cvslog">  <li>Have generated public and private keys via <font color="#008800"><tt>ssh-keygen -t dsa</tt></font> and sent the public key to the LON-CAPA helpdesk for installation on the CVS server.</li>
     <H3>Using CVS: Logging in and out (cvs login; cvs logout)</H3>  </ol>
 <LI><A NAME="cvsget">  <p>
     <H3>Using CVS: Getting files (cvs update -d)</H3>  The first time you use CVS, you need to CHECKOUT the repository.
 <LI><A NAME="cvsupdate">  Generally speaking, you need to checkout <tt>loncapa</tt> only once
     <H3>Using CVS: Updating files (cvs update -d)</H3>  per machine.
 <LI><A NAME="cvssave">  To check-out the repository, use the <tt>checkout</tt> command.
     <H3>Using CVS: Saving files (cvs commit)</H3>  (Otherwise, just enter your CVS directory, <tt>cd loncapa</tt>.)
 <LI><A NAME="cvsadd">  </p>
     <H3>Using CVS: Adding files (cvs add)</H3>  <p>
 <LI><A NAME="cvsadddir">  <font color="#008800">
     <H3>Using CVS: Adding directories (cvs add/import)</H3>  <tt>cvs checkout loncapa</tt><br />
 <LI><A NAME="cvsnotsure">  <tt>cd loncapa</tt>
     <H3>Using CVS: What to do when you're not sure about your files (cvs update)</H3>  </font>
 </UL>  </p>
 <LI><A NAME="makeHTML">  </li>
     <H2>Viewing the software (make HTML)</H2>  <li><a name="cvsupdate" />
 <STRONG>Commands</STRONG>      <h3>Using CVS: Updating files (cvs update -d)</h3>
 <FONT COLOR="#008800">  <p>
 <PRE>  After entering your CVS source tree (<tt>cd loncapa</tt>),
 cd loncom/build  you should frequently update the software changes that
 rm -Rf HTML <I>(or alternatively, "make clean")</I>  other people have made.  This is done with the <tt>update</tt> command.
 make HTML  </p>
 cd HTML  <p>
 <I>(look at the index.html file with a web browser such as Netscape)</I>  <font color="#008800">
 </PRE>  <tt>
 </FONT>  cvs update -d
 <STRONG>General description of what happens</STRONG>  </tt>
 <P>  </font>
   </p>
   <p>
   The <tt>cvs update</tt> command creates output
   as it updates your CVS source tree.  Common flags are
   'U' and 'P'; they indicate that a file in your
   <tt>loncapa</tt> directory is now updated with
   changes made by another programmer.
   </p>
   <p>
   <font color="#880000">
   <tt>`U FILE'</tt></font></p>
   <blockquote><font color="#880000">
   The file was brought up to date in your <tt>loncapa</tt>.
   <br />'U' is done for:
   <br />* any file that exists in the repository but not in your source, and
   <br />* files that you have not changed but are not the most recent versions
   available in the repository.
   <br />The network behavior of 'U' is that the entire new file is uploaded
   from the CVS server.
   </font></blockquote>
   <p><font color="#880000"><tt>
   `P FILE'
   </tt></font></p>
   <blockquote><font color="#880000">
   Like `U', but the CVS server sends a patch instead of an entire file.
   </font></blockquote>
   <p>
   'U' and 'P' essentially accomplish the same thing, just in
   different ways.
   </p>
   <p>
   Usually, when you do not <tt>cvs commit</tt> your code changes,
   the <tt>update</tt> command will tell you that you have modified
   your file with the 'M' flag.
   </p>
   <p><font color="#880000"><tt>
   `M FILE'
   </tt></font></p>
   <blockquote><font color="#880000">
        The file is modified in your working <tt>loncapa</tt> directory.
        This is probably based on changes you made and have not yet
        "cvs commit"-ed.
   </font></blockquote>
   <p>
   Sometimes, it will occur that:
   </p>
   <ul>
   <li>you have modified a file and not yet committed it</li>
   <li>someone else *has* modified a file and *has* committed it</li>
   </ul>
   <p>
   Generally speaking, this is <strong>your</strong> fault.  It is your
   responsibility to resolve conflicts.  <tt>cvs update</tt> informs
   you of a conflict with the 'C' flag.
   </p>
   <p><font color="#880000"><tt>
   `C FILE'
   </tt></font></p>
   <blockquote><font color="#880000">
        A conflict was detected while trying to merge your changes to FILE
        with changes from the source repository.
   </font></blockquote>
   <p>
   You will need to open the file and examine it; CVS will have added in
   markup tags like "&lt;&lt;&lt;&lt;&lt;&lt;" to tell you about the merging
   conflicts.  (Sometimes, CVS will intelligently merge in other changes and
   give you the 'M' flag, but many times you will have to manually edit
   the file as just described.)
   </p>
   </li>
   <li><a name="cvssave" />
       <h3>Using CVS: Saving files (cvs commit)</h3>
   <p>
   <tt>cvs commit</tt> works to submit changes to an <strong>existing</strong>
   file within the repository.  If a file does not currently exist, then you
   will first need to <tt>cvs add</tt> it as described in the following
   section.
   </p>
   Running the <tt>cvs commit</tt> command without additional arguments will
   commit all of your changes within the current directory and subdirectories.
   <p><font color="#008800"><tt>
   cvs commit
   </tt></font></p>
   <p>
   A more precise approach to using <tt>cvs commit</tt> is to pass it specific
   file names.  (Usually you should do this.)
   </p>
   <p><font color="#008800"><tt>
   cvs commit FILENAME
   </tt></font></p>
   <p>
   Note that CVS typically invokes the
   <a href="http://www.eng.hawaii.edu/Tutor/vi.html">vi</a> editor and solicits
   comments about your latest changes to the software.   Your comments should be
   descriptive and informative.  For example:
   </p>
   <ul>
   <li><strong>BAD</strong> - "saving my work"</li>
   <li><strong>GOOD</strong> - "implemented syntax checking of perl scripts
   with -c flag"</li>
   </ul>
   </li>
   <li><a name="cvsadd" />
       <h3>Using CVS: Adding files (cvs add)</h3>
   <p><font color="#008800"><tt>
   cvs add FILENAME
   </tt></font></p>
   <p>
   Then you can run <tt>cvs commit FILENAME</tt> and this file will
   become an "official" part of LON-CAPA.
   </p>
   </li>
   <li><a name="cvsadddir" />
       <h3>Using CVS: Adding directories (cvs add/import)</h3>
   <p><font color="#008800"><tt>
   cvs add DIRECTORYNAME
   </tt></font></p>
   <p>
   There is no need to run <tt>cvs commit</tt>.  Directories immediately
   become part of the LON-CAPA CVS source tree by only using the <tt>cvs add</tt>
   command.
   </p>
   <p>
   You should not ordinarily need to use the <tt>cvs import</tt> command.
   If misused, <tt>cvs import</tt> can lead to the loss of code within
   the repository.
   </p>
   </li>
   <li><a name="cvsnotsure" />
       <h3>Using CVS: What to do when you're not sure about your files
           (cvs update -d)</h3>
   <p>
   Once in a while, multiple programmers may be working on the
   same file.  Most conflicts are avoidable if everybody regularly
   <strong>commits</strong> their changes AND if everybody
   regularly <strong>updates</strong> the CVS source tree they are working on.
   </p>
   <p>
   If you are absent from programming for a few days, and
   <strong>fail</strong> to run <tt>cvs update -d</tt> on your CVS source
   repository, you have only yourself to blame if you find yourself writing
   code in a file that is not up-to-date.
   </p>
   </li>
   </ul></li>
   <li><a name="makeHTML" />
       <h2>Viewing the software (make HTML)</h2>
   <p>
   <strong>Commands</strong>
   </p>
   <p><font color="#008800"><tt>
   cd loncom/build<br />
   rm -Rf HTML <i>(or alternatively, "make clean")</i><br />
   make HTML<br />
   cd HTML<br />
   <i>(look at the index.html file with a web browser such as Netscape)</i>
   </tt></font></p>
   <p>
   <strong>General description of what happens</strong>
   </p>
   <p>
 This is the actual make target code.  This is the actual make target code.
 <FONT COLOR="#880000">  </p>
 <PRE>  <pre>
 <!-- LONCAPA MAKETARGET=HTML START -->  <!-- LONCAPA MAKETARGET=HTML START -->
 HTML:  HTML:
         install -d HTML   install -d HTML
         cp ../../doc/loncapafiles/*.gif HTML   cp $(SOURCE)/doc/loncapafiles/*.gif HTML
         perl parse.pl ../../doc/loncapafiles/loncapafiles.html HTML > HTML/index.html   cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
    perl lpml_parse.pl html development default "$(SOURCE)" '$(TARGET)' \
    > HTML/index.html
 <!-- LONCAPA MAKETARGET=HTML END -->  <!-- LONCAPA MAKETARGET=HTML END -->
 </PRE>  </pre>
 </FONT>  <p>
 What basically happens is that specially marked-up data in the LON-CAPA  What basically happens is that specially marked-up data in the LON-CAPA
 cvs repository file <TT>doc/loncapafiles.html</TT> is parsed into a more  cvs repository file <tt>doc/loncapafiles/loncapafiles.lpml</tt> is parsed
 viewable format by <TT>loncom/build/parse.pl</TT>.  The resulting  into a more viewable format by <tt>loncom/build/lpml_parse.pl</tt>.  The
 file gives a very well organized view of all the files, directories,  resulting file gives a very well organized view of all the files, directories,
 links, ownerships, permissions, and brief documentation of what each  links, ownerships, permissions, and brief documentation of what each
 file does.  file does.
 </P>  </p>
 <LI><A NAME="makebuild">  </li>
     <H2>Compiling the software (make build)</H2>  <li><a name="makebuild" />
 <STRONG>Commands</STRONG>      <h2>Compiling the software (make build)</h2>
 <FONT COLOR="#008800">  <strong>Commands</strong>
 <PRE>  <p><font color="#008800"><tt>
 cd loncom/build  cd loncom/build
 make build  <br />make build
 </PRE>  </tt></font></p>
 </FONT>  <p>
 <STRONG>General description of what happens</STRONG>  <strong>General description of what happens</strong>
 <P>  </p>
   <p>
 This is the actual make target code.  This is the actual make target code.
 <FONT COLOR="#880000">  </p>
 <PRE>  <pre>
 <!-- LONCAPA MAKETARGET=HTML START -->  <!-- LONCAPA MAKETARGET=build START -->
 build:  build: Makefile.build pod2html.sh pod2man.sh
         perl parse.pl ../../doc/loncapafiles/loncapafiles.html build > Makefile.build   echo -n "" > WARNINGS
         make -f Makefile.build all   make -f Makefile.build all
 <!-- LONCAPA MAKETARGET=HTML END -->   make warningnote
 </PRE>  
 </FONT>  Makefile.build: $(SOURCE)/doc/loncapafiles/loncapafiles.lpml lpml_parse.pl
 <TT>loncom/build/parse.pl</TT> reads in all the build information out   cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
 of <TT>doc/loncapafiles/loncapafiles.html.  A new Makefile named   perl lpml_parse.pl build $(CATEGORY) $(DIST) "$(SOURCE)" "$(TARGET)" \
 <TT>loncom/build/Makefile.build</TT> is dynamically constructed.   > Makefile.build
 This dynamically generated Makefile is then run to build/compile  <!-- LONCAPA MAKETARGET=build END -->
 all the software targets from source.  This currently takes 10 minutes  </pre>
 (depends on the speed of the machine you compile with).  <p>
 </P>  <tt>loncom/build/lpml_parse.pl</tt> reads in all the build information out
 <STRONG>Example</STRONG>  of <tt>doc/loncapafiles/loncapafiles.lpml</tt>.  A new Makefile named
 <P>  <tt>loncom/build/Makefile.build</tt> is dynamically constructed.
 Here is information for one file <TT>tth.so</TT> provided in  This dynamically generated Makefile is then used to build and compile
 <TT>doc/loncapafiles/loncapafiles.html</TT>.  all the software targets from source.  This can take several minutes
 <FONT COLOR="#330066">  (it depends on the speed of the machine you compile with).
 <PRE>  </p>
 <BR>&lt;METAGROUP&gt;  <p>
 <BR>&lt;LONCAPA TYPE=LOCATION DIST="redhat6.2" SOURCE="loncom/modules/TexConvert/tthperl/tth.so" TARGET="usr/lib/perl5/site_perl/5.005/tth.so" CATEGORY="system file"&gt;  <strong>Example</strong>
 <BR>&lt;DESCRIPTION&gt;  </p>
 <BR>shared library file for dynamic loading and unloading of TeX-to-HTML functionality  <p>
 <BR>&lt;/DESCRIPTION&gt;  Here is information for one file <tt>tth.so</tt> provided in
 <BR>&lt;BUILD&gt;  <tt>doc/loncapafiles/loncapafiles.lpml</tt>.
 <BR>loncom/modules/TexConvert/tthperl/commands  </p>
 <BR>&lt;/BUILD&gt;  <pre>
 <BR>&lt;DEPENDENCIES&gt;  &lt;file&gt;
 <BR>../tthdynamic/tthfunc.c  &lt;source&gt;loncom/homework/caparesponse/capa.so&lt;/source&gt;
 <BR>../ttmdynamic/ttmfunc.c  &lt;target dist='default'&gt;usr/lib/perl5/site_perl/5.005/capa.so&lt;/target&gt;
 <BR>&lt;/DEPENDENCIES&gt;  &lt;target dist='redhat7 redhat7.1'&gt;usr/lib/perl5/site_perl/5.6.0/capa.so&lt;/target&gt;
 </PRE>  &lt;categoryname&gt;system file&lt;/categoryname&gt;
 </FONT>  &lt;description&gt;
 <TT>loncom/build/parse.pl</TT> sees the <B>BUILD</B> tags and sets up  shared library file for dynamic loading and unloading
 a dynamic file <TT>Makefile.build</TT> to run the command inside the  &lt;/description&gt;
 <B>BUILD</B> tags (currently, <B>DEPENDENCIES</B> is not used for anything  &lt;build trigger='always run'&gt;
 besides documentation).  loncom/homework/caparesponse/commands
 </P>  &lt;/build&gt;
 <P>  &lt;dependencies&gt;
 Here is an example of a dynamically generated <TT>Makefile.build</TT>  caparesponse.c;
 that builds two LON-CAPA files (one of which is <TT>tth.so</TT>.  caparesponse.pm;
 <FONT COLOR="#330066">  README;
 <PRE>  Makefile.PL;
   capa.i;
   commands
   &lt;/dependencies&gt;
   &lt;/file&gt;
   </pre>
   <p>
   <tt>loncom/build/lpml_parse.pl</tt> sees the <b>build</b> tags and sets up
   a dynamic file <tt>Makefile.build</tt> to run the command inside the
   <b>build</b> tags.  The files listed inside the <b>dependencies</b> tags
   are included in the <tt>Makefile.build</tt> so as to determine whether
   or not there is a need to compile.
   </p>
   <p>
   Here is an example of a dynamically generated <tt>Makefile.build</tt>
   that builds two LON-CAPA files (one of which is <tt>tth.so</tt>).
   </p>
   <pre>
 all: ../homework/caparesponse/capa.so ../modules/TexConvert/tthperl/tth.so   all: ../homework/caparesponse/capa.so ../modules/TexConvert/tthperl/tth.so 
   
 ../homework/caparesponse/capa.so:  ../homework/caparesponse/caparesponse.c ../ho  ../homework/caparesponse/capa.so:  ../homework/caparesponse/caparesponse.c ../homework/caparesponse/caparesponse.pm alwaysrun
 mework/caparesponse/caparesponse.pm alwaysrun  
         cd ../homework/caparesponse/; sh ./commands          cd ../homework/caparesponse/; sh ./commands
   
 ../modules/TexConvert/tthperl/tth.so:  ../modules/TexConvert/tthperl/../tthdynam  ../modules/TexConvert/tthperl/tth.so:  ../modules/TexConvert/tthperl/../tthdynamic/tthfunc.c ../modules/TexConvert/tthperl/../ttmdynamic/ttmfunc.c
 ic/tthfunc.c ../modules/TexConvert/tthperl/../ttmdynamic/ttmfunc.c  
         cd ../modules/TexConvert/tthperl/; sh ./commands          cd ../modules/TexConvert/tthperl/; sh ./commands
   
 alwaysrun:  alwaysrun:
 </PRE>  </pre>
 </FONT>  </li><li><a name="loncapafiles" />
 </P>      <h2>Adding/removing files from the LON-CAPA installation
 <LI><A NAME="loncapafiles">          (doc/loncapafiles/loncapafiles.html)</h2>
     <H2>Adding/removing files from the LON-CAPA installation (doc/loncapafiles/loncapafiles.html)</H2>  <p>
 <LI><A NAME="configversusnonconfig">  <strong>To add and remove (and alter)</strong>
     <H2>Configurable files versus non-configurable files</H2>  </p>
 <LI><A NAME="makeinstall">  <p>
     <H2>Updating the non-configurable files on your machine (make install)</H2>  All that you have to do to alter the behavior of the installation is
 <LI><A NAME="makeconfiginstall">  edit a single file (<tt>doc/loncapafiles/loncapafiles.lpml</tt>).
     <H2>Updating the configurable files on your machine (make configinstall)</H2>  Adding, removing, and altering files requires proper attention
 <LI><A NAME="makeRPM">  to the syntax of file format of course.
     <H2>Building RPMs (make RPM)</H2>  </p>
 </OL>  <p>
   <strong>File Format</strong>
   </p>
   <p>
   The preceding <a href="#makebuild">"make build"</a> documentation
   gives an example of a <b>file</b> entry describing one particular file.
   All data within <tt>loncapafiles.lpml</tt> is specified according
   to markup tags.  The format and syntax of <tt>loncapafiles.lpml</tt>
   is currently best described by the HTML documentation code at the beginning of
   loncapafiles.html (as well as, by example, seeing how various
   information is coded).  All in all, the syntax is quite simple.
   </p>
   <p>
   <strong>Philosophy and notes (the thing nobody reads)</strong>
   </p>
   <p>
   Packaging the software from CVS onto a machine file system requires many
   things:
   </p>
   <ul>
   <li>documenting every component of the software,</li>
   <li>handling CVS <u>source</u> to file system <u>target</u> information,</li>
   <li>handling (according to a hierarchical scheme of grouping) file
   ownership and permissions,</li>
   <li>handling (according to a hierarchical scheme of grouping) directory
   ownership and permissions,</li>
   <li>handling symbolic links,</li>
   <li>providing for multiple options of installation targets (e.g. RedHat versus
   Debian),</li>
   <li>providing for different file ownerships and permissions to apply
   to the same file,</li>
   <li>allowing system software documentation to be automatically generated
   (see information on <a href="#makeHTML">"make html"</a>),</li>
   <li>providing information in an easily adjustable form as new demands
   are made on the software packaging system,</li>
   <li>providing software package information (for RPM),</li>
   <li>having information in a format that allows for troubleshooting
   the current status of the machine file system,</li>
   <li>allow for changes to the structure of the CVS repository,</li>
   <li>and something that is simple enough for any one to immediately work with,
   without having to learn any specifics (or hidden traps) of complicated
   Makefile's or a new macro language (m4?).</li>
   </ul>
   <p>
   I looked into, and tried, different ways of accomplishing the above
   including automake and recursive make.  The automake system seemed quite
   complicated (and needlessly so in terms of this project since, by and large,
   it works to coordinate many different types of build/compilation parameters
   whereas we are more concerned with installation parameters).  The other
   alternative, recursive make,
   has significant deficiencies since not all the information
   is kept in one place, and there are significant levels of dependency
   between all the things that must be done to keep software packaging
   up to date.  A particularly convincing article I found when looking into
   much of this was
   <a href="http://www.pcug.org.au/~millerp/rmch/recu-make-cons-harm.html">
   "Recursive Make Considered Harmful" by Peter Miller</a>.  Other complications
   were that, at the time, it was unclear as to what categories
   of software files we had, and whether or not the directory structure
   of CVS would remain constant.  With an ever-developing directory structure
   to CVS, I preferred to organize the information on a per-file basis
   as opposed to a per-directory basis.
   Additionally, a standard big Makefile assumes certain "normalcy" to
   the directory structure of different potential operating system directories
   (RedHat vs. Debian).
   </p>
   <p>
   If you take time to look at <tt>loncapafiles.lpml</tt>
   (and perhaps run the <a href="#makeHTML">make HTML</a> command)
   you will find that the organizing information according to the markup
   syntax in <tt>loncapafiles.lpml</tt> is simple.  Simple is good.
   </p>
   <p>
   <tt>loncom/build/lpml_parse.pl</tt> is the script (invoked automatically
   by the various targets in <tt>loncom/build/Makefile</tt>) that reads
   <tt>doc/loncapafiles/loncapafiles.lpml</tt>.  <tt>lpml_parse.pl</tt>
   is capable of reading and returning different types of information
   from <tt>loncapafiles.lpml</tt> depending on how <tt>lpml_parse.pl</tt>
   is invoked.  <tt>lpml_parse.pl</tt> has yet to have introduced new sources
   of error, and has been tested in quite a number of ways.  As with
   any parser however, I remain paranoid.
   </p>
   <p>
   Finally, some notes on the development.
   <tt>lpml_parse.pl</tt> is very fast and styled after a state-based SAX-like
   approach.  I do eventually want to use a real XML/XSLT approach, however
   I hesitate to make everyone everywhere install something like
   <a href="http://search.cpan.org/search?dist=XML-Xalan">XML::Xalan</a>.
   Also note that <tt>loncapafiles.lpml</tt> has a 
   DTD (<tt>loncom/build/lpml.dtd</tt>) against which it is valid.
   I would also like to use more ENTITY's inside <tt>lpml.dtd</tt> but currently
   the perl XML modules available at CPAN do not digest complex ENTITY's that
   well.
   </p>
   <p>
   The <tt>lpml_parse.pl</tt>-<tt>loncapafiles.lpml</tt> 
   combination has been highly efficient and error-free.
   </p>
   </li><li><a name="configversusnonconfig" />
       <h2>Configurable files versus non-configurable files</h2>
   <p>
   <strong>Machine-specific information is the difference</strong>
   </p>
   <p>
   The current list of configurable files for the LON-CAPA system is
   <tt>/etc/httpd/conf/loncapa.conf</tt>,
   <tt>/etc/ntp.conf</tt>,
   <tt>/etc/krb.conf</tt>,
   <tt>/etc/ntp/step-tickers</tt>,
   <tt>/home/httpd/html/res/adm/includes/copyright.tab</tt>,
   <tt>/home/httpd/html/res/adm/includes/un_keyword.tab</tt>,
   <tt>/home/httpd/hosts.tab</tt>, and
   <tt>/home/httpd/spare.tab</tt>.
   </p>
   <p>
   All of these configurable files contain machine-specific information.
   For instance, the overall LON-CAPA system relies on unique host IDs such
   as msua3, s1, s2, msul1, and 103a1 (specified as a "PerlSetVar lonHostID"
   field within <tt>/etc/httpd/conf/loncapa.conf</tt>).
   Non-configurable files simply do NOT have machine-specific information.
   </p>
   <p>
   <strong>The impact on updating software</strong>
   </p>
   <p>
   What this means in terms of software updating is that:
   </p>
   <ul>
   <li>non-configurable files can be simply overwritten with newer versions
   (without "anything" else to worry about),</li>
   <li>and configurable files must follow these steps to be safely
   overwritten:
   <ol>
   <li>have their machine-specific information saved,</li>
   <li>be overwritten, and then</li>
   <li>have their machine-specific information restored.</li>
   </ol>
   </li>
   </ul>
   </li>
   <li><a name="makeinstall" />
       <h2>Updating the non-configurable files on your machine (make install)</h2>
   <strong>Commands</strong>
   <p><font color="#008800"><tt>
   cd loncom/build
   <br />make install
   </tt></font></p>
   <p>
   <strong>General description of what happens</strong>
   </p>
   <p>
   This is the actual make target code.
   </p>
   <pre>
   <!-- LONCAPA MAKETARGET=install START -->
   install: TEST_hosts_tab Makefile.install Makefile
    echo -n "" > WARNINGS
    make -f Makefile.install SOURCE="$(SOURCE)" TARGET="$(TARGET)" \
    directories
    make -f Makefile.install SOURCE="$(SOURCE)" TARGET="$(TARGET)" files
    make -f Makefile.install SOURCE="$(SOURCE)" TARGET="$(TARGET)" links
    make SOURCE="$(SOURCE)" TARGET="$(TARGET)" \
    NORESTORECONF="$(NORESTORECONF)" configinstall
    make postinstall
    make warningnote
    echo "You can run 'make test' to see if your system is ready to go!"
   
   Makefile.install: $(SOURCE)/doc/loncapafiles/loncapafiles.lpml lpml_parse.pl
    cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
    perl lpml_parse.pl install $(CATEGORY) $(DIST) "$(SOURCE)" \
    "$(TARGET)" > Makefile.install
   <!-- LONCAPA MAKETARGET=install END -->
   </pre>
   <p>
   For safety reasons (so as to preserve a machine's configuration),
   configuration files are NOT installed during this step.  This means
   that files such as <tt>/etc/httpd/conf/loncapa.conf</tt>,
   <tt>/home/httpd/html/res/adm/includes/copyright.tab</tt>, and
   <tt>/home/httpd/spare.tab</tt> are not overwritten, but remain as old,
   non-updated copies.  (To automatically update these files and save/restore
   their encoded machine configuration, you must run "make configinstall").
   </p>
   </li>
   <li><a name="makeconfiginstall" />
     <h2>Updating the configurable files on your machine (make configinstall)</h2>
   <strong>Commands</strong>
   <p><font color="#008800"><tt>
   cd loncom/build
   make configinstall
   </tt></font></p>
   <p>
   <strong>General description of what happens</strong>
   </p>
   <p>
   This is the actual make target code.
   </p>
   <pre>
   <!-- LONCAPA MAKETARGET=configinstall START -->
   configinstall: Makefile.configinstall
    make -f Makefile.configinstall SOURCE="$(SOURCE)" TARGET="$(TARGET)" \
    configfiles
    if (test "0" = $(NORESTORECONF)); then \
    perl loncaparestoreconfigurations suffix .lpmlnew; fi
   
   Makefile.configinstall: $(SOURCE)/doc/loncapafiles/loncapafiles.lpml lpml_parse.pl
    cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
    perl lpml_parse.pl configinstall $(CATEGORY) $(DIST) "$(SOURCE)" \
    "$(TARGET)" > Makefile.configinstall
   <!-- LONCAPA MAKETARGET=configinstall END -->
   </pre>
   <p>
   Configuration files are installed during this step.  This means
   that files such as <tt>/etc/httpd/conf/loncapa.conf</tt>,
   <tt>/home/httpd/html/res/adm/includes/copyright.tab</tt>, and
   <tt>/home/httpd/spare.tab</tt> are overwritten.  Before being overwritten,
   a backup copy is made though.  Information is read out of these
   backup copies and restored to the new files by the
   <tt>loncaparestoreconfigurations</tt> script.  To ensure that
   new file permissions and ownerships are installed, a final set of
   <tt>chown</tt> and <tt>chmod</tt> commands are called for each of 
   the configuration files.
   </p>
   <p>
   <strong>For the truly paranoid</strong>
   </p>
   <p>
   If you are truly paranoid, you can just make the
   <tt>Makefile.configinstall</tt> file and then save, copy,
   and restore all the configuration values yourself.
   <tt>loncaparestoreconfigurations</tt> is pretty smart though, has yet to
   fail, and besides, when needed, backup copies are made.
   </p>
   </li><li><a name="makeRPM" />
       <h2>Building RPMs (make RPM)</h2>
   <p>
   LON-CAPA is currently installed through "intelligent tarballs".
   What I am describing now is part of an earlier (and perhaps future) effort
   involving RPMs.
   </p>
   <p>
   <strong>Commands</strong>
   </p>
   <p><font color="#008800"><tt>
   cd loncom/build<br />
   rm -Rf BinaryRoot <i>(or alternatively, "make clean")</i><br />
   make RPM<br />
   <i>(to subsequently install, you can type commands like
   "rpm -Uvh --force LON-CAPA-base-3.1-1.i386.rpm")</i>
   </tt></font></p>
   <p>
   <strong>Configuration files</strong>
   </p>
   <p>
   Configuration files are automatically saved with the file suffix
   ".rpmsave".  So <tt>/etc/httpd/conf/loncapa.conf</tt> is saved as 
   <tt>/etc/httpd/conf/loncapa.conf.rpmsave</tt>.
   The <tt>loncaparestoreconfigurations</tt> script should work to restore
   configurations in this case.  However, please note that if you install an RPM
   twice without restoring your configuration, you will overwrite the
   ".rpmsave" files.
   </p>
   <p>
   <strong>General description of what happens</strong>
   </p>
   <p>
   This is the actual make target code.
   </p>
   <pre>
   <!-- LONCAPA MAKETARGET=RPM START -->
   RPM: BinaryRoot base_rpm_file_list
    cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
    perl lpml_parse.pl make_rpm $(CATEGORY) $(DIST) $(SOURCE) $(TARGET) \
    > base_customizerpm.xml
    cat base_rpm_file_list.txt | perl make_rpm.pl base 3.2 1 '' '' \
    BinaryRoot base_customizerpm.xml
   
   BinaryRoot: base_rpm_file_list
    make TARGET='BinaryRoot' NORESTORECONF='1' install
   
   base_rpm_file_list:
    cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
    perl lpml_parse.pl rpm_file_list $(CATEGORY) $(DIST) $(SOURCE) \
    'BinaryRoot' | sort > base_rpm_file_list.txt
   <!-- LONCAPA MAKETARGET=RPM END -->
   </pre>
   <p>
   A <tt>BinaryRoot</tt> directory is generated that reflects the locations,
   ownerships, permissions, and contents for all the CVS source
   files, compiled binaries, directories, and links as they should eventually
   occur on the '/' filesystem location.
   </p>
   <p>
   <tt>loncom/build/make_rpm.pl</tt> (also available at
   <a href="http://www.cpan.org">CPAN</a>) is robust (tested over the
   span of months) and, unlike other automated RPM-builders, cleanly
   builds new RPMs without any after-effect of temporary files left
   on the system.  The generated RPM is labeled in the format
   LON-CAPA-base-(VERSION)-(RELEASE).i386.  VERSION is specified inside the
   Makefile.
   </p>
   </li>
   </ol>
   </body>
   </html>
   
   
   
Removed from v.1.9  
changed lines
  Added in v.1.23

FreeBSD-CVSweb <freebsd-cvsweb@FreeBSD.org>