Annotation of loncom/build/readme.html, revision 1.20

1.19      harris41    1: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
                      2:  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
1.17      harris41    3: <!-- The LearningOnline Network with CAPA -->
1.20    ! harris41    4: <!-- $Id: readme.html,v 1.19 2002/07/31 18:14:31 harris41 Exp $ -->
1.17      harris41    5: <html>
                      6: <head>
1.19      harris41    7: <meta http-equiv="Content-Type" content="text/html; charset=utf-8"></meta>
1.17      harris41    8: <meta name="GENERATOR"
1.19      harris41    9:       content="Scott Harrison and Emacs Version 3.14159265358979"></meta>
                     10: <title>LON-CAPA Software Developer Guide</title>
1.17      harris41   11: </head>
                     12: <body>
1.19      harris41   13: <h1>LON-CAPA Software Developer Guide</h1>
1.17      harris41   14: <p>
1.19      harris41   15: <br /><i>Written by Scott,<br />
                     16: <a href="mailto:sharrison@users.sourceforge.net">
                     17: sharrison@users.sourceforge.net</a>,<br />
                     18: January 17, 2001</i>
1.20    ! harris41   19: <br /><i>Last updated, August 14, 2002</i>
1.17      harris41   20: </p>
                     21: <ol>
1.19      harris41   22: <li><a href="#Using_CVS">Using CVS</a>
1.17      harris41   23: <ul>
                     24: <li><a href="#cvslog">Logging in and out (cvs login; cvs logout)</a></li>
                     25: <li><a href="#cvsupdate">Updating files (cvs update -d)</a></li>
                     26: <li><a href="#cvssave">Saving files (cvs commit)</a></li>
                     27: <li><a href="#cvsadd">Adding files (cvs add)</a></li>
                     28: <li><a href="#cvsadddir">Adding directories (cvs add/import)</a></li>
                     29: <li><a href="#cvsnotsure">What to do when you're not sure about your files
                     30: (cvs update)</a></li>
1.19      harris41   31: </ul></li>
1.17      harris41   32: <li><a href="#makeHTML">Viewing the software (make HTML)</a></li>
                     33: <li><a href="#makebuild">Compiling the software (make build)</a></li>
                     34: <li><a href="#loncapafiles">Adding/removing files from the LON-CAPA
                     35: installation (doc/loncapafiles/loncapafiles.lpml)</a></li>
                     36: <li><a href="#configversusnonconfig">Configurable files versus
                     37: non-configurable files</a></li>
                     38: <li><a href="#makeinstall">Updating the non-configurable files on your
                     39: machine (make install)</a></li>
                     40: <li><a href="#makeconfiginstall">Updating the configurable files on your
                     41: machine (make configinstall)</a></li>
                     42: <li><a href="#makeRPM">Building RPMs (make RPM)</a></li>
                     43: </ol>
1.2       harris41   44: 
1.17      harris41   45: <ol>
1.19      harris41   46: 
                     47: <li><a name="Using_CVS" /><h2>Using CVS</h2><br />
1.17      harris41   48: These instructions assume that you are using a Linux or UNIX based
                     49: terminal.
                     50: <ul>
                     51: <li><a name="cvslog" />
                     52:     <h3>Using CVS: Logging in and out (cvs login; cvs logout)</h3>
                     53: <p>
1.19      harris41   54: In order to log into CVS, CVS needs to be part of your system environment.
                     55: You can do this by:
                     56: </p>
                     57: <p>
1.17      harris41   58: <font color="#008800">
1.19      harris41   59: <tt>export CVSROOT=:pserver:USERNAME@install.lon-capa.org:/home/cvs</tt>
1.17      harris41   60: </font>
                     61: </p>
                     62: <p>
                     63: To actually login, you will need to execute the following command:
1.19      harris41   64: </p>
                     65: <p>
1.17      harris41   66: <font color="#008800">
1.19      harris41   67: <tt>cvs login</tt>
1.17      harris41   68: </font>
1.19      harris41   69: </p>
                     70: <p>
                     71: You are then prompted for a password.
1.17      harris41   72: If you do not have a password, or the password is not working, you
1.19      harris41   73: should contact <a href="mailto:helen@lon-capa.org">helen@lon-capa.org</a>.
                     74: </p>
                     75: <p>
                     76: The first time you use CVS, you need to CHECKOUT the repository.
                     77: Generally speaking, you need to checkout <tt>loncapa</tt> only once
                     78: per machine.
                     79: To check-out the repository, use the <tt>checkout</tt> command.
                     80: (Otherwise, just enter your CVS directory, <tt>cd loncapa</tt>.)
1.17      harris41   81: </p>
                     82: <p>
                     83: <font color="#008800">
1.19      harris41   84: <tt>cvs checkout loncapa</tt><br />
                     85: <tt>cd loncapa</tt>
1.17      harris41   86: </font>
                     87: </p>
1.19      harris41   88: <p>After completing work with the CVS repository,
                     89: you can log out:
                     90: </p>
                     91: <p>
1.17      harris41   92: <font color="#008800">
1.19      harris41   93: <tt>cvs logout</tt>
1.17      harris41   94: </font>
                     95: </p>
                     96: </li>
                     97: <li><a name="cvsupdate" />
                     98:     <h3>Using CVS: Updating files (cvs update -d)</h3>
                     99: <p>
                    100: After entering your CVS source tree (<tt>cd loncapa</tt>),
                    101: you should frequently update the software changes that
1.19      harris41  102: other people have made.  This is done with the <tt>update</tt> command.
                    103: </p>
                    104: <p>
1.17      harris41  105: <font color="#008800">
1.19      harris41  106: <tt>
1.17      harris41  107: cvs update -d
1.19      harris41  108: </tt>
1.17      harris41  109: </font>
                    110: </p>
                    111: <p>
1.19      harris41  112: The <tt>cvs update</tt> command creates output
                    113: as it updates your CVS source tree.  Common flags are
                    114: 'U' and 'P'; they indicate that a file in your
                    115: <tt>loncapa</tt> directory is now updated with
                    116: changes made by another programmer.
                    117: </p>
                    118: <p>
1.17      harris41  119: <font color="#880000">
1.19      harris41  120: <tt>`U FILE'</tt></font></p>
                    121: <blockquote><font color="#880000">
                    122: The file was brought up to date in your <tt>loncapa</tt>.
                    123: <br />'U' is done for:
                    124: <br />* any file that exists in the repository but not in your source, and
                    125: <br />* files that you have not changed but are not the most recent versions
                    126: available in the repository.
                    127: <br />The network behavior of 'U' is that the entire new file is uploaded
                    128: from the CVS server.
                    129: </font></blockquote>
                    130: <p><font color="#880000"><tt>
1.17      harris41  131: `P FILE'
1.19      harris41  132: </tt></font></p>
                    133: <blockquote><font color="#880000">
                    134: Like `U', but the CVS server sends a patch instead of an entire file.
                    135: </font></blockquote>
                    136: <p>
                    137: 'U' and 'P' essentially accomplish the same thing, just in
                    138: different ways.
1.17      harris41  139: </p>
                    140: <p>
1.19      harris41  141: Usually, when you do not <tt>cvs commit</tt> your code changes,
                    142: the <tt>update</tt> command will tell you that you have modified
1.17      harris41  143: your file with the 'M' flag.
1.19      harris41  144: </p>
                    145: <p><font color="#880000"><tt>
1.17      harris41  146: `M FILE'
1.19      harris41  147: </tt></font></p>
                    148: <blockquote><font color="#880000">
                    149:      The file is modified in your working <tt>loncapa</tt> directory.
                    150:      This is probably based on changes you made and have not yet
                    151:      "cvs commit"-ed.
                    152: </font></blockquote>
                    153: <p>
1.17      harris41  154: Sometimes, it will occur that:
1.19      harris41  155: </p>
1.17      harris41  156: <ul>
                    157: <li>you have modified a file and not yet committed it</li>
                    158: <li>someone else *has* modified a file and *has* committed it</li>
                    159: </ul>
1.19      harris41  160: <p>
1.17      harris41  161: Generally speaking, this is <strong>your</strong> fault.  It is your
                    162: responsibility to resolve conflicts.  <tt>cvs update</tt> informs
                    163: you of a conflict with the 'C' flag.
1.19      harris41  164: </p>
                    165: <p><font color="#880000"><tt>
1.17      harris41  166: `C FILE'
1.19      harris41  167: </tt></font></p>
                    168: <blockquote><font color="#880000">
1.17      harris41  169:      A conflict was detected while trying to merge your changes to FILE
                    170:      with changes from the source repository.
1.19      harris41  171: </font></blockquote>
                    172: <p>
1.17      harris41  173: You will need to open the file and examine it; CVS will have added in
                    174: markup tags like "&lt;&lt;&lt;&lt;&lt;&lt;" to tell you about the merging
                    175: conflicts.  (Sometimes, CVS will intelligently merge in other changes and
                    176: give you the 'M' flag, but many times you will have to manually edit
                    177: the file as just described.)
                    178: </p>
                    179: </li>
                    180: <li><a name="cvssave" />
                    181:     <h3>Using CVS: Saving files (cvs commit)</h3>
                    182: <p>
                    183: <tt>cvs commit</tt> works to submit changes to an <strong>existing</strong>
                    184: file within the repository.  If a file does not currently exist, then you
                    185: will first need to <tt>cvs add</tt> it as described in the following
                    186: section.
                    187: </p>
1.19      harris41  188: Running the <tt>cvs commit</tt> command without additional arguments will
                    189: commit all of your changes within the current directory and subdirectories.
                    190: <p><font color="#008800"><tt>
1.17      harris41  191: cvs commit
1.19      harris41  192: </tt></font></p>
1.17      harris41  193: <p>
                    194: A more precise approach to using <tt>cvs commit</tt> is to pass it specific
                    195: file names.  (Usually you should do this.)
1.19      harris41  196: </p>
                    197: <p><font color="#008800"><tt>
1.17      harris41  198: cvs commit FILENAME
1.19      harris41  199: </tt></font></p>
1.17      harris41  200: <p>
1.19      harris41  201: Note that CVS typically invokes the
                    202: <a href="http://www.eng.hawaii.edu/Tutor/vi.html">vi</a> editor and solicits
                    203: comments about your latest changes to the software.   Your comments should be
1.17      harris41  204: both short yet uniquely descriptive.  For example:
1.19      harris41  205: </p>
1.17      harris41  206: <ul>
                    207: <li><strong>BAD</strong> - "made some changes and am drinking soda"</li>
                    208: <li><strong>GOOD</strong> - "implemented syntax checking of perl scripts
                    209: with -c flag"</li>
                    210: </ul>
                    211: </li>
                    212: <li><a name="cvsadd" />
                    213:     <h3>Using CVS: Adding files (cvs add)</h3>
1.19      harris41  214: <p><font color="#008800"><tt>
1.17      harris41  215: cvs add FILENAME
1.19      harris41  216: </tt></font></p>
1.17      harris41  217: <p>
                    218: Then you can run <tt>cvs commit FILENAME</tt> and this file will
                    219: become an "official" part of LON-CAPA.
                    220: </p>
                    221: </li>
                    222: <li><a name="cvsadddir" />
                    223:     <h3>Using CVS: Adding directories (cvs add/import)</h3>
1.19      harris41  224: <p><font color="#008800"><tt>
1.17      harris41  225: cvs add DIRECTORYNAME
1.19      harris41  226: </tt></font></p>
1.17      harris41  227: <p>
                    228: There is no need to run <tt>cvs commit</tt>.  Directories immediately
                    229: become part of the LON-CAPA CVS source tree by only using the <tt>cvs add</tt>
                    230: command.
                    231: </p>
1.19      harris41  232: <p>
                    233: You should not ordinarily need to use the <tt>cvs import</tt> command.
                    234: If misused, <tt>cvs import</tt> can lead to the loss of code within
                    235: the repository.
                    236: </p>
1.17      harris41  237: </li>
                    238: <li><a name="cvsnotsure" />
                    239:     <h3>Using CVS: What to do when you're not sure about your files
                    240:         (cvs update -d)</h3>
                    241: <p>
1.19      harris41  242: Once in a while, multiple programmers may be working on the
1.17      harris41  243: same file.  Most conflicts are avoidable if everybody regularly
                    244: <strong>commits</strong> their changes AND if everybody
                    245: regularly <strong>updates</strong> the CVS source tree they are working on.
                    246: </p>
                    247: <p>
1.19      harris41  248: If you are absent from programming for a few days, and
1.17      harris41  249: <strong>fail</strong> to run <tt>cvs update -d</tt> on your CVS source
                    250: repository, you have only yourself to blame if you find yourself writing
                    251: code in a file that is not up-to-date.
                    252: </p>
                    253: </li>
1.19      harris41  254: </ul></li>
1.17      harris41  255: <li><a name="makeHTML" />
1.19      harris41  256:     <h2>Viewing the software (make HTML)</h2>
                    257: <p>
1.17      harris41  258: <strong>Commands</strong>
1.19      harris41  259: </p>
                    260: <p><font color="#008800"><tt>
                    261: cd loncom/build<br />
                    262: rm -Rf HTML <i>(or alternatively, "make clean")</i><br />
                    263: make HTML<br />
                    264: cd HTML<br />
                    265: <i>(look at the index.html file with a web browser such as Netscape)</i>
                    266: </tt></font></p>
                    267: <p>
1.17      harris41  268: <strong>General description of what happens</strong>
1.19      harris41  269: </p>
1.17      harris41  270: <p>
1.6       harris41  271: This is the actual make target code.
1.19      harris41  272: </p>
1.17      harris41  273: <pre>
1.6       harris41  274: <!-- LONCAPA MAKETARGET=HTML START -->
                    275: HTML:
1.17      harris41  276: 	install -d HTML
                    277: 	cp $(SOURCE)/doc/loncapafiles/*.gif HTML
                    278: 	cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
                    279: 	perl lpml_parse.pl html development default "$(SOURCE)" '$(TARGET)' \
                    280: 	> HTML/index.html
1.6       harris41  281: <!-- LONCAPA MAKETARGET=HTML END -->
1.17      harris41  282: </pre>
1.19      harris41  283: <p>
1.7       harris41  284: What basically happens is that specially marked-up data in the LON-CAPA
1.19      harris41  285: cvs repository file <tt>doc/loncapafiles/loncapafiles.lpml</tt> is parsed
                    286: into a more viewable format by <tt>loncom/build/lpml_parse.pl</tt>.  The
                    287: resulting file gives a very well organized view of all the files, directories,
1.7       harris41  288: links, ownerships, permissions, and brief documentation of what each
                    289: file does.
1.17      harris41  290: </p>
1.19      harris41  291: </li>
1.17      harris41  292: <li><a name="makebuild" />
                    293:     <h2>Compiling the software (make build)</h2>
                    294: <strong>Commands</strong>
1.19      harris41  295: <p><font color="#008800"><tt>
1.8       harris41  296: cd loncom/build
1.19      harris41  297: <br />make build
                    298: </tt></font></p>
1.17      harris41  299: <p>
                    300: <strong>General description of what happens</strong>
                    301: </p>
                    302: <p>
1.8       harris41  303: This is the actual make target code.
1.19      harris41  304: </p>
1.17      harris41  305: <pre>
1.13      harris41  306: <!-- LONCAPA MAKETARGET=build START -->
1.17      harris41  307: build: Makefile.build pod2html.sh pod2man.sh
                    308: 	echo -n "" > WARNINGS
                    309: 	make -f Makefile.build all
                    310: 	make warningnote
                    311: 
                    312: Makefile.build: $(SOURCE)/doc/loncapafiles/loncapafiles.lpml lpml_parse.pl
                    313: 	cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
                    314: 	perl lpml_parse.pl build $(CATEGORY) $(DIST) "$(SOURCE)" "$(TARGET)" \
                    315: 	> Makefile.build
1.13      harris41  316: <!-- LONCAPA MAKETARGET=build END -->
1.17      harris41  317: </pre>
1.19      harris41  318: <p>
1.17      harris41  319: <tt>loncom/build/lpml_parse.pl</tt> reads in all the build information out
                    320: of <tt>doc/loncapafiles/loncapafiles.lpml</tt>.  A new Makefile named
                    321: <tt>loncom/build/Makefile.build</tt> is dynamically constructed.
1.19      harris41  322: This dynamically generated Makefile is then used to build and compile
                    323: all the software targets from source.  This can take several minutes
                    324: (it depends on the speed of the machine you compile with).
1.17      harris41  325: </p>
1.19      harris41  326: <p>
1.17      harris41  327: <strong>Example</strong>
1.19      harris41  328: </p>
1.17      harris41  329: <p>
                    330: Here is information for one file <tt>tth.so</tt> provided in
                    331: <tt>doc/loncapafiles/loncapafiles.lpml</tt>.
1.19      harris41  332: </p>
1.17      harris41  333: <pre>
                    334: &lt;file&gt;
                    335: &lt;source&gt;loncom/homework/caparesponse/capa.so&lt;/source&gt;
                    336: &lt;target dist='default'&gt;usr/lib/perl5/site_perl/5.005/capa.so&lt;/target&gt;
                    337: &lt;target dist='redhat7 redhat7.1'&gt;usr/lib/perl5/site_perl/5.6.0/capa.so&lt;/target&gt;
                    338: &lt;categoryname&gt;system file&lt;/categoryname&gt;
                    339: &lt;description&gt;
                    340: shared library file for dynamic loading and unloading
                    341: &lt;/description&gt;
                    342: &lt;build trigger='always run'&gt;
                    343: loncom/homework/caparesponse/commands
                    344: &lt;/build&gt;
                    345: &lt;dependencies&gt;
                    346: caparesponse.c;
                    347: caparesponse.pm;
                    348: README;
                    349: Makefile.PL;
                    350: capa.i;
                    351: commands
                    352: &lt;/dependencies&gt;
                    353: &lt;/file&gt;
                    354: </pre>
1.19      harris41  355: <p>
1.17      harris41  356: <tt>loncom/build/lpml_parse.pl</tt> sees the <b>build</b> tags and sets up
                    357: a dynamic file <tt>Makefile.build</tt> to run the command inside the
                    358: <b>build</b> tags.  The files listed inside the <b>dependencies</b> tags
                    359: are included in the <tt>Makefile.build</tt> so as to determine whether
                    360: or not there is a need to compile.
                    361: </p>
                    362: <p>
                    363: Here is an example of a dynamically generated <tt>Makefile.build</tt>
                    364: that builds two LON-CAPA files (one of which is <tt>tth.so</tt>).
1.19      harris41  365: </p>
1.17      harris41  366: <pre>
1.8       harris41  367: all: ../homework/caparesponse/capa.so ../modules/TexConvert/tthperl/tth.so 
                    368: 
1.17      harris41  369: ../homework/caparesponse/capa.so:  ../homework/caparesponse/caparesponse.c ../homework/caparesponse/caparesponse.pm alwaysrun
1.8       harris41  370:         cd ../homework/caparesponse/; sh ./commands
                    371: 
1.17      harris41  372: ../modules/TexConvert/tthperl/tth.so:  ../modules/TexConvert/tthperl/../tthdynamic/tthfunc.c ../modules/TexConvert/tthperl/../ttmdynamic/ttmfunc.c
1.8       harris41  373:         cd ../modules/TexConvert/tthperl/; sh ./commands
                    374: 
                    375: alwaysrun:
1.17      harris41  376: </pre>
                    377: </li><li><a name="loncapafiles" />
1.19      harris41  378:     <h2>Adding/removing files from the LON-CAPA installation
                    379:         (doc/loncapafiles/loncapafiles.html)</h2>
                    380: <p>
1.17      harris41  381: <strong>To add and remove (and alter)</strong>
1.19      harris41  382: </p>
1.17      harris41  383: <p>
1.11      harris41  384: All that you have to do to alter the behavior of the installation is
1.17      harris41  385: edit a single file (<tt>doc/loncapafiles/loncapafiles.lpml</tt>).
1.11      harris41  386: Adding, removing, and altering files requires proper attention
                    387: to the syntax of file format of course.
1.17      harris41  388: </p>
1.19      harris41  389: <p>
1.17      harris41  390: <strong>File Format</strong>
1.19      harris41  391: </p>
                    392: <p>
                    393: The preceding <a href="#makebuild">"make build"</a> documentation
1.17      harris41  394: gives an example of a <b>file</b> entry describing one particular file.
                    395: All data within <tt>loncapafiles.lpml</tt> is specified according
                    396: to markup tags.  The format and syntax of <tt>loncapafiles.lpml</tt>
1.11      harris41  397: is currently best described by the HTML documentation code at the beginning of
                    398: loncapafiles.html (as well as, by example, seeing how various
                    399: information is coded).  All in all, the syntax is quite simple.
1.17      harris41  400: </p>
1.19      harris41  401: <p>
1.17      harris41  402: <strong>Philosophy and notes (the thing nobody reads)</strong>
1.19      harris41  403: </p>
1.17      harris41  404: <p>
1.11      harris41  405: Packaging the software from CVS onto a machine file system requires many
                    406: things:
1.19      harris41  407: </p>
1.17      harris41  408: <ul>
                    409: <li>documenting every component of the software,</li>
                    410: <li>handling CVS <u>source</u> to file system <u>target</u> information,</li>
                    411: <li>handling (according to a hierarchical scheme of grouping) file
                    412: ownership and permissions,</li>
                    413: <li>handling (according to a hierarchical scheme of grouping) directory
                    414: ownership and permissions,</li>
                    415: <li>handling symbolic links,</li>
1.19      harris41  416: <li>providing for multiple options of installation targets (e.g. RedHat versus
                    417: Debian),</li>
1.17      harris41  418: <li>providing for different file ownerships and permissions to apply
                    419: to the same file,</li>
                    420: <li>allowing system software documentation to be automatically generated
                    421: (see information on <a href="#makeHTML">"make html"</a>),</li>
                    422: <li>providing information in an easily adjustable form as new demands
                    423: are made on the software packaging system,</li>
                    424: <li>providing software package information (for RPM),</li>
                    425: <li>having information in a format that allows for troubleshooting
                    426: the current status of the machine file system,</li>
                    427: <li>allow for changes to the structure of the CVS repository,</li>
                    428: <li>and something that is simple enough for any one to immediately work with,
1.19      harris41  429: without having to learn any specifics (or hidden traps) of complicated
                    430: Makefile's or a new macro language (m4?).</li>
1.17      harris41  431: </ul>
                    432: <p>
1.11      harris41  433: I looked into, and tried, different ways of accomplishing the above
                    434: including automake and recursive make.  The automake system seemed quite
                    435: complicated (and needlessly so in terms of this project since, by and large,
                    436: it works to coordinate many different types of build/compilation parameters
1.19      harris41  437: whereas we are more concerned with installation parameters).  The other
                    438: alternative, recursive make,
                    439: has significant deficiencies since not all the information
1.11      harris41  440: is kept in one place, and there are significant levels of dependency
                    441: between all the things that must be done to keep software packaging
                    442: up to date.  A particularly convincing article I found when looking into
                    443: much of this was
1.17      harris41  444: <a href="http://www.pcug.org.au/~millerp/rmch/recu-make-cons-harm.html">
1.19      harris41  445: "Recursive Make Considered Harmful" by Peter Miller</a>.  Other complications
                    446: were that, at the time, it was unclear as to what categories
1.11      harris41  447: of software files we had, and whether or not the directory structure
                    448: of CVS would remain constant.  With an ever-developing directory structure
                    449: to CVS, I preferred to organize the information on a per-file basis
1.19      harris41  450: as opposed to a per-directory basis.
1.11      harris41  451: Additionally, a standard big Makefile assumes certain "normalcy" to
                    452: the directory structure of different potential operating system directories
                    453: (RedHat vs. Debian).
1.17      harris41  454: </p>
                    455: <p>
1.19      harris41  456: If you take time to look at <tt>loncapafiles.lpml</tt>
1.17      harris41  457: (and perhaps run the <a href="#makeHTML">make HTML</a> command)
1.11      harris41  458: you will find that the organizing information according to the markup
1.17      harris41  459: syntax in <tt>loncapafiles.lpml</tt> is simple.  Simple is good.
                    460: </p>
                    461: <p>
                    462: <tt>loncom/build/lpml_parse.pl</tt> is the script (invoked automatically
                    463: by the various targets in <tt>loncom/build/Makefile</tt>) that reads
                    464: <tt>doc/loncapafiles/loncapafiles.lpml</tt>.  <tt>lpml_parse.pl</tt>
1.11      harris41  465: is capable of reading and returning different types of information
1.17      harris41  466: from <tt>loncapafiles.lpml</tt> depending on how <tt>lpml_parse.pl</tt>
                    467: is invoked.  <tt>lpml_parse.pl</tt> has yet to have introduced new sources
1.11      harris41  468: of error, and has been tested in quite a number of ways.  As with
                    469: any parser however, I remain paranoid.
1.17      harris41  470: </p>
                    471: <p>
1.19      harris41  472: Finally, some notes on the development.
1.17      harris41  473: <tt>lpml_parse.pl</tt> is very fast and styled after a state-based SAX-like
1.19      harris41  474: approach.  I do eventually want to use a real XML/XSLT approach, however
                    475: I hesitate to make everyone everywhere install something like
                    476: <a href="http://search.cpan.org/search?dist=XML-Xalan">XML::Xalan</a>.
                    477: Also note that <tt>loncapafiles.lpml</tt> has a 
                    478: DTD (<tt>loncom/build/lpml.dtd</tt>) against which it is valid.
                    479: I would also like to use more ENTITY's inside <tt>lpml.dtd</tt> but currently
                    480: the perl XML modules available at CPAN do not digest complex ENTITY's that
                    481: well.
1.17      harris41  482: </p>
                    483: <p>
                    484: The <tt>lpml_parse.pl</tt>-<tt>loncapafiles.lpml</tt> 
1.19      harris41  485: combination has been highly efficient and error-free.
1.17      harris41  486: </p>
                    487: </li><li><a name="configversusnonconfig" />
                    488:     <h2>Configurable files versus non-configurable files</h2>
                    489: <p>
                    490: <strong>Machine-specific information is the difference</strong>
                    491: </p>
                    492: <p>
1.12      harris41  493: The current list of configurable files for the LON-CAPA system is
1.19      harris41  494: <tt>/etc/httpd/conf/loncapa.conf</tt>,
                    495: <tt>/etc/ntp.conf</tt>,
                    496: <tt>/etc/krb.conf</tt>,
                    497: <tt>/etc/ntp/step-tickers</tt>,
                    498: <tt>/home/httpd/html/res/adm/includes/copyright.tab</tt>,
                    499: <tt>/home/httpd/html/res/adm/includes/un_keyword.tab</tt>,
                    500: <tt>/home/httpd/hosts.tab</tt>, and
                    501: <tt>/home/httpd/spare.tab</tt>.
1.17      harris41  502: </p>
                    503: <p>
1.12      harris41  504: All of these configurable files contain machine-specific information.
1.19      harris41  505: For instance, the overall LON-CAPA system relies on unique host IDs such
1.12      harris41  506: as msua3, s1, s2, msul1, and 103a1 (specified as a "PerlSetVar lonHostID"
1.19      harris41  507: field within <tt>/etc/httpd/conf/loncapa.conf</tt>).
1.12      harris41  508: Non-configurable files simply do NOT have machine-specific information.
1.19      harris41  509: </p>
                    510: <p>
1.17      harris41  511: <strong>The impact on updating software</strong>
1.19      harris41  512: </p>
1.17      harris41  513: <p>
1.19      harris41  514: What this means in terms of software updating is that:
                    515: </p>
1.17      harris41  516: <ul>
                    517: <li>non-configurable files can be simply overwritten with newer versions
                    518: (without "anything" else to worry about),</li>
                    519: <li>and configurable files must follow these steps to be safely
1.19      harris41  520: overwritten:
1.17      harris41  521: <ol>
1.19      harris41  522: <li>have their machine-specific information saved,</li>
1.17      harris41  523: <li>be overwritten, and then</li>
1.19      harris41  524: <li>have their machine-specific information restored.</li>
1.17      harris41  525: </ol>
1.19      harris41  526: </li>
1.17      harris41  527: </ul>
1.19      harris41  528: </li>
1.17      harris41  529: <li><a name="makeinstall" />
                    530:     <h2>Updating the non-configurable files on your machine (make install)</h2>
                    531: <strong>Commands</strong>
1.19      harris41  532: <p><font color="#008800"><tt>
1.13      harris41  533: cd loncom/build
1.19      harris41  534: <br />make install
                    535: </tt></font></p>
1.17      harris41  536: <p>
                    537: <strong>General description of what happens</strong>
                    538: </p>
                    539: <p>
1.13      harris41  540: This is the actual make target code.
1.19      harris41  541: </p>
1.17      harris41  542: <pre>
1.13      harris41  543: <!-- LONCAPA MAKETARGET=install START -->
1.17      harris41  544: install: TEST_hosts_tab Makefile.install Makefile
                    545: 	echo -n "" > WARNINGS
                    546: 	make -f Makefile.install SOURCE="$(SOURCE)" TARGET="$(TARGET)" \
                    547: 	directories
                    548: 	make -f Makefile.install SOURCE="$(SOURCE)" TARGET="$(TARGET)" files
                    549: 	make -f Makefile.install SOURCE="$(SOURCE)" TARGET="$(TARGET)" links
                    550: 	make SOURCE="$(SOURCE)" TARGET="$(TARGET)" \
                    551: 	NORESTORECONF="$(NORESTORECONF)" configinstall
                    552: 	make postinstall
                    553: 	make warningnote
                    554: 	echo "You can run 'make test' to see if your system is ready to go!"
                    555: 
                    556: Makefile.install: $(SOURCE)/doc/loncapafiles/loncapafiles.lpml lpml_parse.pl
                    557: 	cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
                    558: 	perl lpml_parse.pl install $(CATEGORY) $(DIST) "$(SOURCE)" \
                    559: 	"$(TARGET)" > Makefile.install
1.13      harris41  560: <!-- LONCAPA MAKETARGET=install END -->
1.17      harris41  561: </pre>
1.19      harris41  562: <p>
                    563: For safety reasons (so as to preserve a machine's configuration),
1.13      harris41  564: configuration files are NOT installed during this step.  This means
1.19      harris41  565: that files such as <tt>/etc/httpd/conf/loncapa.conf</tt>,
                    566: <tt>/home/httpd/html/res/adm/includes/copyright.tab</tt>, and
1.20    ! harris41  567: <tt>/home/httpd/spare.tab</tt> are not overwritten, but remain as old,
1.19      harris41  568: non-updated copies.  (To automatically update these files and save/restore
1.13      harris41  569: their encoded machine configuration, you must run "make configinstall").
1.17      harris41  570: </p>
1.19      harris41  571: </li>
1.17      harris41  572: <li><a name="makeconfiginstall" />
                    573:   <h2>Updating the configurable files on your machine (make configinstall)</h2>
                    574: <strong>Commands</strong>
1.19      harris41  575: <p><font color="#008800"><tt>
1.13      harris41  576: cd loncom/build
                    577: make configinstall
1.19      harris41  578: </tt></font></p>
                    579: <p>
1.17      harris41  580: <strong>General description of what happens</strong>
1.19      harris41  581: </p>
1.17      harris41  582: <p>
1.13      harris41  583: This is the actual make target code.
1.19      harris41  584: </p>
1.17      harris41  585: <pre>
1.13      harris41  586: <!-- LONCAPA MAKETARGET=configinstall START -->
1.18      harris41  587: configinstall: Makefile.configinstall
                    588: 	make -f Makefile.configinstall SOURCE="$(SOURCE)" TARGET="$(TARGET)" \
                    589: 	configfiles
                    590: 	if (test "0" = $(NORESTORECONF)); then \
                    591: 	perl loncaparestoreconfigurations suffix .lpmlnew; fi
                    592: 
                    593: Makefile.configinstall: $(SOURCE)/doc/loncapafiles/loncapafiles.lpml lpml_parse.pl
                    594: 	cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
                    595: 	perl lpml_parse.pl configinstall $(CATEGORY) $(DIST) "$(SOURCE)" \
                    596: 	"$(TARGET)" > Makefile.configinstall
1.13      harris41  597: <!-- LONCAPA MAKETARGET=configinstall END -->
1.17      harris41  598: </pre>
1.19      harris41  599: <p>
1.13      harris41  600: Configuration files are installed during this step.  This means
1.19      harris41  601: that files such as <tt>/etc/httpd/conf/loncapa.conf</tt>,
                    602: <tt>/home/httpd/html/res/adm/includes/copyright.tab</tt>, and
                    603: <tt>/home/httpd/spare.tab</tt> are overwritten.  Before being overwritten,
1.13      harris41  604: a backup copy is made though.  Information is read out of these
                    605: backup copies and restored to the new files by the
1.17      harris41  606: <tt>loncaparestoreconfigurations</tt> script.  To ensure that
1.13      harris41  607: new file permissions and ownerships are installed, a final set of
1.17      harris41  608: <tt>chown</tt> and <tt>chmod</tt> commands are called for each of 
1.13      harris41  609: the configuration files.
1.17      harris41  610: </p>
                    611: <p>
                    612: <strong>For the truly paranoid</strong>
                    613: </p>
                    614: <p>
1.13      harris41  615: If you are truly paranoid, you can just make the
1.17      harris41  616: <tt>Makefile.configinstall</tt> file and then save, copy,
1.13      harris41  617: and restore all the configuration values yourself.
1.17      harris41  618: <tt>loncaparestoreconfigurations</tt> is pretty smart though, has yet to
1.19      harris41  619: fail, and besides, when needed, backup copies are made.
1.17      harris41  620: </p>
1.18      harris41  621: </li><li><a name="makeRPM" />
1.17      harris41  622:     <h2>Building RPMs (make RPM)</h2>
1.18      harris41  623: <p>
1.19      harris41  624: LON-CAPA is currently installed through "intelligent tarballs".
                    625: What I am describing now is part of an earlier (and perhaps future) effort
                    626: involving RPMs.
1.17      harris41  627: </p>
                    628: <p>
1.19      harris41  629: <strong>Commands</strong>
1.17      harris41  630: </p>
1.19      harris41  631: <p><font color="#008800"><tt>
                    632: cd loncom/build<br />
                    633: rm -Rf BinaryRoot <i>(or alternatively, "make clean")</i><br />
                    634: make RPM<br />
                    635: <i>(to subsequently install, you can type commands like
                    636: "rpm -Uvh --force LON-CAPA-base-3.1-1.i386.rpm")</i>
                    637: </tt></font></p>
1.17      harris41  638: <p>
                    639: <strong>Configuration files</strong>
                    640: </p>
                    641: <p>
1.13      harris41  642: Configuration files are automatically saved with the file suffix
1.19      harris41  643: ".rpmsave".  So <tt>/etc/httpd/conf/loncapa.conf</tt> is saved as 
                    644: <tt>/etc/httpd/conf/loncapa.conf.rpmsave</tt>.
                    645: The <tt>loncaparestoreconfigurations</tt> script should work to restore
                    646: configurations in this case.  However, please note that if you install an RPM
                    647: twice without restoring your configuration, you will overwrite the
1.13      harris41  648: ".rpmsave" files.
1.17      harris41  649: </p>
                    650: <p>
                    651: <strong>General description of what happens</strong>
                    652: </p>
                    653: <p>
1.13      harris41  654: This is the actual make target code.
1.19      harris41  655: </p>
1.17      harris41  656: <pre>
1.13      harris41  657: <!-- LONCAPA MAKETARGET=RPM START -->
1.17      harris41  658: RPM: BinaryRoot base_rpm_file_list
                    659: 	cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
                    660: 	perl lpml_parse.pl make_rpm $(CATEGORY) $(DIST) $(SOURCE) $(TARGET) \
                    661: 	> base_customizerpm.xml
                    662: 	cat base_rpm_file_list.txt | perl make_rpm.pl base 3.2 '' '' \
                    663: 	BinaryRoot base_customizerpm.xml
                    664: 
                    665: BinaryRoot: base_rpm_file_list
                    666: 	make TARGET='BinaryRoot' NORESTORECONF='1' install
1.13      harris41  667: 
1.17      harris41  668: base_rpm_file_list:
                    669: 	cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
                    670: 	perl lpml_parse.pl rpm_file_list $(CATEGORY) $(DIST) $(SOURCE) \
                    671: 	'BinaryRoot' | sort > base_rpm_file_list.txt
1.16      harris41  672: <!-- LONCAPA MAKETARGET=RPM END -->
1.17      harris41  673: </pre>
1.19      harris41  674: <p>
1.17      harris41  675: A <tt>BinaryRoot</tt> directory is generated that reflects the locations,
1.13      harris41  676: ownerships, permissions, and contents for all the CVS source
                    677: files, compiled binaries, directories, and links as they should eventually
                    678: occur on the '/' filesystem location.
1.17      harris41  679: </p>
                    680: <p>
1.19      harris41  681: <tt>loncom/build/make_rpm.pl</tt> (also available at
                    682: <a href="http://www.cpan.org">CPAN</a>) is robust (tested over the
1.13      harris41  683: span of months) and, unlike other automated RPM-builders, cleanly
                    684: builds new RPMs without any after-effect of temporary files left
1.19      harris41  685: on the system.  The generated RPM is labeled in the format
                    686: LON-CAPA-base-(VERSION)-1.i386.  VERSION is specified inside the
                    687: Makefile.
1.17      harris41  688: </p>
1.18      harris41  689: </li>
1.17      harris41  690: </ol>
                    691: </body>
                    692: </html>
1.12      harris41  693: 
                    694: 
                    695: 

FreeBSD-CVSweb <freebsd-cvsweb@FreeBSD.org>