--- doc/packaging/index.html 2000/05/23 16:33:28 1.2 +++ doc/packaging/index.html 2002/07/18 00:30:42 1.3 @@ -1,321 +1,68 @@ - - -LON-CAPA Packaging - - -

LON-CAPA Packaging and Filesystem Management

-Functional Model Description (Scott Harrison, April 2000) -

-

    -
  1. Synopsis -
  2. Summary of Data Store Objects (e.g. CVS, RPM, LCIC) -
  3. Summary of Processes (e.g. making a CD image, upgrading a system, altering package composition) -
  4. Summary of Actor Objects (e.g. cron jobs, programmers, head packaging -developer) -
  5. Functional Model Diagram -
  6. Description of the Functional Model Diagram -
      -
    1. Data Store Objects -
    2. Processes -
    3. Actor Objects -
    -
  7. Frequently Asked Questions -
-

-

1. Synopsis

-Up to this point, dozens of scripts have been written -to automate various elements of the LON-CAPA installation, upgrade, and software development process. A number of significant issues -relate to this effort. Security on the system is largely a matter of limiting -unnecessary network services, monitoring the filesystem of LON-CAPA -servers to make full account for the presence of every file on the system, -and securely communicating potential problems to system administrators of a given -LON-CAPA server machine. -

-This document is the functional model description which describes the computations -and expected results necessary to manage packaging and filesystem management -for LON-CAPA both now and in the future in a manner that is synchronous -with software development, installations, and upgrades. Promulgation of coding -changes, version control, and maintaining a handle on security and network optimization -issues (such as introducing SAMBA and NetaTalk into the network architecture) have -been some of the reasons for introducing a more strongly defined characterization -of all of these processes involved with LON-CAPA software packaging and file system -management. -

-

2. Summary of Data Store Objects

- -

-

3. Summary of Processes

- -

-

4. Summary of Actor Objects

- -

-

5. Functional Model Diagram

- -

-

6. Description of the Functional Model Diagram

-I will now describe a semi-chronological viewpoint of how all the different processes take place -so as to better understand both the specifics of the functional model, as well as the pertinent -dynamical and object-oriented characteristics of the algorithm. -

-Every week on Thursday at 3 in the morning, a cron job associated with the RPM packaging system on -the main packaging computer updates the build tree of all LON-CAPA source RPMs based on files in the central -CVS file repository. It does this by initiating the process LON_CAPA_update_RPM_build_tree. This process -requests and receives files from the central CVS file repository, and updates all the source RPM file trees. RPM packages -are then "bundled" by issuing build commands to the "Red Hat Package Manager" (rpm -b ...). -

-The LON_CAPA_update_RPM_build_tree process calls the LON_CAPA_make_CD_image process which writes a -CD image into a single file on the main packaging system. This is followed by the process LON_CAPA_post_CD_image -which mounts and posts the CD image file onto a web accessible location. -

-The LON_CAPA_upgrade_sys process is launched by a cron job specific to every individual -computer present in the software development cluster. Each individual computer checks the main packaging computer to see -if a system upgrade is needed. System upgrade information is passed to the LCIC on each specific computer (LON-CAPA -Integrity Checker) so that integrity checking is done as a function of the most recent system upgrade. -

-Finally, on Sunday at 3 in the morning, a cron job associated with each computer launches the LCIC integrity checker -system which resyncs itself based on the RPMs and files present on the system as well as information pertaining to -the expected status of the system (in terms of upgrade, RPM composition, etc). The results of the LCIC integrity checker -"re-syncing" are posted under a password controlled directory. -

-

7. Frequently Asked Questions

-

-

    -
  1. What does LON-CAPA stand for? -
    The LearningOnline Network with a Computer-Assisted Personalized Approach -

    -

  2. What is the general structure of the CVS repository? -
    These are the current directories: -
    loncom - server configuration and application files associated with LON-CAPA library and access servers -
    lonline - lecture online -
    CAPA - files associated with CAPA (homework processing, grading, homework generating, exam and quiz administration) -
    doc - documentation -
    modules - custom built software modules meant for use with the LON-CAPA project -
    rat - the "RAT"; resource assembly tool -
    packaging - files (including this one) specific to packaging and filesystem management of LON-CAPA -

    -

  3. How does a programmer set himself up to use CVS? -
    -
    -you will need to have the CVSROOT enviroment variable set to
-:pserver:yourname@zaphod.lite.msu.edu:/home/cvs
-To do this if you are using csh or tcsh put in your .login:
-setenv CVSROOT :pserver:yourname@zaphod.lite.msu.edu:/home/cvs
-For bash put this in .bash_profile
-export CVSROOT=:pserver:yourname@zaphod.lite.msu.edu:/home/cvs
-
-You can add the "cvs login" command to your .login/.bash_profile if you wish to make this simple.
-And you can add "cvs logout" to your .logout/.bash_logout so you rember to logout.
-
-To use CVS, you must first login (cvs login) and enter in your zaphod password.
-To leave CVS, you must logout (cvs logout).
-
    -

    -

  4. How does a programmer retrieve files from the CVS repository for the first time? -
    cvs checkout [resourcename] -
    [resourcename] is one of the directories above (loncom, rat, doc, and so on) -

    -

  5. How does a programmer update his files from the CVS repository? -
    (In other words, CVS server ----to----your---->> client computer.) -
    cvs update -d [resourcename] -
    [resourcename] is one of the directories above (loncom, rat, doc, and so on) -

    -

  6. How does a programmer send his changes to the central CVS repository? -
    (In other words, client computer ----to----our---->> CVS server.) -
    cvs commit [resourcename] -
    [resourcename] is one of the directories above (loncom, rat, doc, and so on) -

    -

  7. How does a programmer add or remove files from the central CVS repository? -
    Note: this is different from "updating" or "committing" changes. This is not the alteration -of existing files. This is the ADDITION or REMOVAL of files. -
    Within "the" directory, use these two commands to add a file: -
    cvs add [filename]; cvs commit -
    Within "the" directory, use these two commands to remove a file: -
    cvs remove [filename]; cvs commit -

    -

  8. How does a programmer add or remove directories from the central CVS repository? -
    Within the upper directory, use these two commands to add a directory: -
    cvs add [directoryname]; cvs commit -
    Within "the" directory, use these two commands to remove a directory: -
    cvs remove [directoryname]; cvs commit -
    There is a slight catch that you cannot directly "add" a new major directory like -loncom, lonline, rat, doc, and so on. You must first "cvs add" it as a subdirectory, then -move that subdirectory to the top directory structure and "cvs commit". -

    -Better yet, you can cvs import your new directory into the top level area of the CVS repository. -

  9. How does a programmer "inform" the packaging developer of changes to LON-CAPA system file composition? -
    By talking to the packaging developer in person. -
    By making well-formatted changes to the packaging/file_map.txt file in the CVS repository. See below -for more information on how to format the packaging/file_map.txt file. -

    -

  10. What does "run any process" mean in the functional model? -
    Most processes are meant to run automatically and are invoked by cron jobs on various machines. -It may be the case that, for instance, the processes need to be manually invoked so as to prepare -a specific CD release. -

    -

  11. Where are the automated processes located? -
    In the directory /usr/local/bin. -
    By typing ls /usr/local/bin/LON_CAPA_*, you can view all the automated LON-CAPA processes associated with the computer systems. -

    -

  12. Where are the cron jobs located? What are their names and how reliable are they? -
    On every LON-CAPA computer there is a file /etc/cron.d/LON_CAPA_filesystem which launches standard cron jobs. -
    On the LON-CAPA main packaging computer there is a file /etc/cron.d/LON_CAPA_packager which launches cron jobs specific to the -main packaging computer. -
    On LON-CAPA software development computers, there is a file /etc/cron.d/LON_CAPA_upgrader which upgrades all computers in the -LON-CAPA software development cluster. -

    -

  13. Can automated processes be run manually? On the packaging computer? On other LON-CAPA computers? -
    Yes. To correctly manually run an automated process, look to see how the process is invoked in the /etc/cron.d directory. -

    -

  14. What are the specifics associated with the RPM packaging system on the main packaging computer? -
    The RPM packaging system on the main packaging computer introduces the following differences to the main packaging computer compared to -other computers in the software development cluster: - -

    -

  15. What are the specifics associated with the process LON_CAPA_update_RPM_build_tree? -
    This process checkouts LON-CAPA directories and files. This process reads from the packaging/file_map.txt and the -packaging/permissions.txt file. This process reads RPM .spec template files to set up the two LON-CAPA RPMs (LON-CAPA-base and LON-CAPA-auxiliary). -These updated RPMs are placed into a pool of RPMs from which to generate the customized RedHat installation/upgrade CD. -A 'comps' file is generated to customize the grouping of packages on the CD image. The CD image is written and made accessible -on the web. -

    -

  16. What are the specifics associated with the writing and web posting of the LON-CAPA system CD image on the main packaging computer? -
    There are a number of shell commands which need to be launched. (Details coming soon in this document.) -

    -

  17. What are the specifics associated with the process LON_CAPA_make_CD_image? -
    There are a number of shell commands which need to be launched. (Details coming soon in this document.) -

    -

  18. How are upgrades performed on the packaging computer? Is there any way to upgrade/install LON-CAPA on a new machine? -
    Upgrades are performed only based on package composition. New machines can be upgraded via network install or CD install. (More details coming soon.) -

    -

  19. What are the specifics associated with the process LON_CAPA_post_CD_image? -
    There are a number of shell commands which need to be launched. (Details coming soon in this document.) -

    -

  20. What are the specifics associated with the process LON_CAPA_upgrade_sys? -
    LON_CAPA_upgrade_sys does an automated network upgrade of any out-of-date packages based on the main packaging computer. -LON_CAPA_upgrade_sys initializes and invokes the LCIC (integrity checker) based on current upgrade information. -

    -

  21. What output from the LCIC (integrity checker) warrants taking corrective action? -
    Currently, the output from the LCIC consists of two files, TOTAL_FILE.txt and SUMMARY_FILE.txt. There are no immediate plans for -automating any form of response to this output. This does provide information as to whether critical system binaries or configuration -files are being tampered with. Perhaps the SUMMARY_FILE.txt should be e-mailed weekly to a system administrator at the responsible -institution. -

    -

  22. What is the correct format for the packaging/file_map.txt file? -
    Every line refers to a file or directory to install/upgrade on a LON-CAPA system. There are six fields to each line which are tab-separated. -
    -[computer-type]
-[file location on CVS repository]
-[ultimate location on LON-CAPA system]
-[package name]
-[configuration settings that need to be saved?]
-[description of file and what it does]
-
    -

    -

  23. What is the correct format for the packaging/permissions.txt file? -
    Every line refers to a file or directory to install/upgrade on a LON-CAPA system. There are four/five fields to each line which are space-separated. -File listing: -
    -[octal mode]
-[user to own file]
-[group to own file]
-[file name]
-
    -Directory listing: -
    -[octal mode]
-[user to own file]
-[group to own file]
-DIR
-[file name]
-
    - -

    -

  24. What controls version naming in terms of system upgrades? -
    This information is present in packaging/version.txt. This file consists of three tab-separated fields (version number, release number, and version -name). The release number is automatically incremented each week as a function of LON_CAPA_post_CD_image. The version number and release numbers are used -to label the LON-CAPA-base and LON-CAPA-auxiliary RPMs as well as the customized RedHat CD release. -
- - + + + + +LON-CAPA Packaging + + +

LON-CAPA Packaging and Filesystem Management

+Description (Scott Harrison, July 2002) +
    +
  1. Synopsis
    2. +
  2. Diagram
    3. +
+

Synopsis

+ + + + + + + + + + + + + + + + +
Scene 1Scene 2Scene 3
+     +     +a feature
+

++ LON-CAPA
+----------------
+= LON-CAPA (new) +

+		 +

+     +     +LON-CAPA (new)
++ System Administrator
+----------------
+= good times

 +     +     +everything is wonderful
+

+Once upon a time, a little green fellow added a "feature" +to a hypothetical piece of software called LON-CAPA +(for lack of a better term). +

+		 +"Uh-oh" said the system administrator. "I have to upgrade +my LON-CAPA system again?" + +But because the green fellow used XML to specify +the software, there weren't any errors during the +software upgrade, so everybody lived happily ever after. +
+

Diagram

+diagram of the scheme + +