[BACK]Return to loncapasqldatabase.html CVS log [TXT] [DIR] Up to [LON-CAPA] / doc / build

Diff for /doc/build/Attic/loncapasqldatabase.html between versions 1.4 and 1.6

version 1.4, 2001/02/10 18:38:37 version 1.6, 2001/02/12 17:38:13
Line 8 Line 8
 Scott Harrison  Scott Harrison
 </P>  </P>
 <P>  <P>
 Last updated: 02/10/2001  Last updated: 02/12/2001
 </P>  </P>
 <P>  <P>
 This file describes issues associated with LON-CAPA  This file describes issues associated with LON-CAPA
 and a SQL database.  and a SQL database.
 </P>  </P>
 <H3>Latest HOWTO</H3>  <H2>Latest HOWTO</H2>
 <P>  <P>
 <UL>  <UL>
 <LI>Current status of documentation</LI>  <LI>Current status of documentation</LI>
Line 28  and a SQL database. Line 28  and a SQL database.
 <LI>Example sections of code relevant to LON-CAPA</LI>  <LI>Example sections of code relevant to LON-CAPA</LI>
 </UL>  </UL>
 </P>  </P>
 <H3>Current status of documentation</H3>  <H2>Current status of documentation</H2>
 <P>  <P>
 I am going to begin documentation by inserting what notes  I am going to begin documentation by inserting what notes
 I have into this file.  I will be subsequently rearranging  I have into this file.  I will be subsequently rearranging
Line 39  current status of everything is that it Line 39  current status of everything is that it
 been minimally tested, but things need to be cleaned up  been minimally tested, but things need to be cleaned up
 and checked again!  and checked again!
 </P>  </P>
 <H3>Current status of implementation</H3>  <H2>Current status of implementation</H2>
   <P>
   Need to
   <UL>
   <LI>Installation: Fix binary file listings for user permissions and ownership.
   <LI>Installation: Make sure sql server starts, and if database does not
   exist, then create. (/etc/rc.d).
   <LI>Processes: Make sure loncron initiates lonsql on library machines.
   <LI>Read in metadata from right place periodically.
   <LI>Implement tested perl module handler.
   </UL>
 <P>  <P>
 Right now, a lot of "feasibility" work has been done.  Right now, a lot of "feasibility" work has been done.
 Recipes for manual installation and configuration have  Recipes for manual installation and configuration have
Line 70  to the point of getting this SQL softwar Line 80  to the point of getting this SQL softwar
 however there may be more optimal approaches than currently  however there may be more optimal approaches than currently
 exist.  exist.
 </P>  </P>
 <H3>Purpose within LON-CAPA</H3>  <H2>Purpose within LON-CAPA</H2>
 <P>  <P>
 LON-CAPA is meant to distribute A LOT of educational content  LON-CAPA is meant to distribute A LOT of educational content
 to A LOT of people.  It is ineffective to directly rely on contents  to A LOT of people.  It is ineffective to directly rely on contents
Line 131  THE SOLUTION: Line 141  THE SOLUTION:
   processes (lonsql's) handle the MySQL database manipulations.    processes (lonsql's) handle the MySQL database manipulations.
 </PRE>  </PRE>
 </P>  </P>
 <H3>Installation</H3>  <H2>Installation</H2>
 <P>  <P>
 Installation of the LON-CAPA SQL database normally occurs  Installation of the LON-CAPA SQL database normally occurs
 by default when using the LON-CAPA installation CD  by default when using the LON-CAPA installation CD
Line 160  from source or pre-compiled file listing Line 170  from source or pre-compiled file listing
 actual MySQL functionality on the system</TD></TR>  actual MySQL functionality on the system</TD></TR>
 </TABLE>  </TABLE>
 </P>  </P>
 <H3>Installation from source</H3>  <H2>Installation from source</H2>
 <P>  <P>
 The following set of tarballs was found to work together  The following set of tarballs was found to work together
 properly on a LON-CAPA RedHat 6.2 system:  properly on a LON-CAPA RedHat 6.2 system:
Line 174  properly on a LON-CAPA RedHat 6.2 system Line 184  properly on a LON-CAPA RedHat 6.2 system
 Installation was simply a matter of following the instructions  Installation was simply a matter of following the instructions
 and typing the several "make" commands for each   and typing the several "make" commands for each 
 </P>  </P>
 <H3>Configuration (automated)</H3>  <H2>Configuration (automated)</H2>
 <P>  <P>
 Not yet developed.  This will be part of an interface  Not yet developed.  This will be part of an interface
 present on LON-CAPA systems that can be launched by  present on LON-CAPA systems that can be launched by
 entering the command <TT>/usr/sbin/loncapaconfig</TT>.  entering the command <TT>/usr/sbin/loncapaconfig</TT>.
 </P>  </P>
 <H3>Manual configuration</H3>  <H2>Manual configuration</H2>
 <P>  <P>
 This is not complete.  This is not complete.
 </P>  </P>
Line 215  FLUSH PRIVILEGES; Line 225  FLUSH PRIVILEGES;
 Copy support-files/mysql.server to the right place on the system  Copy support-files/mysql.server to the right place on the system
 (/etc/rc.d/...).  (/etc/rc.d/...).
 </P>  </P>
 <H3>Testing</H3>  
 <P>  
 Not yet documented or formalized.  
 </P>  
 <H3>Example sections of code relevant to LON-CAPA</H3>  
 <P>  
 </P>  
   
 <H1>Old notes</H1>  
   
 <H3>How to add a user to the SQL database</H3>  
 <P>  
 <PRE>  
 start the mysql daemon as /usr/local/bin/safe_mysqld &  
 Login as root: mysql -u root -p mysql  
 enter the password as newmysql  
 add the user www: grant all priveleges on *.* to www@localhost identified by 'newmysql' with grant option;  
   
 INSERT INTO user (Host, User, Password)  
 VALUES ('localhost','www',password('newmysql'));  
   
 GRANT ALL PRIVILEGES ON *.* TO www@localhost;  
   
 FLUSH PRIVILEGES;  
   
 Here the user www has the right to grant privileges to other users.  
 This can be changed if required with a simple update command on the grant tables  
   
   
 /home/httpd/perl/perlsql/lonsql  
 /usr/local/mysql/fakeclient  
 </PRE>  
 </P>  
 <H3>To do</H3>  
 <P>  
 <PRE>  
 This is the output from scripts/mysql_install_db...  
 still some todo things (like support-files/mysql.server)  
   
 Creating db table  
 Creating host table  
 Creating user table  
 Creating func table  
 Creating tables_priv table  
 Creating columns_priv table  
   
 To start mysqld at boot time you have to copy support-files/mysql.server  
 to the right place for your system  
   
 PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !  
 This is done with:  
 /usr/local/bin/mysqladmin -u root password 'new-password'  
 See the manual for more instructions.  
   
 Please report any problems with the /usr/local/bin/mysqlbug script!  
   
 The latest information about MySQL is available on the web at http://www.mysql.com  
 Support MySQL by buying support/licenses at http://www.tcx.se/license.htmy.  
 </PRE>  
 </P>  
 <H3>Source Installation and Manual Configuration</H3>  
 <P>  <P>
   <STRONG>The Perl API</STRONG>
 <PRE>  <PRE>
 August, 29 2000; Scott Harrison; LON-CAPA  
   
 These are notes related to a Perl interface and MySQL server installation  
 on Redhat 6.1 and 6.2 boxes.  (Guy Albertelli and Harsha Jagasia  
 contributed significantly to this.)  
   
 ********************  
 * MySQL COMPONENTS *  
 ********************  
   
 There are three components to an effective MySQL installation for the  
 LON-CAPA system.  
   
 Perl::DBI module- the API "front-end"...  
                   database interface module for organizing generic  
                   database commands which are independent of specific  
                   database implementation (such as MySQL, mSQL, Postgres, etc).  
   
 Perl::MySQL module- the API "mid-section"...  
                     the module to directly interface with the actual  
                     MySQL database engine  
   
 MySQL database engine- the "back-end"...  
                        the binary installation (compiled either from source  
                        or pre-compiled file listings) which provides the  
                        actual MySQL functionality on the system  
   
 RedHat Installation-  
   
 Initially done from source:  
 DBI-1.13.tar.gz  Msql-Mysql-modules-1.2209.tar.gz  mysql-3.22.32.tar.gz  
   
 I am now using pre-compiled file listings.  
   
 There were problems with using the RedHat packages since the three  
 different RedHat packages were somewhat noncompatible with each other  
 in terms of expected file locations. (The Debian linux distribution,  
 on the other hand, has a working set of these packages).  
   
 Regardless of how we install these three components, there still remain  
 certain things which need to happen for the configuration.  
   
 *****************  
 * CONFIGURATION *  
 *****************  
   
 (Note: SOMEPASSWORD is actually set to another text string on the current  
 LON-CAPA systems.)  
   
 Configuration is needed to generate the necessary functionality for the  
 MySQL system with LON-CAPA.  
   
 The functionality needed can be understood from this example line  
 of perl code from "lonsql".  
   
    $dbh = DBI->connect( "DBI:mysql:loncapa",     $dbh = DBI->connect( "DBI:mysql:loncapa",
  "www",   "www",
  "SOMEPASSWORD",   "SOMEPASSWORD",
Line 395  FLUSH PRIVILEGES; Line 290  FLUSH PRIVILEGES;
 ** ABILITY for LON-CAPA machines to communicate with SQL databases on  ** ABILITY for LON-CAPA machines to communicate with SQL databases on
    other LON-CAPA machines     other LON-CAPA machines
   
 This is a little more intricate than might first be expected (and I probably  An up-to-date lond and lonsql.
 won't do a perfect job reciting everything in this short synopsis).  Because  </PRE>
 LON-CAPA machines will likely be handling many SQL requests at a time,  </P>
 there were some problems with current MySQL capabilities.  <H2>Testing</H2>
   <P>
 PROBLEM SITUATION:  <PRE>
   <STRONG>** TEST the database connection with my current tester.pl code
   If Server A wants data from Server B, Server A uses a lonc process to  which mimics what command will eventually be sent through lonc.</STRONG>
   send a database command to a Server B lond process.  
     lonc= loncapa client process    A-lonc= a lonc process on Server A  
     lond= loncapa daemon process  
   
                  database command  
     A-lonc  --------TCP/IP----------------> B-lond  
   
   The problem emerges that A-lonc and B-lond are kept waiting for the  
   MySQL server to "do its stuff", or in other words, perform the conceivably  
   sophisticated, data-intensive, time-sucking database transaction.  By tying  
   up a lonc and lond process, this significantly cripples the capabilities  
   of LON-CAPA servers.   
   
   While commercial databases have a variety of features that ATTEMPT to  
   deal with this, freeware databases are still experimenting and exploring  
   with different schemes with varying degrees of performance stability.  
   
 THE SOLUTION:  
   
   A separate daemon process was created that B-lond works with to  
   handle database requests.  This daemon process is called "lonsql".  
   
   So,  
                 database command  
   A-lonc  ---------TCP/IP-----------------> B-lond =====> B-lonsql  
          <---------------------------------/                |  
            "ok, I'll get back to you..."                    |  
                                                             |  
                                                             /  
   A-lond  <-------------------------------  B-lonc   <======  
            "Guess what? I have the result!"  
   
   Of course, depending on success or failure, the messages may vary,  
   but the principle remains the same where a separate pool of children  
   processes (lonsql's) handle the MySQL database manipulations.  
   
   $reply=reply(
       "querysend:SELECT * FROM general_information WHERE Id='AAAAA'",$lonID);
   </PRE>
   </P>
   <H2>Example sections of code relevant to LON-CAPA</H2>
   <P>
 Here are excerpts of code which implement the above handling:  Here are excerpts of code which implement the above handling:
   </P>
 **LONSQL  <P>
   <PRE>
   <STRONG>**LONSQL
 A subroutine from "lonsql" which establishes a child process for handling  A subroutine from "lonsql" which establishes a child process for handling
 database interactions.  database interactions.</STRONG>
   
 sub make_new_child {  sub make_new_child {
     my $pid;      my $pid;
Line 536  sub make_new_child { Line 403  sub make_new_child {
         exit;          exit;
     }      }
 }  }
   </P>
 ** LOND enabling of MySQL requestsw  <P>
   <STRONG>** LOND enabling of MySQL requests</STRONG>
   This code is part of every lond child process in the way that it parses command request syntax  <BR />
   sent to it from lonc processes.  querysend corresponds to B-lonc sending the result of the query.  This code is part of every lond child process in the
   queryreply corresponds to B-lond indicating that it has received the request and will start the  way that it parses command request syntax sent to it
   database transaction (it returns "ok" to A-lonc ($client)).  from lonc processes.  Based on the diagram above, querysend
   corresponds to B-lonc sending the result of the query.
   queryreply corresponds to B-lond indicating that it has
   received the request and will start the database transaction
   (it returns "ok" to
   A-lonc ($client)).
   <PRE>
 # ------------------------------------------------------------------- querysend  # ------------------------------------------------------------------- querysend
                    } elsif ($userinput =~ /^querysend/) {                     } elsif ($userinput =~ /^querysend/) {
                        my ($cmd,$query)=split(/:/,$userinput);                         my ($cmd,$query)=split(/:/,$userinput);
Line 563  sub make_new_child { Line 435  sub make_new_child {
    print $client "error:$!\n";     print $client "error:$!\n";
        }         }
   
   
   
 ** TEST the database connection with my current tester.pl code which mimics what command will eventually be  
    sent through lonc.  
   
 $reply=reply(  
     "querysend:SELECT * FROM general_information WHERE Id='AAAAA'",$lonID);  
 </PRE>  </PRE>
   
 </P>  </P>
 </BODY>  </BODY>
 </HTML>  </HTML>
Removed from v.1.4  
changed lines
  Added in v.1.6

FreeBSD-CVSweb <freebsd-cvsweb@FreeBSD.org>