Last updated: 02/07/2001
+This file describes issues associated with LON-CAPA and a SQL database.
-
+I am going to begin documentation by inserting what notes +I have into this file. I will be subsequently rearranging +them and editting them based on the tests that I conduct. +I am trying to make sure that documentation, installation, +and run-time issues are all consistent and correct. The +current status of everything is that it works and has +been minimally tested, but things need to be cleaned up +and checked again! +
++
+start the mysql daemon as /usr/local/bin/safe_mysqld &
+Login as root: mysql -u root -p mysql
+enter the password as newmysql
+add the user www: grant all priveleges on *.* to www@localhost identified by 'newmysql' with grant option;
+
+INSERT INTO user (Host, User, Password)
+VALUES ('localhost','www',password('newmysql'));
+
+GRANT ALL PRIVILEGES ON *.* TO www@localhost;
+
+FLUSH PRIVILEGES;
+
+Here the user www has the right to grant privileges to other users.
+This can be changed if required with a simple update command on the grant tables
+
+
+/home/httpd/perl/perlsql/lonsql
+/usr/local/mysql/fakeclient
+
+
++
+This is the output from scripts/mysql_install_db... +still some todo things (like support-files/mysql.server) + +Creating db table +Creating host table +Creating user table +Creating func table +Creating tables_priv table +Creating columns_priv table + +To start mysqld at boot time you have to copy support-files/mysql.server +to the right place for your system + +PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER ! +This is done with: +/usr/local/bin/mysqladmin -u root password 'new-password' +See the manual for more instructions. + +Please report any problems with the /usr/local/bin/mysqlbug script! + +The latest information about MySQL is available on the web at http://www.mysql.com +Support MySQL by buying support/licenses at http://www.tcx.se/license.htmy. ++ +
+
+August, 29 2000; Scott Harrison; LON-CAPA
+
+These are notes related to a Perl interface and MySQL server installation
+on Redhat 6.1 and 6.2 boxes. (Guy Albertelli and Harsha Jagasia
+contributed significantly to this.)
+
+********************
+* MySQL COMPONENTS *
+********************
+
+There are three components to an effective MySQL installation for the
+LON-CAPA system.
+
+Perl::DBI module- the API "front-end"...
+ database interface module for organizing generic
+ database commands which are independent of specific
+ database implementation (such as MySQL, mSQL, Postgres, etc).
+
+Perl::MySQL module- the API "mid-section"...
+ the module to directly interface with the actual
+ MySQL database engine
+
+MySQL database engine- the "back-end"...
+ the binary installation (compiled either from source
+ or pre-compiled file listings) which provides the
+ actual MySQL functionality on the system
+
+RedHat Installation-
+
+Initially done from source:
+DBI-1.13.tar.gz Msql-Mysql-modules-1.2209.tar.gz mysql-3.22.32.tar.gz
+
+I am now using pre-compiled file listings.
+
+There were problems with using the RedHat packages since the three
+different RedHat packages were somewhat noncompatible with each other
+in terms of expected file locations. (The Debian linux distribution,
+on the other hand, has a working set of these packages).
+
+Regardless of how we install these three components, there still remain
+certain things which need to happen for the configuration.
+
+*****************
+* CONFIGURATION *
+*****************
+
+(Note: SOMEPASSWORD is actually set to another text string on the current
+LON-CAPA systems.)
+
+Configuration is needed to generate the necessary functionality for the
+MySQL system with LON-CAPA.
+
+The functionality needed can be understood from this example line
+of perl code from "lonsql".
+
+ $dbh = DBI->connect( "DBI:mysql:loncapa",
+ "www",
+ "SOMEPASSWORD",
+ { RaiseError =>0,PrintError=>0});
+
+There is an obvious need to CONNECT to the database, and in order to do
+this, there must be:
+ a RUNNING mysql daemon;
+ a DATABASE named "loncapa";
+ a USER named "www";
+ and an ABILITY for LON-CAPA on one machine to access
+ SQL database on another machine;
+
+So, here are some notes on implementing these configurations.
+
+** RUNNING mysql daemon (safe_mysqld method)
+
+The recommended way to run the MySQL daemon is as a non-root user
+(probably www)...
+
+so, 1) login as user www on the linux machine
+ 2) start the mysql daemon as /usr/local/bin/safe_mysqld &
+
+safe_mysqld only works if the local installation of MySQL is set to the
+right directory permissions which I found to be:
+chown www:users /usr/local/var/mysql
+chown www:users /usr/local/lib/mysql
+chown -R www:users /usr/local/mysql
+chown www:users /usr/local/include/mysql
+chown www:users /usr/local/var
+
+** DATABASE named "loncapa"
+
+As user www, run this command
+ mysql -u root -p mysql
+enter the password as SOMEPASSWORD
+
+This allows you to manually enter MySQL commands.
+The MySQL command to generate the loncapa DATABASE is:
+
+CREATE DATABASE 'loncapa';
+
+** USER named "www"
+
+As user www, run this command
+ mysql -u root -p mysql
+enter the password as SOMEPASSWORD
+
+To add the user www to the MySQL server, and grant all
+privileges on *.* to www@localhost identified by 'SOMEPASSWORD'
+with grant option;
+
+INSERT INTO user (Host, User, Password)
+VALUES ('localhost','www',password('SOMEPASSWORD'));
+
+GRANT ALL PRIVILEGES ON *.* TO www@localhost;
+
+FLUSH PRIVILEGES;
+
+** ABILITY for LON-CAPA machines to communicate with SQL databases on
+ other LON-CAPA machines
+
+This is a little more intricate than might first be expected (and I probably
+won't do a perfect job reciting everything in this short synopsis). Because
+LON-CAPA machines will likely be handling many SQL requests at a time,
+there were some problems with current MySQL capabilities.
+
+PROBLEM SITUATION:
+
+ If Server A wants data from Server B, Server A uses a lonc process to
+ send a database command to a Server B lond process.
+ lonc= loncapa client process A-lonc= a lonc process on Server A
+ lond= loncapa daemon process
+
+ database command
+ A-lonc --------TCP/IP----------------> B-lond
+
+ The problem emerges that A-lonc and B-lond are kept waiting for the
+ MySQL server to "do its stuff", or in other words, perform the conceivably
+ sophisticated, data-intensive, time-sucking database transaction. By tying
+ up a lonc and lond process, this significantly cripples the capabilities
+ of LON-CAPA servers.
+
+ While commercial databases have a variety of features that ATTEMPT to
+ deal with this, freeware databases are still experimenting and exploring
+ with different schemes with varying degrees of performance stability.
+
+THE SOLUTION:
+
+ A separate daemon process was created that B-lond works with to
+ handle database requests. This daemon process is called "lonsql".
+
+ So,
+ database command
+ A-lonc ---------TCP/IP-----------------> B-lond =====> B-lonsql
+ <---------------------------------/ |
+ "ok, I'll get back to you..." |
+ |
+ /
+ A-lond <------------------------------- B-lonc <======
+ "Guess what? I have the result!"
+
+ Of course, depending on success or failure, the messages may vary,
+ but the principle remains the same where a separate pool of children
+ processes (lonsql's) handle the MySQL database manipulations.
+
+Here are excerpts of code which implement the above handling:
+
+**LONSQL
+
+A subroutine from "lonsql" which establishes a child process for handling
+database interactions.
+
+sub make_new_child {
+ my $pid;
+ my $sigset;
+
+ # block signal for fork
+ $sigset = POSIX::SigSet->new(SIGINT);
+ sigprocmask(SIG_BLOCK, $sigset)
+ or die "Can't block SIGINT for fork: $!\n";
+
+ die "fork: $!" unless defined ($pid = fork);
+
+ if ($pid) {
+ # Parent records the child's birth and returns.
+ sigprocmask(SIG_UNBLOCK, $sigset)
+ or die "Can't unblock SIGINT for fork: $!\n";
+ $children{$pid} = 1;
+ $children++;
+ return;
+ } else {
+ # Child can *not* return from this subroutine.
+ $SIG{INT} = 'DEFAULT'; # make SIGINT kill us as it did before
+
+ # unblock signals
+ sigprocmask(SIG_UNBLOCK, $sigset)
+ or die "Can't unblock SIGINT for fork: $!\n";
+
+
+ #open database handle
+ # making dbh global to avoid garbage collector
+ unless (
+ $dbh = DBI->connect("DBI:mysql:loncapa","www","SOMEPASSWORD",{ RaiseError =>0,PrintError=>0})
+ ) {
+ my $st=120+int(rand(240));
+ &logthis("WARNING: Couldn't connect to database ($st secs): $@");
+ print "database handle error\n";
+ sleep($st);
+ exit;
+
+ };
+ # make sure that a database disconnection occurs with ending kill signals
+ $SIG{TERM}=$SIG{INT}=$SIG{QUIT}=$SIG{__DIE__}=\&DISCONNECT;
+
+ # handle connections until we've reached $MAX_CLIENTS_PER_CHILD
+ for ($i=0; $i < $MAX_CLIENTS_PER_CHILD; $i++) {
+ $client = $server->accept() or last;
+
+ # do something with the connection
+ $run = $run+1;
+ my $userinput = <$client>;
+ chomp($userinput);
+
+ my ($conserver,$querytmp)=split(/&/,$userinput);
+ my $query=unescape($querytmp);
+
+ #send query id which is pid_unixdatetime_runningcounter
+ $queryid = $thisserver;
+ $queryid .="_".($$)."_";
+ $queryid .= time."_";
+ $queryid .= $run;
+ print $client "$queryid\n";
+
+ #prepare and execute the query
+ my $sth = $dbh->prepare($query);
+ my $result;
+ unless ($sth->execute())
+ {
+ &logthis("WARNING: Could not retrieve from database: $@");
+ $result="";
+ }
+ else {
+ my $r1=$sth->fetchall_arrayref;
+ my @r2; map {my $a=$_; my @b=map {escape($_)} @$a; push @r2,join(",", @b)} (@$r1);
+ $result=join("&",@r2) . "\n";
+ }
+ &reply("queryreply:$queryid:$result",$conserver);
+
+ }
+
+ # tidy up gracefully and finish
+
+ #close the database handle
+ $dbh->disconnect
+ or &logthis("WARNING: Couldn't disconnect from database $DBI::errstr ($st secs): $@");
+
+ # this exit is VERY important, otherwise the child will become
+ # a producer of more and more children, forking yourself into
+ # process death.
+ exit;
+ }
+}
+
+** LOND enabling of MySQL requestsw
+
+ This code is part of every lond child process in the way that it parses command request syntax
+ sent to it from lonc processes. querysend corresponds to B-lonc sending the result of the query.
+ queryreply corresponds to B-lond indicating that it has received the request and will start the
+ database transaction (it returns "ok" to A-lonc ($client)).
+
+# ------------------------------------------------------------------- querysend
+ } elsif ($userinput =~ /^querysend/) {
+ my ($cmd,$query)=split(/:/,$userinput);
+ $query=~s/\n*$//g;
+ print $client sqlreply("$hostid{$clientip}\&$query")."\n";
+# ------------------------------------------------------------------ queryreply
+ } elsif ($userinput =~ /^queryreply/) {
+ my ($cmd,$id,$reply)=split(/:/,$userinput);
+ my $store;
+ my $execdir=$perlvar{'lonDaemons'};
+ if ($store=IO::File->new(">$execdir/tmp/$id")) {
+ print $store $reply;
+ close $store;
+ print $client "ok\n";
+ }
+ else {
+ print $client "error:$!\n";
+ }
+
+
+
+** TEST the database connection with my current tester.pl code which mimics what command will eventually be
+ sent through lonc.
+$reply=reply(
+ "querysend:SELECT * FROM general_information WHERE Id='AAAAA'",$lonID);
+