-Last updated: 02/07/2001 +Last updated: 02/14/2001
+This file describes issues associated with LON-CAPA and a SQL database.
++
+It might be worthwhile to look at /usr/local/mysql/manual.html. +It is quite in depth. +
++I am going to begin documentation by inserting what notes +I have into this file. I will be subsequently rearranging +them and editing them based on the tests that I conduct. +I am trying to make sure that documentation, installation, +and run-time issues are all consistent and correct. The +current status of everything is that it works and has +been minimally tested, but things need to be cleaned up +and checked again! +
++Need to +
+Right now, a lot of "feasibility" work has been done. +Recipes for manual installation and configuration have +been gathered. Network connectivity of lond->lonsql->lond->lonc +type tests have been performed. A binary installation +has been compiled in an RPM (LON-CAPA-mysql, with perl components +a part of LON-CAPA-systemperl). +The most lacking test in terms of feasibility has +been looking at benchmarks to analyze the load at which +the SQL database can efficiently allow many users to +make simultaneous requests of the metadata database. +
++Documentation has been pieced together over time. But, +as mentioned in the previous section, it needs an +overhaul. +
++The binary installation has some quirks associated with it. +Some of the user permissions are wrong, although this is +benign. Also, other options of binary installation (such +as using binary RPMs put together by others) were dismissed +given the difficulty of getting differing combinations of +these external RPMs to work together. +
-
+LON-CAPA is meant to distribute A LOT of educational content +to A LOT of people. It is ineffective to directly rely on contents +within the ext2 filesystem to be speedily scanned for +on-the-fly searches of content descriptions. (Simply put, +it takes a cumbersome amount of time to open, read, analyze, and +close thousands of files.) +
++The solution is to hash-index various data fields that are +descriptive of the educational resources on a LON-CAPA server +machine. Descriptive data fields are referred to as +"metadata". The question then arises as to how this metadata +is handled in terms of the rest of the LON-CAPA network +without burdening client and daemon processes. I now +answer this question in the format of Problem and Solution +below. +
++
+PROBLEM SITUATION: + + If Server A wants data from Server B, Server A uses a lonc process to + send a database command to a Server B lond process. + lonc= loncapa client process A-lonc= a lonc process on Server A + lond= loncapa daemon process + + database command + A-lonc --------TCP/IP----------------> B-lond + + The problem emerges that A-lonc and B-lond are kept waiting for the + MySQL server to "do its stuff", or in other words, perform the conceivably + sophisticated, data-intensive, time-sucking database transaction. By tying + up a lonc and lond process, this significantly cripples the capabilities + of LON-CAPA servers. + + While commercial databases have a variety of features that ATTEMPT to + deal with this, freeware databases are still experimenting and exploring + with different schemes with varying degrees of performance stability. + +THE SOLUTION: + + A separate daemon process was created that B-lond works with to + handle database requests. This daemon process is called "lonsql". + + So, + database command + A-lonc ---------TCP/IP-----------------> B-lond =====> B-lonsql + <---------------------------------/ | + "ok, I'll get back to you..." | + | + / + A-lond <------------------------------- B-lonc <====== + "Guess what? I have the result!" + + Of course, depending on success or failure, the messages may vary, + but the principle remains the same where a separate pool of children + processes (lonsql's) handle the MySQL database manipulations. ++ +
+I believe (but am not 100% confident) that the following +RPMs are necessary (in addition to the current ones +in rpm_list.txt) to run MySQL. Basically I discovered these +dependencies while trying to do external RPM based installs. +I assume, and sometimes found, that these dependencies apply +to tarball-based distributions too. (So to play it on the +safe side, I am going to include these RPMs as part of the +core, minimal RPM set.) +
+Installation of the LON-CAPA SQL database normally occurs +by default when using the LON-CAPA installation CD +(see http://install.lon-capa.org). It is installed +as the LON-CAPA-mysql RPM. This RPM encodes for the MySQL +engine. Related perl interfaces (Perl::DBI, Perl::Msql-Mysql) +are encoded in the LON-CAPA-systemperl RPM. +
++The three components of a MySQL installation for the +LON-CAPA system are further described immediately below. +
| Perl::DBI module- +the API "front-end"... | |
| database interface module for organizing generic +database commands which are independent of specific +database implementation (such as MySQL, mSQL, Postgres, etc). + | |
| Perl::MySQL module- +the API "mid-section"... | |
| the module to directly interface with the actual +MySQL database engine | |
| MySQL database engine- +the "back-end"... | |
| the binary installation (compiled either +from source or pre-compiled file listings) which provides the +actual MySQL functionality on the system | |
+Note: the mysql site recommends that Linux users install by +using the MySQL RPMs (MySQL-client, MySQL, MySQL-shared, etc). +While these RPMs work, I was unsuccessful at integrating +this RPM-installed database with perl modules from www.cpan.org. +Hence, I strongly recommend that, when installing +from "source", MySQL and the perl components be in fact installed +from their tarballs (.tar.gz, .tgz). (Perl components, when installed +from RPMs, also wound up in incorrect locations on the disk.) +Do not coordinate a source install with externally made RPMs! +It is, of course, okay to use LON-CAPA RPMs such as LON-CAPA-systemperl +and LON-CAPA-mysql since we, in fact, made these RPMs correctly :). +
So, here is exactly how I installed MySQL-3.23. (Note that all files +wind up in /usr/local/mysql-3.23.33-pc-linux-gnu-i686 except for +a link from /usr/local/mysql to /usr/local/mysql-3.23.33-pc-linux-gnu-i686 +and some files involved in system process handling (/etc/rc.d/*/*mysql). +
+This is how I installed the Msql-Mysql-modules perl modules. +
+[root@fenchurch Msql-Mysql-modules-1.2215]# perl Makefile.PL +Which drivers do you want to install? + + 1) MySQL only + 2) mSQL only (either of mSQL 1 or mSQL 2) + 3) MySQL and mSQL (either of mSQL 1 or mSQL 2) + + 4) mSQL 1 and mSQL 2 + 5) MySQL, mSQL 1 and mSQL 2 + +Enter the appropriate number: [3] 1 + + +Do you want to install the MysqlPerl emulation? You might keep your old +Mysql module (to be distinguished from DBD::mysql!) if you are concerned +about compatibility to existing applications! [y] n +[root@fenchurch Msql-Mysql-modules-1.2215]# make +[root@fenchurch Msql-Mysql-modules-1.2215]# make test +make[1]: Entering directory `/home/user/Msql-Mysql-modules-1.2215/mysql' +make[1]: Leaving directory `/home/user/Msql-Mysql-modules-1.2215/mysql' +make[1]: Entering directory `/home/user/Msql-Mysql-modules-1.2215/mysql' +PERL_DL_NONLAZY=1 /usr/bin/perl -I../blib/arch -I../blib/lib -I/usr/lib/perl5/5.00503/i386-linux -I/usr/lib/perl5/5.00503 -e 'use Test::Harness qw(&runtests $verbose); $verbose=0; runtests @ARGV;' t/*.t +t/00base............ok +t/10dsnlist.........ok +t/20createdrop......ok +t/30insertfetch.....ok +t/40bindparam.......ok +t/40blobs...........ok +t/40listfields......ok +t/40nulls...........ok +t/40numrows.........ok +t/50chopblanks......ok +t/50commit..........ok +t/60leaks...........skipping test on this platform +t/ak-dbd............ok +t/akmisc............ok +t/dbdadmin..........ok +t/mysql.............ok +t/mysql2............ok +All tests successful, 1 test skipped. +Files=17, Tests=732, 40 wallclock secs (15.38 cusr + 1.30 csys = 16.68 CPU) +[root@fenchurch Msql-Mysql-modules-1.2215]# make install + +These files are installed. +/usr/bin/dbimon +/usr/lib/perl5/man/man3/Bundle::DBD::mysql.3 +/usr/lib/perl5/man/man3/DBD::mysql.3 +/usr/lib/perl5/man/man3/Mysql.3 +/usr/lib/perl5/site_perl/5.005/i386-linux/Bundle/DBD/mysql.pm +/usr/lib/perl5/site_perl/5.005/i386-linux/DBD/mysql.pm +/usr/lib/perl5/site_perl/5.005/i386-linux/Mysql.pm +/usr/lib/perl5/site_perl/5.005/i386-linux/Mysql/Statement.pm +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBD/mysql/mysql.bs +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBD/mysql/mysql.so +/usr/man/man1/dbimon.1 +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/Msql-Mysql-modules/.packlist ++ +
+This is how I installed the DBI perl modules. +
+[root@fenchurch DBI-1.14]# perl Makefile.PL +*** Note: + The optional PlRPC-modules (RPC::PlServer etc) are not installed. + If you want to use the DBD::Proxy driver and DBI::ProxyServer + modules, then you'll need to install the RPC::PlServer, RPC::PlClient, + Storable and Net::Daemon modules. The CPAN Bundle::DBI may help you. + You can install them any time after installing the DBI. + You do *not* need these modules for typical DBI usage. + +Optional modules are available from any CPAN mirror, in particular + http://www.perl.com/CPAN/modules/by-module + http://www.perl.org/CPAN/modules/by-module + ftp://ftp.funet.fi/pub/languages/perl/CPAN/modules/by-module + +Checking if your kit is complete... +Looks good +Writing Makefile for DBI + + Remember to actually *read* the README file! + Use 'make' to build the software (dmake or nmake on Windows). + Then 'make test' to execute self tests. + Then 'make install' to install the DBI and then delete this working + directory before unpacking and building any DBD::* drivers. + +[root@fenchurch DBI-1.14]# make +[root@fenchurch DBI-1.14]# make test +PERL_DL_NONLAZY=1 /usr/bin/perl -Iblib/arch -Iblib/lib -I/usr/lib/perl5/5.00503/i386-linux -I/usr/lib/perl5/5.00503 -e 'use Test::Harness qw(&runtests $verbose); $verbose=0; runtests @ARGV;' t/*.t +t/basics............ok +t/dbidrv............ok +t/examp.............ok +t/meta..............ok +t/proxy.............skipping test on this platform +t/shell.............ok +t/subclass..........ok +All tests successful, 1 test skipped. +Files=7, Tests=179, 7 wallclock secs ( 6.46 cusr + 0.49 csys = 6.95 CPU) +PERL_DL_NONLAZY=1 /usr/bin/perl -Iblib/arch -Iblib/lib -I/usr/lib/perl5/5.00503/i386-linux -I/usr/lib/perl5/5.00503 test.pl +test.pl +DBI test application $Revision: 1.9 $ +Using /home/user/DBI-1.14/blib +Switch: DBI 1.14 by Tim Bunce, 1.14 +Available Drivers: ADO, ExampleP, Multiplex, Proxy, mysql +dbi:ExampleP:: testing 5 sets of 20 connections: +Connecting... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 +Disconnecting... +Connecting... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 +Disconnecting... +Connecting... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 +Disconnecting... +Connecting... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 +Disconnecting... +Connecting... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 +Disconnecting... +Made 100 connections in 0 wallclock secs ( 0.22 usr + 0.03 sys = 0.25 CPU) + +Testing handle creation speed... +5000 NullP statement handles cycled in 6.6 cpu+sys seconds (762 per sec) + +test.pl done + +[root@fenchurch DBI-1.14]# make install +These files are installed. +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/.packlist +/usr/bin/dbiproxy +/usr/bin/dbish +/usr/lib/perl5/man/man3/Bundle::DBI.3 +/usr/lib/perl5/man/man3/DBD::ADO.3 +/usr/lib/perl5/man/man3/DBD::Multiplex.3 +/usr/lib/perl5/man/man3/DBD::Proxy.3 +/usr/lib/perl5/man/man3/DBI.3 +/usr/lib/perl5/man/man3/DBI::DBD.3 +/usr/lib/perl5/man/man3/DBI::FAQ.3 +/usr/lib/perl5/man/man3/DBI::Format.3 +/usr/lib/perl5/man/man3/DBI::ProxyServer.3 +/usr/lib/perl5/man/man3/DBI::Shell.3 +/usr/lib/perl5/man/man3/DBI::W32ODBC.3 +/usr/lib/perl5/man/man3/Win32::DBIODBC.3 +/usr/lib/perl5/site_perl/5.005/i386-linux/Bundle/DBI.pm +/usr/lib/perl5/site_perl/5.005/i386-linux/DBD/ADO.pm +/usr/lib/perl5/site_perl/5.005/i386-linux/DBD/ExampleP.pm +/usr/lib/perl5/site_perl/5.005/i386-linux/DBD/Multiplex.pm +/usr/lib/perl5/site_perl/5.005/i386-linux/DBD/NullP.pm +/usr/lib/perl5/site_perl/5.005/i386-linux/DBD/Proxy.pm +/usr/lib/perl5/site_perl/5.005/i386-linux/DBD/Sponge.pm +/usr/lib/perl5/site_perl/5.005/i386-linux/DBI.pm +/usr/lib/perl5/site_perl/5.005/i386-linux/DBI/DBD.pm +/usr/lib/perl5/site_perl/5.005/i386-linux/DBI/FAQ.pm +/usr/lib/perl5/site_perl/5.005/i386-linux/DBI/Format.pm +/usr/lib/perl5/site_perl/5.005/i386-linux/DBI/ProxyServer.pm +/usr/lib/perl5/site_perl/5.005/i386-linux/DBI/Shell.pm +/usr/lib/perl5/site_perl/5.005/i386-linux/DBI/W32ODBC.pm +/usr/lib/perl5/site_perl/5.005/i386-linux/Win32/DBIODBC.pm +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/DBI.bs +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/DBI.so +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/DBIXS.h +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/Driver.xst +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/dbd_xsh.h +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/dbi_sql.h +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/dbipport.h +/usr/man/man1/dbiproxy.1 +/usr/man/man1/dbish.1 ++ + old notes in green +
+The following set of tarballs was found to work together +properly on a LON-CAPA RedHat 6.2 system: +
+Installation was simply a matter of following the instructions +and typing the several "make" commands for each +
+ ++Not yet developed. This will be part of an interface +present on LON-CAPA systems that can be launched by +entering the command /usr/sbin/loncapaconfig. +
++This is not complete. +
++Starting the mysql daemon: Login on the Linux +system as user 'www'. Enter the command +/usr/local/bin/safe_mysqld & +
++Set a password for 'root': +/usr/local/bin/mysqladmin -u root password 'new-password' +
++Adding a user: Start the mysql daemon. Login to the +mysql system as root (mysql -u root -p mysql) +and enter the right password (for instance 'newmysql'). Add the user +www +
+INSERT INTO user (Host, User, Password)
+VALUES ('localhost','www',password('newmysql'));
+
+
++Granting privileges to user 'www': +
+GRANT ALL PRIVILEGES ON *.* TO www@localhost; +FLUSH PRIVILEGES; ++ +
+Set the SQL server to start upon system startup: +Copy support-files/mysql.server to the right place on the system +(/etc/rc.d/...). +
++The Perl API +
+ $dbh = DBI->connect( "DBI:mysql:loncapa",
+ "www",
+ "SOMEPASSWORD",
+ { RaiseError =>0,PrintError=>0});
+
+There is an obvious need to CONNECT to the database, and in order to do
+this, there must be:
+ a RUNNING mysql daemon;
+ a DATABASE named "loncapa";
+ a USER named "www";
+ and an ABILITY for LON-CAPA on one machine to access
+ SQL database on another machine;
+
+So, here are some notes on implementing these configurations.
+
+** RUNNING mysql daemon (safe_mysqld method)
+
+The recommended way to run the MySQL daemon is as a non-root user
+(probably www)...
+
+so, 1) login as user www on the linux machine
+ 2) start the mysql daemon as /usr/local/bin/safe_mysqld &
+
+safe_mysqld only works if the local installation of MySQL is set to the
+right directory permissions which I found to be:
+chown www:users /usr/local/var/mysql
+chown www:users /usr/local/lib/mysql
+chown -R www:users /usr/local/mysql
+chown www:users /usr/local/include/mysql
+chown www:users /usr/local/var
+
+** DATABASE named "loncapa"
+
+As user www, run this command
+ mysql -u root -p mysql
+enter the password as SOMEPASSWORD
+
+This allows you to manually enter MySQL commands.
+The MySQL command to generate the loncapa DATABASE is:
+
+CREATE DATABASE 'loncapa';
+
+** USER named "www"
+
+As user www, run this command
+ mysql -u root -p mysql
+enter the password as SOMEPASSWORD
+
+To add the user www to the MySQL server, and grant all
+privileges on *.* to www@localhost identified by 'SOMEPASSWORD'
+with grant option;
+
+INSERT INTO user (Host, User, Password)
+VALUES ('localhost','www',password('SOMEPASSWORD'));
+
+GRANT ALL PRIVILEGES ON *.* TO www@localhost;
+
+FLUSH PRIVILEGES;
+
+** ABILITY for LON-CAPA machines to communicate with SQL databases on
+ other LON-CAPA machines
+
+An up-to-date lond and lonsql.
+
+
++
+** TEST the database connection with my current tester.pl code +which mimics what command will eventually be sent through lonc. + +$reply=reply( + "querysend:SELECT * FROM general_information WHERE Id='AAAAA'",$lonID); ++ +
+Here are excerpts of code which implement the above handling: +
++
+**LONSQL
+A subroutine from "lonsql" which establishes a child process for handling
+database interactions.
+
+sub make_new_child {
+ my $pid;
+ my $sigset;
+
+ # block signal for fork
+ $sigset = POSIX::SigSet->new(SIGINT);
+ sigprocmask(SIG_BLOCK, $sigset)
+ or die "Can't block SIGINT for fork: $!\n";
+
+ die "fork: $!" unless defined ($pid = fork);
+
+ if ($pid) {
+ # Parent records the child's birth and returns.
+ sigprocmask(SIG_UNBLOCK, $sigset)
+ or die "Can't unblock SIGINT for fork: $!\n";
+ $children{$pid} = 1;
+ $children++;
+ return;
+ } else {
+ # Child can *not* return from this subroutine.
+ $SIG{INT} = 'DEFAULT'; # make SIGINT kill us as it did before
+
+ # unblock signals
+ sigprocmask(SIG_UNBLOCK, $sigset)
+ or die "Can't unblock SIGINT for fork: $!\n";
+
+
+ #open database handle
+ # making dbh global to avoid garbage collector
+ unless (
+ $dbh = DBI->connect("DBI:mysql:loncapa","www","SOMEPASSWORD",{ RaiseError =>0,PrintError=>0})
+ ) {
+ my $st=120+int(rand(240));
+ &logthis("WARNING: Couldn't connect to database ($st secs): $@");
+ print "database handle error\n";
+ sleep($st);
+ exit;
+
+ };
+ # make sure that a database disconnection occurs with ending kill signals
+ $SIG{TERM}=$SIG{INT}=$SIG{QUIT}=$SIG{__DIE__}=\&DISCONNECT;
+
+ # handle connections until we've reached $MAX_CLIENTS_PER_CHILD
+ for ($i=0; $i < $MAX_CLIENTS_PER_CHILD; $i++) {
+ $client = $server->accept() or last;
+
+ # do something with the connection
+ $run = $run+1;
+ my $userinput = <$client>;
+ chomp($userinput);
+
+ my ($conserver,$querytmp)=split(/&/,$userinput);
+ my $query=unescape($querytmp);
+
+ #send query id which is pid_unixdatetime_runningcounter
+ $queryid = $thisserver;
+ $queryid .="_".($$)."_";
+ $queryid .= time."_";
+ $queryid .= $run;
+ print $client "$queryid\n";
+
+ #prepare and execute the query
+ my $sth = $dbh->prepare($query);
+ my $result;
+ unless ($sth->execute())
+ {
+ &logthis("WARNING: Could not retrieve from database: $@");
+ $result="";
+ }
+ else {
+ my $r1=$sth->fetchall_arrayref;
+ my @r2; map {my $a=$_; my @b=map {escape($_)} @$a; push @r2,join(",", @b)} (@$r1);
+ $result=join("&",@r2) . "\n";
+ }
+ &reply("queryreply:$queryid:$result",$conserver);
+
+ }
+
+ # tidy up gracefully and finish
+
+ #close the database handle
+ $dbh->disconnect
+ or &logthis("WARNING: Couldn't disconnect from database $DBI::errstr ($st secs): $@");
+
+ # this exit is VERY important, otherwise the child will become
+ # a producer of more and more children, forking yourself into
+ # process death.
+ exit;
+ }
+}
+
+
+** LOND enabling of MySQL requests
+
+This code is part of every lond child process in the
+way that it parses command request syntax sent to it
+from lonc processes. Based on the diagram above, querysend
+corresponds to B-lonc sending the result of the query.
+queryreply corresponds to B-lond indicating that it has
+received the request and will start the database transaction
+(it returns "ok" to
+A-lonc ($client)).
+
+# ------------------------------------------------------------------- querysend
+ } elsif ($userinput =~ /^querysend/) {
+ my ($cmd,$query)=split(/:/,$userinput);
+ $query=~s/\n*$//g;
+ print $client sqlreply("$hostid{$clientip}\&$query")."\n";
+# ------------------------------------------------------------------ queryreply
+ } elsif ($userinput =~ /^queryreply/) {
+ my ($cmd,$id,$reply)=split(/:/,$userinput);
+ my $store;
+ my $execdir=$perlvar{'lonDaemons'};
+ if ($store=IO::File->new(">$execdir/tmp/$id")) {
+ print $store $reply;
+ close $store;
+ print $client "ok\n";
+ }
+ else {
+ print $client "error:$!\n";
+ }
+
+