[BACK]Return to loncapasqldatabase.html CVS log [TXT] [DIR] Up to [LON-CAPA] / doc / build

Diff for /doc/build/Attic/loncapasqldatabase.html between versions 1.6 and 1.12

version 1.6, 2001/02/12 17:38:13 version 1.12, 2001/02/16 12:40:29
Line 8 Line 8
 Scott Harrison  Scott Harrison
 </P>  </P>
 <P>  <P>
 Last updated: 02/12/2001  Last updated: 02/15/2001
 </P>  </P>
 <P>  <P>
 This file describes issues associated with LON-CAPA  This file describes issues associated with LON-CAPA
Line 17  and a SQL database. Line 17  and a SQL database.
 <H2>Latest HOWTO</H2>  <H2>Latest HOWTO</H2>
 <P>  <P>
 <UL>  <UL>
   <LI>Important notes
 <LI>Current status of documentation</LI>  <LI>Current status of documentation</LI>
 <LI>Current status of implementation</LI>  <LI>Current status of implementation</LI>
 <LI>Purpose within LON-CAPA</LI>  <LI>Purpose within LON-CAPA</LI>
   <LI>Dependencies</LI>
 <LI>Installation</LI>  <LI>Installation</LI>
 <LI>Installation from source</LI>  <LI>Installation from source</LI>
 <LI>Configuration (automated)</LI>  <LI>Configuration (automated)</LI>
Line 28  and a SQL database. Line 30  and a SQL database.
 <LI>Example sections of code relevant to LON-CAPA</LI>  <LI>Example sections of code relevant to LON-CAPA</LI>
 </UL>  </UL>
 </P>  </P>
   <H2>Important notes</H2>
   <P>
   The current database is implemented assuming a non-adjustable
   architecture involving these data fields (specific to each version
   of a resource).
   <UL>
   <LI>title</LI>
   <LI>author</LI>
   <LI>subject</LI>
   <LI>notes</LI>
   <LI>abstract</LI>
   <LI>mime</LI>
   <LI>language</LI>
   <LI>creationdate</LI>
   <LI>lastrevisiondate</LI>
   <LI>owner</LI>
   <LI>copyright</LI>
   </UL>
   </P>
   <P>
   These commands create the loncapameta database.
   <PRE>
   mysql> CREATE DATABASE IF NOT EXISTS loncapameta
       -> CREATE TABLE IF NOT EXISTS resourceversions
   
   </PRE>
   </P>
   <P>
   An important quote from the manual:
   <BLOCKQUOTE>
   In MySQL Version 3.23.23 or later, you can also create special FULLTEXT indexes. They are used for full-text search. Only the MyISAM table type supports FULLTEXT indexes. They can be created only from
   VARCHAR and TEXT columns. Indexing always happens over the entire column and partial indexing is not supported. See section 25.2 MySQL Full-text Search for details. 
   </BLOCKQUOTE>
   </P>
   <P>
   I plan on using a MyISAM table type with 11 metadata fields of column 
   type=TEXT.
   </P>
   <P>
   It might be worthwhile to look at /usr/local/mysql/manual.html.
   It is quite in depth.
   </P>
 <H2>Current status of documentation</H2>  <H2>Current status of documentation</H2>
 <P>  <P>
 I am going to begin documentation by inserting what notes  I am going to begin documentation by inserting what notes
Line 55  Right now, a lot of "feasibility" work h Line 99  Right now, a lot of "feasibility" work h
 Recipes for manual installation and configuration have  Recipes for manual installation and configuration have
 been gathered.  Network connectivity of lond->lonsql->lond->lonc  been gathered.  Network connectivity of lond->lonsql->lond->lonc
 type tests have been performed.  A binary installation  type tests have been performed.  A binary installation
 has been compiled in an RPM (LON-CAPA-mysql).  has been compiled in an RPM (LON-CAPA-mysql, with perl components
   a part of LON-CAPA-systemperl).
 The most lacking test in terms of feasibility has  The most lacking test in terms of feasibility has
 been looking at benchmarks to analyze the load at which  been looking at benchmarks to analyze the load at which
 the SQL database can efficiently allow many users to  the SQL database can efficiently allow many users to
Line 141  THE SOLUTION: Line 186  THE SOLUTION:
   processes (lonsql's) handle the MySQL database manipulations.    processes (lonsql's) handle the MySQL database manipulations.
 </PRE>  </PRE>
 </P>  </P>
   <H2>Dependencies</H2>
   <P>
   I believe (but am not 100% confident) that the following
   RPMs are necessary (in addition to the current ones
   in rpm_list.txt) to run MySQL.  Basically I discovered these
   dependencies while trying to do external RPM based installs.
   I assume, and sometimes found, that these dependencies apply
   to tarball-based distributions too.  (So to play it on the
   safe side, I am going to include these RPMs as part of the
   core, minimal RPM set.)
   <UL>
   <LI>egcs-1.1.2-30</LI>
   <LI>cpp-1.1.2-30</LI>
   <LI>glibc-devel-2.1.3-15</LI>
   <LI>zlib-devel-1.1.3-6</LI>
   </UL>
   </P>
 <H2>Installation</H2>  <H2>Installation</H2>
 <P>  <P>
 Installation of the LON-CAPA SQL database normally occurs  Installation of the LON-CAPA SQL database normally occurs
 by default when using the LON-CAPA installation CD  by default when using the LON-CAPA installation CD
 (see http://install.lon-capa.org).  It is installed  (see http://install.lon-capa.org).  It is installed
 as the LON-CAPA-mysql RPM.  This RPM encodes for the MySQL  as the LON-CAPA-mysql RPM.  This RPM encodes for the MySQL
 engine and related perl interfaces (Perl::DBI, Perl::Msql-Mysql).  engine.  Related perl interfaces (Perl::DBI, Perl::Msql-Mysql)
   are encoded in the LON-CAPA-systemperl RPM.
 </P>  </P>
 <P>  <P>
 The three components of a MySQL installation for the  The three components of a MySQL installation for the
Line 172  actual MySQL functionality on the system Line 235  actual MySQL functionality on the system
 </P>  </P>
 <H2>Installation from source</H2>  <H2>Installation from source</H2>
 <P>  <P>
   Note: the mysql site recommends that Linux users install by
   using the MySQL RPMs (MySQL-client, MySQL, MySQL-shared, etc).
   While these RPMs work, I was unsuccessful at integrating
   this RPM-installed database with perl modules from www.cpan.org.
   Hence, I <STRONG>strongly</STRONG> recommend that, when installing
   from "source", MySQL and the perl components be in fact installed
   from their tarballs (.tar.gz, .tgz).  (Perl components, when installed
   from RPMs, also wound up in incorrect locations on the disk.)
   Do not coordinate a source install with externally made RPMs!
   It is, of course, okay to use LON-CAPA RPMs such as LON-CAPA-systemperl
   and LON-CAPA-mysql since we, in fact, made these RPMs correctly :).
   <UL>
   <LI>http://www.cpan.org/authors/id/JWIED/Msql-Mysql-modules-1.2215.tar.gz
   <BR>This tarball Released 20th August 2000
   <LI>http://www.mysql.com/Downloads/MySQL-3.23/mysql-3.23.33-pc-linux-gnu-i686.tar.gz
   <BR>This tarball Last changed 2000-11-11
   <BR>This is actually a binary tarball (as opposed to source code
   that is subsequently compiled).
   <LI>http://www.cpan.org/authors/id/TIMB/DBI-1.14.tar.gz
   <BR>This tarball Released 14th June 2000
   </UL>
   </P>
   <P>So, here is exactly how I installed MySQL-3.23. (Note that all files
   wind up in /usr/local/mysql-3.23.33-pc-linux-gnu-i686 except for
   a link from /usr/local/mysql to /usr/local/mysql-3.23.33-pc-linux-gnu-i686
   and some files involved in system process handling (/etc/rc.d/*/*mysql).
   <PRE>
   (As user=root)
   cd /usr/local/;
   tar xzvf mysql-3.23.33-pc-linux-gnu-i686.tar.gz
   ln -s /usr/local/mysql-3.23.33-pc-linux-gnu-i686 mysql
   cd /usr/local/mysql
   chown -R www /usr/local/mysql/.
   chgrp -R users /usr/local/mysql/.
   chmod -R g-w,g-r,g-x /usr/local/mysql/.
   (probably also want chmod -R a-w,a-r,a-x /usr/local/mysql/.)
   
   Alter <TT>safe_mysqld</TT>
   and <TT>support-files/mysql.server</TT> to use 'localhosts' instead
   of `bin/hostname`.  Also, to use user 'www' instead of 'mysql'.
   
   (These changes could be done with /etc/my.cnf, but
   I think this approach makes sure the database NEVER
   gets screwed up due to somebody forgetting to install /etc/my.cnf).
   
   Change this line in mysql.server from
     pid_file=$datadir/`@HOSTNAME@`.pid
   to
     pid_file=$datadir/localhost.pid
   
   Change this line in safe_mysqld from
   user=root
   to
   user=www
   
   Change this line in safe_mysqld from
     pid_file=$DATADIR/`/bin/hostname`.pid
   to
     pid_file=$DATADIR/localhost.pid
   
   Change this line in safe_mysqld from
   test -z "$err_log"  && err_log=$DATADIR/`/bin/hostname`.err
   to
   test -z "$err_log"  && err_log=$DATADIR/localhost.err
   
   cp -p support-files/mysql.server /etc/rc.d/init.d/mysql
   chmod 755 /etc/rc.d/init.d/mysql
   /sbin/chkconfig --add mysql
   /etc/rc.d/init.d/mysql start
   
   </PRE>
   </P>
   <P>This is how I installed the Msql-Mysql-modules perl modules.
   <PRE>
   [root@fenchurch Msql-Mysql-modules-1.2215]# perl Makefile.PL 
   Which drivers do you want to install?
   
       1) MySQL only
       2) mSQL only (either of mSQL 1 or mSQL 2)
       3)  MySQL and mSQL (either of mSQL 1 or mSQL 2)
   
       4)  mSQL 1 and mSQL 2
       5)  MySQL, mSQL 1 and mSQL 2
   
   Enter the appropriate number:  [3] 1
   
   
   Do you want to install the MysqlPerl emulation? You might keep your old
   Mysql module (to be distinguished from DBD::mysql!) if you are concerned
   about compatibility to existing applications! [y] n
   Where is your MySQL installed? Please tell me the directory that
   contains the subdir 'include'. [/usr/local/mysql] 
   Which database should I use for testing the MySQL drivers? [test] 
   On which host is database test running (hostname, ip address
   or host:port) [localhost] 
   User name for connecting to database test? [undef]
   Password for connecting to database test? [undef]
   [root@fenchurch Msql-Mysql-modules-1.2215]# make
   [root@fenchurch Msql-Mysql-modules-1.2215]# make test
   make[1]: Entering directory `/home/user/Msql-Mysql-modules-1.2215/mysql'
   make[1]: Leaving directory `/home/user/Msql-Mysql-modules-1.2215/mysql'
   make[1]: Entering directory `/home/user/Msql-Mysql-modules-1.2215/mysql'
   PERL_DL_NONLAZY=1 /usr/bin/perl -I../blib/arch -I../blib/lib -I/usr/lib/perl5/5.00503/i386-linux -I/usr/lib/perl5/5.00503 -e 'use Test::Harness qw(&runtests $verbose); $verbose=0; runtests @ARGV;' t/*.t
   t/00base............ok
   t/10dsnlist.........ok
   t/20createdrop......ok
   t/30insertfetch.....ok
   t/40bindparam.......ok
   t/40blobs...........ok
   t/40listfields......ok
   t/40nulls...........ok
   t/40numrows.........ok
   t/50chopblanks......ok
   t/50commit..........ok
   t/60leaks...........skipping test on this platform
   t/ak-dbd............ok
   t/akmisc............ok
   t/dbdadmin..........ok
   t/mysql.............ok
   t/mysql2............ok
   All tests successful, 1 test skipped.
   Files=17,  Tests=732, 40 wallclock secs (15.38 cusr +  1.30 csys = 16.68 CPU)
   [root@fenchurch Msql-Mysql-modules-1.2215]# make install
   
   These files are installed.
   /usr/bin/dbimon
   /usr/lib/perl5/man/man3/Bundle::DBD::mysql.3
   /usr/lib/perl5/man/man3/DBD::mysql.3
   /usr/lib/perl5/man/man3/Mysql.3
   /usr/lib/perl5/site_perl/5.005/i386-linux/Bundle/DBD/mysql.pm
   /usr/lib/perl5/site_perl/5.005/i386-linux/DBD/mysql.pm
   /usr/lib/perl5/site_perl/5.005/i386-linux/Mysql.pm
   /usr/lib/perl5/site_perl/5.005/i386-linux/Mysql/Statement.pm
   /usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBD/mysql/mysql.bs
   /usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBD/mysql/mysql.so
   /usr/man/man1/dbimon.1
   /usr/lib/perl5/site_perl/5.005/i386-linux/auto/Msql-Mysql-modules/.packlist
   </PRE>
   </P>
   <P>
   This is how I installed the DBI perl modules.
   <PRE>
   [root@fenchurch DBI-1.14]# perl Makefile.PL
   *** Note:
       The optional PlRPC-modules (RPC::PlServer etc) are not installed.
       If you want to use the DBD::Proxy driver and DBI::ProxyServer
       modules, then you'll need to install the RPC::PlServer, RPC::PlClient,
       Storable and Net::Daemon modules. The CPAN Bundle::DBI may help you.
       You can install them any time after installing the DBI.
       You do *not* need these modules for typical DBI usage.
   
   Optional modules are available from any CPAN mirror, in particular
       http://www.perl.com/CPAN/modules/by-module
       http://www.perl.org/CPAN/modules/by-module
       ftp://ftp.funet.fi/pub/languages/perl/CPAN/modules/by-module
   
   Checking if your kit is complete...
   Looks good
   Writing Makefile for DBI
   
       Remember to actually *read* the README file!
       Use  'make' to build the software (dmake or nmake on Windows).
       Then 'make test' to execute self tests.
       Then 'make install' to install the DBI and then delete this working
       directory before unpacking and building any DBD::* drivers.
   
   [root@fenchurch DBI-1.14]# make
   [root@fenchurch DBI-1.14]# make test
   PERL_DL_NONLAZY=1 /usr/bin/perl -Iblib/arch -Iblib/lib -I/usr/lib/perl5/5.00503/i386-linux -I/usr/lib/perl5/5.00503 -e 'use Test::Harness qw(&runtests $verbose); $verbose=0; runtests @ARGV;' t/*.t
   t/basics............ok
   t/dbidrv............ok
   t/examp.............ok
   t/meta..............ok
   t/proxy.............skipping test on this platform
   t/shell.............ok
   t/subclass..........ok
   All tests successful, 1 test skipped.
   Files=7,  Tests=179,  7 wallclock secs ( 6.46 cusr +  0.49 csys =  6.95 CPU)
   PERL_DL_NONLAZY=1 /usr/bin/perl -Iblib/arch -Iblib/lib -I/usr/lib/perl5/5.00503/i386-linux -I/usr/lib/perl5/5.00503 test.pl
   test.pl 
   DBI test application $Revision$
   Using /home/user/DBI-1.14/blib
   Switch: DBI 1.14 by Tim Bunce, 1.14
   Available Drivers: ADO, ExampleP, Multiplex, Proxy, mysql
   dbi:ExampleP:: testing 5 sets of 20 connections:
   Connecting... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 
   Disconnecting...
   Connecting... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 
   Disconnecting...
   Connecting... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 
   Disconnecting...
   Connecting... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 
   Disconnecting...
   Connecting... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 
   Disconnecting...
   Made 100 connections in  0 wallclock secs ( 0.22 usr +  0.03 sys =  0.25 CPU)
   
   Testing handle creation speed...
   5000 NullP statement handles cycled in 6.6 cpu+sys seconds (762 per sec)
   
   test.pl done
   
   [root@fenchurch DBI-1.14]# make install
   These files are installed.
   /usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/.packlist
   /usr/bin/dbiproxy
   /usr/bin/dbish
   /usr/lib/perl5/man/man3/Bundle::DBI.3
   /usr/lib/perl5/man/man3/DBD::ADO.3
   /usr/lib/perl5/man/man3/DBD::Multiplex.3
   /usr/lib/perl5/man/man3/DBD::Proxy.3
   /usr/lib/perl5/man/man3/DBI.3
   /usr/lib/perl5/man/man3/DBI::DBD.3
   /usr/lib/perl5/man/man3/DBI::FAQ.3
   /usr/lib/perl5/man/man3/DBI::Format.3
   /usr/lib/perl5/man/man3/DBI::ProxyServer.3
   /usr/lib/perl5/man/man3/DBI::Shell.3
   /usr/lib/perl5/man/man3/DBI::W32ODBC.3
   /usr/lib/perl5/man/man3/Win32::DBIODBC.3
   /usr/lib/perl5/site_perl/5.005/i386-linux/Bundle/DBI.pm
   /usr/lib/perl5/site_perl/5.005/i386-linux/DBD/ADO.pm
   /usr/lib/perl5/site_perl/5.005/i386-linux/DBD/ExampleP.pm
   /usr/lib/perl5/site_perl/5.005/i386-linux/DBD/Multiplex.pm
   /usr/lib/perl5/site_perl/5.005/i386-linux/DBD/NullP.pm
   /usr/lib/perl5/site_perl/5.005/i386-linux/DBD/Proxy.pm
   /usr/lib/perl5/site_perl/5.005/i386-linux/DBD/Sponge.pm
   /usr/lib/perl5/site_perl/5.005/i386-linux/DBI.pm
   /usr/lib/perl5/site_perl/5.005/i386-linux/DBI/DBD.pm
   /usr/lib/perl5/site_perl/5.005/i386-linux/DBI/FAQ.pm
   /usr/lib/perl5/site_perl/5.005/i386-linux/DBI/Format.pm
   /usr/lib/perl5/site_perl/5.005/i386-linux/DBI/ProxyServer.pm
   /usr/lib/perl5/site_perl/5.005/i386-linux/DBI/Shell.pm
   /usr/lib/perl5/site_perl/5.005/i386-linux/DBI/W32ODBC.pm
   /usr/lib/perl5/site_perl/5.005/i386-linux/Win32/DBIODBC.pm
   /usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/DBI.bs
   /usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/DBI.so
   /usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/DBIXS.h
   /usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/Driver.xst
   /usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/dbd_xsh.h
   /usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/dbi_sql.h
   /usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/dbipport.h
   /usr/man/man1/dbiproxy.1
   /usr/man/man1/dbish.1
   </PRE>
   </P>
   <FONT COLOR="green"> old notes in green
   <P>
 The following set of tarballs was found to work together  The following set of tarballs was found to work together
 properly on a LON-CAPA RedHat 6.2 system:  properly on a LON-CAPA RedHat 6.2 system:
 <UL>  <UL>
Line 184  properly on a LON-CAPA RedHat 6.2 system Line 494  properly on a LON-CAPA RedHat 6.2 system
 Installation was simply a matter of following the instructions  Installation was simply a matter of following the instructions
 and typing the several "make" commands for each   and typing the several "make" commands for each 
 </P>  </P>
   </FONT>
 <H2>Configuration (automated)</H2>  <H2>Configuration (automated)</H2>
 <P>  <P>
 Not yet developed.  This will be part of an interface  Not yet developed.  This will be part of an interface
Line 295  An up-to-date lond and lonsql. Line 606  An up-to-date lond and lonsql.
 </P>  </P>
 <H2>Testing</H2>  <H2>Testing</H2>
 <P>  <P>
   To test the backend MySQL database, a number of commands should be
   run after installation.
   <UL>
   <LI><TT>cd /usr/local/mysql/sql-bench; ./run-all-tests --small-test</TT></LI>
   <BR>without the --small-test flag, this test can take more than 10 hours!
   <LI><TT>cd /usr/local/mysql; bin/mysqladmin version</TT></LI>
   <LI><TT>cd /usr/local/mysql; bin/mysqladmin variables</TT></LI>
   <LI><TT>cd /usr/local/mysql; bin/mysqlshow</TT></LI>
   <LI><TT>cd /usr/local/mysql; bin/mysqlshow mysql</TT></LI>
   <LI><TT>cd /usr/local/mysql; bin/mysql -e "select host,db,user from db" mysql</TT></LI>
   <LI><TT>cd /usr/local/mysql/mysql-test; ./test-run-all</TT></LI>
   
   </UL>
   <P>
   These are sections of perl code which helps test the LON-CAPA network.
 <PRE>  <PRE>
 <STRONG>** TEST the database connection with my current tester.pl code  <STRONG>** TEST the database connection with my current tester.pl code
 which mimics what command will eventually be sent through lonc.</STRONG>  which mimics what command will eventually be sent through lonc.</STRONG>
Removed from v.1.6  
changed lines
  Added in v.1.12

FreeBSD-CVSweb <freebsd-cvsweb@FreeBSD.org>