--- doc/build/Attic/loncapasqldatabase.html 2001/02/14 14:33:35 1.8 +++ doc/build/Attic/loncapasqldatabase.html 2001/02/20 22:24:45 1.15 @@ -8,7 +8,7 @@ Scott Harrison

-Last updated: 02/12/2001 +Last updated: 02/15/2001

This file describes issues associated with LON-CAPA @@ -17,6 +17,7 @@ and a SQL database.

Latest HOWTO

+

Important notes

+

+The current database is implemented assuming a non-adjustable +architecture involving these data fields (specific to each version +of a resource). +

+

+

+Security occurs as a function of the user 'www', +and the permissions on the files in the /usr/local/mysql +directory. (These files and directories should +be 700, 600, 500, 400, etc; not allow access to anyone +but user 'www'.) +

+

+These commands create the loncapameta database. +

+mysql> CREATE DATABASE IF NOT EXISTS loncapa;
+mysql> USE loncapa;
+mysql> CREATE TABLE IF NOT EXISTS metadata (title TEXT, author TEXT, subject TEXT, notes TEXT, abstract TEXT, mime TEXT, language TEXT, creationdate TEXT, lastrevisiondate TEXT, owner TEXT, copyright TEXT, FULLTEXT idx_title (title), FULLTEXT idx_author (author), FULLTEXT idx_subject (subject), FULLTEXT idx_notes (notes), FULLTEXT idx_abstract (abstract), FULLTEXT idx_mime (mime), FULLTEXT idx_language (language), FULLTEXT idx_creationdate (creationdate), FULLTEXT idx_lastrevisiondate (lastrevisiondate), FULLTEXT idx_owner (owner), FULLTEXT idx_copyright (copyright)) TYPE=MYISAM;
+mysql> INSERT INTO metadata VALUES ('1','2','3','4','5','6','7','8','9','10','11');
+mysql> SELECT * FROM metadata WHERE title REGEXP "1";
+
+

+

+Current time values for things. +

+fenchurch.lite.msu.edu
+Mem:    46812K av,   45632K used,    1180K free,   14756K shrd,    4292K buff
+Swap: 1148608K av,   11260K used, 1137348K free                   13244K cached
+
+Red Hat Linux release 6.2 (Zoot)
+Kernel 2.2.16-3 on an i586
+
+
Q: How big are data records in test database? A: on average, 1000 bytes each, medline records from PubMed. +
Q: How big is the biggest field? A: on average, 838 bytes each +
Q: How much time to insert 5284 medline records into database? +A: 600 seconds +
Q: What about when using "speed-technique" on page 130? A: 689 seconds (weird, eh?) +
Q: What about REGEXP searching? A: about 1-2 seconds for small fields; +10 to 20 seconds for REGEXP search on "abstract" field +
Q: What about FULLTEXT indexing? A: about 6 seconds for abstract field. + +

+

+An important quote from the manual: +

+In MySQL Version 3.23.23 or later, you can also create special FULLTEXT indexes. They are used for full-text search. Only the MyISAM table type supports FULLTEXT indexes. They can be created only from +VARCHAR and TEXT columns. Indexing always happens over the entire column and partial indexing is not supported. See section 25.2 MySQL Full-text Search for details. +
+

+

+I plan on using a MyISAM table type with 11 metadata fields of column +type=TEXT. +

+

+It might be worthwhile to look at /usr/local/mysql/manual.html. +It is quite in depth. +

Current status of documentation

I am going to begin documentation by inserting what notes @@ -157,10 +229,8 @@ core, minimal RPM set.)

  • egcs-1.1.2-30
  • cpp-1.1.2-30
  • glibc-devel-2.1.3-15
  • -
  • glibc-devel-2.1.3-15
  • zlib-devel-1.1.3-6
  • -

    Installation

    @@ -212,10 +282,233 @@ and LON-CAPA-mysql since we, in fact, ma
    This tarball Last changed 2000-11-11
    This is actually a binary tarball (as opposed to source code that is subsequently compiled). -

  • -
    +
  • http://www.cpan.org/authors/id/TIMB/DBI-1.14.tar.gz +
    This tarball Released 14th June 2000

    +

    So, here is exactly how I installed MySQL-3.23. (Note that all files +wind up in /usr/local/mysql-3.23.33-pc-linux-gnu-i686 except for +a link from /usr/local/mysql to /usr/local/mysql-3.23.33-pc-linux-gnu-i686 +and some files involved in system process handling (/etc/rc.d/*/*mysql). +

    +(As user=root)
    +cd /usr/local/;
    +tar xzvf mysql-3.23.33-pc-linux-gnu-i686.tar.gz
    +ln -s /usr/local/mysql-3.23.33-pc-linux-gnu-i686 mysql
    +cd /usr/local/mysql
    +chown -R www /usr/local/mysql/.
    +chgrp -R users /usr/local/mysql/.
    +chmod -R g-w,g-r,g-x /usr/local/mysql/.
    +(probably also want chmod -R a-w,a-r,a-x /usr/local/mysql/.)
    +
    +Alter safe_mysqld
    +and support-files/mysql.server to use 'localhosts' instead
    +of `bin/hostname`.  Also, to use user 'www' instead of 'mysql'.
    +
    +(These changes could be done with /etc/my.cnf, but
    +I think this approach makes sure the database NEVER
    +gets screwed up due to somebody forgetting to install /etc/my.cnf).
    +
    +Change this line in mysql.server from
    +  pid_file=$datadir/`@HOSTNAME@`.pid
    +to
    +  pid_file=$datadir/localhost.pid
    +
    +Change this line in safe_mysqld from
    +user=root
    +to
    +user=www
    +
    +Change this line in safe_mysqld from
    +  pid_file=$DATADIR/`/bin/hostname`.pid
    +to
    +  pid_file=$DATADIR/localhost.pid
    +
    +Change this line in safe_mysqld from
    +test -z "$err_log"  && err_log=$DATADIR/`/bin/hostname`.err
    +to
    +test -z "$err_log"  && err_log=$DATADIR/localhost.err
    +
    +cp -p support-files/mysql.server /etc/rc.d/init.d/mysql
    +chmod 755 /etc/rc.d/init.d/mysql
    +/sbin/chkconfig --add mysql
    +/etc/rc.d/init.d/mysql start
    +
    +
    +

    +

    This is how I installed the Msql-Mysql-modules perl modules. +

    +[root@fenchurch Msql-Mysql-modules-1.2215]# perl Makefile.PL 
    +Which drivers do you want to install?
    +
    +    1)	MySQL only
    +    2)	mSQL only (either of mSQL 1 or mSQL 2)
    +    3)  MySQL and mSQL (either of mSQL 1 or mSQL 2)
    +
    +    4)  mSQL 1 and mSQL 2
    +    5)  MySQL, mSQL 1 and mSQL 2
    +
    +Enter the appropriate number:  [3] 1
    +
    +
    +Do you want to install the MysqlPerl emulation? You might keep your old
    +Mysql module (to be distinguished from DBD::mysql!) if you are concerned
    +about compatibility to existing applications! [y] n
    +Where is your MySQL installed? Please tell me the directory that
    +contains the subdir 'include'. [/usr/local/mysql] 
    +Which database should I use for testing the MySQL drivers? [test] 
    +On which host is database test running (hostname, ip address
    +or host:port) [localhost] 
    +User name for connecting to database test? [undef]
    +Password for connecting to database test? [undef]
    +[root@fenchurch Msql-Mysql-modules-1.2215]# make
    +[root@fenchurch Msql-Mysql-modules-1.2215]# make test
    +make[1]: Entering directory `/home/user/Msql-Mysql-modules-1.2215/mysql'
    +make[1]: Leaving directory `/home/user/Msql-Mysql-modules-1.2215/mysql'
    +make[1]: Entering directory `/home/user/Msql-Mysql-modules-1.2215/mysql'
    +PERL_DL_NONLAZY=1 /usr/bin/perl -I../blib/arch -I../blib/lib -I/usr/lib/perl5/5.00503/i386-linux -I/usr/lib/perl5/5.00503 -e 'use Test::Harness qw(&runtests $verbose); $verbose=0; runtests @ARGV;' t/*.t
    +t/00base............ok
    +t/10dsnlist.........ok
    +t/20createdrop......ok
    +t/30insertfetch.....ok
    +t/40bindparam.......ok
    +t/40blobs...........ok
    +t/40listfields......ok
    +t/40nulls...........ok
    +t/40numrows.........ok
    +t/50chopblanks......ok
    +t/50commit..........ok
    +t/60leaks...........skipping test on this platform
    +t/ak-dbd............ok
    +t/akmisc............ok
    +t/dbdadmin..........ok
    +t/mysql.............ok
    +t/mysql2............ok
    +All tests successful, 1 test skipped.
    +Files=17,  Tests=732, 40 wallclock secs (15.38 cusr +  1.30 csys = 16.68 CPU)
    +[root@fenchurch Msql-Mysql-modules-1.2215]# make install
    +
    +These files are installed.
    +/usr/bin/dbimon
    +/usr/lib/perl5/man/man3/Bundle::DBD::mysql.3
    +/usr/lib/perl5/man/man3/DBD::mysql.3
    +/usr/lib/perl5/man/man3/Mysql.3
    +/usr/lib/perl5/site_perl/5.005/i386-linux/Bundle/DBD/mysql.pm
    +/usr/lib/perl5/site_perl/5.005/i386-linux/DBD/mysql.pm
    +/usr/lib/perl5/site_perl/5.005/i386-linux/Mysql.pm
    +/usr/lib/perl5/site_perl/5.005/i386-linux/Mysql/Statement.pm
    +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBD/mysql/mysql.bs
    +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBD/mysql/mysql.so
    +/usr/man/man1/dbimon.1
    +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/Msql-Mysql-modules/.packlist
    +
    +

    +

    +This is how I installed the DBI perl modules. +

    $dbh
    +[root@fenchurch DBI-1.14]# perl Makefile.PL
    +*** Note:
    +    The optional PlRPC-modules (RPC::PlServer etc) are not installed.
    +    If you want to use the DBD::Proxy driver and DBI::ProxyServer
    +    modules, then you'll need to install the RPC::PlServer, RPC::PlClient,
    +    Storable and Net::Daemon modules. The CPAN Bundle::DBI may help you.
    +    You can install them any time after installing the DBI.
    +    You do *not* need these modules for typical DBI usage.
    +
    +Optional modules are available from any CPAN mirror, in particular
    +    http://www.perl.com/CPAN/modules/by-module
    +    http://www.perl.org/CPAN/modules/by-module
    +    ftp://ftp.funet.fi/pub/languages/perl/CPAN/modules/by-module
    +
    +Checking if your kit is complete...
    +Looks good
    +Writing Makefile for DBI
    +
    +    Remember to actually *read* the README file!
    +    Use  'make' to build the software (dmake or nmake on Windows).
    +    Then 'make test' to execute self tests.
    +    Then 'make install' to install the DBI and then delete this working
    +    directory before unpacking and building any DBD::* drivers.
    +
    +[root@fenchurch DBI-1.14]# make
    +[root@fenchurch DBI-1.14]# make test
    +PERL_DL_NONLAZY=1 /usr/bin/perl -Iblib/arch -Iblib/lib -I/usr/lib/perl5/5.00503/i386-linux -I/usr/lib/perl5/5.00503 -e 'use Test::Harness qw(&runtests $verbose); $verbose=0; runtests @ARGV;' t/*.t
    +t/basics............ok
    +t/dbidrv............ok
    +t/examp.............ok
    +t/meta..............ok
    +t/proxy.............skipping test on this platform
    +t/shell.............ok
    +t/subclass..........ok
    +All tests successful, 1 test skipped.
    +Files=7,  Tests=179,  7 wallclock secs ( 6.46 cusr +  0.49 csys =  6.95 CPU)
    +PERL_DL_NONLAZY=1 /usr/bin/perl -Iblib/arch -Iblib/lib -I/usr/lib/perl5/5.00503/i386-linux -I/usr/lib/perl5/5.00503 test.pl
    +test.pl 
    +DBI test application $Revision: 1.15 $
    +Using /home/user/DBI-1.14/blib
    +Switch: DBI 1.14 by Tim Bunce, 1.14
    +Available Drivers: ADO, ExampleP, Multiplex, Proxy, mysql
    +dbi:ExampleP:: testing 5 sets of 20 connections:
    +Connecting... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 
    +Disconnecting...
    +Connecting... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 
    +Disconnecting...
    +Connecting... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 
    +Disconnecting...
    +Connecting... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 
    +Disconnecting...
    +Connecting... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 
    +Disconnecting...
    +Made 100 connections in  0 wallclock secs ( 0.22 usr +  0.03 sys =  0.25 CPU)
    +
    +Testing handle creation speed...
    +5000 NullP statement handles cycled in 6.6 cpu+sys seconds (762 per sec)
    +
    +test.pl done
    +
    +[root@fenchurch DBI-1.14]# make install
    +These files are installed.
    +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/.packlist
    +/usr/bin/dbiproxy
    +/usr/bin/dbish
    +/usr/lib/perl5/man/man3/Bundle::DBI.3
    +/usr/lib/perl5/man/man3/DBD::ADO.3
    +/usr/lib/perl5/man/man3/DBD::Multiplex.3
    +/usr/lib/perl5/man/man3/DBD::Proxy.3
    +/usr/lib/perl5/man/man3/DBI.3
    +/usr/lib/perl5/man/man3/DBI::DBD.3
    +/usr/lib/perl5/man/man3/DBI::FAQ.3
    +/usr/lib/perl5/man/man3/DBI::Format.3
    +/usr/lib/perl5/man/man3/DBI::ProxyServer.3
    +/usr/lib/perl5/man/man3/DBI::Shell.3
    +/usr/lib/perl5/man/man3/DBI::W32ODBC.3
    +/usr/lib/perl5/man/man3/Win32::DBIODBC.3
    +/usr/lib/perl5/site_perl/5.005/i386-linux/Bundle/DBI.pm
    +/usr/lib/perl5/site_perl/5.005/i386-linux/DBD/ADO.pm
    +/usr/lib/perl5/site_perl/5.005/i386-linux/DBD/ExampleP.pm
    +/usr/lib/perl5/site_perl/5.005/i386-linux/DBD/Multiplex.pm
    +/usr/lib/perl5/site_perl/5.005/i386-linux/DBD/NullP.pm
    +/usr/lib/perl5/site_perl/5.005/i386-linux/DBD/Proxy.pm
    +/usr/lib/perl5/site_perl/5.005/i386-linux/DBD/Sponge.pm
    +/usr/lib/perl5/site_perl/5.005/i386-linux/DBI.pm
    +/usr/lib/perl5/site_perl/5.005/i386-linux/DBI/DBD.pm
    +/usr/lib/perl5/site_perl/5.005/i386-linux/DBI/FAQ.pm
    +/usr/lib/perl5/site_perl/5.005/i386-linux/DBI/Format.pm
    +/usr/lib/perl5/site_perl/5.005/i386-linux/DBI/ProxyServer.pm
    +/usr/lib/perl5/site_perl/5.005/i386-linux/DBI/Shell.pm
    +/usr/lib/perl5/site_perl/5.005/i386-linux/DBI/W32ODBC.pm
    +/usr/lib/perl5/site_perl/5.005/i386-linux/Win32/DBIODBC.pm
    +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/DBI.bs
    +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/DBI.so
    +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/DBIXS.h
    +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/Driver.xst
    +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/dbd_xsh.h
    +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/dbi_sql.h
    +/usr/lib/perl5/site_perl/5.005/i386-linux/auto/DBI/dbipport.h
    +/usr/man/man1/dbiproxy.1
    +/usr/man/man1/dbish.1
    +
    +

    old notes in green

    The following set of tarballs was found to work together @@ -342,6 +635,21 @@ An up-to-date lond and lonsql.

    Testing

    +To test the backend MySQL database, a number of commands should be +run after installation. +

    +

    +These are sections of perl code which helps test the LON-CAPA network.

     ** TEST the database connection with my current tester.pl code
     which mimics what command will eventually be sent through lonc.