CVS log for loncom/metadata_database/parse_activity_log.pl

Up to [LON-CAPA] / loncom / metadata_database

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

- Bug 6740 Out-of-order recording of submissions (by time).
  - Check for new transactions made immediately before call to lonnet::cstore()
    &Apache::inputtags::hidealldata() called if correct then incorrect,
    where awarded >= 1 when correct (feedback on correctness enabled).
  - Check for transactions made immediately after call to lonnet::cstore()
    if reply from lond::store_handler() is delay:N (where N s the number of
    transactions between the last retrieved in &initialize_storage() and the
    last stored immediately before permanent storage of the current transaction.
    &Apache::grades::makehidden() called if correct then incorrect,
    where awarded >= 1 when correct (feedback on correctness enabled).

-removing non-functional use Apache2::compat
- removing extraneous uses

- max_row_count was never actually being intialized

- I can't believe Matthew did this.  I feel very embarrassed for him. :-)

- safely try to pull in the compat enviironment where needed

Use the same mechanism for logging chunks to store chunks from unparseable
log lines.

Better fix for last commit - if a $chunk is over 20k in length, store it
in a file, "activity.log.unprocessed_chunks", and skip inserting it into the
database.

Deal with uploaded files being stored in the activity.log by skipping them.

- typo fix
- calling &clean_up_and_ext() when we don;'t have the lock ends up deleting the file we have a lock on...

Now hold exclusive lock on activity.log.lock until script exits, and if unable
to lock throw a warning and exit.
Added &clean_up_and_exit to attempt to make sure we die gracefully.
Fixed bug with $newfilename not being set properly (scoping issue).
get_id now uses $dbh->{'mysql_insertid'} to grab the last updated auto-increment column value.
&load_backup_xml_tables now uses $dbh->quote to quote values properly
&xml_store_id_table also uses $dbh->quote as well.

Read xml from a file using a regexp match intead of parsing the XML.
Made xml file an input parameter.

Saves data in "xml" format.  More to come.
Reworked process_courselog SQL insert code to use an accumulator so we can
use the same code to do inserts when loading xml data.

Fix comment.

Many changes:
   corrected help text
   changed command line option 'drop' to 'dropwhendone'
   allow user to specify the name of the logfile
   call lonmysql::fix_table_name
   remove zero length files when done processing
   use "--quote-names " with mysqldump
   increase the amount of logging when table creation fails

Change storage of POST values to escape them and join with '&'.  Also modified
error logging to use '&' instead of ':' as a delimiter.

Use delimiters and escape for @values stored in db.

More exit codes.
Re-included "use lib '/home/httpd/lib/perl/Apache';" because it doesn't
work without it :(

lonmysql:Added &table_information, which returns the metadata mysql keeps
    about the tables.
    Modified &update_table_info to turn the MySQL dates (creation,
    update, and check times) into unix times.
parse_activity_log.pl: Use LONCAPA::Configuration to set configuration
    options instead of the removed subroutine &initialize_configuration
    Modified backup handling code - if a table is missing and any of the
    current tables has been modified since the backup file was written,
    back up the current tables (even though one or more is missing) to
    a filename what will not be overwritten automatically, just to
    be sure no data is being lost.
    &load_backup_tables: Now actually use the filename we pass in instead of
    hard coding a file which may not actually exist.

Added file locking.

Gzip activity log backup files.

Added option to make a backup of the activity.log.

Removed $action_table as MySQL really did not like doing joins with it.
logfile now appears in the lonDaemons directory.  Added indicies to
the activity table on the student_id and time columns.

Fix bug where it would read in the entirety of the activities table
instead of just reading in the id tables.

Added logging code.

Moved from modules/matthew to main LONCAPA directory structure.