CVS log for loncom/lonsql

Up to [LON-CAPA] / loncom

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

- s-nail requires space instead of comma as separator between email addresses.

- Forgot Password utility
  - Support configurable: link lifetime, case sensitivity (for username and
    e-mail address), information required, e-mail types, and custom text.
  - Bug 6648: captcha can be enabled for public-facing web form.

- Replace use of mailto (metamail package) with mail command.
  metamail was in Red Hat 7.3, but is absent from Fedora, CentOS, RHEL etc.

- Domain configuration to override domain's helpdesk settings for e-mail
  recipients, and optionally added text, based on requester's affiliation
  (e.g., faculty, staff or student).

- Bug 5596
  - Message shown that initial retrieval of institutional data may take time.
  - Progress indicator displayed while enrollment in progress
  - Where institutional data are available for a user being added, that
    replaces data included in file upload.
  - Student/employee ID checking for both new users and existing users
    if rule(s) specified in localenroll.pm for domain, and enabled via:
    domain configuration > User Creation > Format rules to check for IDs.

- Bug 5596.
  Add a routine to lonnet.pm -- get_multiple_instusers() which makes one call
  to lond > lonsql > localenroll.pm to retrieve institutional data
  for multiple users when adding users via file upload, to minimize number
  of sleep() commands needed. Supports up to 1s per query, on localenroll.pm
  side if adding more than 100 new users.

- Add new routine to localenroll.pm -- &get_multusersinfo() -- to retrieve
    institutional data for users being added via user file upload.

  Note: if this routine does not exist in localenroll.pm, will fall-back
  to retrieving institutional data using a separate call to &get_userinfo()
  for each user.

- Modify POD to eliminate errors seen with pod2man in perl 5.18

- Re-enable search across domains when querying metadata for resources on
  multi-domain servers. (to accommodate changes in lonsearchat rev. 1.322).

- use lonDocRoot perlvar in place of static string: '/home/httpd/html'.

- scope

Namespacing of usernames.
lonnet.pm
 - &get_instuser() can check the institutional directory for a username or an ID.
                   -returns hash of user information (lastname,firstname etc.)
 - &inst_rulecheck() can check whether a username matches the formats of institutional username rules.
 - &inst_userrules() can retrieve information about username rules defined for the domain.
&get_instuser uses lonsql as a conduit to localenroll::get_userinfo() - operating in mode 1.
&inst_rulecheck() and &inst_userrules() use lond directly as the conduit to localenroll::username_check() and localenroll::username_rules() respectively

lond
 - &get_institutional_user_rules() is interface to localenroll::username_rules()
 - &institutional_username_check() is interface to localenroll::username_check()

lonsql
 - &get_inst_user() retrieves is interface to localenroll::get_userinfo()

lonnet.pm
- usersearch() made more similar to other routines which use lonsql
   (records split on \n not &).
  - srchby, srchtype, and srchterm passed as separate arguments
  - backwards compatibility maintained with older lonsql
lonsql
- usersearch() query processing moved to subroutine.
- searchdomain implicitly from lonsql, isntead of arg in query
- backwards compatibility maintained with older lonnet:usersearch()
- escaping of result more consistent with other lonsql queries (replacement of & with \n in lond::reply_query_handler() now possible).

Institutional directory search now reported as unavailable in the case where a DC has enabled directory searching via "Modify domain configuration" but either:
 (a) localenroll::get_userinfo() has not been customized, or
 (b) an error ioccurred when querying the institutional directory.

lonsql
- &do_inst_dir_search() returns 'unavailable' if the response from localenroll::get_userinfo() is not 'ok'

lonnet.pm
- &inst_directory_query() now returns a scalar and a hash.
  - scalar is the outcome of the query: 'unavailable', 'ok' or ''.
  - hash contains the search results (if any matches)

loncreateuser.pm
- Report to user that institutional directory search is unavailable if response from &lonnet::inst_directory_query() is not 'ok'.

localenroll.pm
- documentation updated
- &get_userinfo() returns 'unavailable' by default

Add "begins with" type searches - matched where target (uname, lastname, etc.) begins with search term.

Remove any spaces adjacent to the comma separator in a lastname,firstname search.

- moving inst dir queries into lonsql

- support domain with - . in them

- Add support for query set to 'usersearch'
- Searches an allusers MySQL table containing: username, domain, lastname, firstname, middlename, generation, id and permanentemail for all users in the domain.
- Can search by username, lastname or lastname,firstname
- Seach can require exact match, or can look for cases where search term is part of a longer string.

- Add support for query set to 'allusers'
- Updates MySQL table with new information for a particular user.
- Added subroutine: allusers_table_update() to access subroutines in LONCAPA::lonmetadata used to update the allusers table.

- switch to using lonnet.pm
- eliminate unneeded code
- remove unused packages

- lonsql
  - Need to escape results from userlog queries (escaping of all results from lonsql eliminated in 1.50).
  - Allow filtering of user's activity.log for 'Role'
  - Include host in information returned for userlog query

- Autoupdate.pl
  - &lonnet::get_my_roles() by default gets roles from nohist_userroles.  Use in Autoupdate wants information about a user's own roles (from roles.db) - use context as argument
  - warning messagea when username changed (assumes fixed student ID). prints to screen
for now (log to file would be preferred)

- lonnet.pm
  - documentation for &userfileupload
  - get_my_roles() now includes optional context which determines which db file is target for dump (default is nohist_userroles, context of 'userroles' sets target to roles.db
  - documentation for &userlog_query() - now includes hostID
  - documentation updated for &get_my_roles().

Renaming and deletion of portfolio files now causes removal of records in MySQL portfolio_metadata and portfolio_addedfield tables for old file name.

Arguments passed to lonsql via a querysend to lond changed - $arg1 now colon-separated uname,domain,group; $arg3 -now action (e.g., delete).

portfolio_metadata, portfolio_addedfields and portfolio_access tables in MySQL updated when portfolio access controls are modified or when portfolio metadata is updated.

Some routines in searchcat.pl moved to LONCAPA::lonmetadata.pm so they are available to lonsql when performing MySQL table updates on the library server which houses the portfolio files.

metadata updates and access control updates from lonmeta and lonnet::modify_access_controls are routed through lonnet::update_portfolio_table() which in turn does a querysend to dispatch a query to lonsql (via lond).

Only access control settings for public and passphrase-protected controls are stored in the portfolio_access MySQL table.

- removing some code duplication

- backport 1.73,74

9- remove log spew

- backport 1.75

- eval the new photo routines so lond/sql don't die in the middle of them

Bug #4531: limit search to queried domain on multidomain machines

Prepare to do filtering of lonsql results based on the domain that lond is
running under

- &reply() needs to connect to the correct location

- style police

Add support for student photo import from an institutional repository. Availability of photos of registered students in a course controlled by course environment parameter: 'internal.studentphoto'.  This may be configured to require initial acceptance of conditions of use by course owner.  View classlist in ENRL, and Automated Enrollment Manager now includes option to display thumbnails of student photos.  Nightly enrollment update can import student photos for students added to a course.  Student photos can be updated via the Automated Enrollment Manager.

Log spam

Bug 4441. Accommodation of non-word characters in institutional codes. Sanity checking now needed.

- if we can connect remove the file the says we can't

- IP removal from hosts.tab

-  4 is enough

Removed some commented out code, specify full path to parse_activity_log.pl,
additional logging code is intentional.

Updates for activity log work.

Changes to support autoenroll calls from a remote server.

lond on the homeserver for the course(s) handles requests from the remote
server for institutional data (e.g., classlists, valid institutional courseIDs,
institutional section numbers for a course code, validation of course owners), by
calling the appropriate functions in the homeserver's localenroll.pm

All replies are made directly with the exception of fetch_enrollment_query, which
is shipped over to lonsql, in case retrieval of institutional classlists is a
protracted process.

lonsql on the homeserver for the course(s) calls localenroll::fetch_enrollment()
and writes XML files of enrollment data to /home/httpd/perl/tmp

Transfer of classlist data occurs later following an autoretrieve call from the
remote server. It is planned to generalize this function and add encryption to the transfer back to the client.

Autoenroll.pl called by cron on a library server, now only carries out updates for
courses in its domain, for which the library server is the course's homeserver. If a domain has multiple library servers Autoenroll.pl will need to be run on each library server.

Bug 3026: Remove logging spew.

Removed extra newline in lonsql return value.

-more xhtmlized

Removed out of date POD.  Report database errors using $sth->errstr() when
there is an error, instead of using $@.

Simple one-line status for cluster-wide automated reporting

SessionThre2.html from the Guts manual.

- cleaning out unnessceary comments

I don't know whether to laugh or cry.

loncapa_apache.conf should not be directly specified (and certainly not
before loncapa.conf)

This appears to work, but frankly I wouldn't trust it.  Fixed enough of the
errors that it runs, forks, and returns search results.  Theoretically, it
should also do the logfile searches, too.

Many, many changes.
THIS CODE SHOULD NOT BE USED AT THIS TIME.  It does not run.

Added POD comments everywhere.
Now use File::Find instead of requiring find.pl, which I could never find.
Now use strict.
Rearranged most of the code.
Added &do_sql_query.
&userlog and &courselog were not modified but were moved.

Removed superflous re-escaping of results.  The offending call to escape
resulted in there being no '&' symbols in the returned file.  lond
translates '&' to "\n" when it writes the file.  lonsearchcat expects
one record per line.

Handle doubly-used delimiter "&" in activity.log, install more filters for
userlog.

Values in logs are escaped

Toward bug 121 - filters start to work now.

lonsearchcat display definitely broken due to "escape" bugfix in lonsql.

Toward bug 121.
lonnet now has routines to query course and user activity logs.
lonsql userlog already does some stuff.
Left to do: implement filters!
BUGFIX: lonsql was not escaping the query result. A ":" in any of the fields
would truncate the reply. lonsearchcat likely has to be adapted to unescape.

New routines for userlog and courselog queries.

BUGFIX: $result had the wrong scope.

Towards bug 121

If the query command is 'logquery' instead of an SQL command, lonsql will
go into another program block. Functionality to query logs should be
implemented there.

Also bugfix: a query-reply must ALWAYS have an "\n" in the end, or the
remote end will wait forever. Previous statement "append if $result" might
lead to lengthy timeout results if database is offline.

Replaced access.conf with loncapa_apache.conf.

using LONCAPA::Configuration::read_conf
BUG 129

adding POD; removing void context map statements -Scott Harrison

updating documentation -Scott

minor edits to documentation -Scott

email admins if their sql setup is bad -Scott

never output regular metadata in custom metadata output...
single or multiple lines...

One forth of the initial prefork

important to be case sensitive when retrieving custom-limited information
-Scott

only looking at current versions for home user resources .. adding
in this filter -Scott

subreply HAS to be called to get the information back to the lond
layer -Scott

 moving extraneous new line -Scott

using sql password present in root read-only /etc/httpd/conf/access.conf

ok.. putting in safeguard to make sure lonsql NEVER starts when
the database is down -Scott

make sure that the child dies immediately if cannot connect to database
-Scott

trying to get custom information without a specific custom query -Scott

wrong argument passed to sql prepare statement -Scott

okay... custom metadata passing should work now -Scott

trying to make sure query is logged for debuggiing.. -Scott

minor improvements... -Scott

forgot a period for adding .meta suffix.. the shame -Scott

enabling debug statement again -Scott

more changes to support bleeping custom metadata -Scott

better changes for finding and passing metadata -Scott

do not send unnecessary newlines and be sure to send the url field -Scott

filtering out relevant part of .meta files to send regarding custom
information -Scott

okay.. need to make sure newline is in right place.. done with
debugging .-Scott

oh.. the newline thing.. -Scott

more debuggin grrrrrrrrrrr -Scott

debugging this sob bug -Scott

trying

am actually passing relevant custom metadata information..
this may change the "logistics" of passing resource metadata
in the future.. -Scott

beginning to traverse the resource file tree for .meta and
actually execute custom metadata query -Scott

commenting in how I will reconcile sql-query results with custom metadata
query results -Scott

grabbing list of all .meta files -Scott

logging attempt at a custom query

accepting input for custom metadata query and commented where I need
to implement directory traversal/*.meta searching -Scott

just adding note about authorship -Scott

alteration to stupid password -Scott

Check if already running

implemented error-free fetching

improving functionality

file for refereeing database/SQL processes