CVS log for loncom/interface/lonsearchcat.pm

Up to [LON-CAPA] / loncom / interface

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

- For 2.11
  Backport 1.359

- For 2.11.4 (modified)
  Include changes in 1.358, 1.359, 1.360

- Wording Change

- Avoid need for access control check for each record.
  Only add to results if access is public or passphrase-protected (guest).

- Wording change.

- Accommodate argument changes in &selectbox() and &prettyinput() in lonmeta.pm
  rev. 1.255.

- For 2.11.4 (modified)
  Include changes in rev. 1.355, 1.356

- Bug 6907
  - When context is LTI Provider, or deep-linked, and display is embedded in
    another web page, set target to _parent for inline menu links, and help
    item when displaying search results within LON-CAPA's own frames.

- Bug 6907
  When context is LTI Provider, or deep-linked, and display is embedded in
  another web page, set target to _parent for inline menu links, and help
  item when displayng search results within LON-CAPA's own frames.

- For 2.11
  Backport 1.354

- Bug 6936

- Trust settings
  loncommon::domain_select() can take optional args -- refs to arrays of
  included domains, or excluded domains; used to populate drop-down list
  for domains in Advanced search interface.

- Remove unused hash.

- For 2.11
  - Backport 1.351

- Need to call &reload_result_frame() if the hit count from the first server
  contacted exceeds the number to show.

- For 2.11
  - Fix 1.331.4.12 backport of 1.347 and 1.349

- For 2.11
  - Backport 1.350

- If user has domain-level bre priv, but not system-level bre priv,
  both basic search and advanced search are restricted to user's domain.

- For 2.11
  - Backport 1.345, 1.346, 1.347, 1.348, 1.349.

- Modification to change in lonsearchcat.pm rev. 1.347
  - display_url() passes onclick action in call to &lonhtmlcommon::crumbs()
    (which now accepts an additional arg).

Limiting access to "private" resources

Private resources will now only show up in search results for the author and only if they are logged into the server on which it was published.

The use of private resources was discontinued in 2008.  This only applies to older resources.

search: fixed bug 6774 + opening previews in a modal, as with import

further bug fixes, optimizations and cleanup for searches

fixed some bugs in search UI, reduced sleeps for speed optimization in search and general network communication, reduced some timeouts

- For 2.11
  Backport 1.344

- Coding style: keys()

- For 2.11
  - Backport 1.341, 1.342.

- For 2.11
  - Backport 1.341, 1.342.

- Consistent behavior for Content Library items on Main Menu page and
  Content Library Search page when no role selected.

Consistent wording / re-use existing phrases

- For 2.11
  - Backport 1.340

- Display of metadata (linked to on results page) uses modal window, and
  loncommon::headtag() does not include &modal_window() in javascript, if
  called for a frameset.

- For 2.11
  - Backport 1.339

- Modify POD to eliminate errors seen with pod2man in perl 5.18

- For 2.11
  - Backport 1.336, 1.337, 1.338

- Remove some trailing white space.
- Punctuation in documentation.

- Wording change and use quant in &mt() call, to avoid awkward phrasing:
  "There are 1 matches to your query".

- Re-enable search across domains when querying metadata for resources on
  multi-domain servers. (to accommodate changes in lonsearchat rev. 1.322).

- For 2.11
  - Backport 1.335.

- &untiehash() subroutine -- used in course search -- was removed in 1.327,
  when course search was moved to lonsearchcourse.pm.
- remove call to &untiehash(), which is no longer needed, from &cleanup()
  routine so search results are reliably written to lower frame.

- For 2.11
  - Backport 1.334.

- Internationalization: Do not translate twice page title
- lonsearchcat.pm 1.313:
    - Corrected unbalanced tags
    - Officially use debugging leftover

- For 2.11
  - Backport 1.332, 1.333.

- For 2.11
  - "Stored links" instead of "wishlist".

- Localization. Single space between sentences in composite keys.

- Added link to metadata for each resource in the search result lists. CCs an DCs have been missing this.
- New sub routine to display available tools (Stored Links and Metadata)
- Error messages: Added error style and internationalization
- Consistent wording

- For 2.11.
  - Reverse changes in 1.331.4.1. Stored Links ('Wishlist') in 2.11.

- For 2.11.
  - Reverse 1.325, 1.328, 1.331. Wishlist not in 2.11.0.

Fix wishlist link in lonindexer
Eliminate some popups in favor of modal windows

Centralize knowledge of where the tmp directory is within LONCAPA.pm

- Changed the way the pop-up-windows for 'Add Link' and 'Add Folder' are generated (have to call lonwishlist here to get the newest information, i.e. about the existing folders, instead of writing the pop-up-window statically via javascript).

- Split lonwishlist into two modules, one for displaying (handler) and one utility-module, to avoid problems with Apache::constants when calling methods from lonwishlist (e.g. need to call lonwishlist-method in lonmenu and lonmenu is used in printout.pl -> outside of mod_perl).
- Moving entries in wishlist: corrected warning shown when no entry is selected
- Some  code optimizations
- Still some things do not work perfect -> in progress

"Search Course" was only inside of searchcat because the Remote did not
have enough buttons.

Separate the two, incorporate content search under "Content"

Declutter lonsearchcat a little bit before necessary major cleanup.

Save string for folder-dropdown-list (when setting a new link viewing a resource) in user-data. So a call from lonwishlist-routine in lonmenu/lonsearchcat is not necessary (i.e. no issue with modules not supporting Apache::Constants).

Added icon for "Set a link for this resource/folder to wishlist". Therefor added a new column in "Browse Resources" containing the icon. In "Search" the icon is just printed after the resource-title. These aren't optimal position for the wishlistlink-icon. The best would be to create a kind of header for resource-previews. There you could place icons like "Set link to wishlist" and "Printout".

- Bug 6251.
  - Third arg is now a ref to a hash, instead of a hash.
  - New (optional) fourth arg, is an additional attribute for the
    select tag, e.g., an onchange action.
- Update calls to loncommon::select_form() accordingly.

Bugfix - Make resource import in course work again when using the search in the course editor:
Corrected XHTML conformity changes in rev. 1.309 - don't escape "&" in JavaScript code.

Multidomain library servers will only queried once during searches instead of
once for each domain.

Overload was not a good idea. Let the operating system take care of this.

- Add missing & to catalogmode element in query strings for breadcrumb links.

Font size standardisation for filenames, links, crumbs / Bug 6114:
- lonhtmlcommon::crumbs:
    - Eliminated size (font resize) parameter: Always generate code which uses standard font size. Removed deprecated font size tag.
    - Eliminated noformat parameter: The routine should generate content, not special styles or formats. If a different style/format is really needed, it can be added from outside. Due to consistency, currently no special styles/formats are used, only LC_filename. loncommon:CSTR_pageheader needs to be revisited anyway.
    - Use LC_filename for path and filename
    - Only single space before target attribute
    - Standard code indention
- Decrease default font size for LC_filename
- No different font size for links (<a>)
- CSTR Header: only display lastitem when not empty
- Optimized line breaks on browse resource / import resource screen

Basic Search:
- Exclude checkbox from &mt() call
- Highlight domain
- de.pm: wording optimized

Correction: Removed debugging changes which were accidently added in rev. 1.316

Replaced special styles for helpdesk request pages by standard pick_box styles.
Actual difference was only a limited width which moved directly to lonsupportreq.pm.

- Header:
    - Added breadcrumbs to Unactionable Search Query for standard (non-popup) mode
    - Corrected heading typo
- Close button:
    - XHTML: <input> closure
    - Only show <hr> when close button is shown
    - Wrapped close button in <p>
- Added missing <form> closure
- Import button: XHTML: <input> closure

Advanced Search - consistent and nicer layout: use pick_box

Optimized options and search result navigation:
- Separate display options and navigation
- Wrapped display options in fieldset
- Replaced navigation button texts by "<" and ">"; added title with text
- Optimized &mt() usage; Added text "Type"; Consistent option layout (first description, followed by option)
- Optimized nowrap behavior and simplified spacing

Basic search page:
    - Display options in normal font size
    - Moved Search button to bottom of each search area
    - Moved Close button down, right next to Search button
    - Close text only lower case

Other optimizations:
- Replaced deprecated center tag and font tag by appropriate styles
- Error page "Unable to retrieve search results...": added start_page call, localization

XHTML:
    - lower case "onclick" attribute
    - Properly include javascript
    - Added type attribute to script tags

Updated translation files (de.pm, zh.pm)

- Display search results in data_table
- Corrected unbalanced tags:
    - <p>: Course Search
    - <form>: &output_date_error
- Added info, warning, error style to messages
- Wording and &mt() usage optimizations, updated translations files de.pm, zh.pm
- Added missing translations entries (de.pm, zh.pm)
- Added alt attribute to images (currently empty)

Added breadcrumbs to Course Search

Removed unneeded label closure

XHTML: Properly escape "&" in URLs

Removed deprecated attributes from <hr> (noshade, width, align)
and XHTML conform tag closure

Removed outdated no_title option for start_page call.
This option will be new standard (title bar will be removed) and is not needed explicitly anymore.

XHTML:
- XHTML conform <img> tag closure
- Added some missing alt attributes to <img>

Box title is now the main title of the next page and the link text or the former page title will be set to the subtitle in the near future.
Group: consistent wording

consistent wording

- Check that navmap object could be created to avoid ISE when search of discussion postings is selected in Course Search.
- Display link to roles page to re-select the course if no navmap object.

- Originally the <nobr></nobr> encompassed both the viewselect and records select boxes (see rev 1.208).

Replaced
  <nobr>...</nobr>
by
  <span class="LC_nobreak">...</span>

- lonsearchcat.pm: Added missing start tag for nobreak area
- lonsurveyreports.pm: Added missing end tag for nobreak area

- As dates/times stored in the mysql tables are UTC-based, comparisons with current time need to use UTC_TIMESTAMP() for current time.
- * allowed in strings used in searches.

Display "Revise search" button to right of data table containing Search progress information so isn't out of sight below the frame boundary.

- Replaced search status table with LON-CAPA standard table
- Set status fields to readonly

Localization:
Added missing &mt() calls

- Fix typo.

Bug #4106: really evaluate custom rights

Bug #4106: don't show "custom" results

NSDL is dead.

- Align column titles with columns.

Localization:
Added missing &mt() for close buttons

Localization:
- Optimized unhandy &mt() usage for Creation and Modification dates
- Added text to inform about input order for Creation and Modification dates

Localization:
- Added some missing &mt() calls
- Optimized &mt() call for display of search fields

- BUG#4168 inhibit the inline menu in the seraching/browsing popup window.

- remove crumbing from portfolio results

- switch Store to Save

Correct misleading wording

- allow searching for the character . in fields
- modify the parser so - has to come after a space for it to mena not
   (can now search for resource by p-a.guy)

	Bug 5218. Pass form, titleelement and element to advanced search
	so javascript can be built properly.

- missing second arg to &mt

- lonnet libserv removal

- get_servers can take an array of valid domains

Prefer a null value as the option value in a select box when the user chooses "Not specified" - searching in MySQL for IS NULL seems better.

In cases where there are multiple ways to enter metadata for a single field, eliminate storage of comma(s) when the user has not specified anything
for the field.

Advanced search for portfolio files can now include the name(s) of added (course-specified) field(s) with/without the value(s).

addedfieldnames and addedfieldvalues eliminated from the portfolio_metadata table in MySQL - searches do a join on the portfolio_addedfields table instead.

- more re fix ups

Some wording changes in breadcrumbs. Advanced search now functional for portfolios.  Needs additional work on searching based on added metadata fields.

Portfolio searching - work in progress.  Basic search functional; advanced search is not.  Results display needs customization.

- style

* Bug #4941: avoid getting trapped when clicking previewing during search
* Make display of external resources less ugly

Bug #4924: related word actually worked, just didn't display.
Also, with new thesaurus, it is important to limit the output

Bug #4857: Apparently, one cannot have targets on links that trigger submit
forms with targets, even if they are the same. Oh well, no more extra window
when going from Search to RES.

- BUG#4849, adding nav controls to inline search display

* don't forget selected items when switching from Search to Import
* Bug #2674, Bug #563: import from bookmarks
* Bug #3154: visually separate Search and Import in DOCS

Jumping from Search to Import: clicking on filename in crumbs makes no sense

Still trying to export the search results into IMPORT ...

Jump from Search into Resource Space

Clean up: no need for three database files.
Next step: being able to flip back and forth between import and search

&Apache::lonnet::unescape -> &unescape
&Apache::lonnet::escape -> &escape

- make htmlcommon's text box encode the values

- upadte breadcrumbs calls

- start_page + frameset

- trying to move APache::Constats dependency from loncommon.pm

 - some start_page work

- using the loncommon 'simple_error_page'

- er typos

-elimintaing some use of &bodytag in favor of &start_page instead

- converted to use the page start end helpers

Bug #563: show preview in Search

Bug #4531: limit search to queried domain on multidomain machines

Appending the domain restriction to the query itself does nothing.

- Apache 2 doesn't like it when PerlCleanupHandlers don't return an Apache::Constant (like OK or DECLINED ...)

Bug #4644 - Advanced search now works without domain restrictions.
I wonder how this code could ever have worked ...

Bigger cluster, more time needed. Trying as work around for Bug #3377

Bug #4487: sorted searches returned one too few results per page

Bug #4480: by default, do not include external (i.e., NSDL) resources
Bug #4479: better default sorting option

Correctly deal with external URLs

- BUG#4170

- increasing the use of loncommon::get_env_multiple

- err commited a conflict .... fixed that

- ENV -> env

- backport 1.242

- securiity clean on $status was to restrictve, revert to just removing any directory sepeartors

Updates to sorting - added "ascending/descending" input, modified internal
logic of sorting so it now uses lonmeta::fields for most field descriptions,
modified compact and summary views to show the data we're sorting on.

Fix bug where unspecified domain restriction in basic search would cause invalid SQL
to be sent out.

Added 'domain' to SQL queries.

- <html> -> &Apache::lonxml::xmlbegin, thus valid doctypes are now getting output, Yeah! StandardsCmpliance

- backing out the encoding changes for now

- some xhtml cleanups
- one icon -> lonhttpd

- backport 1.237

- I've had too much coffee

- backport 1.236

- setect aborted connections
- searching could go into an infinte loop

- backport 1.235

BUG#3736

<label>s around course search checkboxes.

Adding <label> tag around checkboxes and associated text.

Search results are now sortable.

Discussion postings associated with resources and bulletin boards now
searchable.

Copyright and language searches now remember their values.
Was using \ instead of / in some <br /> tags.
Cleaned up search description output.

Breadcrumbs updated with proper help links.

Major changes to search phrase parsing.  Now use recursive descent parser
to make sense of phrases.  Multi-word (quoted) phrases, - (meaning not),
and 'or'd phrases are handled properly.
Added &process_phrase_input, which creates the SQL query for phrase inputs.
Removed &build_SQL_query and &recursive_SQL_query_build
Added &output_unparsed_phrase_error.
Localized &output_blank_field_error.
For basic search fixed bug where basic search phrase was not remembered
and filled in properly when the "revise search" button was hit.

Include menu link in breadcrumbs when we are not doing a groupsearch.
Added 'authorspace' to the list of searchable fields.

Removed debugging code.
&parse_advanced_search: now spell "SELECT" properly.
&detailed_citation_view: now use 'sequsage_list' instead of 'usage_list'.

&parse_basic_search: Modified formatting of pretty html description of
search.
&detailed_citation_view: Now lists goto, comefrom, sequsage fields, with
links to resources.  Also formats numerical output as needed.

Bug 1030: Multiple words search always fails.
Now split whitespace separated words apart.  Quoted items are handled properly.
Added &process_phrase_input to do this.
Simplified &related_version since it gets more regular inputs now.
Modified &parse_basic_search and &parse_advanced_search to use this new
method.

Added breadcrumbs to search results output.
Corrected misspelling of 'Correct' in &detailed_citation_view.

&display_results now uses buttons instead of links for prev/refresh/next,
allows setting of the number of items to display and the output mode
(compact/detailed/xml/fielded/whatever).
Modified &compact_view and &detailed_citation_view.
Modified &prev_next_buttons to produce buttons instead of links.

Made access count searchable.  Expanded output for xml and fielded search
result views.

Fix bug 2537: Once you do a search with related words you always search
with related words, even if you uncheck the box.  Added
&clean_up_environment to set checkbox values to '' so they will overwrite the
values in the persistent data cache.
Modified &parse_domain_restrictions to not restrict the domain when passed
a value of ''.
Modified &handler to not load in saved parameters when we receive a new
search request and instead call &clean_up_environment.

Bug 2940: Searching for just 'AND', 'OR', or 'NOT' caused internal server
error.  Now it just causes the usual obtuse error message.

Fix to make the 'revise search' button work for advanced searches again.

&detailed_citation_view now shows more data.

Removed old help code pending help rewrite.
Added and implemented dynamic metadata search capability.

Checked every current advanced search form and ensured the mysql query is
generated properly.  Reworked handling of date queries.

Remove debug spew.

Mostly interface changes and changes to translation strings.
Added &reset_timing, &time_left, and modified existing &update_seconds to
encapsulate the logic used to determine how long a search takes and when
to output an update to the search time counter.  This makes &run_search
a little easier to digest.
&prev_next_buttons: reworked logic.
&display_results: Changed refresh from 1 second to 2 seconds.

Reworked basic search interface.  No feature changes, it's just less un-pretty.

&run_search: Actually compute the time remaining before outputing it.
&set_up_table_structure: Copy the metadata table description before
modifying it.  The authoritative copy of the table was being prepended
with an 'id' column multiple times.  Since columns must be uniquely
named MySQL barfed on table creation.
&xml_sgml_view: Use HTML::Entities::encode to encode '<>&' instead of
filling the perl code with &lt; and &gt;

Log MySQL table creation error instead of abusing the user with them.

Bug 2910: negative time remaining in search.  Fixed by setting
time_remaining to 0 when it is less than zero.  Increased the frequency of
time updates (appears to work more consistently now) by updating the time
both before and after we undertake very time consuming tasks (instead of
just after).
Also checked for untitled documents since the document title is used for
linking most of the time.

Added calls to breadcrumbs in &print_(basic|advanced)_search_form
subroutines.

Cosmetic interface changes.

Added &search_html_header() to consolidate a little code.  Removed weird
background color in table.

Added &LONCAPA::lonmetadata::metadata_col_to_hash.
Modified lonpublisher.pm and lonsearchcat.pm to use it instead of the
version in lonmeta.pm.

Typos: "separate"

Modified to use LONCAPA::lonmetadata to create the table.  A number of
whitespace and POD changes were made as well.

Help buttons

Domain and file category selection included again - adv search form done

Date setting improvements. Now has "not set"

Cleanup - saving my work

Start consolidating metadata-related subroutines into lonmeta using
lonmysql

Bugs #789 and 1926: Trying to make new fields searchable, just saving my work.

Produced internal server error, commented out.

Removed many global variables and cleaned up POD.  Working towards bug 2537.

Trivial form name changes so it's easier to debug.

- backport 1.194

- removing some logfile spew

Added a few &mt() calls.

- BUG#2392 added icons to search results

- needed for BUG#1473, the 'Select' link can now also get you the title of a resource back

- backport 1.190

Bug 2258: Advanced search could not import into docs sequences.  The problem
was that the "Advanced Search" link did not pass the 'mode' parameter along
to the advanced search page, so $ENV{'form.mode'} eq ''.  The parameter
is now passed along (lonsearchcat.pm) and groupsort.pm has been modified to
assume a 'simple' mode if the mode is not specified.  Should go in 1.0.2.
I do not want to consider how long this has been broken.

Various small internationalization bugfixes.

Some translations.

Internationalization and German localization

Internationalization, German Localization

Bug 1752: import fails with only one resource.  Updated javascript to detect
the case of a non-array of the form element "returnvalues".

Bug 1845 - domain limited search is no longer default on simple search.

Bug 1503 - search now updates the status before contacting servers.

Fix bug which caused the system to ALWAYS search only in the current domain
if the user ever asked for the search to be limited (basic search only).

Automatically referesh the lower frame every 1 sec if there are no results.

Default to only searching current domain on basic search.

I have seen too many people who don't know what to do next ...

Minor cleanups and an attempt at addressing 1503 by updating the
status line prior to doing network calls.

- should be using the _users_ domain not the servers domain, (lonsequence.pm was already doing this, only by luck was this not blowing up.)

Rewinding changes on window registration.
Attempting to overcome JavaScript security by using POST to change window
location.

Bug #1537: make search frameset register with Remote Control.

- BUG#1445b

trivial readability fix.

Updated lonmysql.pm to allow more than one 'KEY', 'INDEX', etc. entry.

lonmysql:
   Added &replace_row, which is analogus to &store_row but it does a
'REPLACE' instead of 'STORE'.
   Changed somewhat the table definition.  PRIMARY KEY, KEY, INDEX,
UNIQUE [INDEX], and FULLTEXT [INDEX] are now able to be specified.
Previously only FULLTEXT was allowed.

lonsearchcat.pm:
   Updated to use new mechanism for specifying FULLTEXT indexing.

Modified data structure expected by lonmysql::create_table.
$table->{'columns'} is now an array ref instead of a hash ref.

Cleaned up POD in lonmysql.pm

lonsearchcat.pm has been tested and does work.

Make upload filetype "filter" case insensitive
Show course search progress and check for aborted connection
Spelling mistake

Continued work on course search

Search course contents

- Perl doesn't like the {} seems ambigious to it

- double declare

DBI module no longer used inside this module (dbi mysql interfacing is
encapsulated inside lonmysql)

Attempt to better catch and log untied hashes by usage of cleanup handlers.

It will be hard to catch every possibility of the module quitting prematurely
and not untieing the hash. Attempt to use cleanup handler.

Fix handling of mime type searches.  Replaced form.mime with form.extension.
The mime type search is now built up from the form.extension and form.category
fields.  This makes it possible to do both 'file extension' and 'category'
searches.  This addresses bug 843.

Bug #928
Forward "catalogmode" to Advanced Search, making it possible to use
"Advanced" for "Group Search"

Towards Bug 916.  Searching is a little smarter - no longer will we see
'waiting on BLANK'.   Also, the timer will count down only after all the
servers have been contacted.

Fixes for bug 775.

lonsearchcat.pm was modified to take a new parameter, cleargroupsort, which
causes it to clear the GDBM file used by the groupsort.pm handler.
&catalogmode_output was modified to take two new parameters which identify
uniquely the radiobutton it produces and the resource the radiobutton
refers to.
The javascript function queue was modified to take two parameters which
correspond to the two new ones passed to &catalogmode_output.

groupsort.pm was modified to pass 'cleargroupsort' to lonspreadsheet.pm
when the 'new search' button is hit.

Overload protection

Backport of initial fix for bug 775.

Changes to address at least part of bug 775.  Changed the logic which
determines when to clean out the groupsearch database file.  Previously it
only occurred if the results were about to be displayed.  This was unfortunate
because the parameter which triggered it is deleted before then.

Made $bodytag a package parameter.  Call to &Apache::loncommon::bodytag
added in &handler.

Starting to implement common header and color scheme for LON-CAPA handlers
(non-content pages).

Instead of <body bgcolor="#...."><h1>... call

   &Apache::loncommon::bodytag(title,[role],[add_body_parms]);

title: what it says in the header
role (OPTIONAL): override role choice
                 ('admin','coordinator','student','author')
add_body_parms: additional parameters to be put into the body tag, for
                example 'onLoad="init();" or stuff

Colors and layout will likely change in the future, including domain
customization, help function calls, (css?)

Fixes one of the bugs found during the demo - the 'select' buttons now work
for search importing to the rat.  Also fixed a bug with the group import to
the rat.  What a mess.  I hate javascript.  This version should be included
in the new stable.

Fixed bug in domain limited searches.  Cleaned up old debugging code.
Commented out custom metadata input and output code, since it doesn't work
right now anyway.

Added 'show', the number of results to show per page, as a form parameter -
Added 'show', the number of results to display per page, as a form parameter
   defaults to 20 but can be selected to be 10,20,50, or 100.
Added a 'sort' phase, which calls the new &print_sort_form function.
   The sort phase is automatically called after a search is run.
Added a 'revise search' button to run_search and the sort phase.
Added parameter to &output_blank_field_error so it can take you to the right
   search phase (disp_basic or disp_adv) after it chides you.
&ensure_db_and_table is a new function made up of code taken from run_search
   that was also needed for the sort form.  This function connects lonmysql
   to MySQL and makes sure our table is there.
&print_sort_form outputs a sorting form.  Or rather, it WILL do that someday.
   Currently just tells the user about their search and prints a
   'revise search' button.
Changed $Datatypes{'id'}->{'type'} and {'restrictions'} because I am learning
    more about MySQL every day.

Many changes to error reporting and better cleanups on errors/aborted
connections.
Groupsearch and interactive imports (such as from editing a randomlabel problem)
are now handled properly (at least on mozilla 1.0 linux, will test on
'alternative' operating systems soon).
Added new results view, &compact_view, which is the default for groupsearch
and interactive imports.
Removed &write_status.
Changed status reporting system in run_search to output general status,
number of hits, and seconds remaining in search instead of 'servers contacted',
'servers read from' and number of hits.
Change parameters to *_view functions.  Now accept 'prefix' html.
Other minor changes.

Corrected attempt to default to basic search.

Now have real-time status update on search progress.  Search for 'e' and
watch it run!
Use parentheses on GDBM functions to avoid causing GDBM to wimper.
Removed some excess debugging code.
Added search-status output routines.
&run_search now outputs and updates the search status.  Also checks for
    aborted connections to the httpd daemon.
&display_results Produces somewhat tidier HTML.

This version should be considered relatively stable.  If you are able to break
it, please let me know.  To test this version out, I suggest doing a basic
search on 'a'.

&handler(): More error messages given to user.
&get_persistent_form_data has been changed a little.
The MySQL table definition has been changed to match the new requirements of
    lonmysql.pm
&run_search() Attempts to keep the user abreast of the search status
    (more to come).
&display_results now places the 'prev refresh next' links at the bottom of
    the page as well as the top.

This code should NOT be considered stable.
However, it seems to work occasionally.
Frames are now used to contact two httpd daemons.  Results are shown in
pages of 20.
&handler was mostly rewritten.  Hopefully the logic is clearer.
Some function names were changed to reflect that they now output html
directly via $r.
Data persistence is a problem.  Rewrites were done of the previous 'persistent'
functions.  Two new functions were added as well.  This still needs work.
&create_results_table is a new function to create the mysql tables.
&write_status is depricated and will be removed once major debugging stops.
Many changes.  Much wailing and gnashing of teeth.

Major Changes:
    Store persistent values in %persistent_db instead of using hidden form
        elements.
    Global $hidden replaced with $hidden_fields.
    Global $closebutton replaced with local variable in &handler.
&handler: Comments changed.
    Read/write %persistent_db.
    Set $hidden_fields to contain the identifier for %persistent_db.
    $closebutton is now passed to many functions.
&advanced_search_form, &basic_search_form, &parse_advanced_search,
    &parse_basic_search - pass $closebutton instead of having it global.
&make_persistent and &reconstruct_persistent_form_data: new functions/rewrites
    of old functions to store and restore $ENV{'form.*'} values to/from the
    %persistent_db.
&run_search: Recommented & moved some code around.
    Now add time if we are able to contact a server within a few seconds.
    Changed status output text.
&display_buttons is still untested and will remain so until the frames
    interface is completed.
&display_results: Now check the database returns and output reasonable
    error messages.  Moved some code to &catalogmode_output.
Removed the following functions:
    &output_results, &make_popwin, &popwin_js, &popwin_imgupdate

Initial restructuring to use lonmysql.pm.

Currently, the search is run and then results are returned to the user.  This
differs from the 'show results as you get them' approach.  Once the search
interface is changed to allow a second httpd to be used to display the results,
this functionality will be an improvement.  Trust me.

&output_results is deprecated and is no longer used.  Will be removed in
the future.  It is to be replaced with &run_search() and &display_results().

&run_search() sends the search query to servers one at a time instead of all
at once.  Getting hung up on crappy connections is inevitable.  Results are
stored in the mysql database.

&display_results() querys the mysql database and provides the results to the
user.  Sorting of results and searching within results is not yet supported.

A few utility functions have been added:

&write_status will eventually be used to provide the user with information
on the status of the current search.  Currently used for debugging.

&display_buttons is not used yet but will be made to output 'previous' and
'next' links like you are used to seeing on search engines.

&copyright_check is used to determine if a search result should be kept.
This function still requires testing.

&parse_row is used to reconstruct the data structures stored away in the
MySQL table.

A large section of data describing the MySQL table used has been added.

Very beginnings of results database started.
&handler, &parse_basic_search, and &parse_advanced_search
    Added support for 'pretty' version of query string which appears at the
    top of the search results page.
&search_results_header
    Print out the pretty query string.

Renamed "%hash" "%groupsearch_db" in an effort to stamp out useless names.
Added &related_version, which does the thesaurus lookup calls.
Added &searchphrasefield_with_related, which is close to &searchphrasefield.
&handler:
    Output html header *before* starting to send requests, thus letting the
    user know what is slowing us down.
&parse_basic_search and &parse_advanced_search now call &related_version.
&advanced_search_form
    Main table of entry values now has three columns instead of two.
    Call &searchphrasefield_with_related where appropriate.
Changes to the documentation for the HTML form helper functions.

loncommon.pm:
  Removed old thesaurus code, including old global variables.
  Added two global variables:
    %Keywords is a hash of words considered 'keywords' by the thesaurus.
    $thesaurus_db_file holds the path to the thesaurus database file.
  Removed initialization of old thesaurus variables from BEGIN block and added
    initialization of new variables.
  Added:
    &get_related_words($keyword), which will return words related to $keyword.
    &initialize_keywords(), which initializes the %Keywords hash on demand.
  Replaced:
    &keyword() now uses the %Keywords hash, after initializing it.
lonsearchcat.pm:
  Added a checkbox on the advanced search to 'use related words', and
  code to add the words to the users query.  This support is preliminary
  and will change.

Fixed POD bug.  pod2html assumes the html title will be listed after a '-'
following the =head1 NAME line.  If it doesn't find that, it dies.

Changed the logic around selecting a view to fix a bug which caused the
view selection field to not save its values for the 'Revise Search Request'.
This change should make it easier to add new views.
Added global hash %Views, which correlates view descriptions with view
    generating functions.
&handler now sets a default view.
&basic_search_form and &advanced_search_form now use &selectbox to generate
    their view selection forms.
&selectbox now accepts a $title of undef, which results in *only* the
    <select> tag being returned, without any "$TITLE: " nonsense around it.
    Also cleanly handles $anyvalue = undef.
&output_results uses %Views to determine $viewfunction.

Removed debugging code.

&output_results:
    Added checks for copyright.  This cannot be done in lonsql because
    lonsql does not know the username & domain of the searcher.
    Resources not able to be used by the current user are not displayed.

&output_results:
    Removed use of $viewselect and replaced with function pointer code.
    Removed $percent variable which was unused for some reason.
    Minor change to comments.

Minor aesthetic cleanups.
Fixed a 'bug' where selecting the category of files would result in values
being set for mime type (aka file extension).

Changed &advancedsearch and &basicsearch to &parse_advanced_search and
&parse_basic_search, both of which now return sql queries.  Calls to
&Apache::lonnet::metadata_query are now done in &handler().
&output_results no longer requires the sql query be passed to it.
&parse_raw_result tries to be intelligent about author names.

&handler:
    Changed setting of $hidden to be a call to the new version of
        &make_persistent.
&advanced_search_form:
    Handle 'revise search request' cleanly.
&make_persistent
    Takes a parmater as input now.
    Correctly handles non-scalar values.  Previously it output references
        into hidden html forms, which did not work for some reason.
&output_results
    Removed debugging code.

Added domain restricted search interface to &advanced_search_form.
Support added in &advancedsearch.
NOTES:
    Assumes no domain will be named 'any'.
    Standalone machines do not get to choose - the dialog box does not appear.
    Selections are not currently restored on 'revise search request'.
        This is my next priority.
    Selecting none of the domains is the same as selecting 'any'.
    Selecting 'any domain' and other domains as well will search all domains.
    There are no plans for machine-level restrictions at this time.

&advanced_search_form
    Added file categories selection list.
&selectbox
    Removed <br /> between text and selection form.
&output_results
    Handle the file categories via mime type.  Currently the mime fill-in
        form and the file categories selection box are mutually exclusive,
        with the latter overriding the former.

Removed the passing of %ENV to subroutines basicsearch, advancedsearch,
    and output_results.
&basic_search_form
    Changed view selection html.
&advanced_search_form
    Changed view selection html.
    Added <form> tag, allowing the advanced search interface to work.
    View selection element can only appear once in the page, otherwise the
        value passed back to the function is an array, which is just a can
        of worms.
&output_results
    Changed page headers to be more concise.
    Changed view selection code to conform to new methods used in the search
        forms.
    Moved the short horizontal line to appear after results, not before.
&search_results_header
    More POD added.
    Now takes the search mode as a parameter, so the headers and title output
        can be a little more customized.
&detailed_citation_view
    Minor cosmetic changes.

Advanced search interface:
    Changed valid field directions.
    Place submit, reset, view select buttons at head and foot of page.
    Removed "Limit by" text (it should be obvious).
    Changed mime type to 'file extension' fill-in box instead of select box.
    Wrapped part of search interface in <table></table>
&searchphrasefield
    Inserted table tags.
    Changed field length from 80 to 50.
    Removed repetative instructions for entering fields.
&selectbox
    Added POD documentation
    changed $value to $default.
    Provide default function for $functionref.

very minor documentation fix

Removed global variables:
   %hostdomains
   %hostips
   %hitcount
   $yourself
Removed BEGIN block as all it did was initalize the globals above.

More POD documentation - still not enough.
Removed some debugging code from &build_custommetatdata_query.
&output_results():
    Replaced 'WLOOP' with logically equivalent for loop.
    Changed calling structure for *_view routines - now takes a hash as
        input instead of an ordered list.
    Moved results parsing to new subroutine &parse_raw_result, which
        returns a hash to be passed to the *_view routines.
    Moved some custom field handling routines to a new subroutine
        &handle_custom_fields.
    Cleaned up a little of the HTML output.
&parse_raw_result unescapes (hopefully intelligently) its inputs.
&detailed_citation_view, &summary_view, &fielded_format_view, and
    &xml_sgm_view all use the new calling structure (hash).
Removed remainder of old POD documentation - most of this was moved to the
per-function POD documentation.

Cleanups to &output_results.
    Moved basic search results HTML header to a new subroutine.
    Moved search progress pop-up window creation to a new subroutine.
    Replaced "BLOOP" loop with something meaningful.
    Replaced "RLOOP" loop with something meaningful.
    "WLOOP"-ectomy is scheduled for the next commit.
&make_popwin (code from &output_results) now addresses the progress icons
    using img_$domain_$server instead of a number.  We do not need to use
    numbers when we have (javascript friendly?) id strings at our disposal.
&popwin_imgupdate changed to address images with img_$domain_$server instead
    of relying on a number.

$hidden, $scrout, and $yourself are no longer global variables or (in
    the latter case) no longer in the code at all.
Split search interface.  By default the user is sent to the basic search
    and must request the advanced search.
Added &basic_search_form and &advanced_search_form functions.
Removed much verbage from the basic search screen.
Added 'keywords' to the basic search query.
Cleaned up &handler, it is now (I hope) easier to tell what is going on.

Removed globals that were not actually global.

More (much more!) POD documentation.
Minor cleanups:
   Changed yesterdays 'unescape' calls to 'escape'.
   Reformatted regular expression writing regular expression in
       &build_custom_metedata_query() so that a2ps can print it
       out instead of hanging.
   Changes to the form outputing routines (which should later be moved
       to loncommon, I suspect) to be a little nicer.
   Tried to make the html produced a little closer to xhtml standard.

Towards rewrite of search interface and internals.
 * POD documentation started.
 * cleanup of BEGIN to use lonnet hashes instead of re-reading hosts.tab.
 * clean up of old POD documentation.
 * beginnings of cleanups of global variables.

adding selected="true" to the view selection lists (basic and
advanced)

Stupid hack to keep people from pressing "Search", "Search", "Search", ...
when slow in response

Passing on the value of "mode" to groupsort

Changes to support calling from construction space (see edit::search and
lonxml for javascript hooks).  Many code cleanups, including the utility
functions popwin_js and popwin_imgupdate, for communicating with the
search status window.

Minor cleanups, preparing for the road ahead.

fixing handling of metadata title so that it gets passed through

sub BEGIN should be BEGIN

updating documentation to remove deleted hashes -Scott Harrison

use Apache::loncommon added -Scott Harrison

changing so that .tab info reading is done through loncommon.pm -Scott Harrison

causing language to default to 'any'; and temp fix for reading
in filetypes.tab -Scott Harrison

altering void context map blocks to be foreach blocks -Scott Harrison

GPL

make sure to check for .end file every now and then

causing loop to not go forever -Scott

updating documentation; entering POD-style tags -Scott

causing to use a distinct GDBM resource (*_searchcat.db) -Scott

forcing untie of hash at beginning of handler -Scott

masking of metadata info based on user.adv works
also improved time-server cycling... will now
last for 5 minutes.. abort button doesn't yet
work, but plan to have mini-handler for this
soon that produces a semaphore-type file which
is monitored by the status looping

it works better now... works with database (tie .db), passes information
on, group search operates now

debugging and improving interface

cleaning up interface a little more

improvements to interface, adding in server grid

causing window to close upon selection

removing version fields from the interface for the time being -Scott

fixing title only checkbox -Scott

basic search expression works again.  Also, the logical 'NOT'
operator works now -sco

restructuring the code to work correctly with the server-by-server
status response plan -Scott

improving minor logical points in the code as well as beginning
to introduce status bar -Scott

updating dates of changing this piece of code (whodunnit) -Scott

adding comments throughout code as well as function descriptions
at top of code -Scott

inserting a record separator when returning search results -Scott

IT WORKS.  custom metadata is correctly searched for, parsed from
reply files, and shown. -Scott

trying to parse in retrieved custom metadata to display on
output -Scott

okay... custom metadata passing should work now -Scott

making sure that blank lines aren't counted.. -Scott

making sure data is output -
-Scott

more changes to support bleeping custom metadata -Scott

small fixes for filtering customdata and splitting apart fields in lines
-Scott

better changes for finding and passing metadata -Scott

fixing misnamed variable -Scott

correctly parsing out custom metadata information and outputting so
as to debug -Scott

trying to better handle detailed_citation_view.. why isn't extra
show working?? -Scott

beginning to implement showing of custom metadata fields..
just a little more parsing.. -Scott

passing custom-metadata query information into the network layer -Scott

ok.. this should work okay enough for now for custom metadata -Scott

!@#$%^&*() more debugging.. -Scott

ok.. now should work -Scott

more debugging needed .. term does not work for multi-valued logic
statements -Scott

the match expression should actually work... now to just traverse the
directories *.meta files and use it! (this is for custom-metadata) -Scott

substituting on the correct matchexp variable -Scott

trying to implement xml tag matching for building custom-metadata query.
also removing single quotes from persistent data fields -Scott

trying to do more to xml-ify the output as well as support preservation
of double-quotes -Scott

grumble.. problems with html understanding a backslashed double quote
it seems.. -Scott

reordering statements to make better sense in make_persistent subroutine -Scott

implemented make_persistent function.. will be sure to backslash double-quotes
to not interfere with html syntax -Scott

beginning to implement custom metadata.. also modularizing the
passing of persistent search data -Scott

fixing date calculation -Scott

fixing missing parenthesis -Scott

full implementation of date processing now -Scot

beginning to implement first stages of metadata date search field
processing -Scott

fixing sql syntax error -Scott

expanding implementation of advanced searching -Scott

tidying up the output formats a little more -Scott

actually implementing the form kinds of output now -Scott

okay.. think I fixed the problem now.. -Scott

more debugging -Scott

debugging test -Scott

adding in sleep command to make sure database has time to make
return .. this is a temporary fix -Scott

fixing some missing variable references in view-formatting subroutines -Scott

adding multiple view formats for the results; just bells and
whistles on top of current interface -Scott

small fix in total sql query formatting -Scott

fixing stupid mistake -Scott

fixing syntax error (missing right brace) -Scott

fixing up bug with the advancedsearch and adding in ability to
select different formats of metadata record output -Scott

implementing more of the advanced search feature -Scott

advanced search is working to the point of hovering 1 inch of the ground
for a brief period of time, or something like that -Scott

adding in mode-based result outputting as well as implementing the building
of sql query expression from logical and/or phrase statements -Scott

filled in missing portions of the beginning glossary.  beginning to
implement mode-based output of results (basic or advanced) -Scott

entered in a commented glossary of functions at the beginning and wrote
skeleton of the build_SQL_query function (which will access Text::Query
module methods).  Also included Text::Query module. -Scott

fixing stupid bug which resulted in missing entry bug for
advanced search -Scott

fixing a few bugs, removing unnecessary statement from
basicsearch subroutine, make advancedsearch subroutine
do something small at first -Scott

beginning to allow for real output for advanced search -Scott

removing unused variable -Scott

removing test output -Scott

altering test output again -Scott

test output of concat argument -Scott

running basic search expression against all fields -Scott

some imperfect logic to enforce filling in some of the advanced
search query fields -Scott

okay.. maybe the scalar variable was nonexistent.. yeah -Scott

why doesn't this work? -Scott

grrr.. -Scott

trying to get the right logic working the right way.. story of my life -Scott

filled method is not working -Scott

modularizing the testing of filled query fields (and length isn't
that good a method for it anyway) -Scott

fixing missing parenthesis -Scott

causing output to end upon finding error and also directing
output of advancedsearch message to right location -Scott

fixing syntax error -Scott

trying to implement error-checking for blank search entry -Scott

removing spurious '/' -Scott

prepending http:// to search result url links for obvious reasons -Scott

fixing source of errors (suppose ENV needs to be global or
something at some point..) -Scott

remodularizing to begin to accommodate multiple responses from
servers.. beginning to implement advanced search .. preparing
to error-check for blank search forms.. -Scott

implementing hyperlinking to search result items
(so that a user can preview a resource before adding
it to a RAT map) -Scott

a better test url reference.. -Scott

trying out a hyperlink (probably won't work here..).. ideally
I want to get base href syntax going -Scott

entering in a field for customized metadata fields -Scott

improper arguments to the new lonnet metadata_query function -Scott

moving the nuts and bolts of the networking of the query
request from lonsearchcat.pm to lonnet.pm -Scott

modularizing the code and adding close button

small fix -Scott

implementing more fields to be present in the metadata
database -Scott

modularizing the code more into subroutines and conditionally
outputting various "things" when in interactive mode -Scott

beginning to pass information back to one of the rat interfaces
now... this isn't the "great" way to do it (with pjump-like
commands), but it works -Scott

barely works -Scott

moving logo in correct position -Scott

adding logo -Scott

introducing a semi-decent interface for the first query page -Scott

the form element interface for the search catalog -Scott

Trying to make ugly things look pretty