CVS log for loncom/publisher/lonpublisher.pm

Up to [LON-CAPA] / loncom / publisher

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

- For 2.11
  Backport 1.300

- For 2.11
  Backport 1.304

- For 2.11.4 (modified)
  Include changes in 1.304

- Bug 6991
  Always include &notify() as one of the handlers in the PerlCleanupHandler
  phase whenever a lock is set.

- When publishing directory in course authoring space custom rights from
  default.rights always apply.

- Breadcrumbs text when creating new file in Course Authoring Space.
- All resources published in Course Authoring Space receive custom
  copyright/distribution, as defined in default.rights file.

- "In course" authoring.
  Copyright/Distibution field for resources in "course author" space is
  'custom', and Custom Distribution File is /res/domain/courseid/default.rights

- For 2.11.4 (modified)
  Include changes in 1.300

- Add public to options when applying common copyright/distribution when
  publishing directory.

- For 2.11
  Backport 1.298, 1.299

- Options for publishing a directory split into types.
- Options to exclude modified files and/or files with modified metadata
  when publishing directory (e.g., to avoid unwanted changes when updating
  copyright and/or source availability for an entire directory.
- For skipped files display reason file was skipped.

- When publishing a directory include option to exclude currently unpublished
  resources (e.g., if only change being made is copyright/distribution or
  source availability for already published files).

- Course Editor has "Standard Problem" item (Grading tab) for creation
  of new problem in user's Authoring Space, or in a Course "Authoring" Space.
- Course Editor has "Import from Course Resources" item (Import tab) to
  import published content from Course "Authoring" Space.
- Course "Authoring" Space
  - default.rights -- course-only access
  - quota is shared with content uploaded directly to course
  - content only browsable in course context
  - metadata not included in searchable meatdata MySQL table
CVs: ----------------------------------------------------------------------

- Allow &batchpublish() to be called by other packages.
  - %addid hash made a global (populated in BEGIN() block.
  - %nokey hash passed as a reference to &batchpublish(),
    &publishdirectory(), &publish().
  - New routine: &getnokey() populates %nokey hash.
  - New arg -- $usebuffer -- passed to &batchpublish() and
    &phasetwo(), when true causes output to be stored in scalar
    instead of sent to $r->print().

- New options when publishing directory -- apply common
  copyright/distribution and/or apply a common source availability.

- Coding style:
  - keys() sort()
  - for readability eliminate use of $_

- Support style of packaging .js dependency used in Camtasia Studio 8.4.2
  *_player.html files.
- Dependency on spritesheet.min.css no longer included as <link> tag in
  Camtasia's top-levl *.html file, so explicitly include in %allow.

- allow tag not needed if image data is included directly in <img> tag's
  src attribute using a "data URI".

- Bug 6671
 - Support changes in packaging of mp4 files between Camtasia 8.1
   and Camtasia 8.4.

- Additional arg for set_allow() - $type, which is the attribute (in the tag)
  which requires an allow to be set.
- For $type = src or href remove any query string appended to the url before
  setting the value of the src in the allow tag.

Improved and consistent navigation and layout by using actionbox and moving button below pick_box

- Indentation in POD

- Modify POD to eliminate errors seen with pod2man in perl 5.18

- Bug 6671. Detect dependencies in javascript used for Camtasia player.
  - Appropriate <allow src="%%path%%"> appended to HTML file in authoring space,
    at first publication step for files referenced in javascript object in text
    within <script></script> via:
      set.FlashPlayerSwf
      set.MediaSrc
      set.XMPSrc
      set.ConfigurationSrc
      set.PosterImageSrc
    e.g., TSC.playerConfiguration.set.FlashPlayerSwf("somepath/file_controller.swf")
    to support upload of Camtasia player directly to a course (Camtasia 8.1).
    - Similar to changes in xml/scripttag.pm rev 1.169 vs. 1.166

- Reverse one change in rev 1.273 (bug 1320) so recursive publication
  of subdirectories works again.

Actions after resource publication:
- Consistent function calls and layout - use actionbox which is now available (improves rev. 1.262 and 1.264)
- 'Diffs with Current Version' link: XHTML

XHTML: action attribute in <form> tag required (dummy attribute added)

- Replace term: "Construction Space" with "Authoring Space" for
  consistency with type of role used to access it, and action taken there.

- Use correct module for &display() routine.

- &constructaccess() moved from loncacc.pm to lonnet.pm
  - to facilitate re-use, and separate from handler,
  - for use in lonnet.pm (use Apache::Constants() not supported there.

Some comments

- Remove debug line.

- &authorspace() routine now takes an argument: $uri
  so "Construction Space" breadcrumb points at appropriate author space
  for resource/directory being viewed or acted om.
  - current role could be different or ("no role" - i.e., cm), so is only
    used to determine authorspace when $uri is unavailable.

- use values from Apache::lonnet::perlvar in place of static strings.

- Fix Construction breadcrumb trails for /adm/cfile, /adm/publish,
  /adm/retrieve, /adm/upload.

- Fix Construction breadcrumb trails for /adm/publish /adm/retrieve
  - loncommon::CSTR_pageheader() takes optional argument
    - name of file or directory used to construct breadcrumb trail
      where $env{'request.filename'} is inappropriate.
- use lonDocRoot perlvar in place of static string: '/home/httpd/html'
- coding style
   - eliminate leaning toothpicks

- Bug 1320 -update POD.

Bug #1320

Bug #1320

Bug #1320

Bug #1320 ...

Read the metafiles

Bug #1320: the passed filename should always be of the type
/priv/domain/user/...
instead of being different for different handlers

Don't remember nothing for title

Clean way to cancel publication

- Backport 1.264 (part), 1.265.

- Customization for GCI_3
  - Equivalent to fix for bug #6139 for .tmp files.
  - Remove local tn-preview files on re-publication.
   (preview files have .tn extension when generated for GCI Test Assembly).

Part of Bug #6139: remove local tmp-preview files on re-publication

- Reverse part of 1.262, as assistant co-authors now have 'bro' priv which allows browsing of /res for areas in which user is a co-author.
- Wrap "functions" footer with loinks to source etc. in &head_subbox().

- Backport 1.263.

- Backport 1.263.

- Backport 1.261.

- Backport 1.252.

- Bug 6122.
  - &notify (includes session lock clean-up) needs to be at start of chain of PerlCleanupHandlers so $env{'user.environment is still defined during clean-up.

- Fix typo in 1.258.2.1.

- Backport 1.261 and part of 1.262.

Optimized post publishing menu:
- Assistent Co-Authors are not allowed to browse resource space, so don't offer link to published resource
- Replaced deprecated font tag, etc. by functionlist and removed now unnecessary <hr> separator lines

- Replace hardcoded path.

Changes related to modification of LON-CAPA screen header.
Added dynamic link to Construction Space to breadcrumbs

Changes related to modification of LON-CAPA screen header.
start_page call:
- Added breadcrumbs
- Added head_subbox and included CSTR standard header
  (crumbs in header doesn't work here - will be checked and changed later)

- Replaced missusage of <h1> to highlight text by standard warning styles
- Corrected <p>-tag closure
- Added standard pick_box to publish directory (inlcuding removal of another wrong usage of <h1>)
Co-Author username display:
    - Consistent username:domain output
    - Now also display plainname and not only username

XHTML conform selected/checked/multiple HTML attributes
and optimized spacing

XHTML:
- Replaced
checked="1"
by
checked="checked"

- Optimized/Corrected white spaces

- Optimized &mt usage and style for "Created directory" messages
- Changed order: First create directory, then display "Created directory..."
- Moved "Done" message to a better place right after the log display but before the menu at the bottom of the page

- Optimized wording
- Optimized &mt usage
- Replaced harcoded styles by LON-CAPA standard styles
- Optimized and corrected some German translations

- XHTML conform checked attributes in input tags
- Optimized white spaces usage

Existing versioned file which _contains_ name of published file can inflate $maxversion check during phase two.
  - Need to check directory contents for /^RegExp$/, not /RegExp$/.

consistent wording, catalog information -> metadata

consistent wording, catalog information -> metadata.

Allow publication of problems that do not have responses if they are
importing libraries, finishing up Bug #5696.
This is just sort of a workaround, assuming that people who import
libraries know what they are doing.

- The past tense of "occur" is occurred.

- Improved data-tabling for keywords table.

- Optimized dependencies warnings (linebreaks; div; only show headline, if warnings occur)
- Don't display horizontal line anymore, if resource is without errors
- Wrapped some messages in div-Tags instead of using linebreaks for nicer output
- Added warning style to "enter metadata" message
- Added missing &mt() calls to store metadata error messages
- Optimized some &mt() calls

this is a work simplification for german authors for present.

Removed debug data (-> 1.243), sorry

Corrected and optimized keyword table:
  - Row counter corrected - now first row with same amount of columns
  - Added missing standard row start / end calls (-> 1.241)
  - Proper ending of table (fill up with "empty" td)

Optimized display of warnings and errors
  - Added headline
  - Use LON-CAPA standard style
  - output as list instead of one after the other
  - optimized &mt() usage
  - Changed order: first warnings, then errors

- Standard LON-CAPA datatable for keywords table.

- Eliminate double line at the end of "Resource Details" and "Metadata Information" tables.

- Shortened main headline and added new sub headline "Resource Details"
- Wrapped resource details in LON-CAPA standard table
- Added info text which asks the author to fill in the metadata
- Added missing and optimzied some &mt() calls
- Moved Finalize/Make Obsolete buttons out of metadata table
- Optimized screen output with some <br /> and <p> tags
- Added error style to output "...document contains errors..."
- Extended output "...without responses..."

- Wrapped main part of publication screen into standard LON-CAPA pick box table
- Removed hardcoded styles
- Moved grade level selections into one row
- Made some attributes XHTML conform

ToDo:
- Tidy up upper part of the screen
- Find better solution for help links

Bug #4106: really evaluate custom rights

Bug #5696: Friends don't let friends publish problems without responses

Bug #5704: have to ignore everything in <answer>-environment

Locking

Work on locks.

- Style.
- Eliminate unhandy &mt() call which included implicit linefeed.

Don't let the user shoot into their foot with variable filenames.

Localization:
Added missing &mt() for 'Select' and 'Search'

Death to "private"

- passing corrected argument 3, BUG#5355

- BUG#5286 warn about rights delay on publish

- some css

- was trying to notify the host 'meta.subscription' that ther was an update

- switch Store to Save

- remove an all_hostnames call

- protexting username...

- starting work on moving to distributed DNS, eliminate usage of the lonnet hostanme global

- BUG#5207

- since _ is valid in metadata tags now, need to use a different temp
  separator in the return from &metaeval, switch to using \0 for this

1. Add missing ;

2. In cases where <image></image> encloses a definition of a dynamic plot via <gnuplot></gnuplot> rather than an actual image file form RES space, a spurious dependency on the user's top level directory could be added on publication because of line feeds.

- removing log spew

- lonid can have . and - now

- more re fix ups

- this change not needed for 2.2.2

- backpor 1.213 1.214

- domain is a required field for a resource to be found

- remove some log spew
- reinstate the announcement of the synching of the metadata database was successful
- update the call to update_metadata to new callind convention

- BUG#5014, when modifying .meta files need to advertise that fact in the
     directoy browser
    - additionaly when doing full dir repbulishes, republish if the .meta file has changed
- also add support for retrieving a previoulsy published .meta file
- add support for getting diffs between previous .meta file revisions

- some @ -> : conversion

- BUG#4911 - select  search links in Publish screen was missing neccessary js

&Apache::lonnet::unescape -> &unescape
&Apache::lonnet::escape -> &escape

- start_page

- when checking for missing  or duped ids, need to be in xml_mode

- lets make the obsolete checkbox actually work

- when publishing a directory it was possible that it didn't do the id number fixups because %addid wasn't initialized

- BUG#4563 message spam

Bug #4264: you don't depend on zombies

- BUG#4304

- adding <label>

Call to &Apache::loncommon::fileembstyle(page) returns 'ssi'.  This was preventing publication of pages.

Saving my work on Bug #4127 - still having problems with "Invalid access"

After recently changing resource version for a course, &symbverify fails
in spite of both the URL and the given symb being correct.

Not sure what else needs to be reset.

COMING SOON!? Probably not ...

Was not dealing consistently with return codes from phasetwo

Bug 3392 - don't allow ':::', etc
Bug 4113 - require extensions

Bug #3956: force directory metadata to override existing
Bug #3352: make obsolete
Bug #3119: edit metadata now actually remembers things

Saving my work: force publication of directories

Work on force publication using different copyright, etc.

- ENV -> env

- extra line

-likely should specifiy a target when doing an ssi, as it was by default wpicking grade, then web which meant it thatought there where 00 rather that 0 warnings.
- inhibit the ok if nothing is wrong, it just looks odd where it is

- BUG#4014, was trying to do ssi on everything, not just ssi able docs

* Be able to make new library files
* Check for existing filename on new file generation was defunct
* Bug #3443: Copyright/Distribution Help was misplaced on screen and outdated

Do document-checking via ssi-call
Do not let people publish documents with errors

- id=" was causing havoc (BUG#3993)

Eventually, we have to stop people from publishing broken stuff.

- backport 1.184

- BUG#3855, there was no way to avoid avoid forced republication of all files.

Aargh ... we clean out the environment when rendering in Phase 1, so extra
environment variables are forgotten between runs during batch.

The beauty of persistent memory: stuff URLs into a global var instead.

Cannot get the push-business to work. Saving my work for later.

Bug #1104: publication takes too long

Put notifications into cleanup phase - seems too good to be true.

- bUG#3530, select/search on publish screen

added <label>s around keywords.

- .tmp files are the thumbnail renderings of a resource

- BUG#3177, forgot to move some of the fix over

- BUG#3177, silly regexp was causing massize slowdown on large <script> blocks

- if a file consisted of
...
</body>
</html>text

(Basically text after the last html tag in the file)

The <allow>s were never inserted.


This should now insert the <allow>s in front of the last tag even if there is text after the last tag in the file

(BUG#2995)

* show sourcerights in dir listing
* dial in the previously set value of sourceavail on re-publish
* fix paths for sourcerights-file the same as for customrights

I don't think anybody used this, but now it gets fixed anyway.

&store_metadata now uses lonmetadata to do the heavy lifting.

More Bug #78 with new description file.. <sprays more stylepolice-repellant>

Aarghh ...

Better fix for Bug #154

Bug #154: multiword_keywords_work_now

-BUG#2994 at least
  - directory publication was not correctly setting some values, like dependencies, gradelevel, MIME TYPE, etc. fixed by removving the 2 code paths and rather just haveiong the one path work for both modes, probably easier to maintian that way

Added &LONCAPA::lonmetadata::metadata_col_to_hash.
Modified lonpublisher.pm and lonsearchcat.pm to use it instead of the
version in lonmeta.pm.

- trying to be UTF8 transparent

- BUG#2454, don't say /adm/navmpss is currently available when publishing a sequence

- backporting support for CSTR on multidomained machines into 1.1.X

- backport 1.163 to 1.1.1 for use on s17

- supporting authors on multidomain machines
  - this works, but requires that all users that author have unique usernames
    ( I.e. albertel@basement and albertel@annarbor will end up accessing the same /home/albertel/public_html)
- I plan on isntalling this on s17.lite.msu.edu to support the serway@brookscole user

- backport part of 1.149, and 1.162

- not the best fix in the word, but rules need to be added together not overwrite.
  probably should create some kind of list of which metadata appends and which overwrite.

- need to do the &parseformatea() after we read the existing meat file and build to old parameters hash, otherwise BUG#2542

Work on bug #2335 - directory catalog info available in RES

Bug #2444: now erases error messages also on republication.

Small bugfixes, standards and gradelevel info on directory level

The uppercasing does not work correctly in all languages.

Start consolidating metadata-related subroutines into lonmeta using
lonmysql

Update metadata entry on re-publication

This was in the wrong place, needed to go into phase 2.

No need to have the same subroutine unsqltime twice - lonmysql is the better
place for it.

Was trying to store UNIX timestamps into MySQL DATE field

a) Cannot do this anymore, since it would destroy dynamic metadata
b) It did not work anyway

Work on Bugs #789 and #1926: gradelevel and standards info

* get nestedness of default.meta files correct
* warn and abort if copyright=custom and no valid rights file given

- BUG#2466 <APPLET CODE="something">  was case sensitive and thus ignoring the CODE

Added a few &mt() calls.

Added some localization statements.

- BUG#1904

Nice big warning message that obsolete parameters are a dangerous thing.

Bug #2350: Oops, had overcompartmentalized this.

Make gathering of metadata hierachical, rather than gather all data encountered
along the way.

Take care of multi-line titles, etc.

-BUG #1984, publication should notice when default.meta has a relative path file and update the relative path

- retabinate

Bug #1715 "obsolete marking" and internationalization

- answering the FIXME

Internationalization and "obsolete" field.

- backport 1.135 to 1.0.2

- need to add <allow>s for <imageresponse> <image>s

Internationalization, German Localization

- now uses ssi to get the metadata,  stops some breakage that was occuring due to not going through the lonhomework.pm handler

- Fixes BUG#2196, batch publication was bleeding values between publications, (If you didn't set a title everything after the first one to have a title got the same title)

- backport 1.130 1.131

- probably should do the same with <tex> btw BUG#2005,2058 are fixed with this

- publishing was screwing up scripts and displays, especially when doing things like multiline strings. use get_all_text_unbalanced now to quickly seek to the end of the script/display tags.

Bug #551: small extra window for previews, etc

Bug #1198
Jay Ihry
Add "Done" and link when done with directory publishing.

Documentation updates.

Missing =back screwing up LaTeX render.

For co-author, log who actually did the change.

Have very limited connectivity, need to do big commit while online.
Will call this JULYone in Bugzilla. Fixes several small bugs.

- from Mark Lucas, BUG# 1897, better fonts in publishing screen, picks up default language more intelligently

- the first one of these fixes BUG #1669, the rest of these I did cause I of BUG#1669

Bug #1346 - default copyright should be "default", not "custom"

- XHTMLizeing

Towards bug #662 - parse rights files for metadata

- trying to use a file named ).problem

- sorry about that

nate
eVS: ----------------------------------------------------------------------

Towards bugs #662 - add customized rights of use (multiple domains,
specific courses, etc).
* Adds new extension ".rights" and handler lonrights.pm for editing and
viewing (not implemented yet)
* Adds new "copyright" metadata value "custom"
* Adds new metadata-field "customdistributionfile" to ".rights" file
(active only if copyright=custom)
* Author can cover large number of resources by one ".rights" file -
if distribution changes, only that one files needs republishing (like
a ".library" file)

- XHTMLing

- if an error occurs during phase one of publishing don't do phasetwo

- sorry committed some debug stuff

- don't need to use lonhomework here, doesn't use it directly, and causes an use circle

Actually print the error message

A file or directory name with "___" would be so bad (symb delimiter) that it
is worth blocking in publication.

best wishes to all.

- fixes BUG#918
- allows \n inside <m>

- Checks for duplicated IDs, dosent allow publishing if this condition exists
- BUG#804

Toward bug 492 - make metadata parser results available to other modules.

- reversed the diff sense, it always seemed backwards to me

Toward bug 839

With recent changes this had lots of double slashes - looks ugly and confusing

Notifies "subscribed" courses.

- retabinate phasetwo
- rflush the publishing output
- fix directory publishing to not skip the /

Toward bug 531.

Can address most recent version.
Can find out with lonnet::getversion what the most recent version is
Shows versions in lonmeta

Fixes Bug #111, toward Bug #531
Can now explicitly address current version of a resource, not just previous
ones.

Does so by generating a symbolic link from the "versioned" filename to the
current file. Deletes symbolic link again when file is republished, and
subsequently replaces it by the now-stale "real" file.

Still has problems with picking up keywords.

Continuing work on mass publication.

Toward bug 482

asymptotically approaching brain explosion as I lower the "truly amazing"
coefficient of this module; POD sections should be in caps;
increasing documentation of various subroutines; XHTML-ifying most of the
output; one spelling fix

Fix 2 bugs: put dates into sql format using &sqltime, like it used to.
Used to only pick up checked keywords if there was more than one.

- fixes bug#257, moved resevaldata.db to nohist_resevaldata.db, and deletes hist file

- After installing these diffs you need to do a make install before restarting the webserver

Fixed bonehead logic error.

POD Documentation update.

Bug 561 - nonexistant metadata table produces a user-visible error now.
Changed to use lonmysql to access the MySQL database and do the insertion/
deletion of elements.
Removed some old POD documentation.

- removing some debugging code

- should handle < and > inside <m> when publishing

- refactorized some of the publish subroutine (only 310 lines long now)

- Now have .subscription files, implemntation should be backward compatible and also should start migrating subscriptions over
- BUG#332

Added help to the Publishing page as a trial run for the help system.
Required splitting the .tex pages a bit more.

Bug 517 - puts the <allow> statements in front of the last closing tag and
hopefully does not add any more spurious \n's. Should in fact clean up
those blank line on re-publish.

- &Apache::lonxml::xmlparse now wants a request obect passed along

- modified all known calls of xmlparse to take pass a request object

- added code to timeout a <script> block that runs more that the configured timeout parameter of seconds. (LON-CAPA defaults to 300)

- supports the image <label>s in a <randomlabel> problem

Modified to use loncommon::get_unprocessed_cgi().  BUG 90.

Fix another bug.

No longer crash when there are no keywords.

Added 'check all' and 'uncheck all' buttons for keywords.

- address BUG#68 and BUG#153, also helps out with BUG#179
- metadata is now stored with Entities escaped, making it easy and possible to put HTML in metadata
- publication protects Entities beofre dumping them into the edit windos this should fix any issues with publishing Entities in metadata.
- attributes stay encoded during publication

Minor verbage change: "view published version" instead of "view target"

- <allow> tags shouldn't be generated for non absolute or relative (internal) hrefs

- stops saying "Currently Not Available" on blank allows (maps often have blanks resources.)
- now sorts depdencies, looks prettier

- can find <IMG SRC="" > for allows
- added bgimg as a type it can find

Not changing source to produce absolute URLs, but allows only. NOT TESTED!!!

fixing chparms, functional reference, and @_[0] warning messages

- crewing up mailto: also

- don't canonconize javascript and # anchors

Fix keywords and auto-identify

changing reference to lonnet to loncommon; adding in POD documentation
-Scott Harrison

using loncommon routines for .tab info; cleaning up map statements -Scott Harrison

Private mode does not sync database. access.conf not read, use lonnet cache

- added back second button

Now also keeps track of resource usage in RAT docs

- added the oft requested second publish button

Stores dependencies to metadata

Stores usage

Check dependencies

- should work with <tag /> properly now.

- now it actually lowercases

- Whoops, for to put the s on the substitution

 - GPL headers added

 - prevent publisher from automatically downcasing all tags

- typo

Length limitation for keyword search

Last couple of my commits trash.

Last version not so hot.

part.id instead of part_id (hopefully)

causing table output of keywords to have consistent number of columns

making a less memory intensive parsing of words

Public publication not an option for maps

Lists dependencies and recognizes background images as such

Accepts filename from query string

Fakes $ENV{'request.uri'} for the sake of recognizing default problem type

Extra commas and lowercase end tags

External URL fixup.

Bug fix - empty URLs do not need fixup.

Deal with codebase in applets, allow access to source dirs

Collects metadata for binary types.

Deal with <allow>

Does URL fixup.

Start on URL cleanup

Gets rid of spurious "obsolete" message for packages

Towards making packages work

More actions in construction space

adding in change date at top

changing sql database connection to just output a warning
instead of terminating the publication

Make londiff work for co-author

Accepts co-author role

Can call diff from here

deglobalizing a few variables, calculating sql-formatted date-time strings,
reading in PerlVar's so that the sql database can be accessed.. -Scott

adding in sql metadata database updating feature -Scott

Handles other MIME types than XML-like

Go Back buttons at end of publication

Separate publish directory, and start "retrieve"

Also update metadata only subscribers when publishing resource.

- I didn't understand what was going on, need that line

- shoudln't need to do that anymore

- chaged includes line

Spare closing bracket in the middle of dirlist

Publish directory

Does not die anymore if there is no previously existing metadata

Now uses File::Copy for file copying to avoid possible buffer overflows.

Now handles xml and html files. Replication not tested. Could have problems
with buffer overflow for large binary files (copy goes by "lines")

Implementing Phase Two after metadata form display

Language selection in alphabetical order
New copyright input
No call to lonhomework, just "use" the right stuff

Added language to metadata input

Input screen

Processes hierachical metadata files

Now calls lonproblem subhandler for metadata on problems

Finds potential keywords

Now fixes IDs and indexes

Reads addid.tab, makes backup copy

Checks file, etc

New publishing handler, now called with post and target parent