[BACK]Return to lpml_parse.pl CVS log [TXT] [DIR] Up to [LON-CAPA] / loncom / build

Diff for /loncom/build/lpml_parse.pl between versions 1.38 and 1.47

version 1.38, 2002/01/31 17:08:40 version 1.47, 2002/04/26 15:53:00
Line 1 Line 1
 #!/usr/bin/perl  #!/usr/bin/perl
   
   # -------------------------------------------------------- Documentation notice
   # Run "perldoc ./lpml_parse.pl" in order to best view the software
   # documentation internalized in this program.
   
   # --------------------------------------------------------- Distribution notice
   # This script is distributed with the LPML software project available at
   # http://lpml.sourceforge.net
   
   # --------------------------------------------------------- License Information
 # The LearningOnline Network with CAPA  # The LearningOnline Network with CAPA
 # lpml_parse.pl - Linux Packaging Markup Language parser  # lpml_parse.pl - Linux Packaging Markup Language parser
 #  #
 # $Id$  # $Id$
 #  #
 # Written by Scott Harrison, harris41@msu.edu  # Written by Scott Harrison, codeharrison@yahoo.com
 #  #
 # Copyright Michigan State University Board of Trustees  # Copyright Michigan State University Board of Trustees
 #  #
Line 37 Line 46
 # 11/4,11/5,11/6,11/7,11/16,11/17 - Scott Harrison  # 11/4,11/5,11/6,11/7,11/16,11/17 - Scott Harrison
 # 12/2,12/3,12/4,12/5,12/6,12/13,12/19,12/29 - Scott Harrison  # 12/2,12/3,12/4,12/5,12/6,12/13,12/19,12/29 - Scott Harrison
 # YEAR=2002  # YEAR=2002
 # 1/8,1/9 - Scott Harrison  # 1/8,1/9,1/29,1/31,2/5,3/21,4/8,4/12 - Scott Harrison
 #  #
 ###  ###
   
Line 68  use HTML::TokeParser; Line 77  use HTML::TokeParser;
 my $usage=<<END;  my $usage=<<END;
 **** ERROR ERROR ERROR ERROR ****  **** ERROR ERROR ERROR ERROR ****
 Usage is for lpml file to come in through standard input.  Usage is for lpml file to come in through standard input.
 1st argument is the mode of parsing.  1st argument is the mode of parsing:
 2nd argument is the category permissions to use (runtime or development)      install,configinstall,build,rpm,dpkg,htmldoc,textdoc,status
 3rd argument is the distribution (default,redhat6.2,debian2.2,redhat7.1,etc).  2nd argument is the category permissions to use:
       typical choices: runtime,development
   3rd argument is the distribution:
       typical choices: default,redhat6.2,debian2.2,redhat7
 4th argument is to manually specify a sourceroot.  4th argument is to manually specify a sourceroot.
 5th argument is to manually specify a targetroot.  5th argument is to manually specify a targetroot.
   
Line 80  Example: Line 92  Example:
   
 cat ../../doc/loncapafiles.lpml |\\  cat ../../doc/loncapafiles.lpml |\\
 perl lpml_parse.pl html development default /home/sherbert/loncapa /tmp/install  perl lpml_parse.pl html development default /home/sherbert/loncapa /tmp/install
   
   For more information, type "perldoc lpml_parse.pl".
 END  END
   
 # ------------------------------------------------- Grab command line arguments  # ------------------------------------------------- Grab command line arguments
   
 my $mode;  my $mode='';
 if (@ARGV==5) {  if (@ARGV==5) {
     $mode = shift @ARGV;      $mode = shift @ARGV;
 }  }
Line 95  else { Line 109  else {
     exit -1; # exit with error status      exit -1; # exit with error status
 }  }
   
 my $categorytype;  my $categorytype='';
 if (@ARGV) {  if (@ARGV) {
     $categorytype = shift @ARGV;      $categorytype = shift @ARGV;
 }  }
   
 my $dist;  my $dist='';
 if (@ARGV) {  if (@ARGV) {
     $dist = shift @ARGV;      $dist = shift @ARGV;
 }  }
   
 my $targetroot;  my $targetroot='';
 my $sourceroot;  my $sourceroot='';
 my $targetrootarg;  my $targetrootarg='';
 my $sourcerootarg;  my $sourcerootarg='';
 if (@ARGV) {  if (@ARGV) {
     $sourceroot = shift @ARGV;      $sourceroot = shift @ARGV;
 }  }
 if (@ARGV) {  if (@ARGV) {
     $targetroot = shift @ARGV;      $targetroot = shift @ARGV;
 }  }
 $sourceroot=~s/\/$//;  $sourceroot=~s/\/$//; # remove trailing directory slash
 $targetroot=~s/\/$//;  $targetroot=~s/\/$//; # remove trailing directory slash
 $sourcerootarg=$sourceroot;  $sourcerootarg=$sourceroot;
 $targetrootarg=$targetroot;  $targetrootarg=$targetroot;
   
 my $logcmd='| tee -a WARNINGS';  my $logcmd='| tee -a WARNINGS';
   
 my $invocation;  my $invocation; # Record how the program was invoked
 # --------------------------------------------------- Record program invocation  # --------------------------------------------------- Record program invocation
 if ($mode eq 'install' or $mode eq 'configinstall' or $mode eq 'build') {  if ($mode eq 'install' or $mode eq 'configinstall' or $mode eq 'build') {
     $invocation=(<<END);      $invocation=(<<END);
Line 135  if ($mode eq 'install' or $mode eq 'conf Line 149  if ($mode eq 'install' or $mode eq 'conf
 END  END
 }  }
   
 # ---------------------------------------------------- Start first pass through  # -------------------------- Start first pass through (just gather information)
 my @parsecontents = <>;  my @parsecontents=<>;
 my $parsestring = join('',@parsecontents);  my $parsestring=join('',@parsecontents);
 my $outstring;  
   
 # Need to make a pass through and figure out what defaults are  # Need to make a pass through and figure out what defaults are
 # overrided.  Top-down overriding strategy (leaves don't know  # overrided.  Top-down overriding strategy (tree leaves don't know
 # about distant leaves).  # about distant tree leaves).
   
 my @hierarchy;  my @hierarchy;
 $hierarchy[0]=0;  $hierarchy[0]=0;
Line 151  my $token; Line 164  my $token;
 $parser = HTML::TokeParser->new(\$parsestring) or  $parser = HTML::TokeParser->new(\$parsestring) or
     die('can\'t create TokeParser object');      die('can\'t create TokeParser object');
 $parser->xml_mode('1');  $parser->xml_mode('1');
 my %hash;  my %setting;
 my $key;  
 while ($token = $parser->get_token()) {  # Values for the %setting hash
   my $defaultset=1; # a default setting exists for a key
   my $distset=2; # a distribution setting exists for a key
                  # (overrides default setting)
   
   my $key=''; # this is a unique key identifier (the token name with its
               # coordinates inside the hierarchy)
   while ($token = $parser->get_token()) { # navigate through $parsestring
     if ($token->[0] eq 'S') {      if ($token->[0] eq 'S') {
  $hloc++;   $hloc++;
  $hierarchy[$hloc]++;   $hierarchy[$hloc]++;
  $key=$token->[1].join(',',@hierarchy[0..($hloc-1)]);   $key=$token->[1].join(',',@hierarchy[0..($hloc-1)]);
  my $thisdist=' '.$token->[2]{'dist'}.' ';   my $thisdist=' '.$token->[2]{'dist'}.' ';
  if ($thisdist eq ' default ') {   if ($thisdist eq ' default ') {
     $hash{$key}=1; # there is a default setting for this key      $setting{$key}=$defaultset;
  }   }
  elsif ($dist && $hash{$key}==1 && $thisdist=~/\s$dist\s/) {   elsif (length($dist)>0 &&
     $hash{$key}=2; # disregard default setting for this key if         $setting{$key}==$defaultset &&
                    # there is a directly requested distribution match         $thisdist=~/\s$dist\s/) {
       $setting{$key}=$distset;
                      # disregard default setting for this key if
                      # there is a directly requested distribution match
                      # (in other words, there must first be a default
              # setting for a key in order for it to be overridden)
  }   }
     }      }
     if ($token->[0] eq 'E') {      if ($token->[0] eq 'E') {
Line 172  while ($token = $parser->get_token()) { Line 197  while ($token = $parser->get_token()) {
     }      }
 }  }
   
 # --------------------------------------------------- Start second pass through  # - Start second pass through (clean up the string to allow for easy rendering)
 undef $hloc;  
 undef @hierarchy;  # The string is cleaned up so that there is no white-space surrounding any
 undef $parser;  # XML tag.  White-space inside text 'T' elements is preserved.
 $hierarchy[0]=0;  
   # Clear up memory
   undef($hloc);
   undef(@hierarchy);
   undef($parser);
   $hierarchy[0]=0; # initialize hierarchy
 $parser = HTML::TokeParser->new(\$parsestring) or  $parser = HTML::TokeParser->new(\$parsestring) or
     die('can\'t create TokeParser object');      die('can\'t create TokeParser object');
 $parser->xml_mode('1');  $parser->xml_mode('1');
 my $cleanstring;  my $cleanstring; # contains the output of the second step
 while ($token = $parser->get_token()) {  while ($token = $parser->get_token()) { # navigate through $parsestring
     if ($token->[0] eq 'S') {      if ($token->[0] eq 'S') { # a start tag
  $hloc++;   $hloc++;
  $hierarchy[$hloc]++;   $hierarchy[$hloc]++;
  $key=$token->[1].join(',',@hierarchy[0..($hloc-1)]);   $key=$token->[1].join(',',@hierarchy[0..($hloc-1)]);
  my $thisdist=' '.$token->[2]{'dist'}.' ';  
    # Surround tagdist (the dist attribute of an XML tag)
    # with white-space to allow for uniform searching a few
    # lines below here.
    my $tagdist=' '.$token->[2]{'dist'}.' ';
   
  # This conditional clause is set up to ignore two sets   # This conditional clause is set up to ignore two sets
  # of invalid conditions before accepting entry into   # of invalid conditions before accepting entry into
  # the cleanstring.   # $cleanstring.
  if ($hash{$key}==2 and  
     !($thisdist eq '  ' or $thisdist =~/\s$dist\s/)) {   # Condition #1: Ignore this part of the string if the tag 
    # has a superior distribution-specific setting and the tag
    # being evaluated has a dist setting something other than
    # blank or $dist.
    if ($setting{$key}==$distset and
       !($tagdist eq '  ' or $tagdist =~/\s$dist\s/)) {
     if ($token->[4]!~/\/>$/) {      if ($token->[4]!~/\/>$/) {
  $parser->get_tag('/'.$token->[1]);   $parser->get_tag('/'.$token->[1]);
  $hloc--;   $hloc--;
     }      }
  }   }
  elsif ($thisdist ne '  ' and $thisdist!~/\s$dist\s/ and   # Condition #2: Ignore this part of the string if the tag has
        !($thisdist eq ' default ' and $hash{$key}!=2)) {   # is not blank and does not equal dist and
    # either does not equal default or it has a prior $dist-specific
    # setting.
    elsif ($tagdist ne '  ' and $tagdist!~/\s$dist\s/ and
          !($tagdist eq ' default ' and $setting{$key}!=$distset)) {
     if ($token->[4]!~/\/>$/) {      if ($token->[4]!~/\/>$/) {
  $parser->get_tag('/'.$token->[1]);   $parser->get_tag('/'.$token->[1]);
  $hloc--;   $hloc--;
     }      }
  }   }
    # In other words, output to $cleanstring if the tag is dist=default
    # or if the tag is set to dist=$dist for the first time.  And, always
    # output when dist='' is not present.
  else {   else {
     $cleanstring.=$token->[4];      $cleanstring.=$token->[4];
  }   }
  if ($token->[4]=~/\/>$/) {  
     $hloc--;  
  }  
     }      }
     if ($token->[0] eq 'E') {      # Note: this loop DOES work with <tag /> style markup as well as
       # <tag></tag> style markup since I always check for $token->[4] ending
       # with "/>".
       if ($token->[0] eq 'E') { # an end tag
  $cleanstring.=$token->[2];   $cleanstring.=$token->[2];
  $hloc--;   $hloc--;
     }      }
     if ($token->[0] eq 'T') {      if ($token->[0] eq 'T') { # text contents inside tags
  $cleanstring.=$token->[1];   $cleanstring.=$token->[1];
     }      }
 }  }
 $cleanstring=&trim($cleanstring);  $cleanstring=&trim($cleanstring);
 $cleanstring=~s/\>\s*\n\s*\</\>\</g;  $cleanstring=~s/\>\s*\n\s*\</\>\</g;
   
 # ---------------------------------------------------- Start final pass through  # -------------------------------------------- Start final (third) pass through
   
 # storage variables  # storage variables
 my $lpml;  my $lpml;
Line 949  sub format_directory { Line 996  sub format_directory {
  my ($chmod,$chown)=split(/\s/,$categoryhash{$categoryname});   my ($chmod,$chown)=split(/\s/,$categoryhash{$categoryname});
  return $directory="\n<tr><td rowspan='2' bgcolor='#ffffff'>".   return $directory="\n<tr><td rowspan='2' bgcolor='#ffffff'>".
     "$categoryname</td>".      "$categoryname</td>".
     "<td rowspan='2' bgcolor='#ffffff'><!-- POSTEVAL [$categoryname] verify.pl directory /$targetdir $categoryhash{$categoryname} -->&nbsp;</td>".      "<td rowspan='2' bgcolor='#ffffff'><!-- POSTEVAL [$categoryname] ".
       "verify.pl directory /$targetdir $categoryhash{$categoryname} -->".
       "&nbsp;</td>".
     "<td rowspan='2' bgcolor='#ffffff'>$chmod<br />$chown</td>".      "<td rowspan='2' bgcolor='#ffffff'>$chmod<br />$chown</td>".
     "<td bgcolor='#ffffff'>$thtml</td></tr>".      "<td bgcolor='#ffffff'>$thtml</td></tr>".
     "<tr><td bgcolor='#ffffff' colspan='[{{{{{DPATHLENGTH}}}}}]'>".      "<tr><td bgcolor='#ffffff' colspan='[{{{{{DPATHLENGTH}}}}}]'>".
Line 1007  sub format_description { Line 1056  sub format_description {
 sub format_files {  sub format_files {
     my $text=$parser->get_text('/files');      my $text=$parser->get_text('/files');
     $parser->get_tag('/files');      $parser->get_tag('/files');
     if ($mode eq 'html') {      if ($mode eq 'MANIFEST') {
    return $text;
       }
       elsif ($mode eq 'html') {
  return $directories="\n<br />&nbsp;<br />".   return $directories="\n<br />&nbsp;<br />".
     "<a name='files' />".      "<a name='files' />".
     "<font size='+2'>Files</font><br />&nbsp;<br />".      "<font size='+2'>Files</font><br />&nbsp;<br />".
Line 1125  sub format_file { Line 1177  sub format_file {
     $categorycount{$categoryname}++;      $categorycount{$categoryname}++;
     if ($source) {      if ($source) {
  $parser->get_tag('/file');   $parser->get_tag('/file');
  if ($mode eq 'html') {   if ($mode eq 'MANIFEST') {
       my $command=$build;
       if ($command!~/\s/) {
    $command=~s/\/([^\/]*)$//;
       }
       else {
    $command=~s/(.*?\/)([^\/]+\s+.*)$/$1/;
       }
       $command=~s/^$sourceroot\///;
       my (@deps)=split(/\;/,$dependencies);
       my $retval=join("\n",($source,
          (map {"$command$_"} @deps)));
       return $retval."\n";
    }
    elsif ($mode eq 'html') {
     return ($file="\n<!-- FILESORT:$target -->".      return ($file="\n<!-- FILESORT:$target -->".
     "<tr>".      "<tr>".
     "<td><!-- POSTEVAL [$categoryname] verify.pl file '$sourcerootarg' ".            "<td><!-- POSTEVAL [$categoryname] verify.pl file '$sourcerootarg' ".
     "'$targetrootarg' ".      "'$targetrootarg' ".
     "'$source' '$target' ".      "'$source' '$target' ".
     "$categoryhash{$categoryname} -->&nbsp;</td><td>".      "$categoryhash{$categoryname} -->&nbsp;</td><td>".
Line 1228  END Line 1294  END
  $logcmd.' && echo "'.   $logcmd.' && echo "'.
  'Configuration source file does not exist '.   'Configuration source file does not exist '.
  ''.$sourceroot.'/'.$source.'"'.   ''.$sourceroot.'/'.$source.'"'.
  "$logcmd); } && perl verifymodown.pl ${targetroot}/${target} \"$categoryhash{$categoryname}\"$logcmd;\n\n";        "$logcmd); } && perl verifymodown.pl ${targetroot}/${target} \"".
    "$categoryhash{$categoryname}\"$logcmd;\n\n";
  }   }
  elsif ($mode eq 'build' && $build) {   elsif ($mode eq 'build' && $build) {
     push @buildall,$sourceroot.'/'.$source;      push @buildall,$sourceroot.'/'.$source;
Line 1335  sub format_fileglob { Line 1402  sub format_fileglob {
     $categorycount{$categoryname}+=scalar(@semi)+1;      $categorycount{$categoryname}+=scalar(@semi)+1;
     if ($sourcedir) {      if ($sourcedir) {
  $parser->get_tag('/fileglob');   $parser->get_tag('/fileglob');
  if ($mode eq 'html') {   if ($mode eq 'MANIFEST') {
            return join("\n",(map {"$sourcedir$_"} split(/\;/,$filenames2)))."\n";
    }
    elsif ($mode eq 'html') {
     return $fileglob="\n<tr>".      return $fileglob="\n<tr>".
  "<td><!-- POSTEVAL [$categoryname] verify.pl fileglob '$sourcerootarg' ".        "<td><!-- POSTEVAL [$categoryname] verify.pl fileglob '$sourcerootarg' ".
  "'$targetrootarg' ".   "'$targetrootarg' ".
  "'$glob' '$sourcedir' '$filenames2' '$targetdir' ".   "'$glob' '$sourcedir' '$filenames2' '$targetdir' ".
  "$categoryhash{$categoryname} -->&nbsp;</td>".   "$categoryhash{$categoryname} -->&nbsp;</td>".
Line 1453  sub format_build { Line 1523  sub format_build {
     if ($text) {      if ($text) {
  $parser->get_tag('/build');   $parser->get_tag('/build');
  $build=$sourceroot.'/'.$text.';'.$tokeninfo[2]{'trigger'};   $build=$sourceroot.'/'.$text.';'.$tokeninfo[2]{'trigger'};
    $build=~s/([^\\])\\\s+/$1/g; # allow for lines split onto new lines
     }      }
     return '';      return '';
 }  }
Line 1545  sub trim { Line 1616  sub trim {
   
 # ----------------------------------- POD (plain old documentation, CPAN style)  # ----------------------------------- POD (plain old documentation, CPAN style)
   
   =pod
   
 =head1 NAME  =head1 NAME
   
 lpml_parse.pl - This is meant to parse files meeting the lpml document type.  lpml_parse.pl - This is meant to parse files meeting the lpml document type.
 See lpml.dtd.  LPML=Linux Packaging Markup Language.  
   
 =head1 SYNOPSIS  =head1 SYNOPSIS
   
 Usage is for lpml file to come in through standard input.  <STDIN> | perl lpml_parse.pl <MODE> <CATEGORY> <DIST> <SOURCE> <TARGET>
   
   Usage is for the lpml file to come in through standard input.
   
 =over 4  =over 4
   
Line 1584  Only the 1st argument is mandatory for t Line 1658  Only the 1st argument is mandatory for t
 Example:  Example:
   
 cat ../../doc/loncapafiles.lpml |\\  cat ../../doc/loncapafiles.lpml |\\
 perl lpml_parse.pl html default /home/sherbert/loncapa /tmp/install  perl lpml_parse.pl html runtime default /home/sherbert/loncapa /tmp/install
   
 =head1 DESCRIPTION  =head1 DESCRIPTION
   
 I am using a multiple pass-through approach to parsing  The general flow of the script is to get command line arguments, run through
 the lpml file.  This saves memory and makes sure the server  the XML document three times, and output according to any desired mode:
 will never be overloaded.  install, configinstall, build, rpm, dpkg, htmldoc, textdoc, and status.
   
   A number of coding decisions are made according to the following principle:
   installation software must be stand-alone.  Therefore, for instance, I try
   not to use the GetOpt::Long module or any other perl modules.  (I do however
   use HTML::TokeParser.)  I also have tried to keep all the MODES of
   parsing inside this file.  Therefore, format_TAG subroutines are fairly
   lengthy with their conditional logic.  A more "elegant" solution might
   be to dynamically register the parsing mode and subroutines, or maybe even work
   with stylesheets.  However, in order to make this the installation back-bone
   of choice, there are advantages for HAVING EVERYTHING IN ONE FILE.
   This way, the LPML installation software does not have to rely on OTHER
   installation software (a chicken versus the egg problem).  Besides, I would
   suggest the modes of parsing are fairly constant: install, configinstall,
   build, rpm, dpkg, htmldoc, textdoc, and status.
   
   Another coding decision is about using a multiple pass-through approach to
   parsing the lpml file.  This saves memory and makes sure the server will never
   be overloaded.  During the first pass-through, the script gathers information
   specific as to resolving what tags with what 'dist=' attributes are to be used.
   During the second pass-through, the script cleans up white-space surrounding
   the XML tags, and filters through the tags based on information regarding the
   'dist=' attributes (information gathered in the first pass-through).
   The third and final pass-through involves formatting and rendering the XML
   into whatever XML mode is chosen: install, configinstall, build, rpm, dpkg,
   htmldoc, textdoc, and status.
   
   The hierarchy mandated by the DTD does not always correspond to the hierarchy
   that is sensible for a Makefile.  For instance, in a Makefile it is sensible
   that soft-links are installed after files.  However, in an LPML document, it
   is sensible that files and links be considered together and the writer of the
   LPML document should be free to place things in whatever order makes best
   sense in terms of LOOKING at the information.  The complication that arises
   is that the parser needs to have a memory for passing values from
   leaves on the XML tree to higher-up branches.  Currently, this memory is
   hard-coded (like with the @links array), but it may benefit from a more
   formal approach in the future.
   
 =head1 README  =head1 README
   
 I am using a multiple pass-through approach to parsing  This parses an LPML file to generate information useful for
 the lpml file.  This saves memory and makes sure the server  source to target installation, compilation, filesystem status
 will never be overloaded.  checking, RPM and Debian software packaging, and documentation.
   
   More information on LPML is available at http://lpml.sourceforge.net.
   
 =head1 PREREQUISITES  =head1 PREREQUISITES
   
Line 1610  linux Line 1722  linux
   
 =head1 SCRIPT CATEGORIES  =head1 SCRIPT CATEGORIES
   
 Packaging/Administrative  UNIX/System_administration
   
   =head1 AUTHOR
   
    Scott Harrison
    codeharrison@yahoo.com
   
   Please let me know how/if you are finding this script useful and
   any/all suggestions.  -Scott
   
 =cut  =cut
Removed from v.1.38  
changed lines
  Added in v.1.47

FreeBSD-CVSweb <freebsd-cvsweb@FreeBSD.org>