Return to lonchart.test_procedure CVS log

Up to [LON-CAPA] / doc / homework

Updated this document to include testing of pages, reuse of the same
.problem, sorting, and the student active status.  Also to fix a couple
of typos in the last submission.  This document corresponds to the class
created by Jason Stredwick called Chart Test on the 103 domain on machine
103l5.

    1: lonchart Test Procedure
    2: 
    3:         The procedure will show the validity of the lonchart code.  A course 
    4: called Chart Test was created to test the possible scenarios due to course 
    5: creation.  Next a series of false students will take the course.  Each student 
    6: will perform differently.  The number of students will be relate directly to
    7: the number of grade related tests required.  The combination of these two sets 
    8: of tests will ensure the proper output of lonchart.
    9:         The course was designed to test the following: problems in the top 
   10: level sequence, problems in a sequece separated by a subsequence, problems with
   11: parts, reuse of the same problem, using a page, and the correctness of a single
   12: problem resource in general.  The course is laid out beginning with a top 
   13: sequence containing in order, start, problem, sequence, problem, page, problem,
   14: and finish resources.  The subsequence contains two problems.  The first 
   15: problem is a general problem containing a simple numeric response.  The second 
   16: problem contains two parts.  The page contains three problems with two parts, 
   17: but all three problems are the same.
   18:         The chart output should contain three columns.  The first should have 
   19: three problems in it, the second should have three, and the third should have 
   20: six problems.  The headings for these columns should be the title of the 
   21: sequences in which they lie.
   22:         The following table outlines a series of students and their performance
   23: on the homework sets.  Each row in the table represents a student in the 
   24: course.  The combinations of results will test the possible outcomes lonchart 
   25: will handle.  When the table specifies just correct, it means on the first try.
   26: 
   27: Homework 2 student responses:
   28: username        problem 1                     problem 2 
   29:                                         part1           part2
   30: 
   31: armadilla       correct                 correct         correct
   32: Bean            skip                    skip            skip
   33: cat             correct(12 tries)       correct         skip
   34: dog             correct(2 tries)        skip            correct
   35: elephant        correct by override     correct after   correct first time
   36:                                           skip first
   37: flamingo        incorrect by override   correct first   correct after skip 
   38:                                           time            first time
   39: gnat            incorrect               incorrect       correct
   40: hippo           excused                 correct         incorrect
   41: iguana          correct second try      incorrect       incorrect
   42:                   after correct by
   43:                   override on first
   44:                   try
   45: 
   46:         For the problems in the top sequence, they will all be correct on first
   47: try, except for bean, who didn't log in.  Also, cat will get problem two and 
   48: three correct on the second try and iguana will be incorrect for problem one. 
   49:         All the problems will be correct on the first try for Homework 3, 
   50: except for the dog which will be correct with the following tries: 121131, and 
   51: cat which will be incorrect on problem one part two, incorrect on problem two 
   52: part two, and correct with two tries on problem one part one.
   53: 
   54: 
   55: Expected output:
   56: 
   57: User Name       Homework 1      Homework 2   Homework 3   Total Solved/
   58:                                                           Total Problems
   59: armadillo       111      3      111      3   111111   6   12 / 12
   60: Bean
   61: cat             122      3      *1       2   2.1.11   6   9 / 12
   62: dog             111      3      2 1      2   121131   4   11 / 12
   63: elephant        111      3      +11      2   111111   6   11 / 12
   64: flamingo        111      3      -11      2   111111   6   11 / 12
   65: gnat            111      3      ..1      1   111111   6   10 / 12
   66: hippo           111      3      x1.      1   111111   6   10 / 11
   67: iguana          .11      3      +..      1   111111   6   10 / 12
   68: 
   69: 
   70: Expect output for student information:
   71: 
   72: User Name   Domain   Section   PID         Full Name                         
   73: armadillo   103      1                     armadillo, a a
   74: bean        103      1                     Bean, b b
   75: cat         103      1                     cat, c c
   76: dog         103      1                     dog, d d
   77: elephant    103      1                     elephant, e e
   78: flamingo    103      1                     flamingo, f f
   79: gnat        103      1                     gnat, g g
   80: hippo       103      1         a123332     Hippopatumous II, h h
   81: iguana      103      1         144         iguana, i i
   82: jason       103                a33377721   Stredwick fourth, Jason Michael
   83: 
   84: 
   85:         One function of lonchart is to sort students by either username, 
   86: section, or last name.  Once you have downloaded the students for this class,
   87: either when no cache exists or recalculate button has been pressed, sort can 
   88: occur.  Begin by selecting sort by section followed by last name then 
   89: username.  The results for each will be listed below.  For the sake of space,
   90: the grade all the information will be left off except for the username.  
   91: The data below will show the correct username order for each of the sortings.
   92: If you want to check for the whole line to be correct, use the above to tables
   93: by matching them to the appropriate username.
   94: 
   95: Sort by section:        Sort by last name:      Sort by username:
   96: armadillo               bean                    armadillo
   97: bean                    hippo                   bean
   98: cat                     jason                   cat
   99: dog                     armadillo               dog
  100: elephant                cat                     elephant
  101: flamingo                dog                     flamingo
  102: gnat                    elephant                gnat
  103: hippo                   flamingo                hippo
  104: iguana                  gnat                    iguana
  105: jason                   iguana                  jason
  106: 
  107: 
  108:         The last piece of functionality is the student status.  It allows
  109: the list of students to include/exclude based on the selection of Active,
  110: Expired, or Any type of student.  The list of students so far have been 
  111: active only.  To see who is expired, select expired followed by one of the
  112: sorting buttons or the recalculate button.  There will only be one user, dunk.
  113: Select the Any option, and redisplay the list of students again.  This time 
  114: all the students including dunk are displayed.
  115: 
  116:         This concludes the test of lonchart.  Deviations from the expected 
  117: output show incorrectness in the code.  Please change the expected output for
  118: new formats, conceptual changes, or feature additions.