scaffold.pl issues - next stage opened by preview, newly opened stage answers marked as incorrect when prior stage answers were submitted and correct

[taniwallach]

When a problem using scaffold.pl is supposed to open new sections only when prior sections are correct there are 2 issues I recently discovered. (The settings in use when I saw these issues appear at the bottom of the post.)

1. Previewing a correct answer will also cause the next section to open. This is probably not intentional, and certainly provides some advantage in terms of the "attempt counter" to students who use preview to open the next stage over those who just submit their answers. This seems related to the fact that preview essentially grades the problem, but hides the "results" which means that in terms of determining if the new section should be opened, the just previewed section is treated as correct - triggering the next section to open. Fixing this issue would probably require a mechanism to track which stages were already open at the time of the preview submission.

2. Under such circumstances, when a submission is made with the correct answers to stage N, then answers in stage N+1 will be marked even though they are usually empty (and typically mark as incorrect). Preventing this behavior would also apparently require a mechanism to track which stages were already open at the time of the submission.

A method which does not track the "open stage" data on the server side, which is likely to be quite difficult to do, would probably depend on some hidden fields to carry the data needed. Such an approach would need to be able to verify that the hidden special data was not tampered with on the client side. As a result, I suspect that fixing these issues is quite likely non-trivial.

---

Settings in use when the issues reported above were observed:

Scaffold::Begin(
      can_open => "when_previous_correct",
      is_open  => "correct_or_first_incorrect",
      instructor_can_open => "always",
      after_AnswerDate_can_open => "always",
      hardcopy_is_open  => "correct_or_first_incorrect",
    );

[taniwallach]

@dpvc Do you have an idea how issue 1 can be fixed?

[taniwallach]

#506 is an attempt to fix the first issue (so far).
It does not yet have a method to prevent tampering with the hidden field used to "preserve" the state information on which was the last open section.