Convenience Method for MultiAnswer

[drdrew42]

MultiAnswer and AnswerEvaluator Attributes

After some discussion about #925, I think the clearer solution is to offer a convenience method for passing settings for individual answer rules.

Thanks to @dpvc for feedback regarding maintaining existing functionality.

setCmpFlags

• The first argument to setCmpFlags is the answer rule to be targeted (starting with 1).
• Following this first argument any number of settings may be provided (as a list, not a HASH ref).

Use Cases

Type Checking:

The original problem I faced involved customizing the error messages when type matching failed. I wrote a post-filter to replace the default error messages, but I also wanted to provide partial credit. However, the presence of the type-match warning prevented my checker from doing so (see #925 for details). Now, with this method I can avoid the default type-match warning and provide my own instead with partial credit in my checker:

$multianswer_obj->setCmpFlags(1, showTypeWarnings => 0);

Custom Context Flags:

Thanks to @drgrice1 for this example:

When working with Context("Fraction"), the studentsMustReduceFractions => 0 setting will automatically reduce student responses before they reach our MultiAnswer->{checker} subroutine. If one wishes to provide partial credit for un-reduced fraction answers, it is necessary to ensure that this flag is true. Before the introduction of this convenience method, code contortions were necessary to configure this setting in a MultiAnswer object.

[dpvc]

I think the clearer solution is to offer a convenience method for passing settings for individual answer rules.

There is another alternative, which would be to allow you to pass a MathObjects answer checker directly to MultAnswer() itself. Currently, you can pass either a MathObject or a string that parses to one, but you could allow an answer checker as well, provided it came from a MathObject. The presence of, say, the correct_value property should be sufficient to identify a MathObject checker from other ones.

The lines that need changing are

pg/macros/parsers/parserMultiAnswer.pl

Lines 189 to 192 in e131356

	foreach my $x (@data) {
	$x = Value::makeValue($x, context => $context) unless Value::isValue($x);
	push(@cmp, $x->cmp(@ans_defaults));
	}

and making them something like

	foreach my $x (@data) {
		if (ref($x) eq 'AnswerEvaluator') {
			Value::Error('Only MathObject answer checkers can be passed to MultiAnswer()')
				unless defined($x->{correct_value});
			push(@cmp, $x);
			$x = $x->{correct_value};
		} else {
			$x = Value::makeValue($x, context => $context) unless Value::isValue($x);
			push(@cmp, $x->cmp(@ans_defaults));
		}
	}

should do the trick (though I haven't tested this).

Then you can use the individual answer checkers to set their options. E.g.,

$ma = MultiAnswer(
	Fraction(1,2)->cmp(studentsMustReduceFractions => 1),
	Real(3)->cmp(showTypeWarnings => 0)
)->with(
	checker  => sub {
		...
	}
);

[drdrew42]

I actually attempted to do this before, supporting AnswerEvaluators passed into the MultiAnswer constructor, but I couldn't get around the retrieval of the correct answer from the evaluator.

In your code suggestion, correct_value is not a property of the AnswerEvaluator object. The AnswerEvaluator does, however, have a ref to the AnswerHash, so this can be resolved (in a way) -- except it seems that the AnswerHash only stores correct_ans as a string. So, in this approach, the author would create a MathObject and store its cmp method. Then, when passed to MultiAnswer, we would pull the stringified MathObject to create a new MathObject which is then stored in the data array of the MultiAnswer. I think the reparsing that this solution requires could/would cause unexpected things to happen with the display of the correct answer.

It would be nice to supplement to the convenience method by supporting this approach as well, but do you know of a better way for us to recover the original MathObject from the AnswerEvaluator?

[dpvc]

In your code suggestion, correct_value is not a property of the AnswerEvaluator object.

Sorry, you are right, it is on the AnswerEvaluator's answer hash, not the AnswerEvaluator itself (as I mentioned, I hadn't tested the code; I was writing from memory).

except it seems that the AnswerHash only stores correct_ans as a string.

True, but you don't want correct_ans, you want correct_value, which is put there by the MathObject answer checkers, and is a reference to the MathObject that created the AnswerEvaluator. That is what you want.

You don't want to re-parse the correct_ans string, as you will lose any properties that are set on the original MathObject (e.g., a Function object might have limits set, or test points or other values that are not carried by the correct_ans string). So you want to use the correct_value that is on the answer hash.

[drgrice1]

I think that this generally looks good. A couple of suggestions though.

I think it would be good to add the new capability of passing an AnswerEvaluator directly to the MultiAnswer constructor to the POD documentation.

Also, you removed the section in the POD that describes how the checker works, and that states that a checker is required. I think that should be added back. The examples don't show everything that was previously in the explanation, and are not clear on the return values of the checker.

I think a comment about the width of answer rules should still be in the POD as well. That should be updated for how to do that with PGML answers.

[drgrice1]

This is a little confusing. If you add code like this to the POD, authors may think that this might work. I think you mean that the first example will throw an error message, and the second will not. But that is not entirely clear. Also, it seems to insinuate that the $success or $failure variables are something other than what they are. The value of the $success variable would actually be the answer hash for the $fraction_obj answer. Of course $failure would not be set because the die would occur. In any case, the result of the MultiAnswer constructor call is never stored in a variable, and so the call would not be particularly useful.

[drgrice1]

This has the same problem as the example code in the setCmpFlags section.

[dpvc]

In light of the comments by @drgrice1, perhaps this method should return $self so that you can chain the calls together. E.g.,

$ma = MultiAnswser("1/2", "sqrt(x-10)")->
         setCmpFlags(1, studentsMustReduceFractions => 1)->
         setCmpFlags(2, limits => [10, 15]);

and still have $ma be the MultiAnswer object.

[drdrew42]

Following the above suggestions, setCmpFlags is now chainable and the POD is updated:

• Previously, information regarding the checker had been moved to its own section under "ATTRIBUTES". Now, "(required)" has been added to its header in that section and an example answer checker was added also.
• success/failure examples have been clarified for setCmpFlags and setMessage
• added descriptions for the use of AnswerEvaluators during construction
• PGML implementation example now includes answer rule width

[drgrice1]

I think this looks good now.

[dpvc]

Looks good.

[drgrice1]

I will merge this since it has two approvals.