Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get rid of has_equal_pivot/chart() in favor of more granular checking #11

Open
filipsch opened this issue Sep 11, 2018 · 0 comments
Open

Comments

@filipsch
Copy link
Contributor

filipsch commented Sep 11, 2018

Some history first: In the past, pythonwhat featured the test_function("my_fun") function, that would look for a call of my_fun() in the solution, look at which arguments were specified in the solution code and what their values were. Next, this function would look at the student, and demand the exact same function call, with the same arguments and same values to be present in the student's submission as well. Even though this allowed for very succinct SCTs, there were some problems with this approach. First, the function was a magic box, that did a bunch of stuff for you behind the scenes. If it broke, you had no clue where or why. Second, the amount of arguments passed to test_function() started to explode to allow for all kinds of ways to customize how the function worked, e.g. to not test some arguments, to not evaluate some arguments but check their values instead, to adapt the automatically generated messages in different error cases, etc. After some time, it was decided to go with more granular checks: instead of test_function(), there is now check_function() to look for the function call, check_args() to see if an argument was specified, and has_equal_value() to see if the arguments correspond:

# call to verify
round(1.1234, ndigits=2)

# old SCT
test_function("round")

# new SCT
Ex().check_function('round').multi(
    check_args(0).has_equal_value(),
    check_args('ndigits').has_equal_value()
)

Even though much more verbose, functions are more atomic, it is now crystal clear what is being tested, and it is easy to customize how things are tested (e.g. drop a check_args() if you don't want to verify an argument).

Looking at sheetwhat with this history in mind, we can see functions like has_equal_pivot() and has_equal_chart() that are basically test_function()-like one-stop-shops that just verify almost everything there is to verify about a pivot table or chart. It is impossible to customize what it's doing, to specify different custom feedback messages, etc.

The proposal, therefore, is to come up with a set of building block functions that make it easy for SCT authors to zoom in on parts of the student data and solution data, verify parts, verify the length of parts, etc. For example (but feel free to propose something else):

Ex().check_pivot().multi(
     check_field('rows').has_equal_length().multi(
         check_element(0).has_equal_value(),
         check_element(1).has_equal_value()
     ),
     check_field('columns').has_equal_length()....
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant