Skip to content

Commit

Permalink
add controls for when grades are sent to the LMS
Browse files Browse the repository at this point in the history
  • Loading branch information
@Alex-Jordan
Alex-Jordan committed Nov 8, 2024
1 parent dd844c7 commit c27f281
Show file tree
Hide file tree
Showing 5 changed files with 257 additions and 44 deletions.
63 changes: 53 additions & 10 deletions conf/authen_LTI.conf.dist
View file
Original file line number Diff line number Diff line change
Expand Up @@ -135,18 +135,58 @@ $LTIGradeMode = '';
#$LTIGradeMode = 'course';
#$LTIGradeMode = 'homework';

# When set this variable sends grades back to the LMS every time a user submits an answer. This
# keeps students grades up to date but can be a drain on the server.
$LTIGradeOnSubmit = 1;

# If $LTICheckPrior is set to 1 then the current LMS grade will be checked first, and if the grade
# has not changed then the grade will not be updated. This is intended to reduce changes to LMS
# records when no real grade change occurred. It requires a 2 round process, first querying the
# current grade from the LMS and then when needed making the grade submission.
# Controls for when to report grades to the LMS.

# This variable can be set to 'open_date', 'reduced_scoring_date', 'close_date', 'answer_date',
# or 'never', corresponding to the open, reduced scoring, close, and answer dates for a set, or
# never. After this type of date has passed, WeBWorK will send grades to the LMS, subject to the
# other controls described below. Prior to this date, WeBWorK will only send grades to the LMS if
# the set has met the condition set with $LTISendGradesEarlyThreshold. For example, if it is set
# to 'close', then once the close date is reached for a set and something triggers a potential
# grade passback, scores will be sent to the LMS regardless of whether or not the set has been
# attempted or has reached a threshold score. Prior to the close date, if something triggers a
# potential grade passback, whether or not the set's scores are passed depends on the value of
# $LTISendGradesEarlyThreshold. In 'course' grade mode, this is controlling whether or not the
# set is included in the overall course grade computation. In 'homework' grade mode, it is only
# about that set's score. For a test, the date that matters is the earliest date among the versions
# or the set template.
$LTISendScoresAfterDate = 'open_date';

# This variable can be set to a number from 0 to 1 inclusive, or set to the string 'attempted'.
# When something triggers a potential grade passback, if it is earlier than $LTISendScoresAfterDate,
# the condition described by this variable must be met or else no grade will be sent. (In 'course'
# grade mode, the set will not be included in the overall grade calculation.) If this variable is
# a number, then the set will need to have a score that reaches or exceeds this number for its
# score to be sent to the LMS (or included in the 'course' grade calcuation). If this variable is
# set to 'attempted', then the set needs to have been attempted for its score to be sent to the LMS
# (or included in the 'course' grade calcuation). For a regular or jitar set, 'attempted' means that
# at least one exercise was attempted. For a test, 'attempted' means that either multiple versions
# exist or there is one version with a graded submission.
$LTISendGradesEarlyThreshold = 0;

# When this variable is set, then each time WeBWorK would send a grade to the LMS, it first
# requests the current score from the LMS. If there is not a significant difference between the
# current score in the LMS and the score that WeBWorK is possibly going to send, then WeBWorK will
# not send a score to the LMS. This is intended to avoid frequent insignificant updates to the
# student's grades in the LMS. With some LMSs, some students receive notifications each time there
# is a grade update, and setting this variable will prevent too many notifications for them. This
# does create a two-phase process, first querying the current grade from the LMS and then when
# needed making the grade submission.
$LTICheckPrior = 0;

# The system periodically updates student grades on the LMS. This variable controls how often
# that happens. Set to -1 to disable.
# When this variable is set then every time a user submits an answer, WeBWorK may send grades back
# to the LMS. This is subject to the above settings. If using 'course' grade mode, WeBWorK will
# possibly send a new overall score for this user. If using 'homework' grade mode, WeBWorK will
# only send a new score for this user for this set. Since this may put a strain on the server, it
# can be turned off and grades will only be sent to the LMS according to the $LTIMassUpdateInterval
# or when an instructor initiates a mass grade passback using LTI Grade Update.
$LTIGradeOnSubmit = 1;

# The system periodically updates student grades on the LMS. This variable controls how often
# that happens. Set this to -1 to disable mass passback. These mass passback events must be triggered
# by someone in the course doing something after enough time has passed, even as simple as loading a
# page. So for example this cannot be set in such a way as to make sure the passback events happen
# at a particular time each day.
$LTIMassUpdateInterval = 86400; #in seconds

################################################################################################
Expand All @@ -170,6 +210,9 @@ $LTIMassUpdateInterval = 86400; #in seconds
#'LTI{v1p3}{LMS_url}',
#'external_auth',
#'LTIGradeMode',
#'LTISendScoresAfterDate',
#'LTISendGradesEarlyThreshold',
#'LTICheckPrior',
#'LTIGradeOnSubmit',
#'LTIMassUpdateInterval',
#'LMSManageUserData',
Expand Down
54 changes: 39 additions & 15 deletions lib/WeBWorK/Authen/LTIAdvanced/SubmitGrade.pm
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ use Digest::SHA qw(sha1_base64);

use WeBWorK::Debug;
use WeBWorK::Utils qw(wwRound);
use WeBWorK::Utils::Sets qw(grade_set grade_gateway grade_all_sets);
use WeBWorK::Utils::Sets qw(grade_set grade_gateway gateway_attempted earliest_gateway_date grade_all_sets get_date);

# This package contains utilities for submitting grades to the LMS
sub new ($invocant, $c, $post_processing_mode = 0) {
Expand Down Expand Up @@ -128,7 +128,8 @@ async sub submit_course_grade ($self, $userID) {
$self->warning("lis_source_did is not available for user: $userID")
if !$user->lis_source_did && ($ce->{debug_lti_grade_passback} || $self->{post_processing_mode});

return await $self->submit_grade($user->lis_source_did, scalar(grade_all_sets($db, $userID)));
return await $self->submit_grade($user->lis_source_did,
scalar(grade_all_sets($db, $userID, $ce->{LTISendScoresAfterDate}, $ce->{LTISendGradesEarlyThreshold})));
}

# Computes and submits the set grade for $userID and $setID to the LMS. For gateways the best score is used.
Expand All @@ -147,18 +148,37 @@ async sub submit_set_grade ($self, $userID, $setID) {
$self->warning('lis_source_did is not available for this set.')
if !$userSet->lis_source_did && ($ce->{debug_lti_grade_passback} || $self->{post_processing_mode});

return await $self->submit_grade(
$userSet->lis_source_did,
scalar(
$userSet->assignment_type =~ /gateway/
? grade_gateway($db, $userSet, $userSet->set_id, $userID)
: grade_set($db, $userSet, $userID, 0)
)
);
my $score;
my $incorrect_attempts = [];
my $attempted;
my $date;

if ($userSet->assignment_type =~ /gateway/) {
$score = scalar(grade_gateway($db, $userSet, $userSet->set_id, $userID));
$attempted = gateway_attempted($db, $userSet, $userSet->set_id, $userID);
$date = earliest_gateway_date($db, $userSet->set_id, $userID, $ce->{LTISendScoresAfterDate});
} else {
my $totalRight;
my $total;
($totalRight, $total, $incorrect_attempts) = (grade_set($db, $userSet, $userID, 0, 1))[ 0, 1, 3 ];
$score = ($total == 0) ? 0 : $totalRight / $total;
$attempted = 1 if ($totalRight || grep $_ > 0, @$incorrect_attempts);
$date = get_date($userSet, $ce->{LTISendScoresAfterDate});
}

my $beforeSendScoresAfterDate = $ce->{LTISendScoresAfterDate} eq 'never' || before($date);
if ($beforeSendScoresAfterDate) {
return if ($ce->{LTISendGradesEarlyThreshold} eq 'attempted' && !$attempted);
return if ($score < $ce->{LTISendGradesEarlyThreshold});
}

# $beforeSendScoresAfterDate needs to be passed so that if LTICheckPrior is set, submit_grade() can decide between calling
# an empty grade equivalent to 0 before the SendScoresAfterDate versus not equivalent after the SendScoresAfterDate
return await $self->submit_grade($userSet->lis_source_did, $score, $beforeSendScoresAfterDate);
}

# Submits a score of $score to the lms with $sourcedid as the identifier.
async sub submit_grade ($self, $sourcedid, $score) {
async sub submit_grade ($self, $sourcedid, $score, $beforeSendScoresAfterDate = 1) {
my $c = $self->{c};
my $ce = $c->{ce};
my $db = $c->{db};
Expand Down Expand Up @@ -301,11 +321,15 @@ EOS
# See: https://webwork.maa.org/moodle/mod/forum/discuss.php?d=5002
debug("LMS grade will be updated. sourcedid: $sourcedid; Old score: $oldScore; New score: $score")
if $ce->{debug_lti_grade_passback};
} elsif ($oldScore ne '' && abs($score - $oldScore) < 0.001) {
} elsif (abs($score - $oldScore) < 0.001
&& ($score != 1 || $oldScore == 1)
&& ($score != 0 || $oldScore ne '' || $beforeSendScoresAfterDate))
{
# LMS has essentially the same score, no reason to update it
debug("LMS grade will NOT be updated - grade unchanges. Old score: $oldScore; New score: $score")
if $ce->{debug_lti_grade_passback};
$self->warning('LMS grade will NOT be updated - grade unchanged. '
debug(
"LMS grade will NOT be updated - grade has not significantly changed. Old score: $oldScore; New score: $score"
) if $ce->{debug_lti_grade_passback};
$self->warning('LMS grade will NOT be updated - grade has not significantly changed. '
. "Old score: $oldScore; New score: $score")
if $ce->{debug_lti_grade_passback} || $self->{post_processing_mode};
return 1;
Expand Down
48 changes: 39 additions & 9 deletions lib/WeBWorK/Authen/LTIAdvantage/SubmitGrade.pm
View file
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ use Time::HiRes;

use WeBWorK::Debug;
use WeBWorK::Utils qw(wwRound);
use WeBWorK::Utils::Sets qw(grade_set grade_gateway grade_all_sets);
use WeBWorK::Utils::Sets qw(grade_set grade_gateway gateway_attempted earliest_gateway_date grade_all_sets get_date);

# This package contains utilities for submitting grades to the LMS via LTI 1.3.
sub new ($invocant, $c, $post_processing_mode = 0) {
Expand Down Expand Up @@ -207,7 +207,8 @@ async sub submit_course_grade ($self, $userID) {
$self->warning('LMS user id is not available for this user.') unless $user->lis_source_did;
$self->warning('LMS lineitem is not available for the course.') unless $lineitem;

return await $self->submit_grade($user->lis_source_did, $lineitem, grade_all_sets($db, $userID));
return await $self->submit_grade($user->lis_source_did, $lineitem,
grade_all_sets($db, $userID, $ce->{LTISendScoresAfterDate}, $ce->{LTISendGradesEarlyThreshold}));
}

# Computes and submits the set grade for $userID and $setID to the LMS. For gateways the best score is used.
Expand All @@ -225,14 +226,36 @@ async sub submit_set_grade ($self, $userID, $setID) {
$self->warning('LMS user id is not available for this user.') unless $user->lis_source_did;
$self->warning('LMS lineitem is not available for this set.') unless $userSet->lis_source_did;

return await $self->submit_grade($user->lis_source_did, $userSet->lis_source_did,
$userSet->assignment_type =~ /gateway/
? grade_gateway($db, $userSet, $userSet->set_id, $userID)
: (grade_set($db, $userSet, $userID, 0))[ 0, 1 ]);
my $totalRight;
my $total;
my $incorrect_attempts = [];
my $attempted;
my $date;

if ($userSet->assignment_type =~ /gateway/) {
($totalRight, $total) = grade_gateway($db, $userSet, $userSet->set_id, $userID);
$attempted = gateway_attempted($db, $userSet, $userSet->set_id, $userID);
$date = earliest_gateway_date($db, $userSet->set_id, $userID, $ce->{LTISendScoresAfterDate});
} else {
($totalRight, $total, $incorrect_attempts) = (grade_set($db, $userSet, $userID, 0, 1))[ 0, 1, 3 ];
$attempted = 1 if ($totalRight || grep $_ > 0, @$incorrect_attempts);
$date = get_date($userSet, $ce->{LTISendScoresAfterDate});
}

my $beforeSendScoresAfterDate = $ce->{LTISendScoresAfterDate} eq 'never' || before($date);
if ($beforeSendScoresAfterDate) {
return if ($ce->{LTISendGradesEarlyThreshold} eq 'attempted' && !$attempted);
return if ($total > 0 && $totalRight / $total < $ce->{LTISendGradesEarlyThreshold});
}

# $beforeSendScoresAfterDate needs to be passed so that if LTICheckPrior is set, submit_grade() can decide between calling
# an empty grade equivalent to 0 before the SendScoresAfterDate versus not equivalent after the SendScoresAfterDate
return await $self->submit_grade($user->lis_source_did, $userSet->lis_source_did, $totalRight, $total,
$beforeSendScoresAfterDate);
}

# Submits scoreGiven and scoreMaximum to the lms with $sourcedid as the identifier.
async sub submit_grade ($self, $LMSuserID, $lineitem, $scoreGiven, $scoreMaximum) {
async sub submit_grade ($self, $LMSuserID, $lineitem, $scoreGiven, $scoreMaximum, $beforeSendScoresAfterDate = 1) {
my $c = $self->{c};
my $ce = $c->{ce};

Expand Down Expand Up @@ -275,9 +298,16 @@ async sub submit_grade ($self, $LMSuserID, $lineitem, $scoreGiven, $scoreMaximum
&& $priorData->[0]{resultMaximum} ? $priorData->[0]{resultScore} / $priorData->[0]{resultMaximum} : 0;

my $score = $scoreGiven / $scoreMaximum;
if (abs($score - $priorScore) < 0.001) {
# we want to update the LMS score if the difference is significant,
# or if the new score is 1 but the LMS score was not 1 (but possibly insignificantly different)
# or if the new score is 0 and the LMS score was empty and it is past the SendScoresAfterDate
if (abs($score - $priorScore) < 0.001
&& ($score != 1 || $priorScore == 1)
&& ($score != 0 || @$priorData && $priorData->[0]{resultScore} ne '' || $beforeSendScoresAfterDate))
{
$self->warning(
"LMS grade will NOT be updated as the grade is unchanged. Old score: $priorScore, New score: $score.");
"LMS grade will NOT be updated as the grade has not significantly changed. Old score: $priorScore, New score: $score."
);
return 1;
}

Expand Down
55 changes: 54 additions & 1 deletion lib/WeBWorK/ConfigValues.pm
View file
Original file line number Diff line number Diff line change
Expand Up @@ -874,7 +874,7 @@ sub getConfigValues ($ce) {
},
'LTI{v1p1}{LMS_url}' => {
var => 'LTI{v1p1}{LMS_url}',
doc => x('A URL for the LMS'),
doc => x('URL for the LMS'),
doc2 => x(
'An address that can be used to log in to the LMS. This is used in messages to users '
. 'that direct them to go back to the LMS to access something in the WeBWorK course.'
Expand Down Expand Up @@ -912,6 +912,59 @@ sub getConfigValues ($ce) {
labels => { '' => 'None', 'course' => 'Course', 'homework' => 'Homework' },
type => 'popuplist'
},
LTISendScoresAfterDate => {
var => 'LTISendScoresAfterDate',
doc => x('When to send scores to the LMS regardless of status'),
doc2 => x(
'After this type of date has passed, WeBWorK will send grades to the LMS, subject to the other '
. 'controls for grade passback. Prior to this date, WeBWorK will only send grades to the LMS if '
. "the set has met the condition set by \$LTISendGradesEarlyThreshold. In 'course' grade mode, "
. 'this is controlling whether or not the set is included in the overall course grade computation. '
. "In 'homework' grade mode, it is only about that set's score. For a test, the date that matters "
. 'is the earliest date among the versions or the set template.'
),
values => [qw(open_date reduced_scoring_date close_date answer_date never)],
labels => {
open_date => 'After the open date',
reduced_scoring_date => 'After the reduced scoring date',
close_date => 'After the close date',
answer_date => 'After the answer date',
never_date => 'Never'
},
type => 'popuplist'
},
LTISendGradesEarlyThreshold => {
var => 'LTISendGradesEarlyThreshold',
doc => x('Condition under which scores can be sent to an LMS early'),
doc2 => x(
"This can be set to a number from 0 to 1 inclusive, or set to the string 'attempted'. When something "
. 'triggers a potential grade passback, if it is earlier than $LTISendScoresAfterDate, the '
. "condition described by this variable must be met or else no grade will be sent. (In 'course' "
. 'grade mode, the set will not be included in the overall grade calculation.) If this variable is '
. 'a number, then the set will need to have a score that reaches or exceeds this number for its '
. "score to be sent to the LMS (or included in the 'course' grade calcuation). If this variable is "
. "set to 'attempted', then the set needs to have been attempted for its score to be sent to the "
. "LMS (or included in the 'course' grade calcuation). For a regular or jitar set, 'attempted' "
. "means that at least one exercise was attempted. For a test, 'attempted' means that either "
. 'multiple versions exist or there is one version with a graded submission.'
),
type => 'text',
},
LTICheckPrior => {
var => 'LTICheckPrior',
doc => x('Check a score in the LMS before sending a score to the LMS'),
doc2 => x(
'When this is set, then each time WeBWorK would send a grade to the LMS, it first requests the current '
. 'score from the LMS. If there is not a significant difference between the current score in the '
. 'LMS and the score that WeBWorK is possibly going to send, then WeBWorK will not send a score to '
. "the LMS. This is intended to avoid frequent insignificant updates to the student's grades in "
. 'the LMS. With some LMSs, some students receive notifications each time there is a grade update, '
. 'and setting this variable will prevent too many notifications for them. This does create a '
. 'two-step process, first querying the current grade from the LMS and then when needed making '
. 'the grade submission.'
),
type => 'boolean'
},
LTIGradeOnSubmit => {
var => 'LTIGradeOnSubmit',
doc => x('Update LMS Grade Each Submit'),
Expand Down
Loading

0 comments on commit c27f281

Please sign in to comment.