You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Since testwhat v4.7.2, different SCT functions do some assertions and verifications to figure out whether the function is appropriately used. You can see it in the CHANGELOG here.
One of these assertions happens in check_object(), to check whether it is being called on the root state. This assertion is only done for courses whose course image contains the TESTWHAT_V2_ONLY env variable (more recent courses), because otherwise they would break 71 exercises that contain SCTs that violate this rule:
course_id chapter_id id number title
1 639 1521 9772 2 R code chunks
2 639 1521 9773 3 Customize R code chunks
3 639 1521 9775 5 Inline R code
4 682 1640 11701 4 Making a scree plot!
5 2399 10069 114116 10 Relative change in demographic trend
6 3022 8205 172631 11 Generate a grid of models
7 3022 8205 172632 12 Evaluate the grid
8 3022 8207 172656 12 Tuning a Random Forest via tree depth
9 3026 8217 99950 12 Rate-limiting
10 3851 11371 135814 14 Evaluate a modeling procedure using n-fold cross-validation
11 3851 11372 140833 6 Modeling an interaction (2)
12 3851 11372 140834 13 Input transforms: the "hockey stick" (2)
13 5798 18624 214577 2 Create value box
14 5798 18624 214578 3 Create gauge
15 5798 18624 214579 4 Color indicators
16 5798 18624 214580 5 Linking
17 5798 18624 214582 7 Static table
18 5798 18624 214583 8 Web-friendly table
19 5798 18624 214584 9 DT::datatable
20 5798 18625 214598 7 Making the Dataframe Reactive
21 5798 18625 315666 13 Create Global Input Sidebar
22 5951 19124 220845 1 Background on modeling for explanation
23 5951 19124 220846 2 Exploratory visualization of age
24 5951 19124 220852 7 The modeling problem for explanation
25 5951 19125 220856 1 Explaining teaching score with age
26 5951 19125 220860 4 Predicting teaching score using age
27 5951 19125 220863 7 Explaining teaching score with gender
28 5951 19125 220868 10 Predicting teaching score using gender
29 5951 19126 220869 1 Explaining house price with year & size
30 5951 19126 220876 7 Explaining house price with size & condition
31 5951 19127 220879 1 Model selection and assessment
32 5951 19127 220882 4 Assessing model fit with R-squared
33 5951 19127 220885 7 Assessing predictions with RMSE
34 5951 19127 220891 10 Validation set prediction framework
35 5951 19125 273100 11 Making predictions using rank
36 5951 19127 273113 13 Conclusion - Where to go from here?
37 5951 19126 298464 10 Predicting house price using size & condition
38 5951 19126 298470 6 Interpreting residuals
39 5951 19127 298652 6 Comparing the R-squared of two models
40 5951 19126 328548 4 Predicting house price using year & size
41 5951 19127 342938 5 Computing the R-squared of a model
42 5951 19127 343228 8 Computing the MSE & RMSE of a model
43 5951 19127 343235 9 Comparing the RMSE of two models
44 5951 19127 343408 11 Fitting model to training data
45 5951 19127 343411 12 Predicting on test data
46 5951 19124 353946 10 The modeling problem for prediction
47 5951 19124 354701 4 Background on modeling for prediction
48 5951 19124 369666 3 Numerical summaries of age
49 5951 19124 369852 5 Exploratory visualization of house size
50 5951 19124 369855 6 Log10 transformation of house size
51 5951 19124 370102 8 EDA of relationship of teaching & "beauty" scores
52 5951 19124 370105 9 Correlation between teaching and "beauty" scores
53 5951 19124 370252 11 EDA of relationship of house price and waterfront
54 5951 19124 370257 12 Predicting house price with waterfront
55 5951 19125 370853 2 Plotting a "best-fitting" regression line
56 5951 19125 370855 3 Fitting a regression with a numerical x
57 5951 19125 371740 5 Making predictions using "beauty score"
58 5951 19125 371744 6 Computing fitted/predicted values & residuals
59 5951 19125 372155 8 EDA of relationship of score and rank
60 5951 19125 372162 9 Fitting a regression with a categorical x
61 5951 19125 373846 12 Visualizing the distribution of residuals
62 5951 19127 374711 2 Refresher: sum of squared residuals
63 5951 19127 374714 3 Which model to select?
64 5951 19126 374757 2 EDA of relationship
65 5951 19126 374760 3 Fitting a regression
66 5951 19126 374968 5 Making predictions using size and bedrooms
67 5951 19126 375305 11 Making predictions using size and waterfront
68 5951 19126 375306 12 Automating predictions on "new" houses
69 5951 19126 375357 8 Parallel slopes model
70 5951 19126 375359 9 Interpreting the parallel slopes model
71 6617 28953 421290 10 Apply the two steps
This issue is a reminder that we have to rewrite the SCTs for these exercises because they are smelly. As soon as that is done, we can remove this v2_only() check.
How I came up with this list
Make a branch on testwhat that does not do the v2 only check
Install this branch of testwhat in docker-r-shared and tag it (with a release candidate format, as it is never the intention to deploy)
This makes the validator go through all exercises with this hypothetical shared image, and spit out the exercises where it goes wrong.
Finally, you can use this script to programmatically a list of exercise ids from the validator endpoint and their corresponding chapters and courses.
The text was updated successfully, but these errors were encountered:
Since
testwhat v4.7.2
, different SCT functions do some assertions and verifications to figure out whether the function is appropriately used. You can see it in the CHANGELOG here.One of these assertions happens in
check_object()
, to check whether it is being called on the root state. This assertion is only done for courses whose course image contains theTESTWHAT_V2_ONLY
env variable (more recent courses), because otherwise they would break 71 exercises that contain SCTs that violate this rule:This issue is a reminder that we have to rewrite the SCTs for these exercises because they are smelly. As soon as that is done, we can remove this
v2_only()
check.How I came up with this list
testwhat
that does not do the v2 only checkdocker-r-shared
and tag it (with a release candidate format, as it is never the intention to deploy)The text was updated successfully, but these errors were encountered: