diff --git a/R/MeasureSurvRCLL.R b/R/MeasureSurvRCLL.R index 471496f45..613551c2d 100644 --- a/R/MeasureSurvRCLL.R +++ b/R/MeasureSurvRCLL.R @@ -16,6 +16,10 @@ #' density function and \eqn{S} the survival function. #' RCLL is proper given that censoring and survival distribution are independent, see Rindt et al. (2022). #' +#' **Note**: Even though RCLL is a proper scoring rule, the calculation of \eqn{f(t)} (which in our case is discrete, i.e. it is a *probability mass function*) for time points in the test set that don't exist in the predicted survival matrix (`distr`), results in 0 values, which are substituted by `"eps"` in our implementation, therefore skewing the result towards \eqn{-log(eps)}. +#' This is the intractable likelihood problem discussed in Rindt et al. (2022), where the authors perform interpolation to get non-zero values for the \eqn{f(t)}. +#' Until this is handled in `mlr3proba` some way, we advise against using this measure for model evaluation. +#' #' @section Parameter details: #' - `na.rm` (`logical(1)`)\cr #' If `TRUE` (default) then removes any NAs in individual score calculations. diff --git a/man/mlr_measures_surv.rcll.Rd b/man/mlr_measures_surv.rcll.Rd index 405555ff9..bfec27951 100644 --- a/man/mlr_measures_surv.rcll.Rd +++ b/man/mlr_measures_surv.rcll.Rd @@ -13,6 +13,10 @@ The RCLL, in the context of probabilistic predictions, is defined by where \eqn{\Delta} is the censoring indicator, \eqn{f} the probability density function and \eqn{S} the survival function. RCLL is proper given that censoring and survival distribution are independent, see Rindt et al. (2022). + +\strong{Note}: Even though RCLL is a proper scoring rule, the calculation of \eqn{f(t)} (which in our case is discrete, i.e. it is a \emph{probability mass function}) for time points in the test set that don't exist in the predicted survival matrix (\code{distr}), results in 0 values, which are substituted by \code{"eps"} in our implementation, therefore skewing the result towards \eqn{-log(eps)}. +This is the intractable likelihood problem discussed in Rindt et al. (2022), where the authors perform interpolation to get non-zero values for the \eqn{f(t)}. +Until this is handled in \code{mlr3proba} some way, we advise against using this measure for model evaluation. } \section{Dictionary}{