The EloML package provides Elo rating system for machine learning models. Elo Predictive Power (EPP) score helps to assess model performance based Elo ranking system.
Find more in the EPP: interpretable score of model predictive power arxiv paper.
Installation time should not exceed 1 minute.
# Install the the development version from GitHub:
# install.packages("devtools")
devtools::install_github("ModelOriented/EloML")The following example takes less than 20 seconds to complete.
Load EloML library and benchmark data. In the example we use the data frame auc_data from the EloML package. The data used for EPP calculations should be a data frame, where first 3 columns correspond to: Player (model), Round (split), Score (auc).
library(EloML)
data(auc_scores)
head(auc_scores)
# model split auc
# 1 catboost_1 1 0.9824724
# 2 catboost_1 2 0.9820267
# 3 catboost_1 3 0.9801000
# 4 catboost_1 4 0.9848932
# 5 catboost_1 5 0.9845456
# 6 catboost_1 6 0.9858062
To calculate EPP use calculate_epp function. For more options see help of the function ?calculate_epp.
calculate_epp(auc_scores)
# Head of Players EPP:
# player epp
# 1 catboost_1 -0.793627
# 2 catboost_2 2.915507
# 3 catboost_3 -1.990134
# 4 gbm_1 -20.381584
# 5 gbm_10 1.664303
# 6 gbm_11 2.714073
# Type of estimation: glmnet