Home

• Data Model
• Orchestrator
• Evaluator

Idomaar evaluation process

Idomaar allows evaluating recommendation tasks via a three-phase process: data splitting, data streaming, result evaluation.

Data splitting

Data splitting creates the data required to run the experiment. An input dataset is processed and split into two main sets: batch training data and on-line streaming data. Batch training data is the set of entities and relations used to bootstrap the computing environment. It basically consists of all the data available to the computing environment for the initial training of the recommendation algorithm. On-line streaming data consists of a set of additional entities and relations, together with a list of recommendation requests to be tested. In fact, entities and relations can be provided later to the computing environment (as they become available, e.g., the injection of new items). Furthermore, a set of recommendation requests is created. Each request consists of the message to be sent to the computing environment (to request a recommendation) and the expected output (i.e., the groundtruth).

Data streaming

The data streaming consists of two main sequential tasks: computing environment bootstrapping and online evaluation. During the task Computing environment bootstrapping, the orchestrator sends the batch training data to the computing environment and waits for

Result evaluation