Overview of entities and relations

The platform contains DataFile, which represent both data (the bits themselves) and metadata. Each DataFile is of a specific DataType. A DataSet is a collection of DataFiles.

An Interface represents the interface of one or many Implementation’s, e.g. it defines the names and DataType’s as Input and defines a single output DataType.

Each Implementation is linked to a Docker image and belongs to (represents different versions of) an Algorithm.

Whenever an Implementation is applied to a Benchmark as a Submission, it will be evaluated. This is done in two stages:

• The Benchmark is linked to a DataSet. The DataSet contains a test_data_file field, Implementation is ran with this test_data_file as input, and it should produce an output according to the Benchmark’s interface.

• The output of the first step, together with the Benchmark’s DataSet.ground_truth_data_file are input to an Implementation defined by the Benchmark’s evaluator. This Implementation should product as output a DataFile of type ‘OutputMetrics’.

Each time an Implementation is run, this creates a Job, which contains status, logs etc.