Competition
Probabilistic track: results
There was a single entry in the probabilistic track of the competition. The entry used the SLIPCOVER system and was by Riccardo Zese, Elena Bellodi and Fabrizio Riguzzi of the University of Ferrara. For details of this entry see this document. A break down of the results is shown below. To see how the system performed on various types of problem, you can use the checkboxes. The table, and average penalty will update accordingly. Note: In the probabilistic track, the penalty paid is the squared difference between the true probability of a trace, and the probability returned by the system.
| Include Problems without: | |
| Include Problems with: | |
| Tasks with small language biases, are tasks where the list of relevant predicates given to the learner contain only the predicates needed to explain the examples. Medium sized language biases contain an extra, irrelevant predicate, and large language biases contain at least two extra irrelevant predicates. Note that in tasks that required predicate invention, the invented predicates were not given to the learner. | |
Note that in the table below, the list of properties is exhaustive. So, for example, if a problem does not have recursion in its list of properties, then it is possible to explain the examples without recursion. You can see details of the problem, together with the test traces and the final attempt made by the learner to estimate the probabilities of these traces by clicking on the "problem details and system output" links.
| Problem Name | Properties | Average penalty | Details |
|---|---|---|---|
| Self destruct | probabilistic; predicate invention; small language bias | 0.10731910869117492 | problem details and system output |
| Self destruct | probabilistic; predicate invention; medium language bias | 1.0 | problem details and system output |
| Self destruct | probabilistic; predicate invention; large language bias | 0.03766469571413892 | problem details and system output |
| security camera | probabilistic; non observational predicate learning; non monotonic; small language bias | 1.0 | problem details and system output |
| security camera | probabilistic; non observational predicate learning; non monotonic; medium language bias | 1.0 | problem details and system output |
| security camera | probabilistic; non observational predicate learning; non monotonic; large language bias | 1.0 | problem details and system output |
| connections | probabilistic; small language bias | 0.43052954036740737 | problem details and system output |
| connections | probabilistic; medium language bias | 0.43059400508573586 | problem details and system output |
| connections | probabilistic; large language bias | 0.02799999999999996 | problem details and system output |
| probabilistic lock cell | probabilistic; non observational predicate learning; non monotonic; small language bias | 0.5246192511301718 | problem details and system output |
| probabilistic lock cell | probabilistic; non observational predicate learning; non monotonic; medium language bias | 0.5343646107651215 | problem details and system output |
| probabilistic lock cell | probabilistic; non observational predicate learning; non monotonic; large language bias | 0.5114627140165431 | problem details and system output |
| stun | probabilistic; predicate invention; non monotonic; recursion; small language bias | 0.16649327715311013 | problem details and system output |
| stun | probabilistic; predicate invention; non monotonic; recursion; medium language bias | 0.20608557101112718 | problem details and system output |
| stun | probabilistic; predicate invention; non monotonic; recursion; large language bias | 0.421630859375 | problem details and system output |
| nonOPL stun | probabilistic; non observational predicate learning; non monotonic; recursion; small language bias | 0.220458984375 | problem details and system output |
| nonOPL stun | probabilistic; non observational predicate learning; non monotonic; recursion; medium language bias | 0.220458984375 | problem details and system output |
| nonOPL stun | probabilistic; non observational predicate learning; non monotonic; recursion; large language bias | 0.220458984375 | problem details and system output |
| boost | probabilistic; non observational predicate learning; recursion; small language bias | 0.35963541666666665 | problem details and system output |
| boost | probabilistic; non observational predicate learning; recursion; medium language bias | 0.35963541666666665 | problem details and system output |
| boost | probabilistic; non observational predicate learning; recursion; large language bias | 0.35963541666666665 | problem details and system output |
| boost: predicate invention | probabilistic; predicate invention; recursion; small language bias | 0.3101159160482062 | problem details and system output |
| boost: predicate invention | probabilistic; predicate invention; recursion; medium language bias | 0.1789400175183276 | problem details and system output |
| boost: predicate invention | probabilistic; predicate invention; recursion; large language bias | 0.319580078125 | problem details and system output |
| tripwire | probabilistic; non observational predicate learning; predicate invention; non monotonic; small language bias | 1.0 | problem details and system output |
| tripwire | probabilistic; non observational predicate learning; predicate invention; non monotonic; medium language bias | 1.0 | problem details and system output |
| tripwire | probabilistic; non observational predicate learning; predicate invention; non monotonic; large language bias | 1.0 | problem details and system output |
| tripwire with off switch | probabilistic; non observational predicate learning; predicate invention; non monotonic; recursion; small language bias | 1.0 | problem details and system output |
| tripwire with off switch | probabilistic; non observational predicate learning; predicate invention; non monotonic; recursion; medium language bias | 1.0 | problem details and system output |
| tripwire with off switch | probabilistic; non observational predicate learning; predicate invention; non monotonic; recursion; large language bias | 1.0 | problem details and system output |
| recursive valid move with walls | probabilistic; non monotonic; small language bias | 0.06187884577600208 | problem details and system output |
| recursive valid move with walls | probabilistic; non monotonic; medium language bias | 0.0618794026576696 | problem details and system output |
| recursive valid move with walls | probabilistic; non monotonic; large language bias | 0.06187808329255562 | problem details and system output |
| invalid_move | probabilistic; non observational predicate learning; non monotonic; small language bias | 0.26881182949292803 | problem details and system output |
| invalid_move | probabilistic; non observational predicate learning; non monotonic; medium language bias | 0.24485829943399126 | problem details and system output |
| invalid_move | probabilistic; non observational predicate learning; non monotonic; large language bias | 0.3321287980344474 | problem details and system output |
| recursive valid move: nonOPL | probabilistic; non observational predicate learning; recursion; small language bias | 0.12799999999999984 | problem details and system output |
| recursive valid move: nonOPL | probabilistic; non observational predicate learning; recursion; medium language bias | 1.0 | problem details and system output |
| recursive valid move: nonOPL | probabilistic; non observational predicate learning; recursion; large language bias | 0.07890820312499987 | problem details and system output |
| energy | probabilistic; non observational predicate learning; small language bias | 1.0 | problem details and system output |
| energy | probabilistic; non observational predicate learning; medium language bias | 1.0 | problem details and system output |
| energy | probabilistic; non observational predicate learning; large language bias | 1.0 | problem details and system output |
| locking cells | probabilistic; predicate invention; non monotonic; recursion; small language bias | 0.0241142473627678 | problem details and system output |
| locking cells | probabilistic; predicate invention; non monotonic; recursion; medium language bias | 0.0241142473627678 | problem details and system output |
| locking cells | probabilistic; predicate invention; non monotonic; recursion; large language bias | 0.14969771934235576 | problem details and system output |