Knowledge Engineering for Autonomous Agents


Oct. 2020 - Our latest work on integrating knowledge-based reasoning about object sizes within ML is now available on ArXiv. Be sure to visit our Github for the related code and dataset.

Sep. 2020 - We presented our work at KR2020! Watch the video here.


HanS is the Health and Safety Robot Inspector in KMi , who is aware of our health and safety guidelines and is able to navigate the lab, checking that these are enforced. In this video, HanS is navigating KMi searching for a set of relevant everyday objects.
HanS' ability to spot the presence of relevant objects in a noisy, real-world environment is a crucial pre-requisite for autonomous sense-making. For instance, by learning to recognise that a pile of paper is sitting right next to an electrical appliance, and by combining this evidence with additional background knowledge about these objects (e.g., "paper is flammable" and "electric appliance is a potential ignition source"), HanS is then able to identify a potential fire hazard.

Currently, HanS is able to recognise many objects commonly found in office spaces automatically (without relying on ARTags), after being trained on only a few shots captured from the target environment.

HanS uses the knowledge of the typical size of objects to correct the object classes predicted through Machine Learning. In this context, we adopt a size knowledge representation which categorises objects qualitatively, based on their front surface area, depth, and aspect ratio.

Introducing knowledge of the typical size of objects, HanS could improve its object recognition performance by up to 7%, in terms of weighted F1 score across 60 object classes commonly found in KMi.


Gianluca Bardaros Photo Gianluca BardaroResearch Associate
Agnese Chiattis Photo Agnese ChiattiPhD Research Student
Enrico Dagas Photo Enrico DagaSenior Project Officer
Enrico Mottas Photo Enrico MottaProfessor of Knowledge Technologies
SciRoc Logo