Download PDFOpen PDF in browser

Classifier Labels as Language Grounding for Explanations

14 pagesPublished: September 17, 2018


Advances in state-of-the-art techniques including convolutional neural networks (CNNs) have led to improved perception in autonomous robots. However, these new techniques make a robot’s decision-making process obscure even for the experts. Our goal is to auto- matically generate natural language explanations of a robot’s perception-based inferences in order to help people understand what features contribute to these classification predic- tions. Generating natural language explanations is particularly challenging for perception and other high-dimension classification tasks because 1) we lack a mapping from features to language and 2) there are a large number of features which could be explained. We present a novel approach to generating explanations, which first finds the important features that most affect the classification prediction and then utilizes a secondary detector which can identify and label multiple parts of the features, to label only those important features. Those labels serve as the natural language groundings that we use in our explanations. We demonstrate our explanation algorithm’s ability on the floor identification classifier of our mobile service robot.

Keyphrases: Dependable Robots, human-robot interaction, interpretability, Language and Vision

In: Daniel Lee, Alexander Steen and Toby Walsh (editors). GCAI-2018. 4th Global Conference on Artificial Intelligence, vol 55, pages 148--161

BibTeX entry
  author    = {Sai Prabhakar Pandi Selvaraj and Manuela Veloso and Stephanie Rosenthal},
  title     = {Classifier Labels as Language Grounding for Explanations},
  booktitle = {GCAI-2018. 4th Global Conference on Artificial Intelligence},
  editor    = {Daniel Lee and Alexander Steen and Toby Walsh},
  series    = {EPiC Series in Computing},
  volume    = {55},
  pages     = {148--161},
  year      = {2018},
  publisher = {EasyChair},
  bibsource = {EasyChair,},
  issn      = {2398-7340},
  url       = {},
  doi       = {10.29007/8h3z}}
Download PDFOpen PDF in browser