Stepwise AI Interpretive Approach for Mutimodal Data Fusion

EasyChair Preprint 13912

Abstract

In recent years, Artificial Intelligence technology has excelled in various tasks and is taking the world by storm. However, the various transformations in neural networks make it difficult to make sense of the reasons why decisions are made. For this reason, trustworthy AI techniques have started gaining popularity. To this end, this study uses AI interpretability as an anchor point to point to the field of data fusion for multimodal AI for in-depth insights. The paper proposed a Stepwise AI Interpretative (SAII) approach using different pairing methods of 'one-to-one' and 'many-to-many' in an attempt to illustrate/demonstrate the interpretability of the process of pairing images and text. A counterfactual instantiation method was used to compare the whole-local relationship between a set of images and their associated descriptive text. The approach was evaluated via ‘task performance’.

Keyphrases: Trustworthy AI, data fusion, multimodal

@booklet{EasyChair:13912,
  author    = {Bowen Long and Enjie Liu and Renxi Qiu and Yanqing Duan},
  title     = {Stepwise AI Interpretive Approach for Mutimodal Data Fusion},
  howpublished = {EasyChair Preprint 13912},
  year      = {EasyChair, 2024}}