Quotation Dieber, Jürgen, Kirrane, Sabrina. 2022. A novel model usability evaluation framework (MUsE) for explainable artificial intelligence. Information Fusion. 81 143-153.


RIS


BibTeX

Abstract

When it comes to complex machine learning models, commonly referred to as black boxes, understanding the underlying decision making process is crucial for domains such as healthcare and financial services, as well as when they are used in connection with safety critical systems such as autonomous vehicles. As a result, interest in explainable artificial intelligence (xAI) tools and techniques has increased in recent years. However, the user experience (UX) effectiveness of existing xAI frameworks, especially concerning algorithms that work with data as opposed to images, is still an open research question. In order to address this gap, we examine the UX effectiveness of the Local Interpretable Model-Agnostic Explanations (LIME) xAI framework, one of the most popular model agnostic frameworks found in the literature, with a specific focus on its performance in terms of making tabular models more interpretable. In particular, we apply several state of the art machine learning algorithms on a tabular dataset, and demonstrate how LIME can be used to supplement conventional performance assessment methods. Based on this experience, we evaluate the understandability of the output produced by LIME both via a usability study, involving participants who are not familiar with LIME, and its overall usability via a custom made assessment framework, called Model Usability Evaluation (MUsE), which is derived from the International Organisation for Standardisation 9241-11:2018 standard.

Tags

Press 'enter' for creating the tag

Publication's profile

Status of publication Published
Affiliation WU
Type of publication Journal article
Journal Information Fusion
Citation Index SCI
Language English
Title A novel model usability evaluation framework (MUsE) for explainable artificial intelligence
Volume 81
Year 2022
Page from 143
Page to 153
Reviewed? Y
URL https://api.elsevier.com/content/article/PII:S1566253521002402?httpAccept=text/xml
DOI http://dx.doi.org/10.1016/j.inffus.2021.11.017
Open Access Y
Open Access Link https://penni.wu.ac.at/papers/IF%202022%20A%20Novel%20Model%20Usability%20Evaluation%20Framework%20for%20Explainable%20Artificial%20Intelligence.pdf

Associations

People
Kirrane, Sabrina (Details)
External
Dieber, Jürgen (Vienna University of Economics and Business, Austria)
Organization
Institute for Information Systems and New Media IN (Details)
Research areas (ÖSTAT Classification 'Statistik Austria')
1108 Informatics (Details)
1109 Information and data processing (Details)
1122 Artificial intelligence (Details)
1138 Information systems (Details)
1146 Management information systems (Details)
5255 Data security and data privacy (Details)
Google Scholar: Search