Interactive Explanations of Internal Representations of Neural Network Layers: An Exploratory Study on Outcome Prediction of Comatose Patients

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

9 Downloads (Pure)

Abstract

Supervised machine learning models have impressive predictive capabilities, making them useful to support human decision-making. However, most advanced machine learning techniques, such as Artificial Neural Networks (ANNs), are black boxes and therefore not interpretable for humans. A way of explaining an ANN is visualizing the internal feature representations of its hidden layers (neural embeddings). However, interpreting these visualizations is still difficult. We therefore present InterVENE: an approach that visualizes neural embeddings and interactively explains this visualization, aiming for knowledge extraction and network interpretation. We project neural embeddings in a 2-dimensional scatter plot, where users can interactively select two subsets of data instances in this visualization. Subsequently, a personalized decision tree is trained to distinguish these two sets, thus explaining the difference between the two sets. We apply InterVENE to a medical case study where interpretability of decision support is critical: outcome prediction of comatose patients. Our experiments confirm that InterVENE can successfully extract knowledge from an ANN, and give both domain experts and machine learning experts insight into the behaviour of an ANN. Furthermore, InterVENE’s explanations about outcome prediction of comatose patients seem plausible when compared to existing neurological domain knowledge.
Original languageEnglish
Title of host publicationKDH 2020
Subtitle of host publication5th International Workshop on Knowledge Discovery in Healthcare Data
EditorsKerstin Bach, Razvan Bunescu, Cindy Marling, Nirmalie Wiratunga
PublisherCEUR
Pages 5-11
Number of pages7
Volume2675
Publication statusPublished - 2020
Event5th International Workshop on Knowledge Discovery in Healthcare Data, KDH 2020 - Virtual Workshop, Santiago de Compostela, Spain
Duration: 29 Aug 202030 Aug 2020
Conference number: 5

Publication series

NameCEUR Workshop Proceedings
PublisherCEUR
Volume2675
ISSN (Print)1613-0073

Workshop

Workshop5th International Workshop on Knowledge Discovery in Healthcare Data, KDH 2020
Abbreviated titleKDH 2020
CountrySpain
CitySantiago de Compostela
Period29/08/2030/08/20

Fingerprint Dive into the research topics of 'Interactive Explanations of Internal Representations of Neural Network Layers: An Exploratory Study on Outcome Prediction of Comatose Patients'. Together they form a unique fingerprint.

Cite this