Skip to main content Skip to main navigation


On the Overlap Between Grad-CAM Saliency Maps and Explainable Visual Features in Skin Cancer Images

Fabrizio Nunnari; Md Abdul Kadir; Daniel Sonntag
In: Andreas Holzinger; Peter Kieseberg; A. Min Tjoa; Edgar Weippl (Hrsg.). Machine Learning and Knowledge Extraction. International IFIP Cross Domain (CD) Conference for Machine Learning & Knowledge Extraction (MAKE) (CD-MAKE-2021), August 17-20, Virtual, Pages 241-253, LNCS, Vol. 12844, ISBN 978-3-030-84060-0, Springer International Publishing, 2021.


Dermatologists recognize melanomas by inspecting images in which they identify human-comprehensible visual features. In this paper, we investigate to what extent such features correspond to the saliency areas identified on CNNs trained for classification. Our experiments, conducted on two neural architectures characterized by different depth and different resolution of the last convolutional layer, quantify to what extent thresholded Grad-CAM saliency maps can be used to identify visual features of skin cancer. We found that the best threshold value, i.e., the threshold at which we can measure the highest Jaccard index, varies significantly among features; ranging from 0.3 to 0.7. In addition, we measured Jaccard indices as high as 0.143, which is almost 50% of the performance of state-of-the-art architectures specialized in feature mask prediction at pixel-level, such as U-Net. Finally, a breakdown test between malignancy and classification correctness shows that higher resolution saliency maps could help doctors in spotting wrong classifications.


Weitere Links