Knowledge-aware deep framework for collaborative skin lesion segmentation and melanoma recognition

Highlights：

• We propose a novel end-to-end deep framework that is able to perform skin lesion segmentation and melanoma recognition jointly, where the clinical knowledge is exploited and transferred with the mutual guidance between these two tasks.

• We design a lesion-based pooling and shape extraction module to transfer the lesion structure information from the skin lesion segmentation task to the melanoma recognition task, which assists the network to learn more informative feature representation for melanoma recognition.

• We propose a diagnosis guided feature fusion scheme to pass the lesion class information from the melanoma recognition task into the skin lesion segmentation task, which generates discriminative representations for different types of skin lesions.

• We design a recursive mutual learning method that further enhances the joint learning ability of the proposed model for both skin lesion segmentation and melanoma recognition.