Zero-shot domain adaptation for natural language inference by projecting superficial words out

摘要

In natural language inference, the semantics of some words do not affect the inference. Such information is considered superficial and brings overfitting. In this paper, we project the superficial information out to learn a more general representation. The generality refer to the adaptation to different domains. The projection is over the dropout framework. To project all information of recurring words out, we propose a parameter-free model (HardDrop). By further noticing that the information of some recurring words needs to be reserved, we propose SoftDrop to learn to cautiously project the information out. Our approaches outperform the competitors over the source domain and over zero-shot target domains. For some target domains (e.g. from RTE to MRPC), the accuracy/f1-score increases from 47.6%/0.501 to 68.1%/0.771.