Handling new target classes in semantic segmentation with domain adaptation

作者:

Highlights:

摘要

In this work, we define and address a novel domain adaptation (DA) problem in semantic scene segmentation, where the target domain not only exhibits a data distribution shift w.r.t. the source domain, but also includes novel classes that do not exist in the latter. Different to “open-set” (Panareda Busto and Gall, 2017) and “universal domain adaptation” (You et al. 2019), which both regard all objects from new classes as “unknown”, we aim at explicit test-time prediction for these new classes. To reach this goal, we propose a framework that leverages domain adaptation and zero-shot learning techniques to enable “boundless” adaptation in the target domain. It relies on a novel architecture, along with a dedicated learning scheme, to bridge the source–target domain gap while learning how to map new classes’ labels to relevant visual representations. The performance is further improved using self-training on target-domain pseudo-labels. For validation, we consider different domain adaptation set-ups, namely synthetic-2-real, country-2-country and dataset-2-dataset. Our framework outperforms the baselines by significant margins, setting competitive standards on all benchmarks for the new task. Code and models are available at: https://github.com/valeoai/buda.

论文关键词:

论文评审过程:Received 15 January 2021, Revised 23 July 2021, Accepted 9 August 2021, Available online 18 August 2021, Version of Record 9 September 2021.

论文官网地址:https://doi.org/10.1016/j.cviu.2021.103258