HMOE: Hypernetwork-based Mixture of Experts for Domain Generalization - Archive ouverte HAL Access content directly
Preprints, Working Papers, ... (Preprint) Year :

HMOE: Hypernetwork-based Mixture of Experts for Domain Generalization

(1, 2) , (2) , (3) , (1, 4) , (2) , (2)
1
2
3
4

Abstract

Due to the domain shift, machine learning systems typically fail to generalize well to domains different from those of training data, which is the problem that domain generalization (DG) aims to address. However, most mainstream DG algorithms lack interpretability and require domain labels, which are not available in many real-world scenarios. In this work, we propose a novel DG method, HMOE: Hypernetwork-based Mixture of Experts (MoE), that does not require domain labels and is more interpretable. We use hypernetworks to generate the weights of experts, allowing experts to share some useful meta-knowledge. MoE has proven adept at detecting and identifying heterogeneous patterns in data. For DG, heterogeneity exactly arises from the domain shift. We compare HMOE with other DG algorithms under a fair and unified benchmark-DomainBed. Extensive experiments show that HMOE can perform latent domain discovery from data of mixed domains and divide it into distinct clusters that are surprisingly more consistent with human intuition than original domain labels. Compared to other DG methods, HMOE shows competitive performance and achieves SOTA results in some cases without using domain labels.
Fichier principal
Vignette du fichier
HMOE_arXiv.pdf (2.34 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03855006 , version 1 (16-11-2022)

Identifiers

Cite

Jingang Qu, Thibault Faney, Ze Wang, Patrick Gallinari, Soleiman Yousef, et al.. HMOE: Hypernetwork-based Mixture of Experts for Domain Generalization. 2022. ⟨hal-03855006⟩
0 View
0 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More