HMOE: Hypernetwork-based Mixture of Experts for Domain Generalization - Institut des Systèmes Intelligents et de Robotique Accéder directement au contenu
Pré-Publication, Document De Travail (Preprint/Prepublication) Année : 2022

HMOE: Hypernetwork-based Mixture of Experts for Domain Generalization

Résumé

Due to the domain shift, machine learning systems typically fail to generalize well to domains different from those of training data, which is the problem that domain generalization (DG) aims to address. However, most mainstream DG algorithms lack interpretability and require domain labels, which are not available in many real-world scenarios. In this work, we propose a novel DG method, HMOE: Hypernetwork-based Mixture of Experts (MoE), that does not require domain labels and is more interpretable. We use hypernetworks to generate the weights of experts, allowing experts to share some useful meta-knowledge. MoE has proven adept at detecting and identifying heterogeneous patterns in data. For DG, heterogeneity exactly arises from the domain shift. We compare HMOE with other DG algorithms under a fair and unified benchmark-DomainBed. Extensive experiments show that HMOE can perform latent domain discovery from data of mixed domains and divide it into distinct clusters that are surprisingly more consistent with human intuition than original domain labels. Compared to other DG methods, HMOE shows competitive performance and achieves SOTA results in some cases without using domain labels.
Fichier principal
Vignette du fichier
HMOE_arXiv.pdf (2.34 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03855006 , version 1 (16-11-2022)

Identifiants

Citer

Jingang Qu, Thibault Faney, Ze Wang, Patrick Gallinari, Soleiman Yousef, et al.. HMOE: Hypernetwork-based Mixture of Experts for Domain Generalization. 2022. ⟨hal-03855006⟩
84 Consultations
198 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More