CoSwID, a Code Switching Identification Method Suitable for Under-Resourced Languages - Archive ouverte HAL Access content directly
Conference Papers Year : 2022

CoSwID, a Code Switching Identification Method Suitable for Under-Resourced Languages

CoSwID, une méthode d'identification de l'alternance codique appropriée aux langues peu dotées

(1)
1
Laurent Kevers

Abstract

We propose a method for identifying monolingual textual segments in multilingual documents. It requires only a minimal number of linguistic resources-word lists and monolingual corpora-and can therefore be adapted to many under-resourced languages. Taking these languages into account when processing multilingual documents in NLP tools is important as it can contribute to the creation of essential textual resources. This language identification task-code switching detection being its most complex form-can also provide added value to various existing data or tools. Our research demonstrates that a language identification module performing well on short texts can be used to efficiently analyse a document through a sliding window. The results obtained for code switching identification-between 87.29% and 97.97% accuracy-are state-of-the-art, which is confirmed by the benchmarks performed on the few available systems that have been used on our test data.
Fichier principal
Vignette du fichier
KeversLaurent_sigul-2022.pdf (181.37 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03722821 , version 1 (13-07-2022)

Identifiers

  • HAL Id : hal-03722821 , version 1

Cite

Laurent Kevers. CoSwID, a Code Switching Identification Method Suitable for Under-Resourced Languages. 1st Annual Meeting of the ELRA/ISCA Special Interest Group on Under-Resourced Languages (SIGUL2022 @LREC2022), European Language Resources Association, Jun 2022, Marseille, France. pp.112-121. ⟨hal-03722821⟩
23 View
14 Download

Share

Gmail Facebook Twitter LinkedIn More