DeepLSH: Deep Locality-Sensitive Hash Learning for Fast and Efficient Near-Duplicate Crash Report Detection

Youcef Remil (1, 2, 3, 4) , Anes Bendimerad (2) , Romain Mathonat (2) , Chedy Raïssi (5) , Mehdi Kaytoue (2)

1 INSA Lyon - Institut National des Sciences Appliquées de Lyon
2 Infologic
3 DM2L - Data Mining and Machine Learning
4 LIRIS - Laboratoire d'InfoRmatique en Image et Systèmes d'information
5 Riot Games

Automatic crash bucketing is a crucial phase in the software development process for efficiently triaging bug reports. It generally consists in grouping similar reports through clustering techniques. However, with real-time streaming bug collection, systems are needed to quickly answer the question: What are the most similar bugs to a new one?, that is, efficiently find near-duplicates. It is thus natural to consider nearest neighbors search to tackle this problem and especially the well-known locality-sensitive hashing (LSH) to deal with large datasets due to its sublinear performance and theoretical guarantees on the similarity search accuracy. Surprisingly, LSH has not been considered in the crash bucketing literature. It is indeed not trivial to derive hash functions that satisfy the so-called locality-sensitive property for the most advanced crash bucketing metrics. Consequently, we study in this paper how to leverage LSH for this task. To be able to consider the most relevant metrics used in the literature, we introduce DeepLSH, a Siamese DNN architecture with an original loss function, that perfectly approximates the locality-sensitivity property even for Jaccard and Cosine metrics for which exact LSH solutions exist. We support this claim with a series of experiments on an original dataset, which we make available. CCS CONCEPTS • Software and its engineering → Software maintenance tools; • Computing methodologies → Randomized search; • Theory of computation → Theory of randomized search heuristics.

Mots clés: Crash deduplication Stack trace similarity Approximate nearest neighbors Locality-sensitive hashing Siamese neural networks; Crash deduplication; Stack trace similarity; Approximate nearest neighbors; Locality-sensitive hashing; Siamese neural networks

Domaines: Informatique [cs]; Intelligence artificielle [cs.AI]; Complexité [cs.CC]; Génie logiciel [cs.SE]

Youcef Remil, Anes Bendimerad, Romain Mathonat, Chedy Raïssi, Mehdi Kaytoue. DeepLSH: Deep Locality-Sensitive Hash Learning for Fast and Efficient Near-Duplicate Crash Report Detection. IEEE/ACM International Conference on Software Engineering, IEEE/ACM, Apr 2024, Lisbon (Portugal), Portugal. ⟨hal-04236085⟩