ConsilientSFL: using preferential voting system to generate combinatorial ranking metrics for spectrum-based fault localization

摘要

Spectrum-based fault localization (SFL) techniques have shown considerable effectiveness in localizing software faults. They leverage a ranking metric to automatically assign suspiciousness scores to certain entities in a given faulty program. However, for some programs, the current SFL ranking metrics lose effectiveness. In this paper, we introduce ConsilientSFL that is served to synthesize a new ranking metric for a given program, based on a customized combination of a set of given ranking metrics. ConsilientSFL can be significant since it demonstrates the usage of voting systems into a software engineering task. First, several mutated, faulty versions are generated for a program. Then, the mutated versions are executed with the test data. Next, the effectiveness of each existing ranking metric is computed for each mutated version. After that, for each mutated version, the computed existing metrics are ranked using a preferential voting system. Consequently, several top metrics are chosen based on their ranks across all mutated versions. Finally, the chosen ranking metrics are normalized and synthesized, yielding a new ranking metric. To evaluate ConsilientSFL, we have conducted experiments on 27 subject programs from Code4Bench and Siemens benchmarks. In the experiments, we found that ConsilientSFL outperformed every single ranking metric. In particular, for all programs on average, we have found performance measures recall, precision, f-measure, and percentage of code inspection, to be nearly 7, 9, 12, and 5 percentages larger than using single metrics, respectively. The impact of this work is twofold. First, it can mitigate the issue with the choice and usage of a proper ranking metric for the faulty program at hand. Second, it can help debuggers find more faults with less time and effort, yielding higher quality software.