At the 19th International Conference on Software Technologies (ICSOFT 2024) in Dijon, France, we presented “Feature Extraction, Learning and Selection in Support of Patch Correctness Assessment” (Csuvik, Horváth and Vidács).
The paper compares features from earlier studies, both hand-crafted heuristics and learned embeddings, for telling correct automatically generated patches from incorrect ones, using several machine learning models. Not all features are equally useful, and the choice of model matters a lot: with 43 selected features an MLP classifier reached 81% F1, against 64% for a decision tree using all 490 features. On a dataset of 903 labelled patches the model reached 100% precision and 97% recall at its peak.
It covers which signals separate a correct patch from one that only passes the tests.
Read the paper: publisher page
