At the 18th International Conference on Software Technologies (ICSOFT 2023) in Rome, Italy, we presented “Can ChatGPT Fix My Code?” (Csuvik, Gyimóthy and Vidács).
We asked ChatGPT to suggest repairs for real bugs sampled from well-known program repair datasets, generated 1,000 inputs and checked every answer by hand. ChatGPT produced fixes on par with those written by developers, but the prompt mattered: on average it proposed a correction in 19% of cases, and the wrong input format dropped this to as low as 6%.
It was an early, measured look at what a chat model can and cannot repair.
Read the paper: publisher page
