

The research, conducted by the non-profit Model Evaluation & Threat Research (METR), set out to measure the real-world impact of advanced AI tools on software development. Over several months in early 2025, METR observed 16 experienced open-source developers as they tackled 246 genuine programming tasks – ranging from bug fixes…