We ran a randomized controlled trial to see how much AI coding tools speed up experienced open-source developers.
The results surprised us: Developers thought they were 20% faster with AI tools, but they were actually 19% slower when they had access to AI than when they didn't.
https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/
This is funny and I can imagine how it could happen.
More discussion here: #1033006
sorry, I did not see that via the dupes feature or the similar feature :/
https://xcancel.com/METR_Evals/status/1943360399220388093