Model Evaluation and Threat Research is an AI research charity that looks into the threat of AI agents! That sounds a bit AI doomsday cult, and they take funding from the AI doomsday cult organisat…
But when a mid-tier or entry level dev can do 60% of what a senior can do
This simply isn’t how software development skill levels work. You can’t give a tool to a new dev and have them do things experienced devs can do that new devs can’t. You can maybe get faster low tier output (though low tier output demands more review work from experienced devs so the utility of that is questionable). I’m sorry but you clearly don’t understand the topic you’re making these bold claims about.
Even pre AI I had to deal with a project where they shoved testing and compliance at juniors for a long time. What a fucking mess it was. I had to go through every commit mentioning Coverity because they had a junior fixing coverity flagged “issues”. I spent at least 2 days debugging a memory corruption crash caused by such “fix”, and then I had to spend who knows how long reviewing every such “fix”.
And don’t get me started on tests. 200+ tests, of them none caught several regressions in handling of parameters that are shown early in the frigging how-to. Not some obscure corner case, the stuff you immediately run into if you just follow the documentation.
With AI all the numbers would be much larger - more commits “fixing coverity issues” (and worse yet fixing “issues” that LLM sees in code), more so called “tests” that don’t actually flag any real regressions, etc.
This simply isn’t how software development skill levels work. You can’t give a tool to a new dev and have them do things experienced devs can do that new devs can’t. You can maybe get faster low tier output (though low tier output demands more review work from experienced devs so the utility of that is questionable). I’m sorry but you clearly don’t understand the topic you’re making these bold claims about.
I think more low tier output would be a disaster.
Even pre AI I had to deal with a project where they shoved testing and compliance at juniors for a long time. What a fucking mess it was. I had to go through every commit mentioning Coverity because they had a junior fixing coverity flagged “issues”. I spent at least 2 days debugging a memory corruption crash caused by such “fix”, and then I had to spend who knows how long reviewing every such “fix”.
And don’t get me started on tests. 200+ tests, of them none caught several regressions in handling of parameters that are shown early in the frigging how-to. Not some obscure corner case, the stuff you immediately run into if you just follow the documentation.
With AI all the numbers would be much larger - more commits “fixing coverity issues” (and worse yet fixing “issues” that LLM sees in code), more so called “tests” that don’t actually flag any real regressions, etc.