Here’s a person, Baldur Bjarnason, @baldur, who thinks I’m really wrong about LLMs and coding. I mostly don’t agree but the argument is well-presented: https://www.baldurbjarnason.com/2025/trusting-your-own-judgement-on-ai/
@baldur Baldur is really right that most issues in software engineering are horribly science-free. For any nontrivial “best practice”, an experiment to test it would require two teams, doing nontrivial work, approaching the same problem with practices that are closely equivalent except for the one under study. Which is probably never gonna happen.
@timbray @baldur a different way to think of it is treating a specialist + a tool as a single unit of comparison, not one or the other. Like, you can't compare programming languages to each other even for a similar task, because a lot depends on whether a programmer or a team likes the language and is efficient at it.
Another analogy is racing: it's long established that you can't compare drivers or cars in isolation, only a particular driver driving a particular car as a whole unit.
@isagalaev @timbray @baldur You can compare programming languages, if you use an abstract method of defining the size of the task. This is what function point analysis does. Of course there are many other variations to take into account besides programming languages. Capers Jones has many good books on the subject of measurement.
