

If someone demands a piece of software that completely erases all available data when a certain button is pressed, no one is told about it, someone presses it by accident and deletes a whole bunch of data, that’s not a software development error. That’s a management failure. The software did exactly what it was supposed to do.
And LLMs wouldn’t make anything better in that regard. An LLM is built to do as the user wishes, with much, much less pushback than a human. So whereas a human getting the demand to write that software might say, “what in the actual fuck are you smoking and can I have some of it”, an LLM will typically say, “that’s a very bold decision!” In fact, in the case of MCAS, the issue of safety was raised by the engineers many times, and was flat out ignored by management. An LLM is much less likely to make a fuss and much more likely to just do it.





Yeah, I just read up on it. Now I’m feeling sick. I’ll take the essay any day.