A toot from mid-way through a thread of a hilarious deconstruction of various LLM foibles. This one discusses the vapidity of the “unit tests” LLMs like to write:

LLMs love to call things smoke tests. […] The test literally does nothing and cannot possibly fail, but says it does at least two things, because to an LLM something saying it does something is the same thing as it actually doing that thing.