I wouldn't say Pangram is broken, but I would say that it's brittle

Summary

So let me get to the nut of this thing before I do my usual meandering. Recently, someone accused me of using AI to write this old post from about a year ago, specifically highlighting the section about Ta-Nehisi Coates. I replied by saying that the post contained no AI writing; nothing I publish has been written or edited by an LLM. My accuser proceeded to say that the AI detection tool Pangram flagged that portion as 100% AI written with high confidence, a result I replicated. (Pangram report.) As the man who wrote that piece, I knew that I wasn’t guilty, but I also knew that wasn’t going to fly with other people. I was sufficiently annoyed that I paid for a Pangram license and found that, when I entered the entire essay, Pangram cleared it as 100% human written with high confidence. (Pangram report.) That is to say, the essay of about 5,000 words was declared 100% human written with high confidence even though it contains the section of about 300 words which was declared 100% AI written with high confidence. Subsequently, I’ve found that I can produce the same sort of inconsistency in the opposite direction - I can break the section that was called 100% AI into pieces that are then called 100% human written. (Pangram report of an example.) So pieces that Pangram call 100% human written are part of a larger piece that Pangram calls 100% AI written which in turn is part of an essay that Pangram calls 100% human written, a Russian doll of contradictory results.If nothing else, there’s a fundamental inconsistency here: it doesn’t make sense for a system to say that it has high confidence that an essay is 100% human written but also that a piece of that essay is 100% AI-written, again with high confidence, and also to say that the piece itself has subsections that are 100% human written…. These claims are mutually exclusive and must necessarily erode our confidence in the instrument. Experimenting, I’ve also found that I can pretty reliably get Pangram to declare larg...