Scanning for Pangram Errors

Summary

Pangram is a new AI detection tool that has made waves in various reader circles for catching literary prize winners using AI generated prose. Pangram claims it can help its users detect AI generated writing. AI detection isn’t new, it has been around as long as AI. Unlike previous generations of detection tools though, Pangram boasts a false positive rate of 1 in a 10,000. That is, if Pangram says a block of text is AI there is only a one in ten thousand chance that it was written by a human.I run a small romance book community. Right now many niche book communities are facing an onslaught of unlabeled AI works, mine included. To solve this problem, I turned to Pangram to identify works as AI written so that I could focus my moderation effort on those works. Pangram isn’t uncontroversial, and many online doubt its accuracy claims. The public has been burned by overconfident claims by AI detection vendors before. I have approached detection with a dose of skepticism myself, and so I tested Pangram in a number of ways.I ran a large number of stories published to RoyalRoad through Pangram, and found the tool did not detect any AI on stories before the release of ChatGPT. This is what we expect to find in a functioning detector, and is an encouraging signal that Pangram doesn’t needlessly flag human writing. I plan to write up my RoyalRoad results later, but that isn’t the focus of this post. In this post I test a different aspect of Pangram: memorization.Pangram is a machine learning system, and one concern with machine learning systems is that they might “memorize” their training set. When training a machine learning model like Pangram, the scientists and engineers give the model many labeled examples of AI and human writing, and the hope is that the model learns what AI writing “looks like” if you give it enough examples. But there is the risk Pangram is simply remembering every example from training and regurgitating its answers, rather than learning to spot AI in ...