Anthropic cofounder hallucinates ghost in the machine after hearing the Pope speak about AI

Summary

OPINION In his encyclical Magnifica Humanitas, Pope Leo XIV warns against equating machine "intelligence" with human intelligence."We must avoid the misconception of equating this type of 'intelligence' with that of human beings," he declared. "These systems merely imitate certain functions of human intelligence."Invited to speak at the event, Chris Olah, a co-founder of Anthropic and the company's interpretability research lead, proceeded to push back on that idea amid his appreciation of the occasion. AI systems, he said, "are not the cold, calculating robots we were promised. They are made from us, from our words – and, as the Holy Father observes, they remain in important ways mysterious even to those of us who train them." It's as if naming the company Anthropic granted a license to anthropomorphize AI models. The notion that there's some AI mystery in the spiritual sense is just hot garbage Literally speaking, there's some truth to Olah's musing. AI systems are not cold – Blackwell chips idle at 32 to 38°C. They are not calculating – they're bad at math. And they're not robots – AI models are specialized binary blobs of tensors and metadata that can be instantiated across multiple servers. But the notion that there's some AI mystery in the spiritual sense is just hot garbage. AI systems are indeed "made from us, from our words" and that is why Anthropic and its rivals have been named in more than 100 lawsuits. One of the reasons those systems remain mysterious is that Anthropic and its rivals don't disclose where they got their training data.In his prior paragraph, Olah leans on the "mystery" of AI even more prominently."AI systems are not engineered the way a bridge or an airplane is engineered," Olah wrote. "We understand an airplane because we designed every part of it and we understand the physics that act on it. AI models are not like that. They are grown, on a structure roughly modeled after the brain, on an enormous inheritance of human thought and spee...