Summary

Cantrill opens with a confession about a college prank where he falsely warned students of a spreading computer virus, causing panic, lost work, and administrative consequences. He draws a direct parallel to AI safety researchers—specifically ex-Anthropic employee Jacob Coxon and Anthropic's Evan Hubinger—who have publicly claimed a greater than 10% probability that AI will kill all humans within a decade. He rebuts these claims from his domain of building computers, arguing that AI's physical-world agency remains constrained by human-engineered systems and that robots simply cannot do what doomers hand-wave away. He concludes that domain experts implicitly hold the public's trust and must not abuse it by making extraordinary claims without commensurate evidence.

Key Insight

AI extinction claims are a contagion of fear spread by experts abusing the public trust their credentials confer—and must be met with Carl Sagan's demand for extraordinary evidence proportional to extraordinary claims.

Spicy Quotes (click to share)

• 9

  I have never seen fear sown so irresponsibly by putative technologists as I have now with respect to AI and the threat of human extinction.

• 6

  When those domain experts sow fear, it is the fear that takes root, propagating much more readily than any evidence that may follow it.

• 3

  As Jonathan Swift famously quipped over three centuries ago, 'Falsehood flies, and the truth comes limping after it.'

• 5

  To the degree that digital systems have agency in the physical world, it is agency that we—we humans with arms and legs and brains and parents and children—permit.

• 4

  Acts of engineering are not acts of intelligence alone.

• 6

  The lesson that I learned (shamefully) decades ago is that domain experts, by way of their expertise, implicitly hold the public's trust—and we must not abuse it.

Tone

critical, confessional, polemical