You can become utterly dependent on a new glamorous technology, be it cyber-space, artificial intelligence. . . It’ll enable you. It’ll move you forward. But does it create a potential achilles heel? Often it does.1
—Admiral James Stavridis
Artificial intelligence (AI) has become the technical focal point for advancing naval and Department of Defense (DoD) capabilities. Secretary of the Navy Carlos Del Toro listed AI first among his priorities for innovating U.S. naval forces. Chief of Naval Operations Admiral Michael Gilday listed it as his top priority during his Senate confirmation hearing.2This focus is appropriate: AI offers many promising breakthroughs in battlefield capability and agility in decision-making.
Yet, the proposed advances come with substantial risk: automation—including AI—has persistent, critical vulnerabilities that must be thoroughly understood and adequately addressed if defense applications are to remain resilient and effective. Current state-of-the-art AI systems undergirding these advances are surprisingly fragile—that is, easily deceived or broken or prone to mistakes in high-pressure use.
