US colonel causes stir over potential challenges when using generative AI in conflict
"US colonel retracts comments on simulated drone attack ‘thought experiment’", 3 June 2023
A US air force colonel “misspoke” when he said at a Royal Aeronautical Society conference last month that a drone killed its operator in a simulated test because the pilot was attempting to override its mission, according to the society.
The confusion had started with the circulation of a blogpost from the society, in which it described a presentation by Col Tucker “Cinco” Hamilton, the chief of AI test and operations with the US air force and an experimental fighter test pilot, at the Future Combat Air and Space Capabilities Summit in London in May.
According to the blogpost, Hamilton had told the crowd that in a simulation to test a drone powered by artificial intelligence and trained and incentivized to kill its targets, an operator instructed the drone in some cases not to kill its targets and the drone had responded by killing the operator.
The comments sparked deep concern over the use of AI in weaponry and extensive conversations online. But the US air force on Thursday evening denied the test was conducted. The Royal Aeronautical Society responded in a statement on Friday that Hamilton had retracted his comments and had clarified that the “rogue AI drone simulation” was a hypothetical “thought experiment”...
...While the simulation Hamilton spoke of did not actually happen, Hamilton contends the “thought experiment” is still a worthwhile one to consider when navigating whether and how to use AI in weapons. “Despite this being a hypothetical example, this illustrates the real-world challenges posed by AI-powered capability and is why the Air Force is committed to the ethical development of AI,” he said in a statement clarifying his original comments.
In a statement to Insider, the US air force spokesperson Ann Stefanek said the colonel’s comments were taken out of context. “The Department of the Air Force has not conducted any such AI-drone simulations and remains committed to ethical and responsible use of AI technology,” Stefanek said.