On October 5th, the New York City Council held a hearing on AI safety issues. Jacob Coxon, a former researcher at Anthropic and an AI safety “whistleblower,” was present to testify. Coxon had previously posted warnings on the X platform that AI “could destroy humanity,” sparking widespread attention with 170 million views in just one day. During the hearing, Coxon once again warned that humanity faces significant risks from AI, advocating strongly for slowing down the current pace of AI development.
“One thing that worries me is that we currently don’t know how to control AI⋯⋯ We don’t understand its motivations, nor do we comprehend why it does what it does,” Coxon stated bluntly at the hearing. “The second concern is the accelerating pace of advancement in AI capabilities. Next year, the models’ abilities will be very powerful, especially as we are approaching the stage where AI can self-improve, meaning they can carry out research previously done by humans.”
Coxon reiterated that while the development of AI can bring significant benefits to humanity, “almost improving everyone’s life,” based on his experience, if development continues at this rate, “humans are likely to lose control over AI, ultimately leading to extinction.”
He revealed that OpenAI’s plan is to achieve a “milestone in automated research” by around 2027 or 2028, a goal that seems achievable in just a few months.
“These companies need to be more transparent with the public and independent experts. They need to prioritize security more than we do and until we can have confidence in artificial intelligence, we should slow down the development of cutting-edge AI,” Coxon concluded before ending his remarks.
AI critics such as Google DeepMind researcher Alex Turner and former OpenAI researcher and current head of the AI Futures Project, Daniel Kokotajlo, also testified remotely, highlighting the significant risks posed by rapidly advancing technology.
Turner expressed concern that a superintelligent group could seize control of human civilization, indicating that the likelihood of AI taking over human civilization is approximately one-third.
He noted that concerns about “AI replacing humans” are not new in the AI community and mentioned that Google seems to have abandoned previous controls on AI risks, such as DeepMind’s technology never being used for military or intelligence purposes or developing autonomous weapon systems. However, by 2025, Google had removed these principles.
Turner welcomed the proposals put forth by the City Council and expressed willingness to provide further suggestions after the hearing.
Kokotajlo used a metaphor likening the relationship between humans and AI to that of a board of directors and a company, citing the increasing reliance on AI-generated explanations to understand internal operations. He emphasized the need for enhanced transparency in the AI industry and to halt the race among leading AI companies for reversed improvements.
Responding to a question from the City Council President about how the U.S. should respond to competition from China if it slows down AI development, Kokotajlo emphasized that artificial intelligence is a more urgent threat than China. He stressed the need to address pressing domestic issues first before ensuring that China does not harm the U.S.
Kokotajlo pointed out that China’s AI products largely originate from the U.S., so if the U.S. slows down development, it will directly impact China’s progress. He highlighted that China’s AI is also moving towards “Recursive Self-Improvement” (RSI), and therefore, the U.S. should take sufficient security measures to prevent models from being stolen by the Chinese military.
At the invitation of New York City Council President Julie Menin, representatives from Anthropologie, OpenAI, Google, and Meta attended the Monday hearing.
Since Coxon resigned from his position at Anthropic last month and issued a chilling warning that “AI could destroy humanity,” discussions on AI protection measures and regulation have intensified.
Legislators and tech executives in New York City are deliberating on a series of bills aimed at regulating AI companies in the city, including mandatory installation of “kill switches” and unprecedented cash rewards for whistleblowers.
Phil Wong, a Chinese-American councilman from Queens, stated to Epoch Times before the hearing that this “all-hands-on-deck” hearing is rare in the New York City Council and marks the first time a hearing on AI safety issues has been held in the United States.
“This issue is quite serious; children are using AI for homework, and adults are relying on AI as lawyers and doctors⋯⋯ this is very dangerous,” Wong expressed. “I am proud that New York City is the first to hold such a hearing nationwide, our council president has done a great job.”