Microsoft CEO Nadella has warned that with the continuous enhancement of Artificial Intelligence (AI) models, companies should not rely solely on security assurances from model developers, but should establish observable, testable, and controllable systems to ensure that models can be paused or shut down when necessary.
Nadella, in a lengthy post on social platform X on October 10, pointed out that the operational mechanisms of modern “Super Intelligence” systems are still difficult to understand, even though the capabilities of the cutting-edge AI models driving these systems have surpassed traditional software systems. However, companies have already begun allowing these systems to access sensitive data and authorizing them to execute important, even critical tasks. He advocates for separating the control mechanisms that determine the access rights and actions of AI models and establishing secure systems that are observable, testable, and independently auditable.
He proposed treating advanced AI models as systems that could make mistakes or be compromised and setting up mechanisms similar to an “emergency brake” from the start, allowing authorized personnel to pause or shut down the model during task execution. He also urged the industry to establish common security standards and to disclose relevant information promptly in the event of incidents.
Microsoft President Smith, in an interview with the British Broadcasting Corporation (BBC), also called for strengthening AI security oversight and proposed introducing government reviews before the release of advanced AI models. He pointed out that the current security regulations for AI are even less than those for washing machines and milk.
Smith said, “Remember, washing machines are subject to more safety regulations than AI at the moment.” He added that packaged milk and commercial aircraft both undergo inspection and certification.