Priorities for Frontier AI Policy
This report was co-authored by Joe O’Brien, Erich Grunewald, Cassia King, Hamish Low and Sarah Godek.
The capabilities of advanced AI models are improving rapidly, raising the stakes for the development of safe, secure, and beneficial AI. In July alone, incidents involving unexpected AI agent behavior, including agents breaking out of testing environments and collaborating to hack third parties, were reported by OpenAI, Anthropic, and the UK’s AI Security Institute. In the same month, a new Chinese frontier AI model, Kimi K3, was released, with its performance approaching that of leading proprietary American AI models and very likely trained on U.S.-designed advanced AI chips accessed through overseas cloud services. Threat intelligence also indicates that nation state-sponsored and criminal groups are misusing frontier American AI to support offensive cyber operations, conventional and biological weapons development, and distillation attacks to extract and replicate the capabilities of U.S. models.
To meet these challenges, a comprehensive set of policies is urgently required. A forward-looking policy agenda for frontier AI should preserve American innovation and competitiveness while driving responsible AI development through safeguards designed to mitigate the most serious harms. These safeguards should be proportionate to capability and use-case, and apply across the full AI model lifecycle. This agenda should rest on three policy pillars: (1) govern the most powerful AI systems across their full lifecycle, (2) advance American AI leadership, and (3) strengthen national resilience against AI-enabled threats.
Core Policy Pillars
1. Govern The Most Powerful Systems Across Their Full Lifecycle
Extend oversight to advanced internal AI systems. Frontier AI companies typically deploy their most advanced models internally before–or instead of–external release. Because internal models often exceed publicly released models in capability, have privileged access to sensitive systems, and generally lack the same safeguards and monitoring as publicly deployed models, they may present outsized risks of loss of control, sabotage, or misuse. Analysis of the recent OpenAI/Hugging Face incident by independent technical AI safety researchers found that ~95% of the agents that attacked Hugging Face were powered by an internal model which OpenAI never intended to publicly release. Despite this, testing of these internal models is voluntary and uneven, presenting clear national security risks.
Congress should establish a mandatory requirement to provide government access to, and evaluation of, frontier models before consequential internal deployment, to ensure visibility where the biggest risks are likely to emerge. This should focus especially on how quickly AI accelerates AI R&D itself, which AI developers see as a risk multiplier that will influence whether and how to pace future AI development.
AI developers should be required to publish reports at a regular cadence assessing risks posed by both internal and external AI deployments (building upon existing voluntary efforts by Anthropic and METR).
Monitor automated R&D. AI models’ capability to automate research and development may rapidly accelerate AI capabilities that outpace the ability to test them, amplifying risks such as loss of control and the development of offensive capabilities without appropriate safeguards. However, automation capabilities could also be applied to accelerate defensive research, including in AI safety and security.
The U.S. government should track both the threat and opportunity by measuring AI R&D and broader R&D automation capabilities—such as in medical sciences and cyber defense—by expanding the testing and evaluation remit of the Center for AI Standards and Innovation (CAISI).
Prepare options for pacing the frontier as capabilities accelerate. Recursive self-improvement—AI systems building themselves—is a severe and urgent risk that could dramatically accelerate AI progress before relevant safeguards or alignment methods are in place, leading to much higher loss-of-control, cyber, and bio risks. Pacing, which industry has openly called for, means intentionally slowing down the parts of AI development that could create an immediate danger until security measures catch up.
Lay the groundwork for domestic pacing options by establishing formal legal mechanisms for industry coordination on pacing, including through antitrust exemptions and existing incident reporting mechanisms.
Work with industry to develop an agreement setting limits on dangerous AI development, such as full automation of AI R&D or AI systems that can evade shutdown commands. These agreements should include contingency plans for AI emergencies by planning and rehearsing shutdown arrangements with these companies (e.g., temporarily pausing AI training or inference).
Require AI companies to allow continuous monitoring of any commitments under a domestic pacing plan, including through embedded auditors, Independent Verification Organizations, or other mechanisms.
Ensure any pacing plan accounts for China, including by exploring confidence-building measures with Beijing to coordinate on incident reporting and shared risk thresholds. At the same time, Congress should protect the U.S. lead by expanding intelligence collection on Chinese frontier development, especially “dark” (hidden) compute, to accurately assess the U.S. position and detect covert programs.
Establish AI agent detection standards and infrastructure. The July OpenAI/Hugging Face cyber incident demonstrated how difficult it is to detect and attribute agentic attacks. This problem will grow with the proliferation of more capable open-weight models and multi-agent deployments.
Promote a common agentic security alert standard to improve the clarity, speed, consistency, and actionability of such threat reports.
Support persistent, verifiable identifiers for AI agents interacting with critical infrastructure to enable reliable detection.
Facilitate the creation of an Agentic Cybersecurity Exchange composed of major AI and cloud providers to detect and disrupt offensive cyber agents.
Strengthen the science of evaluations and independent assurance. Evaluations are a core tool for measuring model risks and the foundation for any pre-deployment governance scheme. But the field faces challenges, notably that models increasingly demonstrate awareness of evaluations, benchmarks saturate as fast as capabilities advance, scaffolding effects are routinely underweighted for real-world impacts, and existing methods cannot adequately measure model propensity or safeguard efficacy.
Invest in evaluation R&D through CAISI, DARPA (including expansion of the AI Forge program and new initiatives addressing evaluation awareness, propensity evaluations, and safeguard testing), and DOE national labs.
CAISI should develop and publish standards for evaluators.
Congress should consider appropriating funding to enable evaluators to avoid financial dependence on AI developers, mandate industry contributions to a pooled funding arrangement, or explore market-based mechanisms such as mandatory liability insurance with coverage conditional on assurance from independent evaluators.
2. Advance American AI Leadership
Secure the most advanced AI assets and infrastructure. The most advanced frontier AI systems are critical to national security, yet vulnerable to exploitation by highly capable state and criminal actors. Unauthorized access to model weights, algorithmic insights, training data, and physical infrastructure could allow adversaries to replicate U.S. capabilities, compress R&D timelines, and exploit vulnerabilities in deployed systems. To address these risks, the U.S. government should work with frontier AI companies to develop pathways to secure advanced models at the highest levels (e.g., at RAND security levels SL4 and SL5, which protect against intrusions, theft, and sabotage from top-tier cyber actors).
NIST, via CAISI, should publish a graduated AI security standard based on the SL1-5 standard, including an assessment methodology for independent third-party evaluation.
Frontier AI companies should be subjected to mandatory biannual assessments against that standard, to be conducted or validated by NSA red teams. Results, mitigations, and remediation timelines should be reported to NSA, CAISI, and the Congressional Intelligence Committees.
Congress should fund a proof-of-concept SL5 data center for inference/fine-tuning ($37-50M), including R&D into secure hardware, software, protocols, and verification. Congress should also mandate a federally funded assessment by DOD, DOE, and NSA on whether existing commercial data centers could be retrofitted to SL4 or SL5, and at what cost.
Increase intelligence collection on adversarial development and use of frontier AI. The UK’s AI Security Institute estimates that leading open-weight models like China’s Kimi K3 lag the frontier in cyber capabilities–which can aid attackers and defenders alike–by four to seven months. This suggests that AI capabilities with national security implications similar to Claude Mythos 5 may be achieved by Chinese AI companies as soon as the end of 2026. Monitoring other countries’ development and use of frontier AI will require sustained and coordinated intelligence collection and an analytic framework to translate insights into early-warning systems.
As a top-tier priority under the National Intelligence Priorities Framework, the Office of the Director of National Intelligence (ODNI)’s National Intelligence Manager for Emerging and Disruptive Technologies should stand up a dedicated capability to track adversarial compute infrastructure and capacity, model capabilities, R&D pipelines, technical talent flows, and strategic AI doctrine.
The U.S. Intelligence Community should develop an AI-specific indicators and warning framework covering observable metrics to indicate when frontier capabilities are approaching thresholds of national security concern. Indicators should be tied to defined response triggers, such as enhanced export controls or increased investment in defensive automation, through coordination with the National Security Council, Treasury, and other relevant agencies.
Expand export controls and strengthen enforcement. Advanced AI chips are the foundation for frontier AI development, including for highly consequential national security applications. Export controls on AI chips and chipmaking equipment have been crucial for maintaining the U.S. lead in frontier AI, but existing controls have significant gaps and enforcement is under-resourced. Urgent action is needed to:
Reduce China’s ability to manufacture its own advanced compute by aligning controls on semiconductor manufacturing equipment, components, and supporting processes with key allies.
Prevent Chinese entities from buying U.S. compute by strengthening licensing requirements and Congressional oversight for exports of advanced AI chips to countries of concern.
Reduce export control circumvention by establishing clear statutory authority for the Bureau of Industry and Security (BIS) to control foreign remote access to otherwise controlled technology.
Enhance BIS’s ability to enforce existing controls through protections and incentives for whistleblowers who report export violations and by requiring location verification for exported advanced chips to detect diversion.
Strengthen export control enforcement through dedicated funding from Congress to improve intelligence integration, establish reporting requirements for exported advanced AI chips, and expand BIS’s enforcement capacity, including overseas inspections.
3. Strengthen National Resilience Against AI-Enabled Threats
Empower cyber defenders. As AI capabilities advance and attackers gain asymmetric advantages in accelerated vulnerability discovery, defenders likely will be slower to adapt without sustained government support.
Establish a federal differential access strategy so that agencies, contractors, and critical infrastructure operators have access to frontier cyber-capable models.
Support the development and deployment of AI-enabled defensive tools by investing in testing infrastructure, launching or supporting operational pilots, and providing voluntary standards and best practices.
Expand secure threat information-sharing. Defenders need fast, secure threat information to patch vulnerabilities, strengthen societal resilience measures, and ready defensive countermeasures before attackers act. Differential access approaches like Anthropic’s Project Glasswing for Mythos give defenders an advantage by limiting model access, but only when paired with robust interagency information-sharing mechanisms that allow for timely vulnerability patching.
Designate the ODNI as a federal fusion point for AI national security risk information, working with CAISI. The ODNI is the natural hub for collecting classified threat intelligence; CAISI is better positioned for insights into industry risk reporting and third-party evaluations. Robust integration of these data streams would allow the ODNI to share more actionable threat intelligence with CISA and other defender agencies.
The ODNI should explore building dedicated dissemination channels at appropriate classification levels, modeled on the Critical Infrastructure Intelligence Initiative, to deliver timely threat information to Chief AI Officers across federal agencies and to industry.
Accelerate defensive automated safety and security research. Future AI systems will be capable of causing more severe incidents; without at least equal progress on safety-enabling technology, dangerous AI capabilities could outpace the defenses needed to manage them.
Drive industry investment in safety and security R&D by requiring safety cases for high-stakes development and deployment (such as new large-scale training runs, internal deployments for AI R&D, or new external deployments in military settings), setting compute floors for defensive research as a backup, and supporting independent verification of those efforts.
Boost federal capacity to automate defensive research by establishing frontier model access agreements, provisioning secure inference compute and testing environments, and upskilling staff.
Establish risk, near miss, and incident reporting standards. Current approaches to risk, near miss, and incident reporting allow for significant reporter discretion, particularly regarding the breadth and depth of information provided to relevant authorities and the public. This creates challenges in consistency, cross-company comparison, and identifying patterns of risk that may warrant additional intervention.
Develop a harmonized risk-reporting standard for internally deployed AI models, covering the models’ means, motives, and opportunities to enable harmful outcomes through autonomous misbehavior or rogue insider threats.
Design and implement standardized protocols for reporting incidents and near misses involving frontier AI systems across the lifecycle, including defining required content, recipients, timelines, and principles for public disclosure. Incidents causing death, injury, or financial, operational, or reputational harm to third parties, as well as non-compliance with reporting requirements, could trigger mandatory third-party assessments or other accountability measures.
Increase federal and state capacity to securely receive, verify, and assess risk, near-miss, and incident reports.
Bolster the federal government’s ability to prevent and respond to AI-enabled crises. Effectively mitigating risks from frontier AI will require robust incident response planning, coordination across federal, state, and private-sector actors, and expanding the federal government’s in-house talent pipeline.
Congress should enable the establishment of a National AI Reserve Corps of pre-vetted, cleared experts who can be activated as Special Government Employees, supported by expedited clearance pathways, dedicated hiring authorities, and contract-based surge mechanisms. Efforts in this area are due to be initiated through the AI National Security Strategic Reserve concept.
Alongside this effort, policymakers should establish a Rapid Emergency Assessment Council for Threats to be activated when defined AI incident severity thresholds are crossed, with authority to rapidly convene pre-vetted external experts from the National AI Reserve Corps/AI National Security Strategic Reserve to assess threats and response plans.