Research
The below database includes IAPS’ public research reports, responses to government requests for information, blog posts, and more.
Featured Research
Location Verification for AI Chips introduces a design for a mechanism that can work on existing chips and verify their location to help enforce export control agreements, while being privacy preserving and reasonably tamper-proof.
Secure Governable Chips details how security features on AI chips could help mitigate national security risks from the development of broadly capable dual-use AI systems, while protecting user privacy.
AI Agent Governance: A Field Guide provides an overview of AI Agents and what to expect. Such systems could autonomously achieve goals in the world, with little to no explicit human instruction about how to do so.
Filter
- Responses to RFIs/RFCs 5
- Asher Brass 2
- Compute governance 9
- Frontier Security 1
- Jam Kraprayoon 6
- Linkpost 8
- Oscar Delaney 2
- Erich Grunewald 7
- Oliver Guest 9
- Shaun Ee 2
- Tao Burga 1
- International governance & China 10
- Policy and Standards 17
- Renan Araujo 3
- Sumaya Nur Adan 1
- Issue Brief 5
- Joe O'Brien 3
- Bill Anderson-Samways 3
- Onni Aarne 1
“Differential access” is a strategy to tilt the cybersecurity balance toward defense by shaping access to advanced AI-powered cyber capabilities. We introduce three possible approaches, Promote Access, Manage Access, and Deny by Default, with one constant across all approaches — even in the most restrictive scenarios, developers should aim to advantage cyber defenders.
Our survey of 53 specialists across 105 AI reliability and security research areas identifies the most promising research prospects to guide strategic AI R&D investment.
Adding location verification features to AI chips could unlock new governance mechanisms for regulators, help enforce existing and future export controls by deterring and catching smuggling attempts, and enable post-sale verification of chip locations. This paper is meant to serve as an initial introduction to location verification use-cases for AI chips with comparison of different methods.
As the administration works towards a strong, streamlined successor to the diffusion rule, we offer recommendations for BIS across three core objectives: (1) Steer the global distribution of American compute to preserve America’s lead in AI; (2) Ensure importing countries—including allies—uphold US export controls or face strict import limits, and use existing technology to address enforcement challenges such as illegal AI chip reexports; and (3) Secure key AI models stored on foreign soil, as model weight theft represents a major potential “compute shortcut” for adversaries.
There has been significant recent speculation about whether the US government will lead a future project to build and acquire advanced AI, or continue to play a more arms-length role. We conducted a forecasting workshop on this question, employing the IDEA protocol to elicit predictions from six professional forecasters and five experts on US AI policy.
This report is an accessible guide to the emerging field of AI agent governance, including an analysis of the current landscape of agent and their capabilities, novel and enhanced risks posed by more agentic systems, and major open questions and agent interventions.
While attention focuses on publicly available models like ChatGPT, the real risk to U.S. national interests is the theft of unreleased “internal models.” To preserve America’s technological edge, the U.S. government must work with AI developers to secure these internal models.
Our comments focus on ways the US AI Action Plan can build trust in American AI, deny advantages to adversaries, and prepare to adapt as the technology evolves.
If the US is serious about outcompeting China in AI, it needs to strengthen, not weaken, its AI chip export regime. A crucial first step is eliminating the widespread occurrence of AI chip smuggling.
This is a linkpost for an article written by IAPS researchers Oscar Delaney and Oliver Guest.
This is a linkpost to a piece that Tao Burga, an IAPS fellow, co-authored with researchers from CNAS (Center for a New American Security).
This paper, published by the Oxford Martin AI Governance Initiative, explores how to determine which actors are best suited to develop AI model evaluations. IAPS staff Renan Araujo, Oliver Guest, and Joe O’Brien were among the co-authors.
This is a link post for a paper which was led by researchers from the Oxford Martin AI Governance Initiative, and on which IAPS researcher Oliver Guest was one of the authors.
A look at U.S. and Chinese policy landscapes reveals differences in how the two countries approach the governance of general-purpose artificial intelligence. Three areas of divergence are notable for policymakers: the focus of domestic AI regulation, key principles of domestic AI regulation, and approaches to implementing international AI governance.
In this commentary, we explore key questions for the International Network of AI Safety Institutes and suggest ways forward given the upcoming San Francisco convening on November 20-21, 2024. What should the network work on? How should it be structured in terms of membership and central coordination? How should it fit into the international governance landscape?
Based on a systematic review of open sources, we identify Chinese “AISI counterparts,” i.e. Chinese institutions doing similar work to the US and UK AISIs and that have relatively close government links.
This post is a copy of IAPS’ response to a BIS request for public comment. It outlines ways to expand the role of other stakeholders in the reporting process for AI models and compute clusters, including third-party evaluators, civil society groups, and other public sector entities.
AI Safety Institutes (AISIs) are a new institutional model for AI governance that has expanded across the globe. In this primer, we analyze the “first wave” of AISIs: the shared fundamental characteristics and functions of the institutions established by the UK, the US, and Japan that are governmental, technical, with a clear mandate to govern the safety of advanced AI systems.
IAPS submitted a response to a National Institute of Standards and Technology (NIST) Request for Comment, outlining practices that could help AI developers better manage and mitigate misuse risks from dual-use foundation models.
IAPS submitted a response to a Department of Defense Request for Information on Defense Industrial Base Adoption of AI for Defense Applications.
This report analyzes the research published by Anthropic, Google DeepMind, and OpenAI about safe AI development, as well as corporate incentives to research different areas. This research reveals where corporate attention is concentrated and where there are potential gaps.
This piece is a link post for a paper which was led by Hadrien Pouget (Carnegie Endowment for International Peace) and Claire Dennis (Centre for the Governance of AI). IAPS staff Renan Araujo and Oliver Guest were among the paper’s co-authors.
This policy memo by Jam Kraprayoon and Bill Anderson-Samways, published by the UKDayOne and the Social Market Foundation, recommends that the UK government implement a targeted market-shaping program mobilizing public and private sector investment to supercharge the UK’s AI assurance technology industry.
This policy memo by Jam Kraprayoon, Joe O’Brien, and Shaun Ee (IAPS), published by the Federation of American Scientists, proposes that Congress should set up an early warning system for novel AI-enabled threats to provide defenders maximal time to respond to a given capability before information about it is disclosed or leaked to the public.
Future AI systems may be capable of enabling offensive cyber operations, lowering the barrier to entry for designing and synthesizing bioweapons, and other high-consequence dual-use applications. If and when these capabilities are discovered, who should know first, and how? We describe a process for information-sharing on dual-use capabilities and make recommendations for governments and industry to develop this process.
This report analyzes the potential impact of high-end consumer GPUs on the efficacy of US export controls on AI chips. It studies three stockpiling scenarios and what AI capabilities those may enable, and makes recommendations for policymakers.
This blog post by Erich Grunewald (IAPS) and Samuel Hammond (the Foundation for American Innovation) argues that Congress should increase the funding of the Bureau of Industry and Security.
This issue brief suggests agenda items for dialogues about advanced AI risks that minimize risk of leaking sensitive information.