Rethink Priorities

Special Projects: RP's Special Projects (SP) team supports priority projects that RP fiscally sponsors, incubates, or directly launches. In 2024, some of SP's notable achievements include: 42 fiscal sponsorship expressions of interest received; 10 projects supported; $6.4m total direct costs of supported projects; 7 sponsee hiring rounds completed; 39 total sponsee staff at year end; 6 visa sponsorship applications (UK & US); 9 in-person events; and 216 total event attendees. In 2024, the following projects were fiscally sponsored or incubated at RP: Epoch, Apollo Research, Consultants for Impact (CFI - previously the Effective Altruism Consulting Network), Quantified Uncertainty Research Institute (QURI), Condor Camp, The Insect Institute, Insect Welfare Research Society, the Vista Institute for AI Policy, Global Challenges Project, and provided general administrative support services to Pow-dr.

AI Policy and Strategy: In 2024, Rethink Priorities' (RP's) AI work was conducted by the Institute for AI Policy and Strategy (IAPS). A brief overview of IAPS' four main workstreams and 2024 accomplishments follows: (1) Compute policy work aims to explore novel interventions and strategic considerations for managing access to massive AI compute resources. In 2024, IAPS work on this area included publishing a landmark paper with the Center for a New American Security (CNAS) focused on using on-chip governance mechanisms to manage national security risks from AI and advanced computing, working on a forthcoming (published in 2025) report on location verification for advanced AI chips, and publishing a report analyzing the potential impact of high-end consumer GPUs on the efficacy of US export controls on AI chips. The work has received significant interest from US policymakers. (2) Frontier Security work aims to identify concrete interventions to address AI-related threats to national defense and security and protect innovation. Accomplishments in 2024 include publications analyzing advanced AI companies' RSPs (responsible scaling plans) and providing recommendations, a systematic analysis of regulatory precedents from all federal agencies, an early warning system for dual-use capabilities which lays out info-sharing paths between AI companies, government, and critical defenders, publications examining safety R&D efforts by advanced AI labs and the role the UK government can play in leading AI assurance, multiple RFI responses on key topics regarding AI, and upcoming reports (published in 2025) on differential access for cybersecurity capabilities, and on promising directions in AI reliability R&D. (3) International Strategy work aims to study trends in international coordination and competition around advanced AI. In 2024, published research included advice on topics for AI Track IIs that minimize risks of leaking sensitive information, an analysis of the first wave of AI safety institutes and key questions for the international network of these institutes, as well as analysis of China including counterparts to AI safety institutes and a comparison of US and China's approaches to governing advanced AI. (4) IAPS programs department ran an AI policy fellowship with 13 fellows, who now hold relevant positions in advanced AI labs, US and UK government, and think tanks.