Center on Long-Term Risk🔒 SSL Verified

Table of Contents

• ↓ A high-level model of AI bargaining
• ↓ Overview of Transformative AI Misuse Risks: What Could Go Wrong Beyond Misalignment
• ↓ Individually incentivized safe Pareto improvements in open-source bargaining
• ↓ Top Alternatives to longtermrisk.org

6-Month Traffic Trend Estimate

Official Contact & Profiles

Visual representation extracted from longtermrisk.org

Content Snippets from longtermrisk.org

"Our goal is to address worst-case risks from the development and deployment of advanced AI systems. We are currently focused on harmful propensities in AI systems and on cooperation failures between such systems."

"We do interdisciplinary research, make and recommend grants, and build a community of professionals and other researchers around our priorities."

"CLR’s overall mission is to reduce the risk of astronomical suffering from powerful AI, or s-risks. We’re primarily concerned with threat models involving the deliberate creation of suffering, and have identified a number of properties that may increase such risks if future powerful AIs have them. We call these s-risk conducive properties, short SRCPs. Our previous empirical research agenda has focused on characterizing and measuring these properties with a particular focus on agential suffering as a result from conflict. While we are still interested in measuring SRCPs we have shifted our focus from evaluating to understanding and steering their emergence, and now also consider threat models that involve motivations for creating suffering outside of conflict. This puts more emphasis on […]"

"Author: Anthony DiGiovanni Executive summary Introduction At the Center on Long-Term Risk (CLR), we're interested in preventing catastrophic cooperation failures between powerful AIs. These AIs might be able to make credible commitments,1 e.g., deploying subagents that are bound to auditable instructions. Such commitment abilities could open up new opportunities for cooperation in high-stakes negotiations. In particular, with the ability to commit to certain policies conditional on each other's commitments, AIs could use strategies like "I'll cooperate in this Prisoner's Dilemma if and only if you're committed to this same strategy" (as in open-source game theory). But credible commitments might also exacerbate conflict, by enabling multiple parties to lock in incompatible demands. For example, suppose two AIs can each lock a […]"

Key Topics & Focus Areas for longtermrisk.org:

A high-level model of AI bargaining

Overview of Transformative AI Misuse Risks: What Could Go Wrong Beyond Misalignment

Individually incentivized safe Pareto improvements in open-source bargaining