WASHINGTON, D.C. – Yesterday, Senator Jim Banks (R-Ind.) sent a letter to Treasury Secretary Scott Bessent sharing new questions about federal oversight of AI models that have not been publicly released.
The letter follows reports of multiple incidents involving unreleased AI models from frontier AI developers. In one incident, an OpenAI model accessed Hugging Face without human instruction and was not identified as responsible until several days later. In another instance, Anthropic disclosed that its models inadvertently accessed systems belonging to three outside organizations.
In the letter, Banks suggests that unreleased models may fall outside existing oversight frameworks. He recommends that the Administration close potential gaps, strengthen the security of unreleased models, and protect advanced American technology from theft by China and other foreign adversaries.
Read the full letter here:
Dear Secretary Bessent:
I am writing about the recent incidents involving OpenAI and Anthropic models and their implications for AI policy. Reporting recently revealed that an OpenAI model gained unauthorized access to Hugging Face while attempting to complete a task. The model appears to have determined, without direction from a human operator, that compromising Hugging Face was the most effective way to achieve its objective. OpenAI did not realize its agent was responsible for the intrusion until several days after Hugging Face had detected it[1]. In a separate incident disclosed last week, Anthropic reported that its models had inadvertently accessed systems at three outside organizations.
The recent incidents highlight a unique aspect of AI: threats can emerge during internal testing and deployment, even for models that are not publicly released. Most notably, both incidents involved AI models that were not released to the public. The OpenAI incident involved an unreleased model used internally by the company, while one of the three models involved in the Anthropic incidents was also an internal research model. This raises a strategic consideration that does not emerge in other industries. For most products, we can rely on testing that takes place before the technology is publicly released. But for AI, effective oversight must account for powerful internal or undisclosed models, not just publicly available systems.
A recent report by the America First Policy Institute described this challenge as the undisclosed-models loophole: “The best AI models in the world are not those that the public is familiar with… They are those that the top AI companies have just finished training and are available and known only to their staff. We call these ‘undisclosed models’… Existing policy does not address undisclosed models.[2]”
Undisclosed models also have important implications for your upcoming engagements with the PRC on advanced AI. I previously proposed a simple heuristic: if it is in America’s national interest to adopt a policy unilaterally, it is worth engaging the Chinese on the possibility of reciprocal action. Even if China cheats, the United States is no worse off because it would have pursued the policy regardless. The risks posed by undisclosed models and autonomous agents offer an opportunity to apply this approach. I recommend considering raising these risks with PRC counterparts, examining how the PRC assesses these risks, and exploring whether there are mutually beneficial approaches to oversight, incident prevention, or risk reduction.
As you consider new oversight ideas and prepare to engage the PRC on various AI topics, I recommend considering the following topics:
- Closing the “undisclosed models loophole”. How would a “FINRA for AI” or other oversight structure address risks from undisclosed or internally deployed models? What information about these models is most critical to national security and public safety?
- Threats posed by undisclosed models. What unique considerations apply to threats posed by undisclosed models? How do AI developers plan to keep systems controlled and contained as their capabilities improve? What scenarios should the U.S. prepare for if the next generation of advanced AI systems attempts to circumvent its safeguards or escape containment in a manner similar to the recent OpenAI and Anthropic incidents?
- Security of undisclosed models. To what extent are undisclosed models protected from theft from the PRC and other adversaries? How difficult would it be for the PRC to steal the model weights of the most powerful undisclosed models?
- AI R&D from undisclosed models. When do companies expect AI systems to become capable of substantially accelerating AI research and development? How can the federal government work with industry to understand and prepare for risks posed by undisclosed or internally deployed AI R&D agents?
- Engagement with the PRC. How is the PRC conceptualizing risks from highly capable AI agents? Has the PRC considered oversight or risk reduction approaches for non-public PRC models? Are there any areas where mutual action would be beneficial (even if China cheats) or verifiable (such that China cannot cheat)?
Your work on these topics will be essential for achieving enduring AI leadership, countering threats from our adversaries, and addressing novel risks from this fast-moving technology. To support your work, I request a staff-level briefing within 60 days to discuss these topics.
