Anthropic's revised S-1/A filing, submitted Monday, warns that the company may soon reach a model capability it defines as recursive self-improvement (RSI), according to the article.
The filing says Anthropic uses AI in its research and development. It cautions that "using AI to help build more capable AI could also pose a considerable safety risk, particularly if it sharply accelerates the pace of progress." The company says this risk is covered by its Responsible Scaling Policy.
The new Risk Factors language says Anthropic has in the past not identified, and may in the future not identify, a significant acceleration in its rate of development until after it has occurred. It also says the company's internal estimates and expectations for future models "indicate that we may imminently meet" the policy's definition of "dramatically accelerated progress attributable to AI."
Anthropic also states that it has not designated any current model as exceeding this threshold and that it continues to assess the underlying data. The filing says the company does not anticipate being able to safely use Claude to fully autonomously design and build successive generations of frontier models without meaningful human supervision. It adds that Anthropic is considering updates to its Responsible Scaling Policy to keep humans in control of frontier model development.
The article notes that an S