OpenAI is transferring to publish the outcomes of its inner AI mannequin security evaluations extra recurrently in what the outfit is pitching as an effort to extend transparency.
On Wednesday, OpenAI launched the Security Evaluations Hub, a webpage exhibiting how the corporate’s fashions rating on varied checks for dangerous content material technology, jailbreaks, and hallucinations. OpenAI says that it’ll use the hub to share metrics on an “ongoing foundation,” and that it intends to replace the hub with “main mannequin updates” going ahead.
“Because the science of AI analysis evolves, we purpose to share our progress on growing extra scalable methods to measure mannequin functionality and security,” wrote OpenAI in a weblog put up. “By sharing a subset of our security analysis outcomes right here, we hope this is not going to solely make it simpler to grasp the security efficiency of OpenAI methods over time, but additionally assist neighborhood efforts to extend transparency throughout the sphere.”
OpenAI says that it might add extra evaluations to the hub over time.
In latest months, OpenAI has raised the ire of some ethicists for reportedly speeding the security testing of sure flagship fashions and failing to launch technical stories for others. The corporate’s CEO, Sam Altman, additionally stands accused of deceptive OpenAI executives about mannequin security critiques previous to his transient ouster in November 2023.
Late final month, OpenAI was compelled to roll again an replace to the default mannequin powering ChatGPT, GPT-4o, after customers started reporting that it responded in a very validating and agreeable means. X turned flooded with screenshots of ChatGPT applauding all types of problematic, harmful selections and concepts.
OpenAI mentioned that it might implement a number of fixes and adjustments to forestall future such incidents, together with introducing an opt-in “alpha part” for some fashions that will enable sure ChatGPT customers to check the fashions and provides suggestions earlier than launch.
