OpenAI releases a cybersecurity model: is it enough?

The vulnerability was found in two hours

Justicz has a casual interest in cybersecurity; he had previously found a vulnerability in the same code. As he told TechCrunch, to search for a new bug, Justicz pointed ChatGPT at the code and asked it to find vulnerabilities at least as serious as one he had found before, which was fixed several years ago. Two hours later, the model produced a result. Justicz said the bug was so obscure that a person would have had difficulty spotting it, but it worked.

He reported the vulnerability to the Debian project, which maintains the relevant software. The project released a fix. There are no known cases of the bug being exploited. But the ease with which Justicz found it suggests that models with similar capabilities could help even novice hackers find vulnerabilities.

In a social media post discussing his discovery, Justicz wrote that highly unusual bugs could be found and exploited in the coming months. He also pointed to the potential for using a powerful model to inspect code or binaries that no one had audited yet. In his assessment, working for longer and having access to less restricted models could produce even more results.

OpenAI expands Daybreak

OpenAI announced ChatGPT Cyber, a model for cybersecurity. It will be available to the highest-tier users in the Daybreak Red program, which gives offensive cybersecurity researchers tools to find vulnerabilities and develop ways to exploit them.

The day before, the company said it had used the same model to inspect the Linux kernel and prepare fixes for the issues it found. The model identified 948 potential vulnerabilities in the kernel, as well as 81 new vulnerabilities in the Firefox browser. OpenAI said it wants to significantly expand Daybreak and bring in organizations of all sizes, independent researchers, and open-source software developers.

The company believes that people responsible for protecting digital infrastructure have limited time to act before models become even better at finding vulnerabilities and access to them becomes easier. Defenders, however, may struggle to handle a growing workload with limited resources. Justicz said he personally spent about $5,000 on subscriptions and tokens to find several vulnerabilities.

Alongside access to models through Daybreak, OpenAI announced a $1 billion support program for cybersecurity professionals. It is intended for participants working to protect critical infrastructure, including power grids and water systems.

Access and defenders’ resources

Justicz believes OpenAI recently restricted Daybreak Blue’s capabilities, making the program less useful for finding vulnerabilities. The restrictions could make things harder for potential attackers. According to Justicz, all he needed to join the program was to upload a photo of his driver’s license.

But restrictions have a downside: cybersecurity professionals worry that it will become harder for them to use advanced technology, while attackers will be able to run open models with their safety restrictions removed. According to OpenAI, only “thousands” of cybersecurity professionals take part in Daybreak — far fewer than the potential hackers, as the source notes.

It is unclear whether leading labs are directing enough resources toward the problem. Eric Wallace, a researcher in OpenAI’s technical staff, said the company allocates a significant share of its advanced models’ computing resources to finding vulnerabilities in open-source code and preparing fixes at no charge.

At the same time, searching the Linux kernel for bugs cost $1.2 million. For comparison, OpenAI spent about $6 million in tokens searching for a possible solution to a well-known problem in fluid dynamics. If the time available to fix vulnerabilities in digital infrastructure really is limited, one new model may not be enough.

Dataist's take: Model access programs and funding for defenders address part of the problem, but their scale is still difficult to compare with the number of potential attackers. The cost of inspecting the Linux kernel also shows that finding and fixing vulnerabilities requires significant resources.