On Monday, OpenAI announced impressive advances in mathematics, computer science and physics, along with what the company says is an evolving approach to communicating its findings. But despite their apparent course correction, researchers who spoke with The edge They are increasingly tired of the company’s antics.
It’s been a turbulent few months between OpenAI and mathematicians, who are still dealing with the hundreds of preprints that were unceremoniously dumped on them in early October. This time, OpenAI says it is working with expert “shepherds” to decide how best to share its latest advances.
The three results concern some of the most important unsolved problems in mathematics, computer science, and physics. Some expand on OpenAI’s previous work and others break new ground. Following the advice of the OpenAI “shepherds”, each one has a different launch strategy. For the mathematical result, OpenAI published the AI-generated result, “leaving researchers free to organize their own efforts to explain the result.” For its discovery in computer science, OpenAI will also bring together an external group of researchers “to explain the result.” Meanwhile, the physics have not been published at all, and OpenAI said it will “defer” to an external group of experts the question of how to publish the materials.
- The mathematical result seems surprising. It aims to solve the Kakeya conjecture in all dimensions. Broadly speaking, the problem asks how little space is needed to rotate a needle or pencil in all directions. A test of the three-dimensional version was one of the achievements for which New York University mathematician Hong Wang was awarded the prestigious Fields Medal earlier this year. Brendan Hassett, a mathematics professor at Brown University, said The edge OpenAI’s claim “seems well supported,” although it emphasized that it had not yet verified Lean formalization, a means of computationally verifying results.
- The physics outcome explores the Standard Model, the most complete and successful theoretical framework in physics. This is quantum chromodynamics, which describes the strong nuclear force, one of the four fundamental forces of nature (along with the gravitational force, the electromagnetic force and the weak nuclear force) and its relationship with the Yang-Mills theory, the cornerstone of modern particle physics. Yang-Mills theory is also at the center of one of the seven Millennium Prize mathematics problems, although OpenAI’s work does not address the “massive gap” unsolved.
- The final result is intended to disprove the exponential time hypothesis, a widely accepted assumption in theoretical computer science that certain problems cannot be solved substantially faster than brute force approaches. It is closely related to P versus NP, another Millennium Prize problem and one of the biggest unsolved problems in the field, which asks whether problems whose solutions can be easily checked are also easy to solve.
Given the varying levels of disclosure and the complexity of evaluating such work, it is difficult for researchers to fully judge the company’s findings. Even with all the information, doing so would take time. But the experts who spoke with The edge He said there are good reasons to take the results seriously, describing them as “huge,” “huge advances” and “significant.”
“It doesn’t make any sense that at first only a few distinguished scientists can examine them.”
But researchers’ enthusiasm for the science was tempered by exasperation with OpenAI’s conduct, a familiar sentiment after months of what they describe as a grueling barrage of AI-generated findings. Many were eager for the broader community to continue the arduous task of examining the AI lab’s findings and questioned why OpenAI was putting up additional barriers by selecting experts to oversee the process.
Most experts The edge He welcomed OpenAI’s decision to consult outside researchers, but argued that publicly announcing the results should also mean making that work available for anyone to examine. “I think it’s good that they publish the problems like they did today,” said Constantin Kogler, a researcher at the Institute for Advanced Studies, celebrating the more manageable batch of three results compared to the hundreds published previously. But he questioned the decision to restrict access to experts chosen by OpenAI. “It doesn’t make any sense that at first only a few distinguished scientists can examine them,” he said.
“The fact that they hand-picked the group is a disaster,” said Andreas Karch, a physics professor at the University of Texas at Austin. The edge. “The big labs already have enough power. They shouldn’t pick winners and losers.” Karch initially suggested selecting researchers through a “competitive process,” although he acknowledged that it would likely produce a similar group, which he described as “top-notch.” He later questioned why OpenAI “can’t publish the tests now,” in addition to naming its team of experts. “This would democratize access and at the same time demonstrate that they care about the results being meaningful and understandable,” he said.
“My feeling is that with any approach, there will be someone who will get angry.”
It’s exactly the kind of scrutiny that the Advisory Group on Mathematics and Artificial Intelligence (AGMAI), a think tank that OpenAI had previously promised to consult on sharing its findings, warned about, just a day before the announcement. “We recommend that AI companies do not adopt this practice,” they said, arguing that selecting certain people to receive the result first is “unnecessary,” borrows the reputation of the selected researchers, and risks unequal access, biased selection, and delays, among other issues. Instead, they recommended that AI labs publish results in academic repositories, with formalized tests and information about how problems were chosen, and provide funding and other support to people working to understand the results.
Following the publication of the latest results, AGMAI urged OpenAI to “promptly make public the natural language and Lean results behind its” announced results in physics:
“While these are central questions in physics, they are also important questions in mathematics. The existence of the quantum Yang-Mills theory is part of a Millennium Prize problem and the construction of quantum field theories that interact in four dimensions has been a goal of mathematical physics for decades.
We have great respect for the scientists OpenAI asked to lead this work. But as we explained in our recent statement, understanding must be developed openly and everyone must have equal access. OpenAI has said it delayed the launch until formalization of Lean was complete. In their own words, this is already complete, so there is no scientific reason to delay the release further.”
Making findings openly available could also lead to better science and encourage broader adoption of new tools like AI, Hassett said. “The fact that many groups around the world struggle with these results supports the rapid adoption of new tools by those who want to use them,” he said.
“The fundamental results are being resolved very quickly without human input… the documents are difficult to read and verify.”
Finding a way to communicate with researchers in the fields that OpenAI is increasingly invading and that satisfies everyone may be impossible. The company says it is experimenting with different approaches and recognizes that each has its advantages and disadvantages. “My feeling is that whatever the approach, there will be someone who will get angry,” said Stanford mathematician Jared Duker Lichtman. “So it’s just a matter of finding the least offensive option.”
Xiaoyu He, a mathematics professor at the Georgia Institute of Technology, believes the problem goes deeper than how OpenAI decides to announce its findings. “The results are very impressive,” he said, “but I think they are making a show of trying to be responsible and thoughtful about how they publish the results without being able to change the main problems: that the fundamental results are being resolved so quickly without human input, and that the papers are difficult to read and verify.”
