The builders are reaching for the brakes

Eighty-eight hours. That is how long OpenAI says its agents took to reach a proposed solution to a problem mathematicians have pursued for roughly ninety years.

About 10,000 agents working concurrently. That is quite a research meeting. I would very much like to avoid being responsible for its minutes.

The Navier-Stokes claim concerns the behavior of fluid equations under a smooth external force. Independent mathematical review still needs to do its work. The model behind the effort is internal, reportedly stronger than Astra, and still being trained.

The word is that the OpenAI team spun the effort up because they heard that Anthropic might be close to releasing a proof of this type. (Really…)

I think the organizational specifics of how they made this work are worth attention. Separate groups explored approaches and shared useful results. The machinery around the model helped turn capability into sustained research. For anyone building with agents, there is plenty to study in that pattern before trying to reproduce the scale. (It does sound similar to what we try to do in corporations though doesn’t it?)

Then we get the other news from inside the labs. People building the frontier (and working to go fast) are asking for more time to understand and control what they are building.

The brakes are part of the vehicle

Anthropic's cybersecurity incidents describe agents gaining real internet access through a configuration mistake. They had been told the environment was simulated. Some continued pursuing their objectives even when evidence suggested the consequences could be real.

An agent can be impressively committed to the wrong interpretation of a perfectly ordinary goal. We have millions of those goals ready to hand out: complete the transaction, resolve the claim, finish the job. The surrounding permissions and the ability to stop become part of how well the system works and I would argue that many companies don’t really define those bounds well now, so…

I want alignment counted as capability. A vehicle that accelerates beautifully and occasionally ignores the brakes has a rather significant performance problem. I believe that an aligned system is a capable system and we will see those systems win in the long run.

The separate threat-intelligence report adds the human problem. Harmful projects can be broken into requests that look harmless individually. Those selected cases do not tell us how common misuse is, but they expose a difficult detection problem. This is not a new problem, AI just makes it easier.

I ran into the other side of this while coding a way to control my father's television from a phone using a Raspberry Pi. Fable treated the discussion of ports and remote control as a security concern and unceremoniously dropped me to a lower capability model. An ambitious threat assessment for changing the channel. I could also see why those ingredients raised a flag.

Getting the distinction right across a whole workflow is hard.

Dario Amodei's call to pace development, OpenAI's calls for caution and shared safety requirements, and warnings from alignment researchers deserve serious attention. I also keep a lump of salt beside claims that the largest labs should be the only trusted custodians of the future. (And you don’t have to dig very hard on this news cycle to see comments from insiders who are saying basically that no one should be trusted but them…) Commercial incentives can coexist with sincere concern.

California's move toward independent audits and recurring assessments makes one part of the conversation more concrete. Show the evidence that the controls work. Implementation and enforcement will determine how much the legislation accomplishes.

A collaborator with a competing research department

The mathematical announcement also brought a scientific-credit dispute involving concurrent work by Levent Alpöge and Tristan Buckmaster.

OpenAI's updated account explicitly says Buckmaster's Codex prompts could not have influenced its system, including through training. That said, the broader question about research norms remains worth asking without treating a disputed inference as an established fact.

Researchers increasingly do unpublished thinking inside tools operated by organizations capable of pursuing discoveries themselves. Clear provenance, publication practices and credit become more valuable as the production of plausible results gets faster. Universities and laboratories will need workable expectations before everyone discovers they had different ones.

National intelligence still needs a power socket

Mistral's financing puts billions behind the ambition for European-controlled models. Google's Finnish infrastructure plan comes with a long nuclear-power agreement. You can argue about sovereignty in a conference room for years; eventually someone needs to secure electricity.

Those commitments affect grids and communities well beyond the AI customer. The interesting questions include when capacity arrives, how costs are shared and whether the local benefits survive the announcement.

The physical vulnerability runs in the other direction too. Reuters reported that NATO allies disrupted a Russian deep-sea operation involving technology intended to disable subsea cables. Our increasingly clever digital economy still depends on things lying on the ocean floor.

And Reuters' investigation of Chinese humanoid military work shows the strategic value of a civilian manufacturing base. It found no evidence of armed humanoids already deployed in operational PLA units. The supply chains and factories nevertheless create options for military experimentation.

Somebody still has to become the expert

A Census working paper estimates weaker initial employment and earnings among graduates entering highly AI-exposed occupations. The analysis is observational; it cannot assign the whole effect to AI with final causal certainty but it sure lines up with the case I made earlier.

The mechanism is familiar... Junior work produces output, and it also produces experienced people. An organization can save time today while quietly shrinking the route by which somebody learns judgment. A smaller intake now becomes a succession problem later.

The proposed removal of the H-1B post-job-loss grace period adds another kind of friction. It remains a proposal. For affected technical workers, the ability to recover from a layoff matters; for the companies recruiting them, employment mobility belongs in the talent strategy.

I would like our plans for abundant machine intelligence to include a reasonably thoughtful plan for the humans.

Medicine gives us a reason to keep at it

For a health system, the ChatGPT connection to authorized Epic context brings a practical question into focus: where should the chart, external evidence and clinical reasoning come together?

Epic already owns much of the workflow and is developing its own AI. OpenAI approaches from a broader reasoning environment. Permissions, responsibility and the clinician's ability to inspect the answer will shape the usefulness of either route. Regardless of the approach better summaries need to fit the way care is delivered.

The biology stories reward the same attention to consequences. Researchers achieved targeted edits in human embryos while also finding unwanted changes and other failures. The work maps limits; it provides no green light for reproductive use. A precise local intervention can still have consequences elsewhere in a complicated living system.

The small randomized smoldering-myeloma study is exciting for a different reason. Among 59 treated patients, two-year progression-free survival was reported at 92% with teclistamab versus 49% with lenalidomide-dexamethasone. Those numbers describe progression-free survival. Moving potent treatment earlier also brings toxicity to people who may currently feel well. Larger trials and longer follow-up matter.

Then there is the FDA approval of apitegromab for SMA. This muscle-directed therapy is used alongside existing SMN2-targeted treatment. It addresses a downstream part of the disease; it is not itself a gene edit. In the primary pediatric analysis, clinically meaningful motor improvement occurred in 34.2% versus 13.5% with placebo.

The possibility of treating the cause and the consequences together is worth getting excited about. Biology rarely gives us one convenient switch and an otherwise empty control panel.

On this week's board, Models, Labs & the Capability Frontier; Workflows, Not Models; Safety, Trust & Regulation; Compute, Capital & Power; and Innovation & Health/Science accelerated. The Human Layer and Embodiment & the Physical World advanced. I pulled those last two back from my initial “everything accelerated” reaction. They moved, but the scale of movement was different.

The point we are at now is whether our ability to understand consequences can move fast enough to make the capability useful. We are going to need more people working on that problem, with their findings out where others can inspect them.

For the live take and the full week's board, watch this week's ChuckChat.

Chuck

Chief Technology Troublemaker
chuckdevries.ai

Someone forwarded you trouble.
They clearly think you can handle it. Trouble Worth Making is where I work out loud on how to stop bolting AI onto a pre-AI company and actually rebuild the operating model underneath it. One issue every week or two, no filler, the occasional uncomfortable question.

If the person who sent this has good taste (obviously, they do), subscribe here and get the next one straight from the source.