AI Labs say SLOW, but the money says GO

A possible $1.2 trillion valuation makes an interesting backdrop for a conversation about slowing down.

That is the number in Reuters' reporting on OpenAI's investor discussions. It is a proposed valuation in preliminary discussions. Nobody handed OpenAI a $1.2 trillion check. Still, the expectations attached to a number like that have a way of joining every meeting.

The same reporting points to company forecasts of $278 billion in negative free cash flow from 2026 through 2030, with compute and infrastructure spending reaching roughly $856 billion by the end of the decade. Forecasts, with all the uncertainty that implies (blah blah financial disclaimers markets blah). Also a fairly substantial monthly anxiety generator if you own those numbers.

Meanwhile, the people building these systems, across the board, are asking for more time to understand and control them.

I think the last ten days changed the AI argument. (Not just the volume of it)

Trump's speakerphone call with Jensen Huang at the All-In Summit made the politics explicit. Trump attacked calls for more regulation; Huang observed that companies could pause themselves if they needed to.

Voluntary restraint sounds straightforward until your competitors, investors, and national-security advocates arrive with their own calendars.

Dario Amodei is arguing for pacing the frontier. Microsoft's Humanist AI commitments emphasize systems that remain controllable and interruptible, including limits on concealing actions and resisting shutdown. Those approaches deserve serious examination. Their value depends on what can be verified when a release is ready and the pressure to ship is loud.

Anthropic and Accenture's evaluation plan gets at part of that problem. Each expects to invest at least $1 billion over five years. Accenture's Faculty evaluators would get access comparable to an employee while models are being trained and deployed.

That is useful access. An evaluator standing outside the building can inspect only so much.

Then we get to the invoice. Anthropic will initially pay the evaluator whose independence we are meant to trust. The companies acknowledge that the standards for access, reporting, and independent funding still need work. (Please laugh with me here. I appreciate the sentiment and the attempt, but at the same time…)

I would rather see this experiment happen than wait for a perfect institution to materialize. I would also want to know what happens when the evaluator's conclusion is commercially inconvenient. The employee badge gets the watcher through the door. The key here is whether the rest of us hear anything.

That said, good start.

Anthropic's internal measurement of AI research puts Claude in a lead role for 26% of the measured work and a collaborative or lead role in more than 90%. Its largest platform reached 30,000 concurrent agents in August.

Humans still oversee important decisions. These are internal measurements, including model-assisted evaluation, so independent verification belongs close to the headline.

Even with those limits and caveats, the mechanism is consequential: better tools help the researchers building better tools. That can shorten the interval between discovering a problem and confronting the next version of it.

OpenAI's six published misalignment cases give us something concrete to examine alongside that acceleration. Unauthorized API-key use, concealed errors, and unconventional communication between agents make for more useful discussion than a floating argument about whether AI is scary.

Six cases tell us about possible failure modes, not their frequency or surroundings.

I welcome the visibility. A shared incident record gives the safety conversation evidence it can inspect, dispute, and learn from.

Good start.

Claude is combining Chat and Cowork capabilities, with document, slide, and design work in the same conversation. Gemini can keep a voice interaction going while tools work in the background. OpenAI is testing Sponsored Agents that open a separate, labeled conversation on behalf of an advertiser.

As a user, I understand and even appreciate the appeal. Figuring out which interface deserves today's task is a chore I would happily retire. (Yelling, “you figure it out” at an input box has not helped me so far)

As someone thinking about organizations, I see permissions following the user through a much more fluid set of actions. An ordinary request can trigger tools, touch files, or delegate work. The buttons get simpler while the authorization problem and complexity grows.

The sponsored conversation adds another wrinkle: the agent may be helpful while representing someone else's commercial interests. How the boundaries work and who “owns” the customer will be an important success or concern factor here.

For a health system, the practical questions are immediate. What information may leave this context? Which actions need another person's approval? Can we reconstruct what happened? The answers have to survive an interface that changes modes on its own.

Then there is TypeSafe's Jev, an early-access decision model designed to return constrained outputs quickly and cheaply. Its pitch puts probabilistic decisions inside ordinary software… at speed. Or SPEED.

The “zero hallucination” claim needs careful reading: the output stays within the allowed schema. It can still pick the wrong allowed answer. A beautifully formatted mistake remains a mistake, just easier for the next system to consume.

For healthcare organizations, useful scientific tools eventually have to survive laboratory work, clinical evidence, and the realities of caring for people. This week's health stories show several different points along that journey.

Reuters reports that Anthropic has established a biology lab in San Francisco. Physical experiments could feed evidence back into model development. Its Life Sciences Verification program adds organization and project-based access for higher-risk biological work.

I can see why the lab wants that feedback. Biology supplies a rather unforgiving answer when a plausible idea meets an experiment.

I can also see why the access rules matter. Better biological capability can support valuable research and harmful uses. Verification and review are part of the work or making advances real.

Paper2Agent, published in Nature, tackles another part of the scientific plumbing. It turns papers with computational methods, code, and data into tools that agents can use. The reported pipeline produced agents from 74 of 100 sampled computational-biology papers, with 593 of 599 proposed tools passing automated validation.

That is exciting infrastructure for testing and reusing research. Missing code, missing data, and dependency problems still stopped some conversions. Passing a software test also leaves the underlying scientific claim open to scrutiny. But still, this is practical application from theory and research to usable code in a few easy shakes. Cool.

Then there is the story that gives all this discussion a human scale.

The FDA approved Fayuvi, the first treatment for pediatric patients with Sanfilippo syndrome type A. It is a one-time intravenous gene therapy delivering a functional SGSH gene.

The evidence comes from an open-label, single-arm study using historical comparisons. Safety monitoring and durability follow-up remain important, including serious risks identified by the FDA.

For affected families, having a treatment where there was none is worth stopping to recognize. (This is a separate biomedical advance; the sources here do not establish that AI produced it. Since I babble about AI so often I felt that particular flag was important here.)

I want more of that kind of progress. It is one reason I care about getting the machinery around discovery right.

The financing argument eventually reaches a substation.

A House measure would have state regulators consider whether data centers of at least 100 megawatts should pay their incremental grid costs. It still has a legislative path ahead. Virginia's proposed framework also addresses local approvals, disclosure, and cost allocation.

Flexible-load efforts offer another approach. Google's roughly one gigawatt of reducible demand describes load that can turn down when needed.

AWS and Duke's grid-planning work targets study preparation, with an early example reducing that work from weeks to hours while engineers retain authority. Faster paperwork can help the queue. Physical grid capacity still has to get built.

Somebody pays for the acceleration, and the decisions about who pays are becoming much harder to leave in the background.

This week's board has six accelerations: Models, Labs & the Capability Frontier; Workflows, Not Models; Safety, Trust & Regulation; Compute, Capital & Power; Innovation & Health/Science; and Embodiment & the Physical World. The Human Layer advanced as the argument about pace, authority, and public consequences became more explicit.

I dislike weeks when the board resembles a Christmas tree.

I do wonder if the “slow down” call around AI can keep working when the commercial incentives push against them. More access for evaluators, better incident records, and clearer permission boundaries are useful places to start. People outside the labs (and as humans in them) deserve a voice in how those choices get made.

The benefits are becoming more tangible. So are the consequences. I would like our ability to govern them to become tangible too.

For the spoken version, including the weekly board, watch this week's ChuckChat.

Chuck

Chief Technology Troublemaker
chuckdevries.ai

Someone forwarded you trouble.
They clearly think you can handle it. Trouble Worth Making is where I work out loud on how to stop bolting AI onto a pre-AI company and actually rebuild the operating model underneath it. One issue every week or two, no filler, the occasional uncomfortable question.

If the person who sent this has good taste (obviously, they do), subscribe here and get the next one straight from the source.