The Morning Download: The Agent Boom Needs New Chips

Good morning. Google plans to unveil its first AI inference chip this week, underscoring the industry shift from training AI models to querying them, the WSJ's Robbie Whelan and Katherine Blunt report.

The announcement of the chip, TPU 8i, represents the eighth generation of Google’s Tensor Processing Units, or TPUs, and comes just weeks after Nvidia unveiled its own inference solution, paired with chips from startup Groq. Chip startup Cerebras recently struck an inference deal with AWS.

Google is also introducing a chip for training models, the TPU 8t.

What can one infer from all this? The AI industry is betting that usage in the enterprise and beyond is expected to explode, and it will increasingly happen through agents, capable of executing increasingly complicated strings of tasks, including writing software.

Case in point, the company this week also introduced Gemini Enterprise Agent Platform for building and deploying agents. Google said Wednesday that three-quarters of the new code created inside the company itself is generated by AI. In the fall, it was 50%, Business Insider reported. 

This shift explains why chip makers are racing to optimize for inference. It's no longer necessarily a question of how much compute to throw at something, but how fast one can get answers.

Different demands, different chips. Inference requires less raw computing power than the GPUs used for training AI models, but more memory. Mark Lohmeyer, vice president of AI and computing infrastructure for Google Cloud, tells the WSJ that a single request to an AI agent generates between 20 and 50 times the “inference transactions” as a query to a chatbot.

“What the customers really care about is how can you drive down the latency,” Lohmeyer said. Read the story.

IBM says it's riding the AI tailwind. Revenue rose to $15.92 billion from $14.54 billion a year prior, amounting to what CFO Jim Kavanaugh said was IBM’s highest first-quarter revenue growth in years. “AI continues to be a tailwind for our business,” he told the Journal.

Revenue in the company’s software segment, which includes its hybrid cloud business, rose to $7.05 billion, up 11%. Consulting revenue rose 4% to $5.27 billion, and infrastructure revenue was up 15% to $3.33 billion.

IBM stock has dropped 15% this year, one of the many victims of a broader software selloff, according to Barron's. Shares were dropping in premarket trading Thursday

ServiceNow posts revenue growth, but cyber deal weighs on margins. The cloud-based software company posted $3.77 billion in revenue for the quarter, which was above analyst expectations and represented 22% growth. 

Despite the beat, shares were down 13% in premarket trading on Thursday. Wall Street likely wanted more than just in-line earnings for the quarter, Barron's said. 

ServiceNow also said Wednesday that the acquisition of cybersecurity startup Armis will result in a roughly 75 basis point headwind to its operating margin for the full year, as well as a roughly 125 basis point headwind to its operating margin in the second quarter.

For the current second quarter, ServiceNow is expecting subscription revenue to grow to roughly $3.82 billion, above analysts' expectations.

• ServiceNow CEO Builds New Business Model Around AI

Nokia reports rising sales from AI and data-center customers. The Finnish telecom-equipment provider said Thursday that first-quarter sales of network infrastructure rose 12% on year, driven by demand from AI and cloud customers in the Americas.

STMicroelectronics shares jump after strong quarter. The European chip maker, which counts Apple and Tesla among its major clients, said revenue climbed 23% on year to $3.10 billion, above analysts’ forecast of $3.06 billion. The company agreed to a deal with Nvidia in the quarter to integrate its sensors, microcontrollers, and motor-control technology with Nvidia’s robotics ecosystem.

Workers sue $10 billion AI startup. Mercor, whose clients have included OpenAI, Anthropic and Meta, has been hit with at least seven class-action lawsuits in recent weeks following a third-party data breach that exposed contractor information, the Journal reports. According to plaintiffs, the company’s practices include monitoring its contractors’ computers and sharing that data with clients, using recorded candidate interviews to train AI models, and training client models on materials potentially owned by other companies.

Starbucks cuts tech jobs. The coffee giant cut some employees Tuesday, part of a restructuring of its tech organization, according to an internal message seen by the Seattle Times. Starbucks did not share the number of layoffs or where the employees were based.  The Seattle-based company is in the midst of a turnaround of its U.S. business, with Chief Executive Brian Niccol aiming to improve service as a means of increasing sales.

An ETF to remember. A new exchange-traded fund with the ticker DRAM has had one of the fastest starts by investing in memory, the Journal reports. The Roundhill Memory ETF has gathered more than $1 billion in assets by tracking the computer-memory and storage industry, an indication of the niche’s huge demand related to the AI infrastructure build-out. More than three-quarters of the fund is exposed to just three stocks: Samsung Electronics, Micron Technology and SK Hynix, which posted record first quarter results.

AI infrastructure build-out reaches the second stage. In the latest sign that the playing field is no longer reserved just for hyperscalers and frontier model companies, Dell signed a $1.44 billion deal with an enterprise AI cloud provider called Boost Run, Barron’s reports. As part of the deal Dell will provide Boost Run with hardware and software “needed to fulfill long-term client commitments and scale capacity,” according to a press release.

What's the hourly rate for AI hallucinations? Attorneys at Sullivan & Cromwell, one of Wall Street’s oldest law firms, have apologized to a U.S. federal judge over a filing that contained “multiple” AI hallucinations, including a misquoting of U.S. bankruptcy code. The firm said the attorneys involved did not follow its AI policies when preparing the document, FT reports. 

The conflict with Iran has entered a damaging new phase—a crippling limbo between war and peace that leaves the Strait of Hormuz closed and the prospect of escalation looming. (WSJ)

Wall Street’s watchdogs are ramping up their inquiries into how much risk has built up in the $3 trillion private-credit industry, just as investor angst has sparked some backers to head to the exits. (WSJ)

Republicans are increasingly worried that a battle President Trump started last summer to redraw congressional district lines has backfired and may hand more seats to Democrats. (WSJ)

The WSJ Tech Council brings together CIOs, CTOs and CISOs advancing innovation and shaping the future. Join this trusted community where tech executives connect with peers to explore emerging trends and gain the perspective they need to stay ahead of disruption.

Request Information

The WSJ CIO Journal Team is Steven Rosenbush, Isabelle Bousquette and Belle Lin.

The editor, Tom Loftus, can be reached at thomas.loftus@wsj.com.