Effective AI agent systems require multiple specialized sub-agents rather than a single monolithic agent to avoid hallucination and improve reliability.

Most Stripe Minions PRs originated from a single engineer who orchestrates 16 agents simultaneously from one screen.

“Most of the PRs came from one guy. He's got Alexander now. He's just got a screen. He's orchestrating 16 agents and he's going a whole lot faster.”

Decagon's token usage per conversation has increased over time because more model calls are added to improve quality, contrary to the assumption that optimization reduces token usage.

“Actually over time, the number of tokens we're using per conversation has gone up because we're actually doing more model calls to make the quality better.”

Even when frontier models dominate general tasks, practitioners route simple classification queries to small self-hosted models (e.g., 800M parameters) for latency and cost reasons.

“We have a little model that will classify the field of study of the query that does not need to be jammed into a giant prompt that does not need to be a 1 second latency API call to any frontier model that should be a self-hosted did, you know, 800 million parameter model that you give a few fine-tuning examples.”

LLM judges frequently disagree with expert judgments produced by AI agents, even when the judges agree with the high-level guidance given to them in context.

“What we found was actually more often than not they didn't agree with the actual judgments produced by the agents even though they agreed with the kind of the highle guidance that was provided to the to the judge in context.”

Databricks had five or six different agentic frameworks built independently by different internal teams before standardization.

“Yeah, I think we had like five or six different Agentic frameworks built by every different team.”

Self-improving agents that integrate their own execution, inspection, and modification are more powerful and expressive than outer-loop self-improvement systems.

“I think it's a more powerful way, a more fully expressive way of doing self-improvement than an outer system.”

Kavak bet on long-running agents with hard goals rather than workflow-based multi-agent systems.

“We realized to bet that longunning agents with hard goals not just workflows could maximize our customers satisfaction and obviously their lifetime value.”

Multi-model routing with consensus may not outperform a single model given equivalent test-time compute, the comparison must control for compute budget.

“It's important to ask like are you going to do better than having that model basically think for longer. like once you control for the amount of test time compute is it is it actually still doing better?”

Multi-agent systems still add minimal capability delta beyond single agents, despite generating excitement.

“The multi- aent thing still has like a bit of exciting sexiness to it which is maybe still beyond still like the actual delta it adds to the capabilities.”

The vast majority of deployed agentic systems use relatively small models handling circumscribed tasks with only 3-4 tool calls in a loop, not long-horizon autonomous agents.

“The vast majority of our customers are deployed with relatively small models. the range of tasks that they use them for are usually quite circumscribed. And so, we're looking at maybe you know, like three or four like calls or tool calls in a loop and then it comes back and you know, it takes gets feedback from a human or whatever or gives its answer back. Not you know, the sort of agents that are that are going to go off and do hundreds of calls and you know, write code and analysis and then you know, come back with sort of like a deep report or well-reasoned answer or something like that.”

Multi-agent negotiation between specialized AI systems (e.g., PCB design, thermal, mechanical) will not happen in hardware in the next couple of years.

“I just don't see that happening in hardware kind of in the next couple of years to be.”