Agent Night panel recap: AGI arrived, or it's a skill issue

At Agent Night on August 12 at the Regency Ballroom in San Francisco, we put three people on stage who spend their days at different ends of the agent economy and asked what had actually changed.

The panel was hosted by Michael Grinich, founder of WorkOS, with Jaya Gupta, Partner at Foundation Capital and author of the context graph piece that went viral earlier this year; Flo Crivello, founder and CEO of Lindy, which shipped its AI teammate product that same week; and swyx, host of the Latent Space podcast and creator of the AI Engineer World's Fair.

We ran a version of this event about three months ago, which in agent time is a long while. The first question was the obvious one: what shifted?

Everyone's spend went up

swyx's answer was a number. He has at least 5x'd his spend on AI tooling and put the figure around $5,000 per person. The $200-a-month subscription that people used to mock is now something teams buy several of. His rule: if you can turn the spend into something useful for your business, throw all the money at it, because it is preferable to hiring a human.

Flo went further. "I think we'll look back on summer 26 as a season when AGI arrived. I think it's here," he said. For a long time there were AI agents that kind of worked and kind of didn't. His claim is that the drop-in replacement for a remote coworker now exists.

The other thing that happened in three months, in his telling, is open source. Lindy switched its main model to DeepSeek and announced it around the last event. He described the pattern he'd been running for years: an open weights release lands, the benchmarks and the press release claim parity at a fraction of the price, Lindy runs it on their own evals, and it fails. That stopped being the outcome.

Jaya confirmed it from the cap table side. Foundation Capital portfolio companies have actively moved off OpenAI and Anthropic onto open source models. Growth investors are asking margin questions they hadn't bothered with for three years, when the answer to everything was to go get more customers. Enterprise buyers blew through their Anthropic budgets fast and are now asking whether they need the smartest available model for the work they're actually doing.

swyx sees inference becoming its own field, with clouds that do no training at all. His non-consensus bet for the rest of the year is American open source models. Meta open sourcing two models after a year of talk about personal superintelligence is the case in point. "It is absurd that America doesn't have an open models champion the same way Europe has Mistral," he said.

On how any of those companies make money, swyx's answer was that it's the old open source playbook: make a splash with the weights, sell the hosted version, MongoDB and Docker style. Jaya's addition was the value-added layer: Palantir takes the models and becomes the control plane on top of them.

AGI is here, except when it isn't

Michael asked whether the models are still the constraint. Flo said no: intelligence is no longer the bottleneck. As a thought experiment, he suggested John von Neumann appearing next to your desk mid-meeting. He'd be useless, because you'd keep talking to your teammate, since the teammate has the context. We now have von Neumann in the room and no onboarding, no tool access, no governance or permissioning, and no context system.

swyx disagreed with all of it, on request. He has tasks he's thrown at frontier models for literal days that remain unsolved. He tried to get one to build GitHub, which is admittedly hard infrastructure. An encyclopedia, Grokipedia-style, fell short too — and even scaled down to auditing and creating skills, the tooling disappointed him. His real complaint was about judgment: a model handed irrelevant or insufficient context should push back, and instead it goes off and does literally what you said. "The fact that you have to care is not AGI".

Jaya took a third position. In plenty of deployments the model is too intelligent for the task. Customer support tickets don't need frontier reasoning, and she cited a public company CEO who didn't want his whole org on the top model because it overthinks most of what they do. The company she had in mind is a large retailer whose engineers don't need IMO-level intelligence to run a daily query.

Three answers on one stage. AGI arrived. AGI is close but un-onboarded. AGI is frequently more than the job requires.

swyx's suggested home experiment for anyone who thinks this is settled: have an image model design a website, then ask your coding model of choice to reproduce it pixel perfect. It won't. He framed that as a job description. If you can reliably articulate where models fall short, you are a more valuable contributor to the AI economy than someone who says models are great.

Harness engineering, and the ML research creep

Lindy has been building on this stuff since 2022 and GPT-3.5, before there was tooling, an eval product to buy, or a third-party harness to adopt. Flo dropped a detail he said he didn't think he'd ever shared publicly: in Q3 2022 they made what he now calls a laughably low acquisition offer to Harrison at LangChain. So Lindy built its own harness and its own eval tooling, and Flo's advice is not to do that today, because good ones exist.

His more interesting claim is that harness quality matters less than people think. The gap between a 25th percentile harness and a 75th percentile harness doesn't straddle the viability line. The bad one still finishes most of your tasks. It's slower, it retries, it gets stuck in loops, and it costs more, but it gets there.

swyx's version: the key tools have already emerged, probably a sandbox and some form of search, and you should adopt the interface your model was post-trained against rather than fight the grain. What's unsettled is memory and continual learning. The goal is that your company or your coding agent never repeats a mistake, and the only current mechanism for that is stored memory, which is still bad.

Flo's prototype answer: when you give Lindy feedback, she edits her memory and her prompt, then writes her own eval for that task and replays it to verify the change improved performance. He thinks agents creating their own evals becomes a standard pattern within six months.

swyx extended the idea. You should also generate your own labeling interface, then kick off multiple runs to compare. Which is to say engineering now looks like ML research: "I am babysitting runs. I am doing labeling. I'm doing evals. And I'm not updating weights, but I am updating the harness". He'll spend $1,000 optimizing a harness, because that work compounds indefinitely.

The context graph, eight months later

Jaya posted the context graph piece on December 22, and skills became a thing in early January, which forced an update.

The revision: a skill is a very small representation of context. Most of the large companies she talked to after the post weren't capturing context at all. Everyone in SF records their coffee chats now; that is not how the Fortune 500 operates. For a physical-world business the relevant context might be factory sensors or camera footage.

The harder problem is that context is trapped in people's heads, and for a lot of employees that context is their IP. Extracting it and turning it into a skill is a political act, and there's been real backlash: people deleting their skill files.

Flo's counter-model is incentive alignment. He was on the fence about a candidate, asked Lindy, and got back a screening breakdown drawn from a skill his recruiter had written without telling him, because it made her own job easier. The mechanism that matters is the default: a skill someone writes to do their own work better lands in the team library automatically. That's the flywheel.

One agent or many

Michael asked what wins: the omnipresent general-purpose agent living in your Slack, or a fleet of specialists.

swyx has made his team try everything, and sees two design philosophies. Claude is being built as its own entity: you set it up with its own Gmail, and a phone number is plausibly next. Others run multiple specialist agents. The pattern nobody has productized, in his view, is the personal chief of staff: everyone hand-rolls a twin. That works until someone owns your bot and can read everything it can read, and the permissioning story there is the wild west.

He also offered the strongest single data point of the night for the general-agent camp. Codex has been the chief marketing officer for Latent Space for about a month, running ads, YouTube, and A/B testing across everything, and it has doubled his subscribers. He can't afford a CMO. He can afford $200 a month.

Lindy has spent something on the order of a thousand cumulative person-hours arguing about this internally, and has swung hard back from many agents to one big agent, with small agents reintroduced behind it as an interface. Flo grants the legitimate reasons to split: privileges and permissions get encapsulated, agents get more shareable, the blast radius of a change stays small, and context boundaries stay clean.

The reasons he thinks are illegitimate are the ones driving most fragmentation. New technology first gets used in the shape of the old one — early TV was radio talk shows with a camera pointed at them. Human orgs specialize because humans have 24 hours a day and can only hold so much in their heads. Agents have neither constraint, so importing the org chart is a category error.

Jaya's answer was that it depends who you are. SF developers run 16 agents at once and switch tools weekly. Large companies use whatever IT provisioned, and switch when a vendor drops prices. On the chief of staff category specifically, she recently went through roughly 50 pitches for AI chief of staff and AI personal assistant products; about 47 of those companies no longer exist in that form. Capabilities are better now. Nobody has cracked the interface.

What changes by December

Michael's closer: we're back here in four months at the WorkOS holiday party. What's your bet?

• Flo: between 40 and 60% of all activities currently performed inside companies will be performable by an agent. Whether they actually get performed that way is a different question.
• Jaya: a shift to local models, as people realize how much doesn't need to go to the cloud.
• swyx: faster chips across the entire semiconductor stack. Cerebras runs 750 tokens per second where the norm is 70, and that's the visible edge. "I encourage you to think about a world in which you have 100,000 tokens per second because that is happening".

Housekeeping

Agent Night also included the Airlock launch from WorkOS and a run of lightning demos.

Our next room is Init, our conference in the first week of October at the SF Jazz Center during SF Tech Week, with Aaron Levie among the speakers. Sign up at workos.com/init.

Watch the full panel and the rest of the event recording here:

Thanks to Jaya, Flo, and swyx for joining us on stage.