My GREP on AI Agents and Workflows

For most of my adult life I've lived with a mild congestion I was so used to that I assumed it was normal. Nothing major, nothing that had me hacking through a conversation or while I was out walking the dog, just a persistent cloudiness I rarely paid any mind to. When I moved to Chicago I mentioned it to my doctor and he recommended a nose spray twice a day.

Day one, morning, check. Evening, check. Day two, morning, check. Day eight, and I'd forgotten the last five entirely. I tried again. Made it three days. Again and again I tried, and I never once made it the full two weeks it takes to know whether the thing actually helps. Four years later I had a drawer of half-used bottles and no answer.

Bored, and right around the time Antigravity launched in November of 2025, I decided to build something that would nag me relentlessly about it. I used AI to build it, and what came out the other side is almost entirely deterministic. A schedule. A text. A confirmation. An escalation if I don't answer. The only part that's genuinely AI is the phone call at the end, which has a bit of a personality and some room to improvise on what it says to me.

It texts me. It waits for me to confirm. If I ignore it long enough, it calls my phone and tells me to go to the bathroom and take the spray.

I've missed four doses in 207 days.

I have spent the better part of a year building tools with AI. Workflows, skills, local apps I run just for myself, agents that call other agents. What I've found is that it's easy to fall into a trap where you think you're optimizing, connecting everything together for greater value, aiming at a future state where it's all handled and you never have to think about the machine behind the OAuth. More often than not I regret letting AI tell me how the tool should work, how the data should flow, and where the edges are that need a person.

I want to be clear about where I'm standing. I'm not an AI skeptic, and at the same time I am. I use these tools every day and they have made me faster at real work. What I've lost faith in is the promise that a system will orchestrate that work reliably without me guiding it at every step. I chased that one hard enough to know that it comes at an annoying cost. It costs a lot of time and energy, and what you get back is fragile, and you end up spending more of your week figuring out what the hell you built than it spends doing work on your behalf. You try to straighten one thing out and wind up with a bigger knot in your lap.

A big 'ol plate of spaghetti

The pitch I'm seeing right now is AI everything. An AI workflow for every task. An agent to monitor and control each workflow. An agent to make sure those agents are doing their job. A big plate of spaghetti.

I've built some decently complex tools and I have wasted a lot of hours refining a fragile pipeline that grew tenfold, or sitting there having a conversation with the AI so it can explain to me what's working and what isn't. More than once I've asked it bluntly, what does this tool even do now?

You wind up with a dozen moving parts saving you a small amount of time, and then you have to ask for a debrief of what happened while you were away. The agents hit a nuanced scenario nobody programmed them for and go about generating the wrong output as fact, confidently, or they fail mid-flight and nothing tells you. Then they recommend some error handling. Then a patch. Then another thing. It never ends.

They are very good at finding work. Point one at a codebase and it will find plenty to fix. What it lacks is any sense of the critical path, so it sends you off on a rabbit chase after something you didn't need. Iterate on that for a few weeks and you have a Frankenstein monster that does a lot of things poorly and one, maybe two, things well.

I've noticed it usually wants to reach for a model to solve every problem, too. Almost always. So you burn tokens and fight context windows to do something a few hundred lines of deterministic code has done reliably since before I joined the workforce. It's easy to wall yourself in as you oblige one reasonable addition after another, until you're on the wrong side of something you built and have to break back through it.

Hitting the (fire)wall

At some point in every one of these builds I hit a wall where I'm not solving the problem I started with anymore. I'm herding digital cats into a swimming pool, chasing enhancements that were never going to give me value.

The research says the same thing, but more precisely.

OSWorld 2.0 benchmarks models against real computer tasks. Seven in ten take a person more than an hour. The best model finished about 21% of them. On the long ones, the tasks running close to three hours, every model tested scored zero.

METR measures how much serial human work an AI can replace. Not how long it can run on its own. As of this spring the frontier could handle roughly a twelve hour task at a 50% success rate. Ask for an 80% success rate and that twelve hours drops to ninety minutes. METR's own note on this says the bar is higher for work that has to be reliable and is hard to check. That work needs to clear 98% before automating it is worth the trouble.

Chain those systems together and the failures multiply. A Berkeley team ran a failure taxonomy over 1,642 traces of multi-agent systems falling over. Most of the labeling was done by another model, so take it as a rough map. About a third of the failures were agents talking past each other, withholding information, or doing something other than what they'd just said they would do. That's human relatable. A handoff between two people almost always carries a caveat at the bottom of the email, or a Slack ping an hour later that starts with one more thing on that report. Why would it be any different for agents, except that there's nobody around to send the second message.

Anthropic published numbers on a multi-agent research system it built in-house. It works, and it costs about fifteen times the tokens of a plain chat. That's what we trade. The more of these things we daisy-chain together, the more we spend to get a result we then have to go verify ourselves. Our resources are better spent elsewhere. That's the view from up high, anyway. Down where the work actually gets done is muddier.

What it feels like in the trenches

Let's chat real-world.

Atlassian surveyed twelve thousand knowledge workers and 173 executives this year. Eighty-nine percent of the executives said AI has increased the speed of the work. Six percent were sure they could point to clear returns across the organization. Both of those can be true at once, and I'd bet the space between them is where a lot of next year's budget goes to die.

Stack Overflow asked developers what frustrates them most about these tools. Two thirds gave the same answer: solutions that are almost right, but not quite. Almost right is the most expensive kind of wrong. It takes longer to find than the obviously-wrong, and you spend a lot of time and money to learn the answer.

One caveat worth thinking about even if conjecture. METR ran a study in 2025 that found experienced developers took about 19% longer on real tasks when they used AI, and the kicker was that those same developers came out of it convinced they had been faster. That matched what I was living at the time. Then METR tried to run it again, couldn't get a clean answer, and said in February that they think developers are probably more sped up than that first number implied. Their point estimates were still negative and they couldn't size the change with the design they had, so they're rebuilding it.

I don't take that as the first result being wrong. My read is that people got better at knowing where the edges are. The developers in that first study were probably doing what I was doing, handing over too much and paying for it on the back end, and a year of hard lessons has taught a lot of us to stop. That's just my hunch. But it lines up with everything I've seen. Pointed at a narrow job, it's fast. Hand it the wheel, it wraps you around the axle.

Asleep at the wheel

None of this is new. In 1983 a researcher named Lisanne Bainbridge wrote five pages about automated control rooms, the kind you find in power plants and chemical plants and cockpits, and those five pages explain my last year better than anything published this decade.

Her point was that automating a job doesn't get rid of the person. It leaves them the two worst parts of it. Sit and watch a machine that hardly ever breaks. Take over the second it does. The machine went in because it does the job better than they did, and now they're the one who has to catch it being wrong. Nobody stays sharp doing that. You lose the skill because you stopped using it, and whoever gets hired after you never picks it up in the first place.

Twenty-seven years later Parasuraman and Manzey went looking at the human side of that arrangement. Two things stuck with me. Twenty years on the job doesn't protect you. Pilots caught more failures than people who had never flown and they still went complacent. Piling on more hours with the system doesn't fix it either. What did help was training where the automation failed in front of them. The second one is worse. What makes people stop checking isn't how good the machine is. It's how steady it is. Operators caught 82% of the failures when the system's reliability wobbled around, and 33% when it hummed along the same way every day.

Picture what that looks like at a desk. Week one you read every word of what the model gives you. By month three it's been steady enough for long enough that you're skimming. By month six you're hitting approve while you eat lunch. Nobody decided to stop paying attention. It just stopped. And that's one person at one desk, which is the smaller half of it.

The other half of this

That is a bigger problem than my frustration with brittle pipelines. If a person is supposed to be the control layer, the honest question is whether that control is real or performative. Most systems that claim a human in the loop have a human near the loop, holding a pen, playing Wordle on their phone, signing whatever comes out the other end.

I've been working on a paper with Nil Redón Orriols, an AI thought leader and mathematician in Barcelona, on a way to actually score that. We call it the Human Agency Index, and it came out of reading through documented cases where AI caused real harm and a human was nominally responsible and functionally powerless. It's still in progress and I'll write about it properly when it's finished. I mention it because it's my problem at a different altitude. At the institutional level, oversight nobody measures is a governance failure. At my desk it's a Tuesday spent debugging a workflow that was supposed to save me an afternoon. Different stakes, same missing piece.

What doesn't get automated

Here's where I land. Orchestration doesn't get automated. Not soon, and I can't see the path to it.

Orchestration isn't picking the next step. Agents do that fine. Orchestration is sitting in a room with a client who asks for a dashboard, and realizing half an hour in that a dashboard won't solve anything. The real issue is that two employees are manually copying the exact same numbers between two disconnected systems every day, duplicating the effort by hand. On top of that, the main person handling all that manual data re-entry is retiring in March–a critical detail that nobody brought up until you started digging deeper with targeted questions. You only uncover those hidden realities by paying attention to what isn't being said, a skill earned from years of sitting in those rooms and being wrong a few times.

The thing getting automated is taking a clear spec and turning it into code. Everything else is still yours. Deciding what to build. Deciding whether it's worth building at all. Knowing what will break once it's live. And the faster the code gets written, the more of that work there is, because now you've shipped five things instead of one and every one of them needs a person who understands it. Trying to replace human review to clear a bottleneck is a losing game. So I quit playing that one.

What I do instead

Small tools that do one thing, built for a specific person with a specific problem, fired off by a human who decided right then that this was the moment.

Go back to the nose spray. It isn't clever, and the one clever part is guardrailed inside a phone call. It has one job and no say in what that job is, and it has never once surprised me. That's the whole feature. It doesn't know anything about my health, it isn't optimizing my morning, and it has never proposed a second agent to supervise it. It just refuses to let me off the hook. Four missed doses in 207 days.

Everything I've built that delivers real value came out the same way. Somebody told me about a problem, usually in a convoluted way, in the middle of talking about something else. I sat with it longer than felt productive. Then I built one narrow thing and fixed it until it quit breaking. The decision about which tool to use, when to use it, and what to do when something is wrong, all of that stayed with me. That isn't a gap a better model closes in a few releases. Making those decisions is the actual work, and that stays with us.

There's one more piece of that nose spray story. The thing that actually changed the outcome wasn't the build. It was a doctor listening to me describe something I had stopped noticing years ago and telling me what the solution should be. Nobody automated that part. All I built was the thing that keeps me honest about it, and I built it dumb on purpose.

The tools are good. They're getting better. I use them constantly and I'd be slower without them.

I just don't think they're going to run the show. Somebody still has to notice the thing you stopped noticing.