Hermes AI Agent Reliability: The Failures I Actually Hit

You make a Hermes agent more reliable by treating most failures as runtime problems, not model problems. In my fleet the recorded Hermes breaks were expired logins, dead provider lanes, a config reading the wrong home directory, and gateway processes that crashed or hung on. Fix those four, verify every repair, and uptime follows.

That's not the answer you usually hear. The usual answer is "write better prompts." For this piece I had my own recorded incident logs pulled, and not one of the Hermes failures in them was the model saying something dumb. Every one was the plumbing.

How many AI agents fail?

Nobody has a trustworthy universal number, and I'm not going to quote one. The failure rate depends completely on what the agent does, what it runs on, and whether anyone is watching it.

What I can give you is the shape of failure in a real fleet. I run Hermes as a live runtime next to OpenClaw, Codex and Claude Code, with a primary assistant profile plus 37 agent profiles on my main box alone. At that size, failure isn't a question of if. Something is always a little broken somewhere. The question is how fast you find it and whether the fix sticks.

At one point my own message to the team was basically that my Mac Hermes and one of my server Hermes installs were not working right at all, and that I thought we had messed up how it works. That's an honest picture of agent reliability at scale. Not a percentage. Just a normal day.

Is AI agent overhyped?

Some of it is. The demos are overhyped. The idea that you set up an agent once and it just runs forever is overhyped. What's not overhyped is what a well-watched agent does once the runtime is stable.

Here's my take. The hype skips the boring layer: auth, config, processes, restarts. The results live in exactly that layer. An agent with a great model and a dead login does nothing. An agent with a decent model and a runtime somebody actually maintains does real work every day.

So is it overhyped? The product demos, yes. The operations work, nobody talks about it enough.

The four failure modes that account for most of it

These four come straight out of my recorded Hermes incidents. Not a theory. The logs.

1. Auth that expired, or never finished

This one shows up over and over. Real messages from my logs:

• "No Codex credentials stored. Run 'hermes auth' to authenticate."
• A gateway reporting that the model login had expired.
• An OAuth device code getting rejected because the flow had timed out, so a fresh code had to be minted.

The sneaky version: on an older Mac build, Hermes was expecting the Codex CLI login, not its own login command. You can run the "right" command all day and it won't fix a runtime that's looking somewhere else.

2. A provider lane that went bad

Hermes talks to model providers through profiles, and a lane can die quietly. From my records:

• The default shell profile was still on one provider and exhausted, while the prod profile was pointed at another provider with an invalid API key. Both lanes bad at once. The fix was switching prod to a lane that was live-tested and working.
• On another box, the failure was a config pointing at a model route that no longer worked. Switching the default provider and model, then running a smoke test, brought it back. The smoke test reply was literally "codex-ok".
• A credential pool where one attempt marked the whole provider pool exhausted, so Hermes fell straight through to its fallback.

That last point is worth knowing. Hermes retries a failed primary, then activates the configured fallback. In one record it retried 3 times before switching. When you see the fallback message, it doesn't mean the fallback broke. It means the primary failed first: a 429 rate limit, a 500-series error, a 401 or 403 auth failure, or empty responses.

3. The right config in the wrong place

This is my favorite failure because it looks impossible. Credentials got imported into the global auth file. But the prod Hermes process was reading the auth file inside its own profile folder, because HERMES_HOME was set to the profile. The fix was correct. It was in the wrong file.

Same family: a manual check once missed exporting HERMES_HOME, so it read the default home instead of prod and reported the wrong thing. And on my Windows box, Hermes moved to a new home folder at one point, and the old folder from before the move is still sitting there. Nothing should point at it. If something does, you're debugging a ghost.

4. The gateway and the processes around it

The gateway is where Hermes meets the outside world, and it has its own ways of dying:

• A gateway crashing on startup because a fresh virtualenv was missing the Telegram adapter, so it had no platform to bind to and exited.
• A stray old Hermes process still holding the Telegram connection, so the new one couldn't take it.
• A dashboard rejecting the HTTPS proxy Host header until the backend launched in a proxy-friendly mode.
• hermes update flagged as needing approval, because it restarts the gateway and kills running agents.

That update one is a reliability rule in itself. Updates are an outage you schedule. Treat them that way.

The symptom that lies

One more from the logs, because it taught me something. A box was throwing socket errors that looked like an auth failure. After the retest, all three accounts came back 3 for 3, 9 for 9 total. The errors were transient network flakiness, not auth. If you "fix" auth on that box, you just broke something that was working.

What are the 7 types of AI agents?

The list you'll usually see is: simple reflex agents, model-based reflex agents, goal-based agents, utility-based agents, learning agents, hierarchical agents, and multi-agent systems. The classic textbook version has the first five. Hierarchical agents and multi-agent systems are the two that popular lists add. It's a useful way to talk about how an agent decides what to do.

Here's the thing though. None of those types predicts reliability. Every failure in the section above would hit a goal-based agent and a learning agent exactly the same, because none of them live in the decision logic. They live in the runtime underneath it. Type tells you how an agent thinks. It tells you nothing about whether its login expired last night.

Who are the big 4 AI agents?

There's no official big four. When people ask, they usually mean the agent products from the largest AI labs, and that list shifts every few months.

The more useful question is which runtimes you actually operate. In my stack that's Claude Code as the default in every terminal, Codex as a second lane, Gemini CLI for search-grounded and huge-context work, and Hermes and OpenClaw as live runtimes. Each one has its own login, its own config and its own way of breaking. Reliability is per runtime, not per brand.

Runtime drift, and the repair loop that catches it

Runtime drift is when an install that worked yesterday quietly stops matching what it should be: a login lapses, an update changes a default, a profile points at a dead lane, a process from last week is still running. Nothing "broke" in a way you'd notice. It drifted.

My answer is a named specialist whose whole job is this. Matteo Jr watches every Hermes, OpenClaw and Codex install, verifies updates, repairs runtime drift, and keeps recovery handoffs from a dedicated remote box. Giving that job to one named role, instead of hoping whoever notices will fix it, is the reliability move I'd copy first.

The loop looks like this:

1. Detect. Status checks and smoke tests, not vibes. A recorded restart check in my logs is literally a gateway restart, a six-second wait, then a gateway status read.
2. Retry inside a limit. The escalation thresholds in my shared memory are explicit. For example, Hermes retries a known workflow 3 times within 30 seconds before escalating to OpenClaw.
3. Repair the cause. Auth, lane, config path or process. Usually one of the four above.
4. Verify the repair. More on that next.
5. Write it down. So the next drift of the same kind takes minutes, not an afternoon.

There's also a pattern written into my shared memory that I think every fleet should steal: a forced failure drill, one per week, where you break things on purpose. Kill Hermes mid-job. Break model provider auth. Inject bad memory entries. Then see whether the system notices. I'm not going to tell you we have run it every single week. I will tell you it's the right design, because drift you rehearse is drift you catch.

If you run Claude Code alongside Hermes, you'll notice the overlap with limits. A provider hitting a 429 in Hermes and a session hitting a wall in Claude Code are the same class of problem. Head back to the rate-limit side of this for that angle.

Proving an agent did the work before you trust the output

A process staying alive isn't proof the agent works. That line is straight from one of my repair sessions: the last check wasn't "is the gateway up," it was confirming Telegram actually bound in the new environment, instead of just the process staying alive.

That's the standard I hold everywhere. Before I trust output, I want evidence:

• A smoke test with a known answer, like that "codex-ok" reply.
• A status read after the restart, not before.
• The actual platform bind, the actual message delivered, the actual file written.

At the fleet level, this is why my chain has a check lane that can say no. Work gets planned, work gets executed, and an independent QA lane can veto the delivery. That's the routing chain that catches failures before they reach me. And every handoff between agents carries its own proof, which is how every handoff gets verified instead of assumed.

Model hallucination gets all the headlines. In my logs, the thing that actually costs uptime is an agent, or a person, reporting "fixed" on a repair nobody verified. Make verification the last step of every repair and a huge chunk of your reliability problem goes away. Not all of it. A lot of it.