← All posts

AI agents · Sep 26, 2026 · 6 min read

The prompt was fine. It never ran.

A lead screener ran out of API credit for eight days while every execution showed green. What I changed so a failure looks like a failure.

AAAde AmosForward deployed engineer
$ npm run evals ✓ 28 / 30 passed ✗ refund_request → needs human $

An agency client had a lead screener in their funnel. New leads came in, a model classified them, and good ones were assigned and alerted in Slack. The workflow ran on schedule. Every execution showed success.

For eight days the API account behind the screener had no credit. The step that called the model returned an error, the workflow caught it, and the lead moved on as if it had passed. Seven junk leads reached the sales team before anyone noticed. Nobody noticed because nothing looked broken.

Green is not the same as done

This is the failure I see most often in automation that was built fast. The status reflects whether the workflow finished, not whether the business result happened. A green run, a saved editor state, a “ready” flag, a fresh timestamp. None of those are the thing you actually wanted.

$ npm run evals ✓ 28 / 30 passed ✗ refund_request → needs human $
Eight days of successful executions. The screening step inside each one had failed.
“The prompt was fine. It never ran.”

What I changed

  • 1A screening failure is now a label on the lead, visible in the CRM and in the Slack alert. The lead still moves, but nobody can mistake “unscreened” for “qualified”.
  • 2The workflow alerts on the failure itself, not on the lead. An API error is a message to a human, every time.
  • 3The check I run before calling anything done: open the downstream record. Not the log. The record.

I did not fail the lead closed. In a speed-to-lead funnel, silence costs more than a junk lead. The point was to make the state honest, not to block it.

Same rule applies to content pipelines. On another build, published articles were still marked “generating” because the status was set by the step that started the job, not by the one that confirmed the output. Verify the artifact.

Have a problem you want solved? A workflow that needs to get done? Contact me.

Contact me →