We wrapped every tool call in retries with backoff and felt good about it. Latency went up, error rates looked fine, and we shipped. Two weeks later we found the agent was silently retrying a broken integration dozens of times per run, burning tokens and returning half-true answers. The retry layer had turned a hard failure into a soft one nobody saw.
Curious how others handle this. Do you log every attempt and surface retry counts as a first-class metric, or fail loudly after the first error? And where do you draw the line between retrying a flaky API and masking a real bug in your own tool schema?
Curious how others handle this. Do you log every attempt and surface retry counts as a first-class metric, or fail loudly after the first error? And where do you draw the line between retrying a flaky API and masking a real bug in your own tool schema?