We shipped a MuleSoft API that retried every failure — 400, 401, 403, all 3 times each. Same bad request, same error, 3x log noise, and the downstream system throttled us.
It took us a while to notice, because nothing looked broken at first. The API was responding. The flows were completing. But our logs were full of the same failed request, repeated three times, and the downstream team started asking why we were hammering their system with requests that could never succeed.
A 400 is not a flaky network. It is a bad request. Sending it again does not fix the payload, the missing field, or the invalid value — it just creates more noise, more alerts, and more load on a system that already told us no.
Wrong vs Right
Wrong: retry everything, three times, no questions asked.
Right: retry only transient errors, with backoff — and fail fast on client errors.
<!-- Retry ONLY transient errors -->
<!-- 408, 500, 502, 503 -- with backoff: 2s, 4s, 8s -->
<!-- Never retry: 400, 401, 403 -->
<until-successful maxRetries="3"
millisBetweenRetries="2000"
doc:name="Retry transient errors only">
<http:request method="GET" path="/orders" />
</until-successful>
<!-- In your error handler, decide before you retry: -->
<!-- shouldRetry = [408, 500, 502, 503] contains httpStatus -->
<!-- 400 -> bad request, fix the request -->
<!-- 401 -> unauthorized, fix the credentials -->
<!-- 403 -> forbidden, fix the permissions -->
<!-- 408, 500, 502, 503 -> wait 2s, then 4s, then 8s, then give up -->
In practice, that meant moving the retry decision out of a blanket policy and into a simple status check: if the status is transient, retry with increasing backoff. If it is a client error, stop immediately, log it once, and surface it clearly.
The fix took about ten minutes. The production noise dropped immediately.
3 Rules
- Retry only transient errors. 408, 500, 502, and 503 can be temporary. A retry might actually help.
- Never retry a 400 — the request is wrong. The same request will fail the same way every time. Retrying it is just repeating the mistake.
- Backoff, don't hammer. Wait 2s, then 4s, then 8s. Give the downstream system time to recover instead of adding to the overload.
Closing
Most MuleSoft APIs don't fail in DataWeave, they fail in production on the retry policy nobody reviewed.
What retry mistake have you seen in prod?
Top comments (1)
This is the retry-policy version of a lesson EDI teams learn from 997 rejections: a content error retried three times is not resilience, it is three copies of the same defect plus a throttling problem. Classifying by "could this exact payload ever succeed?" is a cleaner test than status-code lists alone, because a few 4xx cases — a 409 mid-deploy, a 429 — really are transient. Fail fast on the deterministic ones and save the backoff budget for the ones that can heal.