Configuring Exponential Backoff for 5xx Errors

This page is part of the Retryable vs Non-Retryable Errors reference, itself a section of the Error Contracts & Resilience Mapping guide.

Exponential backoff is the standard mechanism for spreading client retries over increasing intervals so a struggling upstream has room to recover. Without it, every in-flight client retries simultaneously the moment a 503 or 504 arrives — turning a brief overload spike into a self-reinforcing storm. This page covers the exact parameters, code, and contract fixtures you need to ship a correct backoff policy.

When You Need This

Backoff misconfiguration typically surfaces in one of three ways:

If you see any of these, the sections below give you a reproducible path from a blank terminal to a validated policy.

Spec Snippet

Annotate every retryable operation in your OpenAPI 3.1.0 spec with an x-backoff-strategy extension. This gives SDK generators and API gateways a machine-readable source of truth for retry parameters.

openapi: "3.1.0"
info:
  title: Payments API
  version: "1.0.0"
paths:
  /v1/transactions:
    post:
      operationId: createTransaction
      x-retryable: true
      x-backoff-strategy:
        base_delay_ms: 1000
        multiplier: 2
        max_retries: 3
        jitter: "full"          # "full" | "equal" | "none"
        max_delay_ms: 10000
        retry_on:
          - 429
          - 500
          - 502
          - 503
          - 504
      responses:
        "202":
          description: Transaction accepted
        "503":
          description: Service temporarily unavailable
          headers:
            Retry-After:
              schema:
                type: integer
              description: Seconds until the service expects to recover

The jitter: "full" value selects the AWS-recommended full-jitter formula (random(0, min(cap, base * 2^attempt))), which gives the best load distribution under high concurrency.

Step-by-Step Resolution Guide

Step 1 — Classify which 5xx codes are retryable

Not every server error warrants a retry. Blindly retrying 501 Not Implemented wastes quota and masks a genuine contract bug.

Map this table to the retry_on list in your spec extension and in your client implementation.

Step 2 — Implement full-jitter backoff in TypeScript

The following Axios interceptor reads the retry parameters from the request config so the values stay consistent with the spec extension.

import axios, { AxiosInstance, AxiosRequestConfig, AxiosError } from "axios";

const RETRYABLE_STATUSES = new Set([429, 500, 502, 503, 504]);
const BASE_DELAY_MS = 1_000;
const MULTIPLIER = 2;
const MAX_DELAY_MS = 10_000;
const MAX_RETRIES = 3;

interface RetryConfig extends AxiosRequestConfig {
  _attempt?: number;
}

function fullJitterDelay(attempt: number): number {
  const exponential = BASE_DELAY_MS * Math.pow(MULTIPLIER, attempt);
  const cap = Math.min(exponential, MAX_DELAY_MS);
  return Math.random() * cap; // uniform random in [0, cap]
}

function parseRetryAfterMs(header: string | undefined): number | null {
  if (!header) return null;
  const seconds = parseInt(header, 10);
  if (!isNaN(seconds) && seconds > 0) return seconds * 1_000;
  const date = Date.parse(header);
  if (!isNaN(date)) return Math.max(0, date - Date.now());
  return null;
}

export function attachRetryInterceptor(client: AxiosInstance): void {
  client.interceptors.response.use(null, async (error: AxiosError) => {
    const config = error.config as RetryConfig | undefined;
    if (!config) return Promise.reject(error);

    const status = error.response?.status ?? 0;
    const attempt = config._attempt ?? 0;

    if (!RETRYABLE_STATUSES.has(status) || attempt >= MAX_RETRIES) {
      return Promise.reject(error);
    }

    // Respect server-provided Retry-After before falling back to algorithm
    const retryAfterHeader = error.response?.headers["retry-after"] as string | undefined;
    const delayMs = parseRetryAfterMs(retryAfterHeader) ?? fullJitterDelay(attempt);

    await new Promise<void>((resolve) => setTimeout(resolve, delayMs));

    return client.request({ ...config, _attempt: attempt + 1 });
  });
}

Step 3 — Implement full-jitter backoff in Python

import asyncio
import random
import time
from typing import Optional

import httpx

RETRYABLE_STATUSES = {429, 500, 502, 503, 504}
BASE_DELAY_S = 1.0
MULTIPLIER = 2
MAX_DELAY_S = 10.0
MAX_RETRIES = 3


def full_jitter_delay(attempt: int) -> float:
    """Return a uniformly random delay in [0, min(cap, base * multiplier^attempt)]."""
    cap = min(MAX_DELAY_S, BASE_DELAY_S * (MULTIPLIER ** attempt))
    return random.uniform(0, cap)


def parse_retry_after(header: Optional[str]) -> Optional[float]:
    """Return seconds to wait from a Retry-After header, or None if absent/unparseable."""
    if not header:
        return None
    try:
        seconds = float(header)
        if seconds > 0:
            return seconds
    except ValueError:
        pass
    # HTTP-date format
    from email.utils import parsedate_to_datetime
    try:
        dt = parsedate_to_datetime(header)
        remaining = dt.timestamp() - time.time()
        return max(0.0, remaining)
    except Exception:
        return None


async def request_with_backoff(
    client: httpx.AsyncClient,
    method: str,
    url: str,
    **kwargs,
) -> httpx.Response:
    for attempt in range(MAX_RETRIES + 1):
        response = await client.request(method, url, **kwargs)
        if response.status_code not in RETRYABLE_STATUSES or attempt == MAX_RETRIES:
            response.raise_for_status()
            return response

        retry_after = parse_retry_after(response.headers.get("retry-after"))
        delay = retry_after if retry_after is not None else full_jitter_delay(attempt)
        await asyncio.sleep(delay)

    raise RuntimeError("Unreachable")

Step 4 — Parse and honour Retry-After

Retry-After can be either a delta-seconds integer or an HTTP-date string (RFC 7231 § 7.1.3). Both implementations above handle both formats. The server-provided value must always win over the algorithm — it reflects actual upstream capacity and scheduled maintenance windows.

HTTP/1.1 503 Service Unavailable
Retry-After: 30
Content-Type: application/problem+json

{
  "type": "https://api.example.com/errors/service-unavailable",
  "status": 503,
  "title": "Service Temporarily Unavailable",
  "detail": "Upstream dependency degraded. Retry after the indicated interval."
}

The error body follows the RFC 7807 Problem+JSON contract, which your client should validate before deciding whether to retry.

Step 5 — Enforce the policy with contract tests

Pin the retry behaviour in CI so generator updates or dependency bumps cannot silently regress it.

// tests/retry-backoff.contract.test.ts
import { setupServer } from "msw/node";
import { http, HttpResponse } from "msw";
import { attachRetryInterceptor } from "../src/retry";
import axios from "axios";

const server = setupServer();
beforeAll(() => server.listen());
afterAll(() => server.close());

test("honours Retry-After header and does not fall back to jitter", async () => {
  let callCount = 0;
  const timestamps: number[] = [];

  server.use(
    http.get("http://test/resource", () => {
      callCount++;
      timestamps.push(Date.now());
      if (callCount < 3) {
        return new HttpResponse(null, {
          status: 503,
          headers: { "Retry-After": "1" },
        });
      }
      return HttpResponse.json({ ok: true });
    })
  );

  const client = axios.create({ baseURL: "http://test" });
  attachRetryInterceptor(client);

  await client.get("/resource");

  expect(callCount).toBe(3);
  // Each wait should be ~1000 ms; allow 200 ms tolerance
  expect(timestamps[1] - timestamps[0]).toBeGreaterThanOrEqual(800);
  expect(timestamps[2] - timestamps[1]).toBeGreaterThanOrEqual(800);
});

test("does not retry on 501", async () => {
  server.use(
    http.get("http://test/unsupported", () =>
      new HttpResponse(null, { status: 501 })
    )
  );

  const client = axios.create({ baseURL: "http://test" });
  attachRetryInterceptor(client);

  await expect(client.get("/unsupported")).rejects.toMatchObject({
    response: { status: 501 },
  });
});

Backoff Timing Diagram

The SVG below illustrates three retry attempts with full-jitter delay: each wait is a random value between zero and the exponential cap, so simultaneous clients spread their retries across the window rather than piling up at the same instant.

RFC and Standard Compliance

The backoff strategy above aligns with these specifications:

The retry_on list in your spec extension should include 429 alongside the 5xx codes above because 429 is structurally identical in backoff treatment: the server is overwhelmed and has provided recovery guidance.

Idempotency and Safety Implications

Automatic retry is safe by default only for idempotent HTTP methods (GET, PUT, DELETE). For POST and PATCH, a retry that arrives after the server completed the first attempt silently duplicates the operation. Guard these with an idempotency key — a client-generated UUID sent in a header such as Idempotency-Key: <uuid> — so the server can deduplicate on the key rather than the body. Without that guarantee, scope your retry interceptor to exclude POST unless the endpoint explicitly opts in via the x-retryable: true extension.

SDK and Codegen Downstream Effect

OpenAPI generators do not emit retry logic from x-backoff-strategy by default. The generated client looks like this:

// Generated (no retry awareness)
async function createTransaction(body: TransactionRequest): Promise<Transaction> {
  const response = await fetch("/v1/transactions", {
    method: "POST",
    body: JSON.stringify(body),
  });
  if (!response.ok) throw new ApiError(response);
  return response.json();
}

After patching the Mustache template or wrapping the client with the interceptor above, the diff is:

-async function createTransaction(body: TransactionRequest): Promise<Transaction> {
-  const response = await fetch("/v1/transactions", { method: "POST", body: JSON.stringify(body) });
-  if (!response.ok) throw new ApiError(response);
-  return response.json();
-}
+async function createTransaction(body: TransactionRequest): Promise<Transaction> {
+  return requestWithBackoff("POST", "/v1/transactions", body, {
+    retryOn: [429, 500, 502, 503, 504],
+    maxRetries: 3,
+    baseDelayMs: 1000,
+    jitter: "full",
+    maxDelayMs: 10000,
+  });
+}

Add a CI lint step that greps the generated output for retryOn and fails the build if it is absent:

grep -q "retryOn" src/generated/client.ts || \
  (echo "Generated client is missing retry configuration" && exit 1)

Common Mistakes

FAQ

Related

• Retryable vs Non-Retryable Errors — the parent reference covering classification rules for all HTTP error categories
• Error Contracts & Resilience Mapping — the top-level guide for designing error contracts across your API surface
• RFC 7807 Problem+JSON Implementation — structure the 503 and 502 error bodies that your retry logic consumes
• Idempotency Key Implementation — prerequisite for enabling safe retries on POST endpoints
• Client Fallback Strategies — what to do when retries are exhausted and no recovery is possible