How to handle API errors in Python (status codes, timeouts, retries)

A demo API call works on the good day it was written. A useful one works on every other day too — which means handling the two failure classes separately: the server answered with an error (an HTTPError), and the server effectively didn't answer at all (a URLError or a timeout). urllib keeps the ladder short, but the order matters.

The ladder, and why the order matters

HTTPError is a subclass of URLError. Catch HTTPError first or the generic parent swallows every meaningful status code and you learn nothing. Here is the pattern worth copying — a retry wrapper that handles the two retryable cases politely (rate limits and server hiccups) and fails fast on everything else:

import json
import time
import urllib.error
import urllib.request

def fetch_with_retry(url, retries=3):
    req = urllib.request.Request(url, headers={"User-Agent": "bryme-demo/1.0"})
    for attempt in range(retries):
        try:
            with urllib.request.urlopen(req, timeout=10) as resp:
                return json.loads(resp.read().decode("utf-8"))
        except urllib.error.HTTPError as e:
            detail = e.read().decode("utf-8", errors="replace")
            print(f"HTTP {e.code}: {detail[:120]}")
            if e.code == 429:
                wait = e.headers.get("Retry-After", "2")
                time.sleep(int(wait) if wait.isdigit() else 2)
                continue
            if e.code >= 500 and attempt < retries - 1:
                time.sleep(2 ** attempt)
                continue
            raise
        except urllib.error.URLError as e:
            print(f"Network problem: {e.reason}")
            raise
    raise RuntimeError("retries exhausted")

Four habits are embedded in those lines. Read the error body: e.read() — good APIs put the actual explanation in the error response ("rate limit exceeded, key quota"), and printing it turns mystery into diagnosis. Respect 429: a rate-limit response with a Retry-After header is the server telling you exactly when to come back; sleeping that long (with a sane fallback) is the difference between a client and a nuisance — the status-code field guide covers what each code means. Back off on 5xx: a server that just errored needs seconds, not an immediate retry; 2 ** attempt is exponential backoff in one expression. Re-raise the rest: a 403 or 404 will not fix itself — crash loudly with the detail instead of retrying into a wall (the base calling pattern is in the previous piece).

Timeouts are errors too

The timeout=10 from the calling pattern raises when a server stalls — in current Python, a TimeoutError (which, like HTTPError, sits under URLError's family tree in older versions; catching URLError plus TimeoutError covers both across versions). Treat it like a network failure: one retry with backoff is reasonable for a flaky cell link; a loop is not.

The trap after the status code

One last habit separates professionals: some APIs return 200 OK with an error in the body ({"error": "..."}). The transport succeeded; the request didn't. After parsing, check for the API's own error key before celebrating — the status code says the pipe works, the payload says whether the answer is real. Log both whenever something puzzles you; the combination of status, body and attempt number is what you'll want in the bug report.

Sources

• Python docs — urllib.error (HTTPError, URLError)
• IETF RFC 9110 — HTTP semantics, incl. Retry-After