API Integration Explained: Connecting to a Customer's Systems

API integration means connecting two systems through their APIs so data or actions flow between them automatically. For a forward deployed engineer, it usually means pulling data from a customer’s system (tickets, orders, records) or pushing results back into it. The code is rarely complicated. The difficulty is in the details: authentication, pagination, rate limits, errors and changes on the other side.

What to find out first

• Authentication: API key, OAuth token, or a service account? How are credentials issued and rotated?
• Endpoints and data: which ones you need, and what the responses really look like (make a few test calls).
• Pagination: page numbers, offsets, or cursors?
• Rate limits: how many requests per second or per day? What happens when you exceed them?
• Incremental access: can you ask for records changed since a time, so you do not reload everything?
• Contacts: who to call when the API changes or breaks?

A reliable client

This example starts a small local API that behaves like a customer’s system: it needs a token, returns results in pages with a cursor, and rate-limits one request. The client follows the cursor, waits and retries on the rate limit, and stops immediately on a bad token:

import json, threading, time
from http.server import BaseHTTPRequestHandler, HTTPServer
from urllib.parse import urlparse, parse_qs
from urllib.request import Request, urlopen
from urllib.error import HTTPError

# --- A small local API that behaves like a customer's system: token auth, pages, one rate-limit hit ---
TICKETS = [{"id": i, "status": "open" if i % 3 else "closed"} for i in range(1, 8)]
calls = {"n": 0}

class FakeCustomerAPI(BaseHTTPRequestHandler):
    def do_GET(self):
        calls["n"] += 1
        if self.headers.get("Authorization") != "Bearer test-token":
            return self.reply(401, {"error": "unauthorized"})
        if calls["n"] == 2:                                   # the second call is rate-limited once
            return self.reply(429, {"error": "slow down"}, {"Retry-After": "1"})
        q = parse_qs(urlparse(self.path).query)
        cursor, size = int(q.get("cursor", ["0"])[0]), 3
        page = TICKETS[cursor:cursor + size]
        nxt = cursor + size if cursor + size < len(TICKETS) else None
        self.reply(200, {"data": page, "next_cursor": nxt})
    def reply(self, code, body, headers={}):
        self.send_response(code)
        for k, v in headers.items():
            self.send_header(k, v)
        self.end_headers()
        self.wfile.write(json.dumps(body).encode())
    def log_message(self, *args):
        pass

server = HTTPServer(("127.0.0.1", 0), FakeCustomerAPI)
threading.Thread(target=server.serve_forever, daemon=True).start()
BASE = f"http://127.0.0.1:{server.server_port}/tickets"

# --- The integration client ---------------------------------------------------------------------
def get_json(url, token, max_attempts=4):
    for attempt in range(1, max_attempts + 1):
        try:
            with urlopen(Request(url, headers={"Authorization": f"Bearer {token}"}), timeout=10) as resp:
                return json.load(resp)
        except HTTPError as e:
            if e.code in (429, 500, 502, 503) and attempt < max_attempts:   # temporary: wait and retry
                wait = float(e.headers.get("Retry-After") or 2 ** attempt)
                print(f"  got {e.code}, retrying in {wait}s")
                time.sleep(wait)
                continue
            raise                                                           # 401, 404...: stop and report

def fetch_all(token):
    cursor, rows = 0, []
    while cursor is not None:
        page = get_json(f"{BASE}?cursor={cursor}", token)
        rows += page["data"]
        print(f"  page with {len(page['data'])} rows, next cursor {page['next_cursor']}")
        cursor = page["next_cursor"]
    return rows

rows = fetch_all("test-token")
print(f"fetched {len(rows)} tickets in {calls['n']} requests")
try:
    get_json(BASE, "wrong-token")
except HTTPError as e:
    print("bad token ->", e.code, "(not retried)")
server.shutdown()

  page with 3 rows, next cursor 3
  got 429, retrying in 1.0s
  page with 3 rows, next cursor 6
  page with 1 rows, next cursor None
fetched 7 tickets in 4 requests
bad token -> 401 (not retried)

The client respected the Retry-After header on the rate limit (429) and fetched all pages. A wrong token (401) failed at once, because retrying would never fix it; that error needs a person. See also working with APIs in the Python course.

Rules for production integrations

1. Retry only temporary errors (timeouts, 429, 5xx), with growing waits, and a maximum number of attempts.
2. Store the raw responses before transforming them, so you can reprocess without calling the API again.
3. Load incrementally where the API allows it, using a “changed since” filter and a saved watermark.
4. Make reruns safe: write results with upserts on the record ID, so running twice does not duplicate data. See pipeline reliability and retries.
5. Keep secrets in a secret manager, never in code or logs.
6. Log and alert: requests, failures, rate-limit waits and record counts, with an alert when a run fails or returns far fewer records than usual.

Common mistakes

1. Assuming the documentation is accurate; always inspect real responses.
2. Retrying everything, including errors that can never succeed.
3. Ignoring rate limits until the customer’s API blocks you.
4. No contact on the customer side for when their API changes.

Key takeaway

API integration is mostly about the details: authentication, pagination, rate limits, retries and safe reruns. Inspect real responses, retry only temporary errors, store raw data, load incrementally and monitor every run. Next: making the data fit, in integrating customer data.