Mocking HTTP in practice

Use the responses library to intercept requests calls in tests — assert the right URL was called, return fake JSON, and test your error-handling code with a simulated 500.

The previous lesson explained why responses is the right tool for unit-testing HTTP-dependent pipeline code. Here you will build and test a small fetch_records function that has both a happy path and an error path.

Because the responses library is not available in the in-browser runtime, the demo below simulates its core behaviour using unittest.mock. The API and assertions are identical to what you would write with the real library — swap the import and decorator to use it in your own test files.

Python — editable, runs in your browser

import json
from unittest.mock import patch, MagicMock

# ── The function under test ──────────────────────────────────────────────────
import urllib.request
import urllib.error

def fetch_records(url: str) -> list:
  """Fetch a JSON array from url. Raises RuntimeError on non-200 status."""
  try:
      with urllib.request.urlopen(url, timeout=10) as resp:
          if resp.status != 200:
              raise RuntimeError(f"HTTP {resp.status}")
          return json.loads(resp.read().decode())
  except urllib.error.HTTPError as exc:
      raise RuntimeError(f"HTTP {exc.code}") from exc

# ── Simulated responses-style helpers ───────────────────────────────────────
class FakeResponse:
  def __init__(self, data, status=200):
      self.status = status
      self._data = json.dumps(data).encode()
  def read(self):
      return self._data
  def __enter__(self): return self
  def __exit__(self, *_): pass

# ── Tests ────────────────────────────────────────────────────────────────────
def test_fetch_records_happy_path():
  """fetch_records returns parsed JSON on a 200 response."""
  fake_data = [{"id": 1, "name": "alpha"}, {"id": 2, "name": "beta"}]
  fake_resp = FakeResponse(fake_data, status=200)

with patch("urllib.request.urlopen", return_value=fake_resp) as mock_open:
      result = fetch_records("https://api.example.com/records")

# Assert return value
  assert len(result) == 2
  assert result[0]["name"] == "alpha"

# Assert the right URL was called
  mock_open.assert_called_once_with("https://api.example.com/records", timeout=10)
  print("PASS: happy path returns correct data and calls correct URL")

def test_fetch_records_http_error():
  """fetch_records raises RuntimeError on a 500 response."""
  import urllib.error

http_error = urllib.error.HTTPError(
      url="https://api.example.com/records",
      code=500,
      msg="Internal Server Error",
      hdrs={},
      fp=None,
  )

with patch("urllib.request.urlopen", side_effect=http_error):
      try:
          fetch_records("https://api.example.com/records")
          print("FAIL: expected RuntimeError")
      except RuntimeError as exc:
          assert "500" in str(exc), f"Expected '500' in error message, got: {exc}"
          print(f"PASS: RuntimeError raised on 500: {exc}")

test_fetch_records_happy_path()
test_fetch_records_http_error()

Using the real responses library

In your actual test file, replace the mock with responses:

import responses
import requests

def fetch_records(url: str) -> list:
    resp = requests.get(url, timeout=10)
    resp.raise_for_status()
    return resp.json()

@responses.activate
def test_fetch_records_happy_path():
    responses.add(
        responses.GET,
        "https://api.example.com/records",
        json=[{"id": 1, "name": "alpha"}, {"id": 2, "name": "beta"}],
        status=200,
    )
    result = fetch_records("https://api.example.com/records")
    assert len(result) == 2
    assert len(responses.calls) == 1
    assert responses.calls[0].request.url == "https://api.example.com/records"

@responses.activate
def test_fetch_records_500_raises():
    responses.add(
        responses.GET,
        "https://api.example.com/records",
        status=500,
    )
    try:
        fetch_records("https://api.example.com/records")
        assert False, "Expected an exception"
    except Exception as exc:
        assert "500" in str(exc)

By default, responses raises ConnectionError for any URL that is not registered. This is a feature: it prevents your code from silently hitting live endpoints during a test run. If you need to allow passthrough for specific URLs, use responses.add_passthrough(pattern).

Asserting request details

responses.calls gives you the full request history. You can assert headers and request body:

assert responses.calls[0].request.headers["Authorization"] == "Bearer test-token"
assert json.loads(responses.calls[0].request.body) == {"filter": "active"}

This matters for testing authentication flows: verify that your code adds the correct Authorization header rather than trusting that it "probably" does.

Where to go next

Next: lab — test suite — combine filesystem mocking, HTTP mocking, and error path testing into a complete test suite covering every function in a pipeline script.

Finished reading? Mark it complete to track your progress.

Using the real responses library

Asserting request details

Where to go next

On this page