Config files

A well-written script does not have its parameters baked in. Output paths, API URLs, log levels, batch sizes — all of these change depending on where the script runs and who is running it. Separating those values into a config file means you can change the script's behaviour without touching the code, which also means you can safely share the code without accidentally sharing production settings.

The common formats

TOML is the current standard in the Python ecosystem. pyproject.toml uses it; tomllib (Python 3.11+) reads it from the standard library without any install. TOML is typed — numbers are numbers, booleans are true/false — and it handles nested structure cleanly:

[database]
host = "localhost"
port = 5432

[output]
path = "/tmp/reports"
format = "json"

YAML is widely used in DevOps tooling (Kubernetes, GitHub Actions, Ansible). It is human-readable and handles nested structures well. The catch: indentation is meaningful, and it has a long list of surprising implicit type coercions (yes becomes True, NO becomes False, bare numbers without quotes become integers). Use it when the ecosystem demands it; avoid it for new config files you control.

INI is the oldest format. Python's configparser in the standard library reads it. Flat key/value sections with no nesting, no types — everything is a string. Still useful for simple scripts where you want zero-dependency config reading.

JSON is already parsed by json (standard library) and is familiar to anyone who has worked with APIs. Its weakness for config: no support for comments, and trailing commas are invalid. Fine as a config format when the data is already JSON-shaped, but TOML is usually more pleasant for humans to edit.

Reading each format

# TOML (Python 3.11+)
import tomllib
with open("config.toml", "rb") as f:
    config = tomllib.load(f)

# YAML (requires pyyaml: pip install pyyaml)
import yaml
with open("config.yaml") as f:
    config = yaml.safe_load(f)

# INI
import configparser
config = configparser.ConfigParser()
config.read("config.ini")

# JSON
import json
with open("config.json") as f:
    config = json.load(f)

Note that tomllib.load() requires the file to be opened in binary mode ("rb"). This is intentional — it enforces UTF-8 and avoids platform encoding issues.

The layered override pattern

The most robust config approach has three layers, evaluated in order:

1. Defaults in code — values that work out of the box, used if nothing else is set
2. Config file — project-specific or environment-specific overrides
3. Environment variables — deployment-specific or secret overrides (highest priority)

import os
import tomllib

# Layer 1: defaults
config = {
    "log_level": "INFO",
    "output_path": "/tmp/output",
    "batch_size": 100,
}

# Layer 2: file overrides
try:
    with open("config.toml", "rb") as f:
        file_config = tomllib.load(f)
    config.update(file_config)
except FileNotFoundError:
    pass   # no config file is fine — defaults apply

# Layer 3: environment variable overrides
if os.environ.get("LOG_LEVEL"):
    config["log_level"] = os.environ["LOG_LEVEL"]
if os.environ.get("OUTPUT_PATH"):
    config["output_path"] = os.environ["OUTPUT_PATH"]

Environment variables always win. This means you can deploy the same script to staging and production with different config files, and still override individual values at runtime without editing any file.

Where to go next

Next: config and logging — reading a TOML config in practice, wiring the resolved log level into Python's logging module, and seeing how the override pattern plays out in a runnable example.