Why scripts need tests
Automation scripts run unattended, touch real systems, and fail silently — making them the most dangerous category of code to ship without tests.
- Explain why automation scripts are high-risk without tests — unattended execution, real side effects, silent failures
- Identify what is worth testing in a pipeline — transformations, config loading, error-handling logic
- Identify what not to test — the OS, the network, the language runtime itself
A web server that crashes emits a 500 and moves on. An automation script that fails silently at step two may corrupt a week's worth of data before anyone notices. Scripts run unattended, often at night, against production systems, with no user watching the output. That combination makes untested automation code disproportionately risky compared to its apparent simplicity.
Three reasons scripts are high-risk
1. Unattended execution. A web endpoint is called by a human who notices something wrong. A cron job runs at 03:00 and your first indication of a problem is a missing report at 09:00 — or, worse, a warehouse full of duplicate rows.
2. Real side effects. Scripts routinely write to databases, move or delete files, call paid APIs, and send emails. A bug in a cleanup script cannot be retried; the files are gone.
3. Silent failures. Python scripts exit 0 by default even if they catch and swallow an exception. A pipeline that "completes" while producing an empty output file looks identical to one that completed correctly, unless you wrote an assertion to check.
What to test
The goal is not to achieve 100 % coverage of every line. It is to assert the invariants that matter: does the data come out in the right shape? Does the config loading handle missing keys gracefully? Does the error path actually surface the exception it is supposed to, rather than swallowing it?
Test these:
- Data transformations. A function that takes raw JSON and returns a cleaned list of dicts is pure and straightforward to unit-test.
- Config loading. A function that reads a TOML or environment-variable config
should raise a clear error on missing required keys, not produce
Nonesilently. - Error-handling logic. If your
exceptclause re-raises, logs, and sends an alert, test that the right thing happens for each exception type. - Path construction. Bugs like
output_dir / filenameproducing the wrong path are common and easy to catch with a single assertion.
Do not test these:
- Python's
open(),os.path.join,json.loads— the standard library is tested by the Python core team; testing it is wasted effort. - The network. Do not hit real APIs in unit tests; mock them (covered in the next two lessons).
- The clock or random number generation. If your code depends on
time.time(), inject it as a parameter so tests can pass a fixed value.
A useful heuristic: test your code, not their code. Anything you did not write should be mocked at the boundary rather than exercised for real.
The testing stack
For automation scripts, three tools cover almost everything:
| Tool | Purpose |
|---|---|
pytest | Test runner, fixture system, assertion introspection |
pytest tmp_path fixture | Temporary directory for file I/O tests; cleaned up automatically |
unittest.mock | Mock any calleable or object attribute in your module |
responses library | Intercept and fake requests calls at the transport layer |
You will use all four in the remaining lessons of this module.
Where to go next
Next: mocking the filesystem — using pytest's tmp_path fixture to test
file-writing functions without touching the real filesystem.
Lab: harden a pipeline
Take a brittle three-step pipeline and make it production-ready — add idempotency checkpoints, atomic writes, exponential-backoff retry logic, and failure alerting.
Mocking the filesystem
Use pytest's tmp_path fixture and unittest.mock.patch to test file-writing functions cleanly, without leaving files behind or touching production directories.