Binary data and bytes

Files on disk and data on the wire are bytes — raw sequences of numbers from 0 to 255. Most of the time Python hides this behind str and file objects, but whenever you touch image files, network packets, database wire formats, or any binary protocol, you work with bytes directly. This lesson demystifies the three types Python provides and the tools to convert between bytes and text safely.

bytes, bytearray, and memoryview

Python has three byte-level types, each with a distinct role:

• bytes — immutable, like str but for raw data. Most I/O returns this.
• bytearray — mutable sequence of bytes; useful when you need to patch a header in place without reallocating.
• memoryview — a window into another buffer's memory with no copy; the right tool when slicing large buffers cheaply (image pixels, audio frames).

# bytes literal: prefix b, values 0-255
header = b"\x89PNG\r\n\x1a\n"
print(header[0])      # 137  (an integer, not a character)
print(header[1:4])    # b'PNG'

# hex literals are equally valid, easier to read for binary formats
magic = b"\xFF\xD8\xFF"   # JPEG magic bytes

# bytearray: mutable
buf = bytearray(b"hello")
buf[0] = ord("H")
print(buf)            # bytearray(b'Hello')

# memoryview: zero-copy slice of a buffer
data = bytearray(range(256))
view = memoryview(data)[64:128]  # 64 bytes, no copy made

Encoding and decoding text

A str is a sequence of Unicode code-points; bytes is a sequence of raw octets. Moving between them requires an encoding — a rule for translating each code-point into one or more bytes.

text = "café"

# encode: str  →  bytes
raw = text.encode("utf-8")       # b'caf\xc3\xa9'
raw_latin = text.encode("latin-1")  # b'caf\xe9'  (one byte per char)

# decode: bytes  →  str
print(raw.decode("utf-8"))        # 'café'

# always name the encoding; never rely on the platform default

UTF-8 is almost always the right choice: it handles every Unicode character, is self-synchronising, and is the default for the web and most modern APIs. Latin-1 appears in legacy systems and some network protocols — it maps bytes 0–255 one-to-one, which is occasionally useful but loses anything outside that range. Specify the wrong encoding and you get a UnicodeDecodeError or, worse, silently corrupted text.

struct: packing and unpacking binary formats

Many binary formats (image headers, network packets, archive formats) lay fields out at exact byte offsets with specific integer widths. The struct module lets you read and write them by a format string instead of bit-shifting by hand:

import struct

# Pack: Python values  →  bytes
# ">IH" = big-endian, 4-byte unsigned int, 2-byte unsigned short
packet = struct.pack(">IH", 0xDEADBEEF, 42)
print(packet.hex())   # 'deadbeef002a'

# Unpack: bytes  →  Python values  (returns a tuple)
version, count = struct.unpack(">IH", packet)
print(version, count)   # 3735928559 42

# calcsize tells you how many bytes a format uses
print(struct.calcsize(">IH"))   # 6

Format string characters you'll encounter most: B (1-byte unsigned), H (2-byte unsigned), I (4-byte unsigned), Q (8-byte unsigned), s (raw bytes, prefix with length like 4s). Prefix > forces big-endian (network order); < forces little-endian (most desktop hardware).

Reading binary files

Open with mode "rb" — the b keeps Python from attempting any encoding conversion:

def read_png_dimensions(path):
    with open(path, "rb") as f:
        header = f.read(24)           # PNG stores width/height at bytes 16-24
    # PNG stores dimensions as big-endian 4-byte ints at offset 16
    width, height = struct.unpack(">II", header[16:24])
    return width, height

This pattern — open in "rb", read a fixed number of bytes, unpack with struct — covers reading any fixed-format binary header, from ZIP local-file records to BMP image headers to network protocol frames.

Where to go next

You can now read raw binary data; the next step is storing structured data. Next: Database IO with sqlite3.