Connections — The TCP Session

Connections — The TCP Session #

An AMQP connection is a long-lived TCP session between a client and a broker. Every message published, every message consumed, every acknowledgment travels over a connection. The connection is the outermost container — it exists before any channel, exchange, or queue, and its closure terminates everything inside it.

The AMQP 0-9-1 connection lifecycle has four distinct phases: protocol negotiation, authentication, tune negotiation, and open. Each phase is a specific sequence of method frames exchanged between client and broker. Understanding this sequence is essential for diagnosing connection problems, implementing clients, and reasoning about failure recovery.

Phase 1: Protocol Header #

The client initiates by sending the AMQP protocol header — not an AMQP frame, but a literal 8-byte octet sequence:

'A' 'M' 'Q' 'P' 0 0 9 1

These bytes identify the protocol version: AMQP 0-9-1. The broker reads this header and responds with either:

• connection.start — the broker accepts AMQP 0-9-1 and starts the handshake.
• A 5-byte error response containing the supported protocol version, then closes the connection.

The error response tells the client what the broker expects. If the client sent the wrong version header, it must reconnect with the correct one.

Client → Broker: 'AMQP\x00\x00\x09\x01'
Broker → Client: connection.start (method frame, channel 0)

All connection-level methods travel on channel 0 — a reserved channel dedicated to connection management. No application data can be sent on channel 0.

The connection.start Method #

connection.start carries:

• version-major / version-minor: broker’s AMQP version (0 and 9 for AMQP 0-9-1).
• server-properties: a table of key-value pairs describing the broker: product, version, platform, capabilities (see below), information.
• mechanisms: space-separated list of supported SASL authentication mechanisms (e.g., PLAIN AMQPLAIN EXTERNAL).
• locales: space-separated list of supported locales (e.g., en_US).

Server capabilities: the capabilities sub-table in server-properties is the broker’s way of advertising optional protocol extensions:

Phase 2: Authentication (SASL) #

The client responds with connection.start-ok:

• client-properties: equivalent to server-properties — identifies the client library and its capabilities.
• mechanism: the SASL mechanism the client has selected from the broker’s list.
• response: the SASL initial response (credentials), encoded per the selected mechanism.
• locale: the locale the client has selected.

PLAIN mechanism (the most common): the response is the null-delimited string \0username\0password:

import base64

def sasl_plain(username: str, password: str) -> bytes:
    return f'\x00{username}\x00{password}'.encode('utf-8')

AMQPLAIN mechanism (RabbitMQ extension): credentials encoded as a longstr-encoded field table:

\x00\x00\x00\x0E  (length of the following field table)
\x05LOGIN  S  \x00\x00\x00\x05guest
\x08PASSWORD  S  \x00\x00\x00\x05guest

AMQPLAIN predates SASL PLAIN in some RabbitMQ clients but is functionally equivalent for most uses.

EXTERNAL mechanism: the client asserts its identity via TLS client certificate; the broker validates against configured CA. No username/password in the SASL exchange. Used for certificate-based authentication.

The broker validates the credentials. If authentication fails, it closes the connection with a 403 ACCESS_REFUSED error before sending connection.tune.

Phase 3: Tune Negotiation #

On successful authentication, the broker sends connection.tune:

• channel-max: maximum number of channels the broker is willing to allow (0 = no limit, but clients should treat as 65535).
• frame-max: maximum frame size in bytes (minimum 4096; 0 = no limit, treat as 131072).
• heartbeat: broker’s proposed heartbeat interval in seconds.

The client responds with connection.tune-ok, proposing its own values. The negotiated value for each parameter is the minimum of the client’s and broker’s proposed values:

channel-max = min(client_channel_max, server_channel_max)
frame-max   = min(client_frame_max, server_frame_max)
heartbeat   = min(client_heartbeat, server_heartbeat)

Why frame-max matters: the maximum frame size determines how large a single content body frame can be. For a 10 MB message with a 131072-byte frame-max, the body requires ~80 body frames. The broker and client both buffer at least frame-max bytes. Setting this too small creates excessive framing overhead; too large wastes memory for connections that send small messages.

Typical values:

• channel-max: 2047 (RabbitMQ default)
• frame-max: 131072 bytes (128 KB, RabbitMQ default)
• heartbeat: 60 seconds (RabbitMQ default)

# pika (Python AMQP client) tune negotiation
connection_params = pika.ConnectionParameters(
    host='localhost',
    channel_max=128,
    frame_max=131072,
    heartbeat=60
)

Phase 4: connection.open #

After tune, the client sends connection.open with the virtual host path:

connection.open: virtual-host='/'

The broker validates that the authenticated user has access to the vhost. If not, it sends connection.close with reply-code 530 (NOT_ALLOWED). If access is granted, the broker sends connection.open-ok and the connection is ready for use.

The full handshake sequence:

Client → Broker: AMQP\x00\x00\x09\x01 (protocol header)
Broker → Client: connection.start
Client → Broker: connection.start-ok (credentials)
Broker → Client: connection.tune
Client → Broker: connection.tune-ok
Client → Broker: connection.open (vhost)
Broker → Client: connection.open-ok
--- connection ready ---
Client → Broker: channel.open (channel 1)
Broker → Client: channel.open-ok
--- channel 1 ready ---

Heartbeats #

AMQP 0-9-1 includes a heartbeat mechanism to detect broken TCP connections. After tune negotiation, both sides agree to send heartbeat frames at the negotiated interval. If no frame (method, content, or heartbeat) has been received for twice the heartbeat interval, the connection is presumed dead and should be closed.

The heartbeat frame is the simplest AMQP frame: a 7-byte sequence with frame type = 8, channel = 0, payload size = 0.

| type=8 | channel=0 (2 bytes) | payload-size=0 (4 bytes) | frame-end=0xCE |

Heartbeat semantics:

• Any frame sent/received resets the heartbeat timer. You do not need to send heartbeats when actively publishing or consuming.
• Heartbeats are only necessary during idle periods.
• TCP keepalives operate at a much longer interval (hours by default on most OSes) and are not a substitute.
• A connection with heartbeat=0 has no heartbeat — broken connections are not detected until the next TCP write fails.

Heartbeat in client libraries: most AMQP client libraries handle heartbeats transparently in a background thread. In pika (Python), the BlockingConnection handles heartbeats automatically if you use process_data_events periodically.

# pika heartbeat — must call process_data_events regularly
connection = pika.BlockingConnection(
    pika.ConnectionParameters(heartbeat=60)
)
# If blocked for > 120s (2x heartbeat), connection will be detected as dead

Closing a Connection #

A connection can be closed gracefully or abruptly.

Graceful close — the connection.close / connection.close-ok handshake:

Client → Broker: connection.close (reply-code=200, reply-text='Normal shutdown')
Broker → Client: connection.close-ok
--- TCP connection closed ---

The client sends connection.close with a reply code and text. The broker stops accepting new channels and methods, drains any in-flight operations, and responds with connection.close-ok. The client then closes the TCP socket.

Broker-initiated close: the broker can initiate connection.close when it detects a protocol error, authentication failure, or resource limit. Common reply codes:

Abrupt close: if the TCP connection drops (network failure, OS kill), neither side sends connection.close. The broker detects the disconnection via heartbeat timeout or TCP error and cleans up the connection’s state — canceling consumers, releasing channel resources.

Connection Recovery #

Most client libraries implement automatic connection recovery — when a connection drops, the library reconnects and re-declares channels, consumers, and topologies.

Connection recovery must handle:

1. Reconnect with backoff: exponential backoff on reconnect attempts to avoid thundering herd when a broker restarts.
2. Re-open channels: after reconnect, all channels are gone — they must be re-opened.
3. Re-declare consumers: any basic.consume subscriptions are gone — consumers must re-subscribe.
4. Re-declare topology: exchanges, queues, and bindings may need to be re-declared (or validated) after reconnect, especially if auto-delete queues were used.

# pika automatic recovery
connection = pika.BlockingConnection(
    pika.ConnectionParameters(
        host='localhost',
        heartbeat=60,
        blocked_connection_timeout=300,
    )
)

In RabbitMQ Java client:

ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
factory.setAutomaticRecoveryEnabled(true);
factory.setNetworkRecoveryInterval(5000);  // retry every 5s

Blocked Connections #

When the RabbitMQ broker is under resource pressure (memory high-watermark or disk free space low), it sends a connection.blocked frame to all publishing connections:

Broker → Client: connection.blocked (reason='low on memory')

A blocked connection still delivers messages to consumers, but the broker will not accept new publishes — basic.publish frames from a blocked connection are queued by the broker but not processed until resources recover. The broker then sends connection.unblocked when the pressure eases.

Clients that handle connection.blocked can pause their publishing logic (avoiding buffer buildup) rather than being silently slow.

The Channel Model #

Once a connection is open, all application work happens on channels. Chapter 3 covers channels in detail. The key connection-level constraint: a connection can have at most channel-max open channels simultaneously. Channels are opened with channel.open (see next chapter) and closed with channel.close. When a connection closes, all channels close too.

Summary #

An AMQP connection is a long-lived TCP session established through a four-phase handshake: protocol version negotiation, SASL authentication, tune parameter negotiation, and virtual host selection. Heartbeats detect broken connections during idle periods. The connection negotiates frame-max (affecting message streaming) and channel-max (limiting concurrency). All protocol methods flow through frames; all frames for connection management use channel 0. When a connection drops, clients must re-establish the full handshake — no protocol state persists across reconnects.