WebSockets, a two-way connection that stays open over HTTP

WebSockets are a bidirectional, full-duplex communication channel over a single TCP connection. RFC 6455 (2011) standardised them. The connection starts as HTTP and upgrades to WebSocket via the Upgrade header; once upgraded, both client and server can send messages at any time. Modern realtime apps (chat, collaboration tools, live dashboards) use WebSockets, with HTTP/3 and Server-Sent Events as alternatives for one-way pushes.

For fifteen years, HTTP was strictly half-duplex: the browser asked, the server answered, and that was the entire conversation. If the server had news — a chat message, a stock tick, the other player’s move — it had no way to say so. It waited to be asked.

The workarounds were increasingly baroque. Polling — hammer the server every second; waste 95 % of the requests. Long polling — hold the request open and answer it the moment something happens; then the client re-opens instantly. Hidden iframes — stream an infinite HTML document and parse <script> tags as they arrived. Each trick traded bandwidth, latency, or complexity for the one thing the protocol wouldn’t give you: a server-initiated message.

WebSockets ended the workaround era. RFC 6455, published in December 2011, defines a way to start a connection as HTTP, talk the server into abandoning HTTP mid-request, and keep the underlying TCP socket open for as long as both sides want. After the handshake, either side can send at any time. The request/response shape is gone.

The handshake is not a hack — it is a deliberate piece of engineering theatre. WebSockets live on ports 80 and 443, wearing HTTP clothes, specifically so corporate firewalls that allow "web traffic" allow them. The disguise is load-bearing: the protocol could have defined its own port, as FTP (21) or SSH (22) did. It didn't, because a protocol no firewall permits is a protocol no one can deploy.

Three headers carry the whole conversion. Connection: Upgrade and Upgrade: websocket signal the protocol switch. Sec-WebSocket-Key is a random 16-byte nonce, base64-encoded — and what the server does with it is the entire point.

The simulator below plays the whole lifecycle — the upgrade, a handful of data frames in both directions, a keepalive pair, and the close. Use the step controls to scrub. Click Show wire bytes to see the exact frame layout the network sees.

Every WebSocket message is one or more frames. The frame header is where almost all of the protocol’s intelligence lives — opcodes, length, masking, fragmentation — crammed into as few as two bytes. Compare to HTTP, where headers routinely hit 800.

Read the header top-to-bottom. FIN is a single bit: set it when this is the last frame of a message (so you can split big messages across frames without them interleaving with other messages). RSV1-3 are reserved for extensions; RSV1 carries permessage-deflate compression when negotiated. Opcode is four bits, which we spell out below. MASK is a single bit: always 1 from client to server, always 0 from server to client. Payload length is the clever part — seven bits that can stand in for 16-bit or 64-bit lengths depending on a sentinel value, so short messages pay for only seven bits of length while long messages can still express eight exabytes.

The four-bit opcode field has sixteen possible values; RFC 6455 assigns seven and reserves the rest. Three carry application payloads (TEXT, BINARY, CONTINUATION). Three are control frames sent by the protocol itself (CLOSE, PING, PONG). Control frames must fit in a single unfragmented frame of ≤125 bytes — they are never part of a streaming message.

Every frame from the client carries a 32-bit masking key. Every byte of the payload is XORed with a rotating byte of that key before it hits the wire. The server reverses the XOR on receipt. The payload’s content is unchanged in net effect — so why bother?

Because of cache-poisoning intermediaries. A sufficiently clever attacker, controlling a browser via JavaScript, could try to craft a WebSocket payload whose bytes look like a valid HTTP GET request to a transparent proxy sitting between the client and the server. The proxy — not knowing WebSockets — might interpret those bytes as a new request, fetch the response, and cache it under a URL the attacker chose. Later users get the poisoned cache entry.

Masking makes this attack impossible. The attacker controls the plaintext but not the random key; the bytes that actually traverse the intermediary are unpredictable per frame. A proxy that decodes them as HTTP sees gibberish. Server-to-client frames are never masked because there are no dangerous intermediaries in that direction — the server does not live behind the victim’s corporate proxy.

A stateless HTTP API is embarrassingly easy to scale. Put fifty machines behind a load balancer; any request goes to any machine; if one dies, another answers in milliseconds. WebSockets subvert every assumption in that sentence.

Because the socket lives for minutes or hours, a user is physically tethered to one server — call it Server A — for the duration of the session. The load balancer cannot round-robin individual frames. If Server A restarts, every connected user is violently disconnected and has to reconnect, potentially to a different server. The state that server held — room membership, typing indicators, subscription filters — is gone unless it was written somewhere durable.

This creates two separate scaling problems. The first is the C10K problem — how many idle sockets can one box hold? In 1999 the answer was about ten thousand, limited by the threads-per-connection model. Modern runtimes — Node, Go, Rust’s Tokio, Java’s Netty — hold each socket as a few hundred bytes of epoll state, and a single machine can comfortably serve a million. The second is fan-out: when a message arrives for a user connected to Server A, and someone else typing on Server B wants to send it, how does it get there?

The handshake is a plain HTTP GET. Browsers attach cookies to HTTP GETs — same-origin or not, unless SameSite says otherwise. The implication: a malicious page at evil.example can execute new WebSocket("wss://yourbank.com/api") in a background script; the browser cheerfully sends the user’s authenticated session cookie along with the upgrade request; the bank’s server accepts; the attacker now has a live, authenticated WebSocket to someone else’s bank session.

This is Cross-Site WebSocket Hijacking — CSWSH — and it is the CSRF of the real-time era. The twist that matters: CORS does not apply to WebSockets. The browser will not block the upgrade on Origin mismatch the way it would block a cross-origin fetch. The defense has to live on the server.

One node holds many sockets. Modern Linux easily handles 1M+ idle TCP sockets per box (with ulimits and sysctl tuned). The harder problem is the application: each socket holds buffer memory; the application must process inbound frames; per-connection ping/pong eats CPU. Real numbers from production: Phoenix LiveView (Elixir) reports ~2M concurrent connections per node; Go services ~1M; Node.js ~300-500k.

Load-balancer stickiness. WebSockets are long-lived and stateful — the same client must keep hitting the same server. L7 load balancers (NGINX, Envoy, HAProxy) handle this with cookie-based or hash-based affinity. AWS ALB and GCP LB both support WebSocket out of the box; sticky sessions need to be enabled explicitly.

Reconnect storms. A node failure or deploy causes every connected client to reconnect at once. Without staggered backoff, the surviving nodes get a stampede. Production systems implement exponential backoff with jitter on the client (pause 1s, then 2s, then 4s with ±50% jitter). Slack's Flannel and Discord's gateway both ship documented stagger strategies.

Cross-node fan-out. When user A on node 1 sends a message to user B on node 17, the application needs a pub/sub layer. Common patterns: Redis Pub/Sub (simplest, lossy under load), Kafka (durable, higher latency), NATS (low-latency in-memory pub/sub). Choose by whether messages must survive a node restart.