Network Programming in Go¶

Below net/http¶

Most Go services never touch a raw socket, and most performance and reliability problems in Go services are still socket problems: missing deadlines, unbounded per-connection memory, a write per small message, connections never reused, slow clients holding resources. This page goes one level below net/http — how the netpoller makes blocking-style code scale, how to write a TCP server that survives real clients, and which socket options matter. The protocol background is in TCP and I/O Models and Event Loop.

Measurements: Go 1.26.4, Apple M1 Pro, loopback.

The netpoller: blocking code, non-blocking sockets¶

Go code reads a socket as if it blocked:

n, err := conn.Read(buf) // looks blocking

Underneath, every socket is in non-blocking mode and registered with the OS readiness API — epoll on Linux (edge-triggered), kqueue on macOS/BSD, IOCP on Windows:

This is the I/O model of an event loop (one epoll set per process, readiness-based), presented as goroutines with straight-line code. The cost is per-goroutine memory instead of callbacks; the benefit is that every connection's logic is ordinary sequential code with ordinary stack traces. It is why goroutine-per-connection is the default design in Go, and why it scales to hundreds of thousands of connections — as long as per-connection memory stays small.

What one connection costs¶

5,000 idle TCP connections to a simple echo server (io.Copy(conn, conn) per connection), client and server in the same process, measured:

The heap number is the surprise: io.Copy allocates a 32 KB buffer for every call, and each connection's goroutine holds one for its whole life, idle or not. At 100,000 connections that is 3.2 GB of buffers for connections that are mostly silent. Remedies:

• Use smaller buffers (io.CopyBuffer with a 4 KB buffer, or a bufio.Reader of a deliberate size) for chatty, low-volume protocols.
• Allocate buffers only while a request is in flight and return them to a sync.Pool between requests.
• For extreme connection counts (millions), event-loop libraries (gnet, cloudwego/netpoll) avoid the goroutine-and-buffer per connection, at the cost of callback-style code.

Kernel memory per connection (socket buffers, often 4–128 KB each direction with autotuning) comes on top and is visible in ss -tm, not in Go's metrics.

Batching writes: the biggest easy win¶

Sending 200,000 messages of 16 bytes over loopback:

Every conn.Write is a system call plus a TCP segment. bufio coalesces the messages into 64 KB writes: 700x faster. The design question is when to flush: after each logical response (simple, correct), or when no more responses are queued (lower overhead under load — see the server in Designing a Binary Protocol). Forgetting to Flush is the classic bug: the last response sits in the buffer and the client waits forever.

On the read side, wrap the connection in a bufio.Reader so parsing small headers does not cost one syscall per field.

Nagle's algorithm and TCP_NODELAY¶

Nagle's algorithm holds back small segments while earlier data is unacknowledged. Combined with the receiver's delayed ACK (up to 40 ms on Linux), a write-write-read pattern — header and body sent as two writes, then waiting for the reply — can stall for tens of milliseconds per request on a real network.

Go disables Nagle by default: every *net.TCPConn gets TCP_NODELAY set when it is created (setNoDelay(fd, true) in net/tcpsock.go). So in Go you get low latency by default and should batch with bufio instead of relying on the kernel. Measured write-write-read over macOS loopback: 28.8 µs per round trip with NoDelay(true), 41.0 µs with SetNoDelay(false) — slower, but without the 40 ms stall, which depends on the peer's delayed-ACK behaviour and does not reproduce on this loopback. Re-enable Nagle only for bulk transfers made of many tiny writes you cannot batch.

A TCP server that survives real clients¶

func serve(ctx context.Context, ln net.Listener, handle func(context.Context, net.Conn) error) error {
    var wg sync.WaitGroup
    defer wg.Wait()                     // graceful: wait for connections to finish
    sem := make(chan struct{}, 10_000)  // cap concurrent connections

    go func() { <-ctx.Done(); ln.Close() }() // shutdown: stop accepting

    for {
        conn, err := ln.Accept()
        if err != nil {
            if errors.Is(err, net.ErrClosed) {
                return nil
            }
            if errors.Is(err, syscall.EMFILE) || errors.Is(err, syscall.ENFILE) {
                time.Sleep(10 * time.Millisecond) // out of fds: back off, don't spin or exit
                continue
            }
            return err
        }
        select {
        case sem <- struct{}{}:
        default:
            conn.Close() // over capacity: shed load at the door
            continue
        }
        wg.Add(1)
        go func() {
            defer func() { conn.Close(); <-sem; wg.Done() }()
            if err := handle(ctx, conn); err != nil && !errors.Is(err, io.EOF) {
                log.Printf("conn %v: %v", conn.RemoteAddr(), err)
            }
        }()
    }
}

And inside handle, every blocking operation has a deadline:

for {
    conn.SetReadDeadline(time.Now().Add(idleTimeout))  // idle clients are dropped
    req, err := readRequest(r)
    if err != nil {
        return err
    }
    conn.SetWriteDeadline(time.Now().Add(writeTimeout)) // slow readers can't pin us
    if err := writeResponse(w, handleReq(ctx, req)); err != nil {
        return err
    }
}

Deadlines are absolute times, not durations, and apply to all future I/O until changed — reset them per operation. Without a read deadline, a client that vanishes without a FIN or RST (pulled cable, NAT entry expired) leaves the goroutine blocked until TCP keepalive notices — with Go's defaults (15 s idle, 15 s interval, 9 probes), about 2.5 minutes; with keepalive disabled, forever. Without a write deadline, a client that stops reading fills the socket buffers and the server's Write blocks forever: a classic way for one bad client to leak goroutines.

For HTTP, the same rules are http.Server fields: always set ReadHeaderTimeout (protects against Slowloris-style slow headers), ReadTimeout/WriteTimeout or per-handler deadlines via http.ResponseController, and IdleTimeout.

Socket options and kernel limits worth knowing¶

Zero-copy paths¶

io.Copy checks for ReaderFrom/WriterTo. On Linux:

• file → socket uses sendfile(2): the kernel copies page-cache pages to the socket without a round trip through user space;
• socket → socket (*TCPConn.ReadFrom with a TCP/Unix source) uses splice(2) through a pipe;
• file → file uses copy_file_range(2).

A reverse proxy or file server that passes plain *os.File and *net.TCPConn values to io.Copy gets these for free; one that wraps them (bufio, a counting reader, TLS) does not — TLS in particular must encrypt in user space (except with kernel TLS, which Go does not use).

Client side¶

• Reuse connections. http.Client/http.Transport pool them; the pool only works if response bodies are fully read and closed (Goroutine Leaks shows the leak when they are not). Set MaxIdleConnsPerHost — the default of 2 is far too low for a service calling one backend heavily, and causes a storm of new connections (and TIME_WAIT sockets) under load.
• Timeouts at every layer: dial (net.Dialer.Timeout), TLS handshake, response headers, and an overall context deadline. http.Client{} with no Timeout waits forever.
• DNS: the pure-Go resolver (default when cgo is off) is goroutine-friendly; the cgo resolver blocks a thread per lookup. Go does not cache DNS answers itself — a high-rate client dialing new connections hits DNS each time, another reason to pool.

Common pitfall¶

A server without write deadlines and a slow consumer. A streaming service pushes events to subscribers with conn.Write from one goroutine per subscriber. One subscriber on a bad mobile network stops reading; its socket send buffer fills, and the server's Write blocks. If events are fanned out from a shared loop that writes to each subscriber in turn, that one client now blocks every other subscriber. Even with a goroutine per subscriber, each stuck one holds its buffers forever. The fix has three parts: a write deadline on every write, a bounded per-subscriber queue between the producer and the writer goroutine, and a policy for when the queue is full — disconnect the slow client (and let it resume from an offset) rather than block or buffer without limit. Backpressure from a slow peer must end at that peer.

Review questions¶

1. Why can goroutine-per-connection scale to 100,000 connections, and what per-connection cost limits it in practice?
2. Where did the 33 KB of heap per idle connection come from in the echo-server experiment?
3. Go sets TCP_NODELAY by default. What should you do instead of relying on Nagle to batch small writes?
4. Why is a read deadline needed even with TCP keepalive enabled, and what does a missing write deadline let a slow client do?
5. Which wrapping of io.Copy's arguments disables sendfile/splice, and why?