From TCP connection to Handler
Go's net/http package is a popular choice for building HTTP servers, and for good reason—it provides a clean, composable model for request handling. To understand how it works, let's trace a request through a minimal server:
package main
import (
"fmt"
"net/http"
)
func hello(w http.ResponseWriter, req *http.Request) {
fmt.Fprintf(w, "hello\n")
}
func headers(w http.ResponseWriter, req *http.Request) {
for name, headers := range req.Header {
for _, h := range headers {
fmt.Fprintf(w, "%v: %v\n", name, h)
}
}
}
func main() {
http.HandleFunc("/hello", hello)
http.HandleFunc("/headers", headers)
http.ListenAndServe(":8090", nil)
}
The entry point is http.ListenAndServe, which handles the low-level details of accepting TCP connections and dispatching them to goroutines:
func ListenAndServe(addr string, handler Handler) error
A simplified view of the call flow looks like this:
The actual sequence in the source code involves more intermediate calls, but the essential loop is straightforward: accept a connection, spawn a goroutine for it, then repeatedly parse HTTP requests from the connection and dispatch each one to a handler.
A handler in Go is any value implementing the http.Handler interface:
type Handler interface {
ServeHTTP(ResponseWriter, *Request)
}
The default mux
In the example above, ListenAndServe is called with nil as the handler argument. The net/http package handles this case internally with an adapter:
type serverHandler struct {
srv *Server
}
func (sh serverHandler) ServeHTTP(rw ResponseWriter, req *Request) {
handler := sh.srv.Handler
if handler == nil {
handler = DefaultServeMux
}
if req.RequestURI == "*" && req.Method == "OPTIONS" {
handler = globalOptionsHandler{}
}
handler.ServeHTTP(rw, req)
}
When no handler is provided, http.DefaultServeMux is used—a package-level instance of http.ServeMux. Calls like http.HandleFunc register routes on this default mux. The same server can be written explicitly with a mux object, with no change in behavior:
func main() {
mux := http.NewServeMux()
mux.HandleFunc("/hello", hello)
mux.HandleFunc("/headers", headers)
http.ListenAndServe(":8090", mux)
}
A mux is just a handler
It's easy to assume ListenAndServe expects a mux as its second argument, but it actually accepts any http.Handler. You can skip routing entirely:
type PoliteServer struct {
}
func (ms *PoliteServer) ServeHTTP(w http.ResponseWriter, req *http.Request) {
fmt.Fprintf(w, "Welcome! Thanks for visiting!\n")
}
func main() {
ps := &PoliteServer{}
log.Fatal(http.ListenAndServe(":8090", ps))
}
This server responds identically to every request, regardless of path or method. It can be simplified further with http.HandlerFunc:
func politeGreeting(w http.ResponseWriter, req *http.Request) {
fmt.Fprintf(w, "Welcome! Thanks for visiting!\n")
}
func main() {
log.Fatal(http.ListenAndServe(":8090", http.HandlerFunc(politeGreeting)))
}
The trick is this adapter type, which turns a function with the right signature into an http.Handler:
// The HandlerFunc type is an adapter to allow the use of
// ordinary functions as HTTP handlers. If f is a function
// with the appropriate signature, HandlerFunc(f) is a
// Handler that calls f.
type HandlerFunc func(ResponseWriter, *Request)
// ServeHTTP calls f(w, r).
func (f HandlerFunc) ServeHTTP(w ResponseWriter, r *Request) {
f(w, r)
}
http.ServeMux itself is an http.Handler. Internally, it keeps a sorted slice of pattern/handler pairs. Its ServeHTTP method finds the handler matching the request path and forwards the request to it. Because a mux delegates to other handlers, it's a form of middleware.
Middleware in practice
Middleware is easiest to understand by contrast. Without it, the flow from server to user handler is direct:
With middleware, an extra handler is inserted into the chain:
In Go, middleware is just a handler that wraps another handler: it can do work before calling the wrapped handler, after it, or both. ServeMux is one example; path-based routing is its preprocessing step, with no postprocessing.
A more typical example is logging middleware:
type LoggingMiddleware struct {
handler http.Handler
}
func (lm *LoggingMiddleware) ServeHTTP(w http.ResponseWriter, req *http.Request) {
start := time.Now()
lm.handler.ServeHTTP(w, req)
log.Printf("%s %s %s", req.Method, req.RequestURI, time.Since(start))
}
type PoliteServer struct {
}
func (ms *PoliteServer) ServeHTTP(w http.ResponseWriter, req *http.Request) {
fmt.Fprintf(w, "Welcome! Thanks for visiting!\n")
}
func main() {
ps := &PoliteServer{}
lm := &LoggingMiddleware{handler: ps}
log.Fatal(http.ListenAndServe(":8090", lm))
}
Here LoggingMiddleware is an http.Handler that stores the user handler as a field. When invoked, it records a timestamp, calls the wrapped handler, then logs the request and elapsed time.
Middleware composes naturally, since the wrapped handler can itself be middleware. The idiomatic signature is a function taking an http.Handler and returning a new one, usually leveraging a closure:
func politeGreeting(w http.ResponseWriter, req *http.Request) {
fmt.Fprintf(w, "Welcome! Thanks for visiting!\n")
}
func loggingMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
start := time.Now()
next.ServeHTTP(w, req)
log.Printf("%s %s %s", req.Method, req.RequestURI, time.Since(start))
})
}
func main() {
lm := loggingMiddleware(http.HandlerFunc(politeGreeting))
log.Fatal(http.ListenAndServe(":8090", lm))
}
The standard library includes middleware of this form, such as http.TimeoutHandler:
func TimeoutHandler(h Handler, dt time.Duration, msg string) Handler
Wrapping a handler with it layers on a timeout:
handler = http.TimeoutHandler(handler, 2 * time.Second, "timed out")
Middleware chains are straightforward to build by nesting calls:
handler = http.TimeoutHandler(handler, 2 * time.Second, "timed out") handler = loggingMiddleware(handler)
The same pattern applies internally in net/http—the serverHandler adapter mentioned earlier is middleware for handling the nil handler case. The composability of this design lets you keep business logic in your own handlers while orthogonal concerns live in reusable middleware.
Concurrency and panic recovery
Two additional behaviors are worth understanding when writing Go HTTP servers.
Concurrency. Each accepted connection is served in its own goroutine. This means a handler can block on I/O—a database query, for example—without stalling other requests. The trade-off is that shared state between handlers needs proper synchronization.
Panic handling. Since each connection runs in its own goroutine, a panic in a handler won't propagate to main—it would crash the process. Adding recover to main doesn't help either, because by the time it would run, the server has already stopped. For this reason, net/http installs a recovery mechanism per serving goroutine. Consider this handler:
func hello(w http.ResponseWriter, req *http.Request) {
fmt.Fprintf(w, "hello\n")
}
func doPanic(w http.ResponseWriter, req *http.Request) {
panic("oops")
}
func main() {
http.HandleFunc("/hello", hello)
http.HandleFunc("/panic", doPanic)
http.ListenAndServe(":8090", nil)
}
Requesting /panic produces this server log, while the connection is closed:
2021/02/16 09:44:31 http: panic serving 127.0.0.1:52908: oops goroutine 8 [running]: net/http.(*conn).serve.func1(0xc00010cbe0) /usr/local/go/src/net/http/server.go:1801 +0x147 panic(0x654840, 0x6f0b80) /usr/local/go/src/runtime/panic.go:975 +0x47a main.doPanic(0x6fa060, 0xc0001401c0, 0xc000164200) [... rest of stack dump here ...]
The server itself keeps running and continues to serve other requests. This built-in protection is better than a crash, but it's minimal: it only closes the connection and logs the error. Returning a meaningful error response to the client requires custom middleware—an exercise that's straightforward given the handler-wrapping pattern described above.
