Python CGI programming — the architecture, and a first program
CGI is the oldest way to make a web server run a program, and understanding it explains why every framework since has been built the way it is. One process per request is a beautifully simple model with exactly one fatal property.
- 2020-05-31
- Programmatic DIB
- Python · Web fundamentals · CGI
The model
A request arrives. The server starts a new process, hands it the request through environment variables and standard input, and reads the response off standard output. The process exits. Nothing is shared and nothing survives.
That isolation is the model's great virtue. There is no shared state to corrupt, a crash affects exactly one request, and the program is trivially easy to reason about because it starts clean every time.
Why nothing works this way any more
Process creation is expensive, and CGI pays that cost on every single request. Under load the server spends more time forking than it does answering. Everything that followed — FastCGI, mod_python, WSGI, ASGI and the application servers built on them — exists to keep the interpreter alive across requests and amortise the startup.
The trade is that state now persists between requests, which is where a large share of modern web bugs come from. CGI did not have that problem because it did not have that opportunity.
Why it is still worth understanding
The request/response contract CGI established — environment for metadata, stdin for the body, stdout for the response — is essentially still the interface WSGI and ASGI present, with the process boundary removed. Learning it makes the abstractions above it legible rather than magical.
- CGI runs one fresh process per request: perfect isolation, terrible throughput.
- Everything since exists to amortise interpreter startup across requests.
- Persistent state is the cost of that optimisation, and the source of new bug classes.
- The request/response contract it defined still shapes WSGI and ASGI.