2. Understanding HTTP for Backend Engineers: Where It All Starts
1. Core Principles of HTTP
Definition
HTTP (HyperText Transfer Protocol) is an application-layer network protocol that powers communication across the World Wide Web. It operates on a standard Client–Server Model, where a client dispatches an intent-driven request and a server returns a structured response.
Statelessness: The Foundational Invariant
HTTP is inherently stateless. The server does not automatically retain memory or session context between successive requests from the exact same client.
Plaintext
Request 1 (Client -> Server) ===> Server processes & completes
Request 2 (Client -> Server) ===> Server has NO native memory of Request 1
Request 3 (Client -> Server) ===> Server treats this as a brand-new transactionPlaintext
+----------+ Request 1: GET /profile +-------------------------+
| | ---------------------------------> | |
| Client | Request 2: POST /checkout | Backend Server |
| (Browser | ---------------------------------> | |
| /App) | Request 3: GET /orders | [No Native Session RAM] |
| | ---------------------------------> | (Each request isolated) |
+----------+ +-------------------------+Because the server does not store who you are between requests, every single request must carry all the metadata and authorization credentials necessary to process it independently:
• Authentication tokens (Authorization: Bearer JWT)
• Session identifiers and Cookies (Cookie: session_id=xyz)
• Client metadata and device headers
Architectural Benefits of Statelessness
• Simpler Server Architecture: No shared session RAM tables required across machines.
• Effortless Horizontal Scaling: Any server instance behind a load balancer can handle any incoming request.
• Failure Resilience: If Server A crashes, the load balancer reroutes the next request to Server B without losing active user context.
2. Transport Layer: TCP vs UDP
HTTP is an application-layer protocol; it relies on transport-layer protocols to move packets across physical network wires.
Plaintext
+-------------------------------------------------------------+
| HTTP (Application Layer) |
+-------------------------------------------------------------+
| |
v v
+-------------------------------+ +-------------------------+
| TCP (Transmission) | | UDP (User) |
+-------------------------------+ +-------------------------+
| - Guaranteed Delivery (ACKs) | | - Connectionless |
| - In-Order Byte Stream | | - No Retransmission |
| - 3-Way Handshake Overhead | | - Near-Zero Latency |
+-------------------------------+ +-------------------------+
| |
(HTTP/1.0, 1.1, 2) (HTTP/3 via QUIC)TCP (Transmission Control Protocol)
Traditionally, HTTP (v1.0, v1.1, v2) runs on top of TCP: • Reliable Delivery: Every dropped packet is acknowledged (ACK) or automatically retransmitted.
• Ordered Data: Packets arriving out-of-order are reassembled in sequence before the application reads them.
• Connection Management: Requires an initial 3-Way Handshake (SYN → SYN-ACK → ACK) before sending any application bytes.
UDP (User Datagram Protocol)
• Fire-and-Forget: Fast, lightweight, with zero handshake delay.
• No Built-in Guarantees: Packets can arrive out-of-order or get lost in transit.
• Modern Evolution: HTTP/3 replaces TCP entirely by running over QUIC (a custom multiplexed, encrypted protocol built on top of UDP).
3. The Evolution of HTTP Versions
Plaintext
+---------------------------------------------------------------------------------------------------------+
| HTTP PROTOCOL EVOLUTION |
+---------------------------------------------------------------------------------------------------------+ [ HTTP/1.0 ] [ HTTP/1.1 ] [ HTTP/2 ] [ HTTP/3 ]
One TCP conn per req Persistent Connections Multiplexing Over TCP QUIC Protocol Over UDP
-------------------- ---------------------- --------------------- ----------------------
Req 1 -> [Open TCP] One TCP Connection Single TCP Connection Single UDP Connection
Res 1 -> [Close TCP] |-- Req 1 -> Res 1 (Wait) |-- Stream 1: Req/Res |-- Independent Streams
Req 2 -> [Open TCP] |-- Req 2 -> Res 2 (Wait) |-- Stream 2: Req/Res |-- No Transport HOL Block
Res 2 -> [Close TCP] \-- Req 3 -> Res 3 |-- Binary Framing |-- 0-RTT Handshakes
(Slow & Resource Heavy) (Head-of-Line Blocking) (TCP HOL Blocking on Loss) (Built-in Encryption)Plaintext
[HTTP/1.0 (1996)]
- New TCP connection opened and closed for every single request.
- High latency and excessive network overhead.
|
v
[HTTP/1.1 (1997)]
- Persistent connections (Keep-Alive).
- Pipelining introduced, but suffered from Application Head-of-Line (HoL) blocking.
|
v
[HTTP/2 (2015)]
- Binary framing and true multiplexing over a single TCP connection.
- HPACK header compression and Server Push.
|
v
[HTTP/3 (2022)]
- QUIC over UDP replaces TCP.
- Completely eliminates TCP-level Head-of-Line blocking.
- 0-RTT connection resumption and seamless network migration across IP changes.Protocol Trade-offs
VersionTransportKey FeatureMajor Bottleneck / LimitationHTTP/1.0TCPHeaders, status codes, multi-type assetsOpens/closes a brand-new TCP connection for every asset.HTTP/1.1TCPConnection: keep-alive, chunked transfer, cachingApplication Head-of-Line Blocking: Responses must return in the exact order requested.HTTP/2TCPBinary framing, multiplexed streams, HPACK compressionTCP Head-of-Line Blocking: If 1 packet drops, TCP halts all multiplexed streams until retransmitted.HTTP/3UDP (QUIC)Stream isolation, 0-RTT handshakes, connection migrationRequires UDP support across enterprise firewalls; higher server CPU usage during packet processing.
4. HTTP Message Anatomy
Every HTTP transaction consists of an explicit request and response separated by strict boundaries.
1. HTTP Request Structure
Plaintext
METHOD /path/resource HTTP-VERSION\r\n
Header-Name: Header-Value\r\n
Header-Name: Header-Value\r\n
\r\n <--- Mandatory Blank Line (CRLF)
[Optional Request Payload / Body]Raw Request Example:
HTTP
POST /api/v1/users HTTP/1.1
Host: api.example.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64)
Content-Type: application/json
Content-Length: 32
Authorization: Bearer eyJhbGciOi...{
"name": "Sumit",
"role": "admin"
}2. HTTP Response Structure
Plaintext
HTTP-VERSION STATUS-CODE STATUS-TEXT\r\n
Header-Name: Header-Value\r\n
Header-Name: Header-Value\r\n
\r\n <--- Mandatory Blank Line (CRLF)
[Optional Response Body / Payload]Raw Response Example:
HTTP
HTTP/1.1 201 Created
Date: Wed, 29 Oct 2026 16:40:00 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 42
Connection: keep-alive{
"id": 101,
"status": "user_registered"
}The Blank Line (CRLF): A strict protocol requirement. It indicates the exact boundary where header metadata terminates and the raw body payload begins.
5. HTTP Headers Classification
Headers are case-insensitive key-value pairs providing metadata that governs caching, security, encoding, and authentication.
Plaintext
HTTP HEADERS
|
+-------------------+---------------+---------------+-------------------+
| | | |
v v v v
Request Headers Response Headers Representation Security Headers
- User-Agent - Server - Content-Type - Strict-Transport-Security
- Authorization - Set-Cookie - Content-Length - Content-Security-Policy
- Accept - Access-Control-Allow-Origin - Content-Encoding - X-Frame-Options1. Request Headers
Dispatched by the client to convey its identity and expectations:
• User-Agent: Identifies client software, OS, and rendering engine.
• Authorization: Passes credentials (e.g., Bearer tokens, Basic auth).
• Accept: MIME types the client can parse (e.g., application/json, text/html).
2. General Headers
Apply broadly to both requests and responses, independent of the payload:
• Date: The timestamp at which the message was generated.
• Connection: Manages connection persistence (keep-alive or close).
• Cache-Control: Directives governing intermediate proxies and caches.
3. Representation Headers
Explicitly describe the format, encoding, and size of the body:
• Content-Type: MIME type of the payload (application/json, image/png, multipart/form-data).
• Content-Length: Exact size of the body in bytes.
• Content-Encoding: Compression algorithm applied (gzip, br, deflate).
4. Security Headers
Instruct browsers to enforce defensive boundaries:
• Strict-Transport-Security (HSTS): Forces the browser to communicate exclusively over HTTPS.
• Content-Security-Policy (CSP): Restricts origins from which scripts, images, and styles can execute.
• X-Frame-Options: Mitigates Clickjacking by preventing the site from being rendered inside an iframe.
6. HTTP Methods & Idempotency
HTTP methods define the semantic intent of the client operation.
Plaintext
HTTP METHODS
|
+-------------------+-------------------+
| |
v v
Safe & Idempotent Unsafe / Non-Idempotent
- GET (Fetch data) - POST (Create resource / action)
- HEAD (Fetch headers only) - PATCH (Partial update)
- PUT (Replace whole resource)
- DELETE (Remove resource)Method Summary Matrix
MethodPrimary PurposeCarries Request Body?Safe?Idempotent?GETRetrieve a resourceNoYesYesPOSTCreate a resource / trigger state mutationYesNoNoPUTCompletely replace a resource (or create if absent)YesNoYesPATCHPartially update specific fields of a resourceYesNoDepends / NoDELETERemove a designated resourceOptionalNoYesOPTIONSInspect supported methods and CORS capabilitiesNoYesYesHEADSame as GET, but returns headers without bodyNoYesYes
Understanding Idempotency
An operation is idempotent if executing it once produces the exact same server state as executing it multiple times consecutively:
State(f(x)) = State(f(f(x)))
Why POST is Non-Idempotent:
Plaintext
POST /api/orders --> Order #101 created ($50 charged)
POST /api/orders --> Order #102 created ($50 charged again! Duplicate charge)Why PUT is Idempotent:
Get Sumitkun’s stories in your inbox
Join Medium for free to get updates from this writer.
Plaintext
PUT /api/users/42 { "name": "Sumit" } --> User 42 name updated to "Sumit"
PUT /api/users/42 { "name": "Sumit" } --> User 42 name remains "Sumit" (No side effect change)Why DELETE is Idempotent:
Plaintext
DELETE /api/posts/99 --> Post 99 removed (200 OK or 204 No Content)
DELETE /api/posts/99 --> Post 99 already gone (404 Not Found, but server state unchanged)7. CORS: Cross-Origin Resource Sharing
The Security Foundation: Same-Origin Policy (SOP)
The browser enforces the Same-Origin Policy to protect users. A web application loaded from Origin A cannot access private responses from Origin B unless Origin B explicitly grants permission.
An Origin is defined strictly by three elements:
Origin = Protocol + Domain (Host) + Port
Plaintext
https://example.com:443/app/users
\___/ \_________/ \_/
| | |
Protocol Host PortOrigin Comparison Examples
Compared URLSame Origin as https://example.com:443?Reasonhttps://example.com/api/dataYESProtocol, host, and port match exactly.http://example.comNOProtocol mismatch (http vs https).https://api.example.comNOSubdomain mismatch (api.example.com vs example.com).https://example.com:8080NOPort mismatch (8080 vs 443).
8. CORS Request Types & The Preflight Flow
When JavaScript running on https://frontend.com makes an asynchronous request to https://api.backend.com, the browser evaluates whether the request is Simple or requires a Preflight Inspection.
Plaintext
Incoming Fetch / Axios Request
|
v
Does it meet Simple criteria?
- Method: GET, POST, or HEAD
- Headers: Standard safe headers only
- Content-Type: form-urlencoded, multipart, or text/plain
|
+-------------------+-------------------+
| YES | NO
v v
[ SIMPLE REQUEST ] [ PREFLIGHT REQUIRED ]
Dispatches directly to Browser dispatches HTTP OPTIONS
Backend Server Handshake before actual requestPreflight Request (OPTIONS)
If a request uses: • Methods like PUT, DELETE, or PATCH
• Custom headers like Authorization or X-Custom-Header
• Content-Type: application/json
The browser automatically halts the operation and dispatches an OPTIONS preflight request first:
Plaintext
Browser (frontend.com) Backend API (api.backend.com)
| |
| ------ Step 1: Preflight Handshake (OPTIONS) ---------> |
| Origin: https://frontend.com |
| Access-Control-Request-Method: DELETE |
| Access-Control-Request-Headers: Authorization |
| |
| <----- HTTP/1.1 204 No Content ------------------------ |
| Access-Control-Allow-Origin: https://frontend.com|
| Access-Control-Allow-Methods: GET, POST, DELETE |
| Access-Control-Allow-Headers: Authorization |
| Access-Control-Max-Age: 86400 |
| |
| ------ Step 2: Actual Request (DELETE /users/42) -----> |
| Authorization: Bearer token |
| |
| <----- HTTP/1.1 200 OK { "deleted": true } ------------ |Essential CORS Response Headers:
• Access-Control-Allow-Origin: Explicit domain allowed to read data (e.g., https://frontend.com or *).
• Access-Control-Allow-Methods: Permitted HTTP verbs (GET, POST, PUT, DELETE, OPTIONS).
• Access-Control-Allow-Headers: Allowed custom request headers (Authorization, Content-Type).
• Access-Control-Allow-Credentials: Required (true) when sharing cookies or auth tokens across origins.
• Access-Control-Max-Age: Caches the preflight response in the browser (in seconds) to avoid redundant OPTIONS calls.
Crucial Rule: CORS is enforced strictly by the browser engine. Backend tools (curl, Postman, server-side scripts) do not enforce CORS headers and bypass them completely.
9. HTTP Status Codes Classification
Status codes indicate the outcome of the client HTTP request:
• 1xx (Informational): Request received, process continuing.
• 2xx (Success): Action successfully received, understood, and accepted.
• 3xx (Redirection): Further action required to complete request.
• 4xx (Client Error): Request contains bad syntax or cannot be fulfilled by client fault.
• 5xx (Server Error): Server failed to fulfill an apparently valid request.
Essential Status Codes Reference
CodeStatus TextArchitectural Meaning100ContinueInitial request headers received; client should proceed to send request body.101Switching ProtocolsServer agrees to upgrade connection protocol (used in WebSocket handshakes).200OKStandard successful response for GET, PUT, or POST.201CreatedResource successfully created (standard response for POST). Returns resource location.204No ContentRequest succeeded, but response body is intentionally empty (common for DELETE/OPTIONS).301Moved PermanentlyResource relocated permanently. Search engines update SEO ranking to target URL.302Found (Temporary)Temporary redirect; clients should retain the original URL for future requests.304Not ModifiedCaching response. Cached resource is fresh; client loads from local storage.400Bad RequestMalformed JSON syntax, invalid query parameters, or client payload validation failure.401UnauthorizedAuthentication required or invalid/expired credentials provided.403ForbiddenServer recognizes identity, but client lacks necessary authorization/permissions.404Not FoundRequested URI path does not exist on the server.405Method Not AllowedThe endpoint exists, but does not support that HTTP verb (e.g., POST on a read-only route).409ConflictMutation conflicts with existing state (e.g., registering an already-used email address).429Too Many RequestsRate limit threshold breached. Usually accompanied by a Retry-After header.500Internal Server ErrorUnhandled exception or fatal crash inside backend application code.501Not ImplementedServer does not recognize the request method or lacks capacity to fulfill it.502Bad GatewayReverse proxy (Nginx) received an invalid response or crash signal from upstream app.503Service UnavailableServer is temporarily overloaded, down for scheduled maintenance, or failing health checks.504Gateway TimeoutUpstream application server failed to respond to the reverse proxy within the timeout window.
10. HTTP Caching & Conditional Requests
Caching eliminates redundant network traffic, lowers bandwidth consumption, and slashes response latency.
1. Freshness Lifetime: Cache-Control
Defines caching policy across browsers and intermediate CDNs:
• Cache-Control: max-age=3600 — Resource is fresh for 1 hour.
• Cache-Control: no-cache — Forces cache validation with the origin server before serving.
• Cache-Control: no-store — Never store any part of the request/response (used for banking and sensitive PII).
• Cache-Control: public, s-maxage=86400 — Instructs shared CDNs (Cloudflare/CloudFront) to cache for 24 hours.
2. Validation: ETags & Last-Modified
When a cached asset expires, the client validates if the underlying data actually changed:
Plaintext
Browser Origin Server
| |
| ------------- Initial Request (Cold Cache) -----------> |
| GET /app.js |
| |
| <------------ HTTP 200 OK ----------------------------- |
| ETag: "v1.2.4" |
| Last-Modified: Wed, 21 Oct 2026 07:28 GMT |
| [Full 2MB Payload] |
| |
| (Browser saves "v1.2.4" to local cache) |
| |
| ------------- Subsequent Request (Revalidation) ------> |
| GET /app.js |
| If-None-Match: "v1.2.4" |
| |
| [Case A: Unchanged] |
| <------------ HTTP 304 Not Modified ------------------- |
| (Zero payload bytes transferred!) |
| |
| [Case B: Changed] |
| <------------ HTTP 200 OK (New ETag: "v1.2.5") -------- |
| [New 2MB Payload] |• ETag (Entity Tag): A unique cryptographic hash/fingerprint of the resource content.
• 304 Not Modified: Saves network bandwidth by instructing the client to reuse its local cache without re-downloading the body.
11. Content Negotiation
Allows the client to request specific representations of a resource, while the server determines the appropriate format:
Plaintext
Content Negotiation Flow
|
Client specifies capabilities via Request Headers:
- Accept: application/json, text/xml
- Accept-Language: en-US, en;q=0.9, fr;q=0.8
- Accept-Encoding: gzip, deflate, br
|
v
Server matches preferences & returns Response Headers:
- Content-Type: application/json
- Content-Language: en-US
- Content-Encoding: br• MIME Type Negotiation: Accept: application/json vs Accept: text/csv.
• Locale Negotiation: Accept-Language: es requests localized Spanish data.
• Compression Negotiation: Accept-Encoding: gzip, br indicates support for Gzip and Brotli compression.
12. Compression: Gzip vs Brotli
Transferring large plain-text payloads (JSON, HTML, CSS, JavaScript) wastes bandwidth and increases latency.
Plaintext
Raw JSON Payload (26 MB)
|
v Gzip / Brotli Compression Algorithm
Compressed Payload (3.8 MB) ===> Transferred over wire (85% Bandwidth Reduction)
|
v Browser Decompresses on Arrival
Parsed JavaScript Object• Gzip: Universal compression supported by virtually all clients and servers.
• Brotli (br): Modern compression algorithm developed by Google; achieves 15–25% higher compression ratios than Gzip for text assets.
13. Handling Large Payloads
1. File Uploads: multipart/form-data
Used for transmitting binary assets (images, videos, PDF documents) alongside structured metadata:
HTTP
POST /api/upload HTTP/1.1
Host: api.example.com
Content-Type: multipart/form-data; boundary=---------------------------974767299852498929531610575-----------------------------974767299852498929531610575
Content-Disposition: form-data; name="description"Profile avatar
-----------------------------974767299852498929531610575
Content-Disposition: form-data; name="avatar"; filename="photo.png"
Content-Type: image/png<Binary Octet Stream Data...>
-----------------------------974767299852498929531610575--
Boundary: A unique delimiter string separating each distinct part of the payload, preventing binary collisions.
2. Large Downloads: Chunked Transfer Streaming
When downloading massive datasets or streaming real-time video, waiting to buffer the entire file before transmission causes high latency and memory spikes:
Plaintext
Server Memory (Streams chunks progressively)
|-- Chunk 1 (Size: 4KB) ----> Client receives immediately
|-- Chunk 2 (Size: 8KB) ----> Client processes
|-- Chunk 3 (Size: 16KB) ---> Client processes
\-- 0 (Zero Chunk) ---------> Transmission completesq• Uses Transfer-Encoding: chunked.
• Eliminates the requirement to compute Content-Length ahead of transmission, allowing real-time data streaming.
14. Transport Security: SSL, TLS & HTTPS
Plaintext
+-------------------------------------------------------------+
| HTTP (Application Messages) |
+-------------------------------------------------------------+
|
v
+-------------------------------------------------------------+
| TLS 1.3 (Encryption & Authentication) |
+-------------------------------------------------------------+
|
v
+-------------------------------------------------------------+
| TCP (Transport Connection) |
+-------------------------------------------------------------+HTTPS = HTTP + TLS (Transport Layer Security)
Core Guarantees of TLS:
• Confidentiality (Encryption): Protects packets in transit from eavesdropping (MITM attacks) using symmetric AES-GCM or ChaCha20 encryption.
• Integrity: Cryptographic message authentication codes (MAC) verify that packets are not modified or injected mid-flight.
• Authentication: Digital X.509 Certificates validate that the server is legitimately owned by the domain claim (preventing DNS spoofing).
15. Complete Architectural Summary & Mental Model
Plaintext
+---------------------------------------------------------------------------------------------------------+
| END-TO-END HTTP REQUEST LIFECYCLE |
+---------------------------------------------------------------------------------------------------------+
CLIENT (Browser / App)
|
| 1. Resolves DNS & establishes TCP 3-Way Handshake + TLS 1.3 Session
| 2. Checks CORS: Dispatches OPTIONS Preflight if custom headers or JSON payload
v
HTTP REQUEST SENT
|-- Request Line : METHOD (GET, POST, PUT, DELETE) + URL + HTTP/Version
|-- Headers : Host, Authorization, Content-Type, Accept-Encoding
|-- Blank Line : CRLF (\r\n) separator
\-- Body Payload : JSON / Form-Data / Binary stream
v
REVERSE PROXY & BACKEND APPLICATION
|-- Validates CORS Origins & Authenticates Authorization Header
|-- Executes Business Logic & Interacts with Database
|-- Formulates Status Code (2xx, 3xx, 4xx, 5xx) & Response Headers
v
HTTP RESPONSE RETURNED
|-- Status Line : HTTP/Version + Status Code (200 OK, 201 Created, 404 Not Found)
|-- Headers : Content-Type, Cache-Control, ETag, Content-Encoding: gzip
|-- Blank Line : CRLF (\r\n) separator
\-- Body Payload : JSON result / HTML string / Binary stream
v
CLIENT CACHE & RENDER
|-- Reuses cached resources on 304 Not Modified
\-- Renders data to user or completes mutationThe Senior Backend Engineer’s Mental Model
HTTP is a stateless, extensible message protocol built on text and binary streams. Because servers retain no native memory of clients, every request operates as an isolated transaction authenticated via headers. Backend engineers govern performance, security, and scale by leveraging connection persistence (HTTP/1.1 vs HTTP/2/3), idempotent method design, strict CORS validation, content compression, and deterministic caching with ETags.
