Load balancing is the process of distributing incoming network traffic across multiple servers so no single server gets overwhelmed. When you visit a website or use an app, a load balancer sits between you and a group of servers, deciding which one should handle your request. The goal is simple: keep things fast, keep things running, and use every available server efficiently.
How a Load Balancer Works
Think of a load balancer like a traffic controller at an airport. Planes (your requests) arrive constantly, and the controller directs each one to an open gate (a server). Without that coordination, some gates would be packed while others sit empty.
In practice, a load balancer is a piece of software (or sometimes a physical device) that listens for incoming connections. When your browser sends a request to a website, it hits the load balancer first. The load balancer picks a server from a pool of available machines, forwards your request there, and sends the server’s response back to you. You never see this happen. From your perspective, you’re just talking to one website.
Common Distribution Algorithms
The rules a load balancer uses to pick a server are called algorithms. Three of the most common ones cover the vast majority of real-world setups.
- Round robin sends each new request to the next server in a fixed rotation. Server A, then B, then C, then back to A. It’s the simplest approach and works well when all your servers have similar capacity.
- Least connections is smarter. Instead of rotating blindly, it checks which server currently has the fewest active connections and sends the next request there. This helps when some requests take much longer to process than others.
- IP hash takes the visitor’s IP address and runs it through a formula that always maps to the same server. This means a returning visitor consistently lands on the same machine, which matters for certain applications that store session data locally.
Layer 4 vs. Layer 7 Load Balancing
Load balancers can operate at different levels of the networking stack, and the two you’ll encounter most are Layer 4 and Layer 7.
A Layer 4 load balancer works at the transport level. It sees IP addresses and port numbers, but it has no idea what’s inside the request. It doesn’t know if you’re loading a webpage, streaming video, or uploading a file. Because it skips all that inspection, it’s extremely fast and adds very little overhead. The tradeoff is that it can’t make intelligent decisions based on what you’re actually requesting.
A Layer 7 load balancer operates at the application level. It can read the full request: URLs, headers, cookies, even the type of content being requested. This lets it do things like route all image requests to one set of servers and all API calls to another, or direct mobile users to servers optimized for smaller payloads. The cost is higher CPU and memory usage, since inspecting every request takes more processing power. For most modern web applications, Layer 7 is the default choice because the routing flexibility is worth the overhead.
Health Checks and Fault Tolerance
A load balancer doesn’t just distribute traffic. It also monitors whether servers are actually healthy. It does this by sending periodic test requests, called health checks, to every server in the pool. If a server fails to respond (or responds with an error) a set number of times in a row, typically two consecutive failures by default, the load balancer pulls it out of rotation. No more traffic goes to that machine until it recovers.
Recovery works the same way in reverse. Once a failed server starts passing health checks again, usually requiring five consecutive successes, the load balancer adds it back to the pool automatically. This entire cycle happens without any manual intervention and without visitors noticing downtime. It’s one of the most important benefits of load balancing: if a server crashes at 3 a.m., the remaining servers absorb the traffic seamlessly.
SSL Termination
Encrypting and decrypting web traffic (the “S” in HTTPS) is computationally expensive. Every time a visitor connects securely, the server has to do significant math to establish and maintain that encrypted connection. When you multiply that by thousands of simultaneous visitors, encryption alone can eat up a large share of your server’s processing power.
Load balancers can handle this work instead, a technique called SSL termination. The load balancer decrypts incoming encrypted traffic, forwards the plain request to the backend server, receives the response, re-encrypts it, and sends it back to the visitor. The backend servers never touch the encryption at all, freeing them to focus entirely on running the application. This noticeably improves response times under heavy traffic.
Sticky Sessions
Some applications need a visitor to keep talking to the same server across multiple requests. If you’re filling out a multi-step form or have items in a shopping cart stored in server memory, getting bounced to a different server mid-session could lose your data. Sticky sessions solve this.
The load balancer assigns each visitor to a specific server, usually by setting a cookie in the browser or by tracking the visitor’s IP address. Every subsequent request from that visitor goes to the same server for a defined period of time. There are two flavors: duration-based, where the load balancer sets a time limit on the stickiness, and application-controlled, where the application itself decides how long the session should last. Sticky sessions are a practical necessity for older or stateful applications, though modern architectures increasingly store session data in shared databases instead, making stickiness unnecessary.
Hardware vs. Software Load Balancers
Hardware load balancers are dedicated physical appliances built specifically for the job. They use specialized processors optimized for high throughput and often include built-in security features like firewalls. They handle massive traffic volumes reliably, but they’re expensive upfront and difficult to scale, since scaling means buying more physical boxes.
Software load balancers run on standard servers or virtual machines. They cost far less, scale easily by spinning up additional instances, and integrate naturally with cloud environments. For most organizations today, especially those running cloud-based applications, software load balancers are the default choice. Hardware appliances still have a place in large enterprises with extremely high traffic demands and strict security requirements, but the industry has shifted decisively toward software.
Load Balancing Across Regions
Everything above describes balancing traffic within a single data center or server cluster. Global server load balancing (GSLB) takes this a step further by distributing traffic across data centers in different geographic regions. If your company has servers in New York, London, and Tokyo, GSLB directs each visitor to the closest or fastest location.
Simple approaches use DNS to rotate visitors across data centers somewhat randomly. Smarter techniques analyze real-time data, like network latency, to route each request to the server that will respond fastest. This reduces load times for users worldwide and provides disaster recovery: if an entire data center goes offline, traffic automatically shifts to the remaining locations.
Cloud Load Balancing Services
All three major cloud providers offer managed load balancing services that handle configuration, scaling, and health checks automatically. AWS provides Elastic Load Balancing, Microsoft offers Azure Load Balancer, and Google Cloud has Cloud Load Balancing. These services eliminate the need to set up and maintain your own load balancer, and they scale on demand as your traffic grows. For teams running applications in the cloud, managed load balancers are typically the fastest path from zero to production-ready infrastructure.