How does Google load balance requests?

The answer is Maglev.

Maglev is a L3/L4 software load balancer. A load balancer such as Nginx, and HAProxy distributes requests among multiple backend services.

Goals

It should allow seamless scalability so a set of load-balancers can be added/removed and the user does not notice any connection drop. A set of backend services can go on/off without affecting a large set of users.

Design

• Backend Service: A service has an assigned VIP.