# Load balancing with Nginx across several app servers

> Spread a site's traffic over several app servers with an Nginx load balancer in Vimonto Deploy: methods, weights, backup servers, HTTPS and the visitor's IP.

**Load balancing** spreads the visitors of one site over several app servers. A load balancer server sits in front: your domain points to it, it answers on HTTP and HTTPS, and Nginx passes every request on to one of the app servers behind it. If an app server stops answering, the others take over.

In Vimonto Deploy, a load balancer is a server of its own with a load-balanced site on it. On that site's **Load balancing** page you choose the app servers, how traffic is divided between them, and which ones only step in when the others are down.

![The Load balancing page of a site with the balancing method, two app servers and their weights](https://ops.vimonto.com/docs-media/en/site-load-balancing.webp?v=161e760d "A load-balanced site")

## How does load balancing work?

| Part | What it does |
|---|---|
| **Load balancer server** | A server of the type **Load balancer**, with only Nginx. Your domain's DNS points to it. |
| **Load-balanced site** | The site on the load balancer, made from the **Load balancer** preset. It has the domain, the certificate and the list of app servers, but no code. |
| **App servers** | App or web servers in your organization, each with a site for the same domain. They run your code and are deployed as usual. |

A visitor connects to the load balancer. Nginx there picks an app server, passes the request on with the original `Host` and `X-Forwarded-*` headers, and sends the answer back. The app servers see the visitor's IP address and whether the visitor used HTTPS, because their sites trust this load balancer (see [what the app servers need](#what-do-the-app-servers-need)).

## Set up a load balancer

1. [Create a server](https://ops.vimonto.com/docs/servers/create-a-server) of the type **Load balancer**. It gets only Nginx; see [server types](https://ops.vimonto.com/docs/servers/server-types).
2. Make sure each app server has a site for your domain, for example `shop.example.com`, and that it is deployed. Use the same domain or add it as an alias under [domains and SSL](https://ops.vimonto.com/docs/sites/domains-and-ssl).
3. On the load balancer's **Sites** page, click **New site** and choose **Load balancer** under **Load balancing**.
4. Enter the same domain with **Use a custom domain** and click **Create site**. A load-balanced site has no repository, no database and no deploys.

> [!NOTE]
> The load balancer needs a domain of your own before you can add app servers: the sites on your app servers cannot answer for an `on-deploy.link` name. Until it has one, the Load balancing page says **Add a domain first**. Once it has one, requests that come in on its `on-deploy.link` address still work: the load balancer passes them on with your own domain as the `Host`.
5. Open the new site's **Load balancing** page and add your app servers (see below).
6. Point the domain's DNS to the load balancer's IP address, and turn on HTTPS on the load balancer's site under [domains and SSL](https://ops.vimonto.com/docs/sites/domains-and-ssl).

Until you add an app server, the load balancer answers every request with `503 Service Unavailable`, and the site's overview says **No app servers yet. Visitors get a 503 until you add one.**

A load balancer server only holds load-balanced sites, and the **Load balancer** preset only goes on a load balancer server: the form lists only the servers that fit.

## Add an app server

1. On the site's **Load balancing** page, click **Add server**.
2. Choose the **Server**. The list holds the active app and web servers of your organization that you have access to.
3. Set the **Port** the app server's site listens on: `80` by default.
4. Set the **Weight**, from 1 to 100: a server with weight 3 gets three times as many requests as one with weight 1.
5. Switch on **Backup server** if this server should only get traffic when the other servers are down.
6. Click **Add server**.

The load balancer reaches the app server over the [private network](https://ops.vimonto.com/docs/servers/network) when both servers are on the same one, and over its public IP address otherwise. The **Servers** list shows each app server with the address and port the load balancer uses, its weight and a **Backup** badge.

A server can be added more than once, on different ports.

## Edit an app server

Click **Edit** next to the server, change its **Port**, **Weight** or **Backup server** setting and click **Save**. The server itself stays the same; to use another one, add it and take this one out. The change is applied right away.

## Choose the balancing method

Under **Method**, click one of the three options. The change is applied right away.

| Method | How Nginx picks an app server |
|---|---|
| **Round robin** (default) | Each server in turn, by weight. |
| **Least connections** | The server with the fewest open connections, for requests that take very different amounts of time. |
| **IP hash** | Each visitor sticks to one server, by IP address. Use it only when your app keeps something on one server. |

Nginx does not accept backup servers together with **IP hash**, so Vimonto Deploy does not let you combine them. With IP hash chosen, **Backup server** cannot be switched on and the form says **IP hash can't use backup servers. Choose another method first.** With a backup server in the list, choosing IP hash is refused with **IP hash can't use backup servers**: edit your backup servers first so they are no longer backups.

## What happens when an app server goes down?

The load balancer checks the app servers through the requests it sends; there are no separate health checks. When a request to an app server fails with a connection error, a timeout or a `502`, `503` or `504`, Nginx tries the next server, so the visitor still gets an answer. After 3 failures within 10 seconds, the server gets no traffic for 10 seconds, and then Nginx tries it again.

Backup servers only get traffic when every other server is unavailable.

## Take an app server out

Click **Take out** next to the server and confirm. The load balancer stops sending traffic to it, and the sites on that server stop trusting the load balancer (see below). Nothing else changes on the server, and its site keeps working on its own address.

The same cleanup happens on its own when you delete the load-balanced site or the load balancer server: the sites on the app servers stop trusting it. When you delete an app server, the load balancer stops sending traffic to it.

## What do the app servers need?

Every change on the Load balancing page also writes the Nginx configuration of the sites on the app servers that serve the same domain. Those sites then trust the load balancer, and only it:

- **The visitor's IP address.** Nginx takes the address from `X-Forwarded-For`, but only when the request comes from the load balancer (`set_real_ip_from`). Your app sees the visitor's IP address as the remote address, without proxy settings of its own.
- **HTTPS.** When the load balancer received the request over HTTPS, PHP gets `HTTPS=on` and a Node.js app gets `X-Forwarded-Proto: https`. Laravel and other frameworks then build `https://` links without trusted-proxy configuration.

Requests from anywhere else cannot fake these headers. Taking a server out, or deleting the load-balanced site or the load balancer, removes the trust again. A site with a [custom Nginx configuration](https://ops.vimonto.com/docs/sites/nginx#what-changes-when-you-use-your-own-configuration) is left alone; add these lines yourself.

Your app also has to work on several servers at once:

- Keep **sessions and cache** in Redis or the database, on a shared [database or cache server](https://ops.vimonto.com/docs/servers/server-types), not in files on one server. For Laravel that means `SESSION_DRIVER` and `CACHE_STORE` set to `redis` or `database` in the [.env](https://ops.vimonto.com/docs/sites/environment) of every app server, pointing to the same Redis or database.
- Keep **uploads** in object storage such as S3, not in `storage/` of one server.
- **Deploy** each app server's site. Every app server has its own release and deploy, so deploy them all when you release a new version.
- Run the **scheduler** on one app server only, unless your scheduled tasks are safe to run more than once.

HTTPS belongs on the load balancer, which talks plain HTTP to the app servers. A site on an app server may still have a certificate of its own, for example to reach it directly: it redirects plain HTTP to HTTPS for everyone except the load balancer, which it serves over plain HTTP, so there is no redirect loop.

## HTTPS on the load balancer

The load-balanced site has its own **Domains and SSL** page. Because your domain points to the load balancer, a free Let's Encrypt certificate is requested there, as for any site, and renews automatically. The load balancer then answers on HTTPS, redirects HTTP to HTTPS and passes the requests on to the app servers over HTTP, with `X-Forwarded-Proto: https`.

Traffic between the load balancer and the app servers is not encrypted. Put them on the same [private network](https://ops.vimonto.com/docs/servers/network) so it stays inside your provider's network.

## What the Nginx configuration looks like

The load-balanced site's configuration has an `upstream` block with the app servers and a `location` that passes requests to it:

```nginx
upstream site_7_servers {
    least_conn;
    server 10.0.0.3:80 weight=3 max_fails=3 fail_timeout=10s;
    server 10.0.0.4:80 max_fails=3 fail_timeout=10s backup;
    keepalive 32;
}

server {
    # listen, server_name, certificate and logs as for every site

    location / {
        proxy_pass http://site_7_servers;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Port $server_port;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade_keepalive;
        proxy_next_upstream error timeout http_502 http_503 http_504;
        proxy_read_timeout 120;
    }
}
```

Connections to the app servers are kept open and reused, and WebSocket connections are passed through. Every change is written as a background task, tested with `nginx -t` and only then loaded; if Nginx rejects it, the previous configuration stays active. See [Nginx](https://ops.vimonto.com/docs/sites/nginx).

If you saved a configuration of your own on the load-balanced site's Nginx page, changes on the Load balancing page are saved but not applied: you see **Saved, not applied**. Restore the default configuration on the Nginx page to use them.

## What else does a load-balanced site have?

The sidebar of a load-balanced site shows **Overview**, **Load balancing**, **Domains and SSL**, **Logs**, **Nginx** and **Settings**. It has no Deployments or Environment page, because it has no code. Its **Settings** page only has **Delete site**: there is no web directory, deploy setting or isolation to choose. The site header shows how many app servers it passes traffic to.

- **Logs** shows the load balancer's own Nginx error and request logs. Your app's logs are on the sites of the app servers.
- **Password protection**, in the framework menu of the site header, protects every app server behind the load balancer at once. It is the only [site feature](https://ops.vimonto.com/docs/sites/site-features) of a load-balanced site.

Every member can see the Load balancing page. Adding, editing and taking out servers and changing the method need the permission to manage sites; see [members and roles](https://ops.vimonto.com/docs/organization/members-and-roles). Each change is recorded in the [audit log](https://ops.vimonto.com/docs/organization/audit-log).

## Frequently asked questions

### How many app servers can I put behind a load balancer?

As many as you like. Each one is a line in the `upstream` block.

### Do I need sticky sessions?

Not when sessions are kept in Redis or the database, which every app server reads. Then **Round robin** or **Least connections** works best. Use **IP hash** only when something really lives on one server.

### Can I use one load balancer for several domains?

Yes. Create one load-balanced site per domain on the load balancer, each with its own app servers.

### Why does my domain show as pointing elsewhere on the app servers?

Because it points to the load balancer, as it should. Only the load balancer's site needs the DNS record and the certificate.