GPT-Load

• In the Azure Portal, search the Marketplace for GPT-Load on Ubuntu 24.04 LTS by SecureAnyCloud and select Create.
• Choose your subscription, resource group and region.
• Set the VM size to Standard_B2s or larger.
• Set the authentication type to SSH public key and supply your key.
• Under Inbound port rules, allow SSH (22) and HTTP (80).
• On the Disks tab, choose StandardSSD_LRS for a good balance of cost and performance.
• Select Review + create, then Create.

First boot takes about a minute. During that minute, the appliance generates this VM's credentials and proves that its gateway rejects unauthenticated requests before it opens port 80.

az vm image terms accept --urn sac1647283583153:gpt-load-ubuntu-24-04:default:latest
az vm create \
  --resource-group my-resource-group \
  --name gpt-load-relay \
  --image sac1647283583153:gpt-load-ubuntu-24-04:default:latest \
  --size Standard_B2s \
  --storage-sku StandardSSD_LRS \
  --admin-username azureuser \
  --generate-ssh-keys \
  --public-ip-sku Standard
# then open the ports your applications need
az vm open-port \
  --resource-group my-resource-group \
  --name gpt-load-relay \
  --port 80 --priority 1001

In production, restrict that rule to your own address ranges. GPT-Load spends money on your behalf, so keep the set of networks that can reach port 80 as small as you can.

Connect over SSH:

ssh azureuser@<vm-ip>

Check the services:

sudo systemctl is-active gpt-load.service nginx.service

Expected output:

active active

The relay itself binds loopback only. nginx on port 80 is the sole ingress and the security gate. Check the listeners:

sudo ss -lntp | grep -E ':(80|3001)\b'

Expected output:

LISTEN 0 511 0.0.0.0:80 0.0.0.0:* users:(("nginx",pid=13533,fd=5),("nginx",pid=13532,fd=5)) LISTEN 0 4096 127.0.0.1:3001 0.0.0.0:* users:(("gpt-load",pid=13130,fd=6)) LISTEN 0 511 [::]:80 [::]:* users:(("nginx",pid=13533,fd=6),("nginx",pid=13532,fd=6))

The line for 127.0.0.1:3001 is the important one. GPT-Load is not reachable from the network at all, only through the gate.

nginx answers its own liveness literal without a credential, and it never touches the relay:

curl -s http://127.0.0.1/nginx-health

Expected output:

OK

Every credential on this appliance was generated for this VM on its first boot. Nothing is baked into the image. Read them from the credentials file:

sudo cat /root/gpt-load-credentials.txt

The file holds four values:

• GPTLOAD_GATE_USERNAME: the SecureAnyCloud gateway's HTTP Basic username, for a browser
• GPTLOAD_GATE_PASSWORD: the SecureAnyCloud gateway's HTTP Basic password, for a browser
• GPTLOAD_AUTH_KEY: the one credential your applications use, and the dashboard's Auth Key
• GPTLOAD_ENCRYPTION_KEY: encrypts your pooled provider keys at rest

Keep GPTLOAD_ENCRYPTION_KEY. Your pooled provider keys are encrypted with it inside the SQLite database. If you change it without running upstream's key migration, the stored pool becomes unreadable.

Browse to http://<vm-ip>/.

The SecureAnyCloud gateway challenges your browser first. Sign in with GPTLOAD_GATE_USERNAME and GPTLOAD_GATE_PASSWORD. The gateway then issues an HttpOnly, SameSite=Lax cookie that the dashboard's own requests use.

The GPT-Load dashboard then asks for its Auth Key. Paste GPTLOAD_AUTH_KEY.

The dashboard opens with nothing configured: no groups, no keys and no providers. This is deliberate, because the image never ships a provider key.

This appliance is an API key pool relay. An unauthenticated instance is therefore far worse than an unauthenticated web app. It would be a free proxy spending your provider credit, and it is machine-callable by design.

During this image's own build, the stock upstream build was measured, and these routes answer a completely anonymous request. The evidence ships on the instance:

sudo cat /usr/share/gpt-load/stock-exposure.txt

The SecureAnyCloud image therefore puts nginx in front, with default deny at server scope. Every route inherits the gate: the dashboard, the admin API, /health and every /proxy/* relay path. Exactly one location is unauthenticated, nginx's own static /nginx-health literal, which never touches the relay.

Prove it. Every one of these requests returns 401 when it carries no credential:

for p in / /health /api/settings /api/groups /proxy/default/v1/models; do printf '%s -> %s\n' "$p" "$(curl -s -o /dev/null -w '%{http_code}' "http://127.0.0.1$p")" done

Expected output:

/ -> 401 /health -> 401 /api/settings -> 401 /api/groups -> 401 /proxy/default/v1/models -> 401

One credential, five ways to send it

A plain password prompt would be wrong for this product for two reasons. A browser's stored HTTP Basic credential travels in the same Authorization header that the dashboard's own JavaScript overwrites. And an OpenAI, Anthropic or Google SDK needs that header for its API key.

So the gate accepts your one GPTLOAD_AUTH_KEY in every idiom GPT-Load itself accepts, plus the browser cookie and HTTP Basic:

KEY=$(sudo grep -m1 'GPTLOAD_AUTH_KEY=' /root/gpt-load-credentials.txt | cut -d= -f2-) for h in "Authorization: Bearer $KEY" "X-Api-Key: $KEY" "X-Goog-Api-Key: $KEY"; do printf '%-34s -> %s\n' "${h%%:*}" "$(curl -s -o /dev/null -w '%{http_code}' -H "$h" http://127.0.0.1/api/settings)" done printf '%-34s -> %s\n' "?key=" "$(curl -s -o /dev/null -w '%{http_code}' "http://127.0.0.1/api/settings?key=$KEY")"

Expected output:

Authorization -> 200 X-Api-Key -> 200 X-Goog-Api-Key -> 200 ?key= -> 200

These are the same idioms that GPT-Load's own authentication reads, so your application sends one credential that satisfies both the SecureAnyCloud gate and GPT-Load's own check. There is no double authentication and no SDK change.

Every comparison is an exact string match against a value generated for this VM. A one-character mutation is refused:

curl -s -o /dev/null -w '%{http_code}\n' -H "Authorization: Bearer ${KEY}x" http://127.0.0.1/api/settings

($KEY is still set from the previous block.)

Expected output:

401

The gate cannot be started unconfigured

nginx.service carries three ConditionPathExists guards and an ExecStartPre guard. First boot creates the first marker only after it has run a check against a throwaway, loopback-only listener that uses this VM's real gate map. The check proves that an unauthenticated request is answered with exactly 401 and that a one-character mutation of every credential is refused.

sudo systemctl cat nginx.service | grep -E 'ConditionPathExists|ExecStartPre'

Expected output:

ExecStartPre=/usr/sbin/nginx -t -q -g 'daemon on; master_process on;' ConditionPathExists=/var/lib/sac/data/gpt-load-gate.ready ConditionPathExists=/etc/nginx/gpt-load.htpasswd ConditionPathExists=/etc/nginx/conf.d/gpt-load-gate-map.conf ExecStartPre=/usr/local/sbin/gpt-load-preflight.sh --nginx

If you take the credential away, nginx refuses to start rather than serve an open relay. gpt-load.service is guarded the same way. It also refuses to start if AUTH_KEY is empty or looks like a default, or if the relay is configured to bind anything other than loopback.

A group is one provider plus a pool of keys for it.

• In the dashboard, open Keys and select Create Group.
• Give the group a name. The name becomes part of your endpoint URL, so keep it short and URL-safe, for example gemini.
• Choose the Channel Type that matches the provider: openai, gemini or anthropic.
• Set the Upstream Address to the provider's own API base, for example https://generativelanguage.googleapis.com for Gemini.
• Save the group, then paste your API keys into it, one per line or comma-separated. This is the pool.

The relay writes your keys to the SQLite store encrypted with this VM's GPTLOAD_ENCRYPTION_KEY. Confirm that the running relay's key matches its own store:

KEY=$(sudo grep -m1 'GPTLOAD_AUTH_KEY=' /root/gpt-load-credentials.txt | cut -d= -f2-)
curl -s -H "Authorization: Bearer $KEY" -H 'Accept-Language: en' \
  http://127.0.0.1/api/dashboard/encryption-status | python3 -m json.tool

Expected output. has_mismatch: false means the key the relay is running with is the key its stored data was encrypted with:

{ "code": 0, "message": "Success", "data": { "has_mismatch": false, "message": "", "scenario_type": "", "suggestion": "" } }

Keep the Accept-Language: en header on any curl request to this API. Without it, the relay answers in its default language.

Your endpoint is http://<vm-ip>/proxy/<your-group-name> and your API key is GPTLOAD_AUTH_KEY.

OpenAI Python SDK:

from openai import OpenAI client = OpenAI( base_url="http://<vm-ip>/proxy/<your-group-name>/v1", api_key="<GPTLOAD_AUTH_KEY>", ) resp = client.chat.completions.create( model="gpt-4o-mini", messages=[{"role": "user", "content": "Hello"}], ) print(resp.choices[0].message.content)

Anthropic Python SDK:

from anthropic import Anthropic client = Anthropic( base_url="http://<vm-ip>/proxy/<your-group-name>", api_key="<GPTLOAD_AUTH_KEY>", )

curl, against a Gemini group:

curl -s "http://<vm-ip>/proxy/<your-group-name>/v1beta/models/gemini-2.0-flash:generateContent" \
  -H "X-Goog-Api-Key: <GPTLOAD_AUTH_KEY>" \
  -H 'Content-Type: application/json' \
  -d '{"contents":[{"parts":[{"text":"Hello"}]}]}'

The OpenAI SDK sends Authorization: Bearer, the Anthropic SDK sends x-api-key and the Google SDK sends x-goog-api-key. The SecureAnyCloud gate accepts all three, and they are GPT-Load's native idioms, so no SDK needs modifying.

Note the path shape: everything after the group name is passed through to the provider unchanged. /proxy/gemini/v1beta/models/... becomes https://generativelanguage.googleapis.com/v1beta/models/... with one of your pooled keys attached.

Key management is the core of what GPT-Load does:

• Rotation. Each request takes the next key from the pool, so load is spread across every key you own rather than hammering one.
• Validation. Select Validate on a group. GPT-Load tests every key in the pool against the upstream and reports which are alive.
• Blacklisting. A key that fails repeatedly is moved out of rotation automatically, so a dead or rate-limited key stops poisoning your traffic.
• Restore. Restore all invalid puts blacklisted keys back into rotation, which is useful after a quota window resets.
• Aggregate groups. A group can contain sub groups with weights, so you can blend several providers behind one endpoint.

Every request is logged with its group, the key used, the status and the latency. Logs and Dashboard show request volume, success rate and per-key behaviour over time.

Everything GPT-Load stores is in one directory, on the OS disk, readable only by root:

sudo ls -la /var/lib/gpt-load/data/
sudo stat -c '%a %U:%G %n' /var/lib/gpt-load/data

Expected output:

700 root:root /var/lib/gpt-load/data

gpt-load.db is the whole application state: your groups, your encrypted key pool, the settings table and the request log. The instance has no database server and no cache server, so nothing else is listening and nothing else needs patching.

Back up the database by stopping the relay and copying the file. Back up /root/gpt-load-credentials.txt with it, because the database is useless without the encryption key:

sudo systemctl stop gpt-load.service
sudo tar czf /root/gpt-load-backup.tar.gz /var/lib/gpt-load/data /root/gpt-load-credentials.txt
sudo systemctl start gpt-load.service

GPT-Load is MIT licensed. The upstream licence ships on the instance:

head -3 /usr/share/gpt-load/LICENSE.upstream

Expected output:

MIT License Copyright (c) 2024 GPT-Load Contributors

The binary proves its own source tree. Go stamps the source commit into the executable, and this appliance ships the exact commit the upstream v1.4.9 tag points at, built from an unmodified working tree:

LC_ALL=C grep -a -o 'vcs\.revision=[0-9a-f]\{40\}' /usr/local/bin/gpt-load | head -1 LC_ALL=C grep -a -o 'vcs\.modified=[a-z]*' /usr/local/bin/gpt-load | head -1

Expected output:

vcs.revision=3028b1ff9d455e83340b1b8b2aa7123d5cb6a8ac vcs.modified=false

Every dependency compiled into that binary was licence checked at build time and the inventory ships with the image:

awk -F'\t' 'NR>2 {print $4}' /usr/share/gpt-load/dependency-licences.txt | sort | uniq -c | sort -rn

Expected output:

288 MIT 20 Apache-2.0 15 BSD 13 ISC 10 BSD-2-Clause 3 BSD-3-Clause 1 Python-2.0 1 MPL-2.0 1 (MIT OR CC0-1.0)

The build fails if any dependency declares AGPL, Affero, SSPL, BUSL, Business Source, Elastic (ELv2), Commons Clause, non-commercial, PolyForm, Prosperity or RSAL terms. It also fails closed on any dependency whose licence cannot be resolved. Everything in the image is MIT, Apache-2.0, BSD, ISC, MPL-2.0 or Python-2.0.

The binary links database client drivers (SQLite, MySQL, PostgreSQL, Redis), but this appliance installs and runs no database or cache server. That keeps the image clear of MongoDB's SSPL, Elasticsearch's ELv2 and the RSALv2 relicensing of Redis 7.4 and later:

sudo ss -lntH | awk '{print $4}' | sort -u

Expected output: the gate on 80, sshd on 22, GPT-Load on loopback 3001, and the platform's own DNS stub on loopback. Nothing else is listening at all:

0.0.0.0:22 0.0.0.0:80 127.0.0.1:3001 127.0.0.53%lo:53 127.0.0.54:53 [::]:22 [::]:80