Grafana Alloy 1.21.3

In the AWS Marketplace listing, choose Continue to Subscribe, then Continue to Configuration. Pick Grafana Alloy 1.21.3 on Ubuntu 24.04 as the software version and select your Region. Choose Launch through EC2.

Select m5.large (or another type that fits your traffic), your key pair and a security group as above. Alloy keeps its write-ahead log and the shipped pipeline's output file under /var/lib/alloy on the root volume, so 20 GiB is a sensible start.

To launch with the AWS CLI instead, replace the values in angle brackets:

aws ec2 run-instances --region <region> --image-id <ami-id-from-the-listing> \
  --instance-type m5.large --key-name <your-key-pair> \
  --security-group-ids <your-security-group> --subnet-id <your-subnet> \
  --block-device-mappings 'DeviceName=/dev/sda1,Ebs={VolumeSize=20,VolumeType=gp3}' \
  --metadata-options HttpTokens=required,HttpEndpoint=enabled \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=alloy}]'

First boot takes a few seconds once the instance is running. Alloy and nginx start straight after it.

Connect over SSH as the ubuntu user, the SSH user for Grafana Alloy 1.21.3 on Ubuntu 24.04:

ssh -i <your-key.pem> ubuntu@<public-ip>

Confirm that first boot has finished and that all four services are running. First boot generates the two passwords and the certificate, then starts Alloy and nginx:

systemctl is-active alloy nginx alloy-firstboot sac-alloy-egress
sudo test -f /var/lib/sac/data/alloy-firstboot.done && echo FIRSTBOOT_DONE

All four units are active and the first-boot marker is present:

active active active active FIRSTBOOT_DONE

List the listening ports. Only SSH and nginx listen on network ports, with one nginx entry per nginx process (the master and one worker per vCPU). Alloy listens on the loopback address only:

sudo ss -lntp | awk 'NR>1 {print $4, $6}' | grep -v ':53 ' | sed -E 's/,pid=[0-9]+,fd=[0-9]+//g' | sort

0.0.0.0:22 users:(("sshd"),("systemd")) 0.0.0.0:443 users:(("nginx"),("nginx"),("nginx")) 0.0.0.0:80 users:(("nginx"),("nginx"),("nginx")) 127.0.0.1:12345 users:(("alloy")) 127.0.0.1:4317 users:(("alloy")) 127.0.0.1:4318 users:(("alloy")) [::]:22 users:(("sshd"),("systemd")) [::]:443 users:(("nginx"),("nginx"),("nginx")) [::]:80 users:(("nginx"),("nginx"),("nginx"))

Run the health check that nginx serves without a login, then Alloy's own readiness check on the loopback address. Both are reachable from your administrator shell:

curl -sk https://127.0.0.1/healthz
sudo curl -s http://127.0.0.1:12345/-/ready

ok Alloy is ready.

Print the credentials file as root:

sudo cat /var/lib/sac/data/alloy-credentials.txt

The file is readable by root only. Its contents look like this:

# Grafana Alloy 1.21.3 - generated on first boot by alloy-firstboot.service. Both passwords are unique to this
# instance; neither was baked into the image. Keep this file secret.
# The Alloy UI (component graph, health, live debugging): HTTP Basic login ALLOY_UI_URL=https://<public-dns-name>/ ALLOY_UI_USER=admin ALLOY_UI_PASSWORD=<ALLOY_UI_PASSWORD>
#
# OTLP/HTTP ingest for instrumented applications and OpenTelemetry exporters (HTTP Basic login 'otlp'):
# POST ${OTLP_HTTP_ENDPOINT}/v1/traces | /v1/metrics | /v1/logs
# OpenTelemetry SDKs: OTEL_EXPORTER_OTLP_ENDPOINT=<OTLP_HTTP_ENDPOINT>, OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf,
# OTEL_EXPORTER_OTLP_HEADERS=Authorization=<OTLP_AUTHORIZATION_HEADER, with the space as %20>. The certificate is
# self-signed: give exporters OTLP_CA_CERTIFICATE (OTEL_EXPORTER_OTLP_CERTIFICATE) or use your own certificate. OTLP_HTTP_ENDPOINT=https://<public-dns-name>/otlp OTLP_USER=otlp OTLP_PASSWORD=<OTLP_PASSWORD> OTLP_AUTHORIZATION_HEADER=<OTLP_AUTHORIZATION_HEADER> OTLP_CA_CERTIFICATE=/etc/ssl/alloy/alloy.crt
#
# Alloy's configuration: /etc/alloy/config.alloy (reload:
sudo systemctl reload alloy). Received telemetry is written to
# /var/lib/alloy/otlp/telemetry.json until you point the pipeline at your own backend.
# New UI password:
sudo alloy-ui-password New ingest password:
sudo alloy-otlp-password
# Self-test:
sudo alloy-selftest Change the served name:
sudo alloy-set-hostname <your-domain>
# User guide: https://secureanycloud.com/stacks/grafana-alloy-1-21-3/

OTLP_AUTHORIZATION_HEADER is the ready-made Basic ... value of the otlp login. Use it with exporters that take a header rather than a user name and password.

The UI is served under the instance's public DNS name, taken from the EC2 instance metadata. The certificate also names the public and private IP addresses and the private DNS name. UI requests to the IP addresses redirect to the name. OTLP ingest is accepted on the IP addresses too, so senders inside your VPC can use the private address.

If you stop and start the instance without an Elastic IP, the boot service picks up the new name and issues a new certificate. To serve Alloy under your own DNS name, see Step 10.

Open the URL in your browser. The browser warns once about the self-signed certificate. Accept it, or install your own certificate (Step 10). When the browser asks, sign in as admin with the password from the credentials file.

The UI opens on the list of components in the pipeline (see the screenshot at the top of this guide). Choose Graph to see how they connect, from the OTLP receiver through the batch processor to the file exporter:

Before you connect your own applications, send one span the way an OpenTelemetry SDK does: OTLP/HTTP with JSON, through nginx on port 443 with the otlp login. The command reads the login from the credentials file into a header file descriptor, so it never appears on a command line. It then reads the span back from the file the shipped pipeline writes:

TRACE_ID="$(openssl rand -hex 16)"; SPAN_ID="$(openssl rand -hex 8)"; NOW="$(date +%s%N)" cat > /tmp/alloy-span.json <<EOF {"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"checkout"}}]}, "scopeSpans":[{"scope":{"name":"sac-guide"},"spans":[{"traceId":"${TRACE_ID}","spanId":"${SPAN_ID}", "name":"GET /cart","kind":2,"startTimeUnixNano":"$((NOW - 150000000))","endTimeUnixNano":"${NOW}"}]}]}]} EOF curl -sk -o /dev/null -w 'HTTP %{http_code}\n' -X POST -H 'Content-Type: application/json' \
  -H @<(sudo sed -n 's/^OTLP_AUTHORIZATION_HEADER=/Authorization: /p' /var/lib/sac/data/alloy-credentials.txt) \
  --data-binary @/tmp/alloy-span.json https://127.0.0.1/otlp/v1/traces sleep 3
sudo grep -F "${TRACE_ID}" /var/lib/alloy/otlp/telemetry.json | grep -oE '"(stringValue":"checkout|name":"GET /cart)"' | sort -u

nginx accepts the span with the login, Alloy batches it and the file exporter writes it out:

HTTP 200 "name":"GET /cart" "stringValue":"checkout"

Without the login, or with the UI login, nginx refuses the same request before it reaches Alloy:

curl -sk -o /dev/null -w '%{http_code}\n' -X POST -H 'Content-Type: application/json' \
  --data-binary @/tmp/alloy-span.json https://127.0.0.1/otlp/v1/traces
curl -sk -o /dev/null -w '%{http_code}\n' https://127.0.0.1/otlp/v1/traces

401 405

The second request is a GET, and the ingest accepts only POST.

In the UI, choose View on a component to open its page, which shows its health, the arguments it runs with and what it exports. The OTLP receiver listens on the loopback address only:

To watch data live, open otelcol.processor.batch.sac and choose Live debugging. Send the test trace again, and each span appears in the page as it passes through the processor.

Point applications and collectors at https://<public-dns-name>/otlp and have them send OTLP/HTTP with the otlp login. You can use the private IP address or your own DNS name instead, as long as the certificate names it. Copy the instance's certificate, /etc/ssl/alloy/alloy.crt, to the senders so they can verify it, or install a certificate your systems already trust (Step 10).

OpenTelemetry SDKs (Java, Python, Go, Node.js, .NET and others) read the standard environment variables. Use OTLP_AUTHORIZATION_HEADER from the credentials file as the header value, with its space written as %20:

OTEL_SERVICE_NAME=my-service OTEL_EXPORTER_OTLP_ENDPOINT=https://<public-dns-name>/otlp OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf OTEL_EXPORTER_OTLP_HEADERS=Authorization=Basic%20<base64-of-otlp:password> OTEL_EXPORTER_OTLP_CERTIFICATE=/etc/ssl/certs/alloy.crt

For an OpenTelemetry Collector or another Alloy, use the otlphttp exporter with the otlp login:

extensions: basicauth/alloy: client_auth: username: otlp password: ${env:ALLOY_OTLP_PASSWORD} exporters: otlphttp/alloy: endpoint: https://<public-dns-name>/otlp auth: authenticator: basicauth/alloy tls: ca_file: /etc/otelcol/alloy.crt service: extensions: [basicauth/alloy] pipelines: traces: receivers: [otlp] exporters: [otlphttp/alloy]

The ingest refuses a request that carries browser signals (an Origin, Referer, Sec-Fetch-* header or a cookie) with HTTP 403. It is for machine clients only, so a web page cannot post telemetry on a visitor's behalf. Failed logins from one address are rate limited (HTTP 429 after a burst).

Keep the password in a secrets store or a file only the sender can read. Never put it in source code or in an image you publish.

The shipped pipeline in /etc/alloy/config.alloy receives OTLP, batches it and writes it to /var/lib/alloy/otlp/telemetry.json (100 MB per file, at most six files). To send telemetry on, replace the otelcol.exporter.file block with an exporter for your backend and point the batch processor's outputs at it.

This example sends traces to Grafana Tempo (or any OTLP endpoint) and metrics to a Prometheus-compatible remote write endpoint such as Mimir:

otelcol.processor.batch "sac" { timeout = "1s" output { metrics = [otelcol.exporter.prometheus.backend.input] traces = [otelcol.exporter.otlp.tempo.input] } } otelcol.exporter.otlp "tempo" { client { endpoint = "tempo.example.internal:4317" } } otelcol.exporter.prometheus "backend" { forward_to = [prometheus.remote_write.mimir.receiver] } prometheus.remote_write "mimir" { endpoint { url = "https://mimir.example.internal/api/v1/push" } }

Edit the file as root. It must stay root:alloy with mode 0640, which the pre-start check requires. Then check it with alloy validate, using the same stability level the service runs at, and reload Alloy:

sudo -u alloy /usr/local/bin/alloy validate --stability.level=public-preview /etc/alloy/config.alloy && echo CONFIG_OK
sudo systemctl reload alloy sleep 2
sudo curl -s http://127.0.0.1:12345/-/ready

CONFIG_OK Alloy is ready.

If the reload is refused, Alloy keeps running the previous configuration and logs the reason (journalctl -u alloy -n 20). The UI shows a component that fails as unhealthy, with its error. Backends on a private address need an egress allowance first (Step 9). The Alloy components reference lists every exporter, processor and receiver, including loki.write for logs and the Prometheus scrape components.

OTLP over HTTP on port 443 works with every OpenTelemetry SDK and collector. If your senders must use OTLP over gRPC, have nginx forward the three OTLP gRPC services to Alloy's gRPC receiver behind the same otlp login. The command adds one location to the ingest configuration that both HTTPS servers include:

sudo tee -a /etc/nginx/alloy-ingest.conf >/dev/null <<'EOF'
# OTLP/gRPC ingest (traces, metrics, logs) with the 'otlp' login, added from the user guide (Step 8) location ~ ^/opentelemetry\.proto\.collector\.(trace|metrics|logs)\.v1\.(Trace|Metrics|Logs)Service/Export$ { if ($alloy_ingest_browser) { return 403; } auth_basic "Grafana Alloy OTLP ingest"; auth_basic_user_file /etc/nginx/alloy-otlp.htpasswd; client_max_body_size 20m; grpc_set_header Authorization ""; grpc_pass grpc://127.0.0.1:4317; } EOF
sudo nginx -t 2>&1 | tail -n 1 && sudo systemctl reload nginx

nginx: configuration file /etc/nginx/nginx.conf test is successful

Check it with an empty export request, which is the gRPC framing of an empty message. With the login, nginx forwards the request to Alloy, which answers 200 with grpc-status: 0. Without it, nginx answers 401 and gRPC clients report UNAUTHENTICATED:

printf '\0\0\0\0\0' > /tmp/alloy-empty.grpc curl -sk --http2 -D - -o /dev/null -X POST -H 'Content-Type: application/grpc' -H 'TE: trailers' \
  -H @<(sudo sed -n 's/^OTLP_AUTHORIZATION_HEADER=/Authorization: /p' /var/lib/sac/data/alloy-credentials.txt) \
  --data-binary @/tmp/alloy-empty.grpc \
  https://127.0.0.1/opentelemetry.proto.collector.trace.v1.TraceService/Export | grep -iE '^(HTTP/2|grpc-status)' | tr -d '\r' curl -sk --http2 -o /dev/null -w '%{http_code}\n' -X POST -H 'Content-Type: application/grpc' \
  --data-binary @/tmp/alloy-empty.grpc https://127.0.0.1/opentelemetry.proto.collector.trace.v1.TraceService/Export

HTTP/2 200 grpc-status: 0 401

Senders then use <public-dns-name>:443 as a gRPC endpoint with TLS and basic auth. In the OpenTelemetry Collector, use the otlp exporter with the same basicauth/alloy authenticator as in Step 6. In SDKs, set OTEL_EXPORTER_OTLP_PROTOCOL=grpc with the endpoint https://<public-dns-name>:443 and the same Authorization header.

To close the gRPC ingest again, delete the block from /etc/nginx/alloy-ingest.conf and reload nginx.

The alloy account cannot open connections to the instance metadata service, link-local addresses or any private network, including your VPC. Public endpoints such as Grafana Cloud need no change. If Alloy sends to a backend or scrapes a target on a private address, add that range first, one CIDR per line:

echo '10.0.12.0/24' | sudo tee -a /etc/sac/alloy-egress-allow
sudo systemctl reload sac-alloy-egress

The file accepts only private ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 100.64.0.0/10 and fc00::/7). Metadata and link-local addresses can never be allowed.

Point a DNS record at the instance (an Elastic IP keeps the address stable), then set the name Alloy is served under. The command creates a new self-signed certificate for the name, updates nginx, the cross-site and cross-origin guards and the credentials file, and keeps the name across reboots:

sudo alloy-set-hostname <your-domain>

Give senders the new /etc/ssl/alloy/alloy.crt, or update their endpoint if they used the old name. To use a certificate from a public or private certificate authority instead, copy it over the generated pair and reload nginx. Running alloy-set-hostname again replaces it with a self-signed one:

sudo install -m 0644 <your-domain>.crt /etc/ssl/alloy/alloy.crt
sudo install -m 0600 <your-domain>.key /etc/ssl/alloy/alloy.key
sudo nginx -t && sudo systemctl reload nginx

Run this command to set a new UI password:

sudo alloy-ui-password

The command generates a new 24-character password for the UI login admin, keeps only its bcrypt hash in nginx's login file, writes the password to the credentials file and reloads nginx. Your browser asks for the new password on the next request.

To issue a new ingest password, for example when someone who knew it leaves, run sudo alloy-otlp-password. Every sender still using the old password is refused from then on, so update OTLP_AUTHORIZATION_HEADER (or the password) on each one. Read the new values from the credentials file (Step 4).

Run the shipped self-test. It signs in to the UI through nginx and proves that no login, a wrong password and the ingest login are refused. It checks that the ingest refuses no login, a wrong password, the UI login, a GET and a browser request. It sends a test span with the ingest login and finds it in the file exporter's output.

It also checks the cross-site, cross-origin and unknown-host guards, that Alloy listens on the loopback address only, and that the instance metadata service cannot be reached from inside Alloy or nginx. Passwords reach curl only through root-only temporary files, never a command line:

sudo alloy-selftest

SELFTEST-OK https 301 + healthz + Alloy UI 401 none/wrong/ingest login, 200 per-instance admin + ready + otlp->batch->file pipeline healthy + OTLP ingest 401 none/wrong/UI login, 405 GET, 403 browser, 200 ingest login + span <id> in the file exporter + 403 cross-site /api and /-/reload + 403 cross-origin POST + 444 unknown host + loopback-only alloy (12345/4317/4318) + metadata 000 inside alloy and nginx (control 200)

The metadata block is a systemd setting on each service:

systemctl show -p IPAddressDeny alloy nginx

IPAddressDeny=fd00:ec2::254/128 168.63.129.16/32 169.254.169.254/32 IPAddressDeny=fd00:ec2::254/128 168.63.129.16/32 169.254.169.254/32

Usage reporting is off, and Alloy's UI and API, support bundle and profiling endpoints are fixed by its start command:

systemctl show -p ExecStart alloy | grep -oE -- '--(disable-reporting|server\.http\.[a-z.-]+(=[^ ;]+)?)'

--server.http.listen-addr=127.0.0.1:12345 --disable-reporting --server.http.disable-support-bundle --server.http.enable-pprof=false

A request for a host name the instance does not serve gets no answer. nginx closes the connection, so curl exits with code 52:

curl -sk --http1.1 -o /dev/null -w '%{http_code}\n' -H 'Host: attacker.example' https://127.0.0.1/healthz; echo "curl exit $?"

000 curl exit 52

Confirm that the certificate was generated on this instance for its own names and addresses:

sudo openssl x509 -in /etc/ssl/alloy/alloy.crt -noout -subject -ext subjectAltName

subject=CN = <public-dns-name>, O = sac X509v3 Subject Alternative Name: DNS:<public-dns-name>, IP Address:<public-ip>, IP Address:<private-ip>, DNS:<private-dns-name>, IP Address:127.0.0.1, DNS:localhost

Print the installed versions and the build record:

/usr/local/bin/alloy --version | head -n 1 nginx -v 2>&1 grep -E '^(Grafana Alloy|build) ' /usr/share/doc/sac-alloy/BUILD-INFO.txt | cut -c1-70

alloy, version v1.21.3 (branch: HEAD, revision: f2108cd) nginx version: nginx/1.24.0 (Ubuntu) Grafana Alloy 1.21.3 (commit f2108cd783accbd491b24843484c33abba6da build Go 1.27.3 (go1.27.3.linux-amd64.tar.gz sha256 65efbc

Key files and what they hold:

• /usr/local/bin/alloy: Grafana Alloy, run by alloy.service as the alloy user.
• /etc/alloy/config.alloy: the pipeline. Reload it with sudo systemctl reload alloy.
• /var/lib/alloy/data: Alloy's storage (write-ahead logs and component state).
• /var/lib/alloy/otlp/telemetry.json: the shipped pipeline's output file (Step 7).
• /etc/nginx/sites-available/sac-alloy and /etc/nginx/alloy-ingest.conf: nginx configuration for HTTPS, the UI proxy with the admin login and the OTLP ingest with the otlp login.
• /etc/nginx/alloy-ui.htpasswd and /etc/nginx/alloy-otlp.htpasswd: the bcrypt hashes of the two logins.
• /etc/ssl/alloy/: this instance's certificate and private key.
• /var/lib/sac/data/alloy-credentials.txt: URL, endpoints and both passwords (root only).
• /etc/sac/alloy-egress-allow: private ranges Alloy may reach (Step 9).
• /usr/share/doc/sac-alloy/: the licences, the build record with every checksum, the shipped configuration and the source offer.

• Open port 443 only to the people who use the UI and the networks your applications send from, and port 22 only to your admin network. Never open 12345, 4317 or 4318.
• Use your own DNS name and a certificate from a certificate authority your senders trust (Step 10).
• Keep both passwords in a secrets store and change them (Step 11) when people leave.
• Treat telemetry as sensitive data, because it can carry URLs, user identifiers and SQL statements. Use Alloy's processors (otelcol.processor.attributes, otelcol.processor.transform) to drop or mask what you do not need before it leaves the instance.
• The instance applies Ubuntu security updates automatically. To update Alloy, read the release notes, back up /etc/alloy/config.alloy, launch a newer version from the listing and point your senders at it.