Zipkin

Important: trace storage is in-memory by default

This appliance ships Zipkin's default STORAGE_TYPE=mem. Traces live in the JVM's memory only and are lost whenever the zipkin service restarts or the VM reboots. This keeps the appliance simple and free of dependencies out of the box. It is fine for development, demos and short-lived debugging sessions.

For production tracing that you need across restarts, point Zipkin at your own persistent backend: MySQL, Elasticsearch or Cassandra. Edit /etc/systemd/system/zipkin.service (Environment=STORAGE_TYPE=mysql plus the corresponding MYSQL_HOST / MYSQL_TCP_PORT / MYSQL_DB / MYSQL_USER / MYSQL_PASS variables, per the upstream Zipkin server documentation). Then run sudo systemctl daemon-reload && sudo systemctl restart zipkin.

The dedicated data disk at /var/lib/zipkin holds only Zipkin's own application log file, not trace data.

Prerequisites

• An active Azure subscription, an SSH key pair, and a VNet with a subnet in the target region
• Standard_B2s (2 vCPU, 4 GiB RAM) as a comfortable starting point for light-to-moderate tracing volume. For higher throughput, size up the JVM heap (JAVA_OPTS in the systemd unit) and the VM.
• Network security group inbound rules that allow 22/tcp from your management network and 80/tcp for the UI and query API
• 9411/tcp if your instrumented apps need to send spans over the network. Skip this rule if all reporting apps run on the same VM and can reach 127.0.0.1:9411 (see Step 4).
• 443/tcp if you add TLS
• Your own domain and TLS in production, because Zipkin serves plain HTTP. Also restrict :9411 to your application subnets.

Step 1 - Deploy from the Azure Marketplace

Sign in to the Azure Portal, choose Create a resource, search the Marketplace for Zipkin by SecureAnyCloud, and select Create.

On the Basics tab, pick your subscription, resource group, region and size. Under Administrator account, choose SSH public key and paste your key. Under Inbound port rules, allow SSH (22) and HTTP (80). Add Custom (9411) if apps outside the VM will send spans.

Review the dedicated data disk on the Disks tab. Then select Review + create, then Create.

Step 2 - Deploy from the Azure CLI

# Accept the Marketplace terms once per subscription (required before the first CLI deploy)
az vm image terms accept --urn sac1647283583153:zipkin-ubuntu-24-04:default:latest
az vm create \
  --resource-group <your-rg> \
  --name zipkin \
  --image sac1647283583153:zipkin-ubuntu-24-04:default:latest \
  --size Standard_B2s \
  --admin-username azureuser \
  --ssh-key-values ~/.ssh/id_ed25519.pub \
  --vnet-name <your-vnet> --subnet <your-subnet> \
  --public-ip-sku Standard
az vm open-port --resource-group <your-rg> --name zipkin --port 80 --priority 1010
az vm open-port --resource-group <your-rg> --name zipkin --port 9411 --priority 1020

Step 3 - Connect to your VM

ssh azureuser@<vm-public-ip>

Step 4 - Confirm the services are running

systemctl is-active zipkin.service nginx.service

Both report active.

Zipkin serves its web UI and full REST API on the loopback address 127.0.0.1:19411 only. This is a private port, not the public 9411. nginx fronts port 80 with the per-VM HTTP Basic Auth gate for the UI and query API. nginx also fronts the well-known port 9411 separately, with an unauthenticated, ingest-only proxy that accepts POST /api/v2/spans and rejects everything else.

If every service that reports traces runs on this same VM, point them at http://127.0.0.1:9411/api/v2/spans directly, with no network exposure at all. Open 9411 in the network security group only if reporting apps live elsewhere.

Step 5 - Retrieve your web UI password

nginx protects the Zipkin UI and query API with HTTP Basic Auth. The username is admin. A unique password is generated on the first boot of your VM and written to a root-only file:

sudo cat /root/zipkin-credentials.txt

The file contains ZIPKIN_USERNAME, ZIPKIN_PASSWORD, the ZIPKIN_URL to open in a browser, and ZIPKIN_SPAN_INGEST_URL for configuring your tracing exporters.

The password is stored on disk only as a bcrypt hash in /etc/nginx/.zipkin.htpasswd, so no plaintext password ships in the image. Store the password somewhere safe.

Step 6 - Confirm the health endpoint

nginx serves an unauthenticated health endpoint for load balancers and probes:

curl -s http://localhost/healthz

It returns ok. This endpoint never requires authentication, so you can use it for an Azure Load Balancer health probe.

Step 7 - Confirm authentication on the UI and query path

Because a password is set on first boot, an unauthenticated request to the UI returns HTTP 401, so nobody reaches the trace data without the password.

The commands below read the per-VM password from the credentials file and prove the round trip. An unauthenticated request is rejected, a wrong password is rejected, and the correct password authenticates:

PW=$(sudo grep '^ZIPKIN_PASSWORD=' /root/zipkin-credentials.txt | cut -d= -f2-)
echo "unauth : $(curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1/)"
echo "wrongpw : $(curl -s -o /dev/null -w '%{http_code}' -u admin:wrong-pw http://127.0.0.1/)"
echo "authed : $(curl -s -o /dev/null -w '%{http_code}' -u admin:$PW http://127.0.0.1/)"

It prints unauth : 401, wrongpw : 401, authed : 200. Only the per-VM password reaches the Zipkin UI and query API, because Zipkin itself is bound to loopback and nginx port 80 is the only way in.

Step 8 - Send a test span and query it back

Zipkin's collector accepts spans at POST /api/v2/spans. Instrumented apps normally send spans continuously. The commands below post one span by hand to confirm the pipeline end to end. The span goes through the authenticated :80 API. The unauthenticated, ingest-only :9411 path accepts the same span format:

PW=$(sudo grep '^ZIPKIN_PASSWORD=' /root/zipkin-credentials.txt | cut -d= -f2-)
TID=$(openssl rand -hex 16)
TS=$(($(date +%s%N)/1000))
# authenticated path (port 80)
curl -s -o /dev/null -w 'POST :80/api/v2/spans -> %{http_code}\n' -u admin:$PW \
  -H 'Content-Type: application/json' \
  -d "[{\"traceId\":\"$TID\",\"id\":\"$(openssl rand -hex 8)\",\"name\":\"checkout\",\"timestamp\":$TS,\"duration\":45000,\"localEndpoint\":{\"serviceName\":\"web-frontend\"}}]" \
  http://127.0.0.1/api/v2/spans
curl -s -u admin:$PW "http://127.0.0.1/api/v2/trace/$TID" | python3 -m json.tool

The POST returns 200 (or 202), and the trace query returns the span you just sent: serviceName: web-frontend, name: checkout, and the duration in microseconds.

Point real instrumentation at ZIPKIN_SPAN_INGEST_URL (http://<vm-ip>:9411/api/v2/spans, no credentials required). For apps running on this same VM, use http://127.0.0.1:9411/api/v2/spans.

Step 9 - Sign in and search traces

Browse to http://<vm-public-ip>/. Your browser prompts for a username and password. Enter admin and the password from Step 5.

Zipkin opens on the trace search screen. Pick a service name, an optional span name and a lookback window. Then choose Run Query to list matching traces sorted by duration.

Step 10 - Read a trace detail and span waterfall

Click any trace in the results to open its detail view. The waterfall shows every span in the trace, nested by parent and child relationship, with exact start offsets and durations. You can see precisely which downstream call added the latency.

Click any span to expand its tags and its local and remote endpoint annotations.

Step 11 - View the service dependency graph

Open Dependencies from the top navigation and pick a date. Zipkin aggregates all traces from that day into a graph. The graph shows every service that called another, with call counts and error rates on each edge.

It is a quick way to see your architecture as it actually behaves in production, not as documented.

Maintenance

• Password. The UI and API password is set on first boot and stored as a bcrypt entry in /etc/nginx/.zipkin.htpasswd. To change it, run sudo htpasswd -B /etc/nginx/.zipkin.htpasswd admin and then sudo systemctl reload nginx.
• Storage is in-memory by default. Traces are lost on zipkin service restart or VM reboot. For durable storage, set STORAGE_TYPE (and the matching backend variables) in /etc/systemd/system/zipkin.service to mysql, elasticsearch or cassandra3 per the upstream storage docs. Then run sudo systemctl daemon-reload && sudo systemctl restart zipkin.
• Span ingest port. :9411 is intentionally unauthenticated because tracing libraries cannot supply Basic Auth, but nginx restricts it to POST /api/v2/spans only. Every other path, including the query API, returns 403. In production, restrict :9411 in your network security group to your application subnets.
• Restrict access. Zipkin serves plain HTTP on port 80. In production, restrict access to trusted IP ranges in your network security group, and front it with TLS terminating on :443, for example with certbot and your own domain.
• Loopback binding. Zipkin is bound to 127.0.0.1:19411 in /etc/systemd/system/zipkin.service (--armeria.ports[0].ip=127.0.0.1), so nginx is the only path in for both the UI and query API and the public ingest port. Do not change the bind address to a public interface.
• JVM sizing. The default heap is -Xms256m -Xmx512m (JAVA_OPTS in /etc/systemd/system/zipkin.service). For higher trace volume, increase the heap and the VM size, then run sudo systemctl daemon-reload && sudo systemctl restart zipkin.
• Logs. Zipkin's own application log is written to /var/lib/zipkin/zipkin.log on the dedicated data disk. journalctl -u zipkin also shows recent output.
• Security patches. Unattended upgrades stay enabled, so the OS continues to receive security updates automatically.

Support

SecureAnyCloud provides 24/7 expert support for this image. Email sales@secureanycloud.com.

Deploy on Azure

Launch Zipkin Distributed Tracing on Ubuntu 24.04 LTS on Azure with 24/7 support from SecureAnyCloud.

View this image

Need Help?

Our support team is available 24/7.

sales@secureanycloud.com