Jaeger

What you get

• Jaeger: Jaeger 1.76 (jaeger-all-in-one), installed at /usr/local/bin/jaeger-all-in-one
• Storage: the embedded Badger key-value store on a dedicated EBS data volume mounted at /var/lib/jaeger
• Receivers: OTLP gRPC on :4317, OTLP HTTP on :4318 and the legacy Jaeger collector on :14268
• Query UI: an nginx reverse proxy on :80 with HTTP Basic Auth, proxying to 127.0.0.1:16686
• First-boot credential: a per-instance UI password generated on first boot and stored in /root/jaeger-credentials.txt (mode 0600, root only)
• Systemd units: jaeger.service, nginx.service and jaeger-firstboot.service (one-shot)

Launch and connect to your instance

Connect with the SSH login user for your OS variant. The command examples below use ubuntu, because the current variant of this image is Ubuntu 24.04.

• Ubuntu 24.04: SSH user ubuntu, for example ssh -i your-key.pem ubuntu@<instance-public-ip>

The recommended instance type is m5.large. The product's pricing covers every current-generation x86_64 instance type, with a 7-day free trial.

Security group

The launched AMI listens on the ports below. In production, restrict each one to trusted networks.

• 22/TCP: SSH
• 80/TCP: the Jaeger Query UI through nginx (HTTP Basic Auth enforced)
• 4317/TCP: the OpenTelemetry Protocol (OTLP) gRPC receiver
• 4318/TCP: the OpenTelemetry Protocol (OTLP) HTTP receiver

The collector port 14268 and the Jaeger query gRPC port 16685 listen on localhost only. They are not exposed externally.

Step 1 - Retrieve the per-instance Jaeger UI password

Connect to your instance over SSH and read the credentials file:

sudo cat /root/jaeger-credentials.txt

The file is generated on first boot and is unique to your instance. It looks like this:

# Jaeger 1 — generated on first boot by jaeger-firstboot.service.
# This UI password is unique to this instance. Store it somewhere safe;
# it is shown in plain text only here. jaeger.ui.url=http://<your-instance-ip>/ jaeger.ui.user=admin jaeger.ui.pass=<JAEGER_UI_PASSWORD>
# OpenTelemetry endpoints (no authentication — restrict via security group): jaeger.otlp.grpc=<your-instance-ip>:4317 jaeger.otlp.http=http://<your-instance-ip>:4318 jaeger.collector.legacy=http://<your-instance-ip>:14268/api/traces

Copy the password somewhere safe, because it appears in plain text only in this file. The file is created with mode 0600 and is readable only by root.

Step 2 - Sign in to the Jaeger Query UI

Open your instance address on port 80 in a browser:

http://<your-instance-public-ip>/

When the browser prompts for HTTP Basic Auth, sign in as admin with the password you just retrieved.

The search page lists every service that has reported a span to the back end, along with the operations seen on each service.

Step 3 - Send some traces

For a first end-to-end check, POST an OpenTelemetry span directly to the OTLP HTTP receiver. Run this on the instance itself:

# Read the UI password into an env var (basic auth only protects the UI, not OTLP)
PASS=$(sudo grep '^jaeger.ui.pass=' /root/jaeger-credentials.txt | cut -d= -f2-)
# Push a synthetic span (use real OpenTelemetry SDKs in production)
NOW_NS=$(date +%s%N)
END_NS=$((NOW_NS + 1000000))
TRACE_ID=$(head -c 16 /dev/urandom | xxd -p -c 16)
SPAN_ID=$(head -c 8 /dev/urandom | xxd -p -c 8)
curl -fsS -X POST http://127.0.0.1:4318/v1/traces \
  -H 'Content-Type: application/json' \
  -d @- <<JSON { "resourceSpans":[{ "resource":{"attributes":[{"key":"service.name","value":{"stringValue":"my-first-service"}}]}, "scopeSpans":[{"scope":{"name":"demo"},"spans":[ {"traceId":"${TRACE_ID}","spanId":"${SPAN_ID}","name":"hello-world","kind":1, "startTimeUnixNano":"${NOW_NS}","endTimeUnixNano":"${END_NS}"} ]}] }] } JSON
# Confirm the service is visible (this hits nginx, so basic auth applies)
curl -fsS -u admin:"${PASS}" http://127.0.0.1/api/services

The output lists the new service:

{"data":["my-first-service","jaeger-all-in-one"],"total":2,"limit":0,"offset":0,"errors":null}

In a real application, use the OpenTelemetry SDK for your language and point it at the OTLP endpoints: <instance-ip>:4317 for gRPC and http://<instance-ip>:4318 for HTTP.

Step 4 - Explore traces in the Query UI

Refresh the search page. The service dropdown now lists your services. Pick one, optionally narrow the search by operation and tags, and select Find Traces to see a scatter plot of trace durations and the underlying trace list.

Select a single trace from the list to see its full timeline. The timeline shows every span from every service that took part in the request, ordered by start time, with durations drawn as bars.

The trace timeline is the main tool for investigating latency. The critical path is the longest unbroken chain of spans, and any span much wider than its peers is the latency hotspot.

Step 5 - View the system architecture (dependencies graph)

Once your services emit spans with parent and child relationships across services, the System Architecture tab shows the inferred service dependency graph. Each arrow is annotated with the count of cross-service calls.

This is the fastest way to see the actual runtime call topology of a microservices deployment, including any unexpected calls that have crept in.

OpenTelemetry SDK quick reference

Point any OpenTelemetry SDK at your Jaeger instance with the OTLP exporter. The examples below cover Python, Node.js and Go.

Python

pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-http

from opentelemetry import trace from opentelemetry.sdk.resources import Resource from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import BatchSpanProcessor from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter resource = Resource.create({"service.name": "my-python-service"}) provider = TracerProvider(resource=resource) provider.add_span_processor( BatchSpanProcessor( OTLPSpanExporter(endpoint="http://<instance-ip>:4318/v1/traces") ) ) trace.set_tracer_provider(provider) tracer = trace.get_tracer("my-python-service") with tracer.start_as_current_span("hello"): print("traced!")

Node.js

npm install @opentelemetry/sdk-node @opentelemetry/exporter-trace-otlp-http \
  @opentelemetry/auto-instrumentations-node @opentelemetry/resources \
  @opentelemetry/semantic-conventions

const { NodeSDK } = require('@opentelemetry/sdk-node'); const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-http'); const { resourceFromAttributes } = require('@opentelemetry/resources'); const { ATTR_SERVICE_NAME } = require('@opentelemetry/semantic-conventions'); const sdk = new NodeSDK({ resource: resourceFromAttributes({ [ATTR_SERVICE_NAME]: 'my-node-service', }), traceExporter: new OTLPTraceExporter({ url: 'http://<instance-ip>:4318/v1/traces', }), }); sdk.start();

Go

go get go.opentelemetry.io/otel \
  go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp \
  go.opentelemetry.io/otel/sdk

The same exporter pattern applies. Point WithEndpoint("<instance-ip>:4318") at your Jaeger instance, and the SDK does the rest.

Day-to-day operations

Service status

sudo systemctl is-active jaeger.service nginx.service
sudo systemctl status jaeger.service --no-pager | head -10

The jaeger-firstboot.service is a one-shot service that runs only on the first boot. Once the sentinel file /var/lib/sac/data/jaeger-firstboot.done exists, the service shows as inactive. This is the expected state after the first boot has completed.

Logs

# Jaeger collector + query
sudo journalctl -u jaeger.service -f
# nginx reverse proxy
sudo journalctl -u nginx.service -f sudo tail -f /var/log/nginx/access.log /var/log/nginx/error.log

Storage

Trace data lives under /var/lib/jaeger on the dedicated EBS data volume:

df -h /var/lib/jaeger
sudo du -sh /var/lib/jaeger/keys /var/lib/jaeger/values

The volume is gp3 and can be resized from the AWS console. After the EBS expansion completes, run sudo resize2fs /dev/nvme1n1. No other Jaeger restart is needed.

Rotating the UI password

NEW_PASS="$(head -c 32 /dev/urandom | base64 | tr -d '/+=' | head -c 24)"
sudo htpasswd -bB /etc/nginx/.htpasswd admin "${NEW_PASS}"
sudo systemctl reload nginx
sudo sed -i "s|^jaeger.ui.pass=.*|jaeger.ui.pass=${NEW_PASS}|" /root/jaeger-credentials.txt

Adding a second user

sudo htpasswd -B /etc/nginx/.htpasswd alice
sudo systemctl reload nginx

Enabling HTTPS

Out of the box, the UI is served over HTTP only. For production deployments, terminate TLS in front of nginx. Two common options are:

• Let's Encrypt and Certbot. Install certbot and the nginx plugin (sudo apt-get install certbot python3-certbot-nginx), and point a DNS name at the instance. Then run sudo certbot --nginx -d your-domain. Certbot rewrites the nginx config to serve HTTPS on :443 with auto-renewal.
• An AWS Application Load Balancer in front. Terminate TLS on the load balancer with an ACM certificate. Set the target group health check to /healthz on :80, which is unauthenticated in this image for exactly this purpose, and forward to the instance on port 80.

Production hardening checklist

• Restrict :80, :4317 and :4318 to your application VPC or known IP ranges.
• Rotate the UI password (see above) after any operator change.
• Place a load balancer in front and terminate TLS there, or run Certbot for a per-instance certificate.
• Resize the /var/lib/jaeger EBS volume from gp3 30 GiB to whatever your retention needs require. Badger is a single-binary key-value store, so trace volume scales with disk size.
• For long-term retention, switch the storage back end from Badger to Elasticsearch or OpenSearch. Set SPAN_STORAGE_TYPE=elasticsearch in /etc/jaeger/jaeger.env, point the relevant ES_* environment variables at your cluster, and restart jaeger.service.

Troubleshooting

• Browser returns 401 unauthorised: nginx Basic Auth is rejecting the login. Use admin and the password from /root/jaeger-credentials.txt.
• Browser returns 502 Bad Gateway: jaeger.service is not running. Start it with sudo systemctl start jaeger.service.
• /api/services returns {"data":null}: no spans have arrived yet. Push a test span (Step 3).
• A service appears but Find Traces returns nothing: adjust the Lookback in the search panel, because traces older than 1 hour are hidden by default.
• jaeger-firstboot.service failed: inspect journalctl -u jaeger-firstboot.service. The usual cause is that the EBS data volume failed to mount.

Useful links

• Jaeger documentation: https://www.jaegertracing.io/docs/
• OpenTelemetry instrumentation guides: https://opentelemetry.io/docs/languages/
• Jaeger on GitHub: https://github.com/jaegertracing/jaeger
• Cloud Native Computing Foundation: https://www.cncf.io/projects/jaeger/

For SecureAnyCloud support, email sales@secureanycloud.com with your AWS account, region and instance ID.

Screenshots

The Jaeger Query UI search page, fronted by nginx HTTP basic authentication and served on first boot with no manual setup.

Choosing a service to inspect from the Jaeger Query UI service dropdown.

The Jaeger trace timeline showing the spans that make up a single distributed request.

Need Help?

Our support team is available 24/7.

sales@secureanycloud.com