OpenObserve 1.1.2 Vector 0.60.0

In the AWS Marketplace listing, choose Continue to Subscribe, then Continue to Configuration. Pick OpenObserve 1.1.2 + Vector 0.60.0 on Ubuntu 24.04 as the software version, select your Region, and choose Launch through EC2.

Choose m5.large or larger, your key pair and a security group as described above. OpenObserve caches recent data in memory, so give busy instances more RAM.

The AMI includes a 20 GiB data volume. Increase its size on the Configure storage step if you expect to keep a lot of data. Under Advanced details, keep Metadata version at V2 only and set the Metadata response hop limit to 1.

To launch with the AWS CLI instead, replace the values in angle brackets:

aws ec2 run-instances --region <region> --image-id <ami-id-from-the-listing> \
  --instance-type m5.large --key-name <your-key-pair> \
  --security-group-ids <your-security-group> --subnet-id <your-subnet> \
  --metadata-options HttpTokens=required,HttpEndpoint=enabled,HttpPutResponseHopLimit=1 \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=openobserve-vector}]'

Add --iam-instance-profile Name=<profile> only if you plan to configure Vector to write to AWS services (Step 10).

Connect over SSH as ubuntu, the default user for the OpenObserve 1.1.2 + Vector 0.60.0 on Ubuntu 24.04 version:

ssh -i <your-key.pem> ubuntu@<public-ip>

The login message shows the web address and where to find the credentials.

First boot takes under a minute. It creates the credentials and the certificate, starts OpenObserve on an empty store, has it issue Vector's ingestion token, starts Vector and proves the pipeline end to end. Then nginx starts in front of everything. Check the services and the installed versions:

systemctl is-active openobserve-vector-firstboot openobserve vector nginx
sudo test -f /var/lib/sac/data/openobserve-vector-firstboot.done && echo FIRSTBOOT_DONE /usr/local/bin/openobserve --version vector --version | cut -d' ' -f1,2 curl -s http://127.0.0.1:5080/config | jq -r '"build_type=\(.build_type) commit=\(.commit_hash)"'

active active active active FIRSTBOOT_DONE openobserve v1.1.2 vector 0.60.0 build_type=opensource commit=d16d693ba99bc20621a58b948ecbcad7ec7fa882

The build_type=opensource value and the commit of the upstream v1.1.2 tag confirm the open-source build of OpenObserve.

Only SSH and nginx listen on public addresses. OpenObserve and Vector listen on loopback:

sudo ss -Hlntp | awk '{print $4, $6}' | grep -v '127.0.0.5' | sed -E 's/,pid=[0-9]+,fd=[0-9]+//g' | sort

0.0.0.0:22 users:(("sshd"),("systemd")) 0.0.0.0:443 users:(("nginx"),("nginx"),("nginx")) 0.0.0.0:80 users:(("nginx"),("nginx"),("nginx")) 127.0.0.1:4317 users:(("vector")) 127.0.0.1:4318 users:(("vector")) 127.0.0.1:5080 users:(("openobserve")) 127.0.0.1:5081 users:(("openobserve")) 127.0.0.1:8686 users:(("vector")) 127.0.0.1:9880 users:(("vector")) [::]:22 users:(("sshd"),("systemd")) [::]:443 users:(("nginx"),("nginx"),("nginx")) [::]:80 users:(("nginx"),("nginx"),("nginx"))

The data lives on its own EBS volume:

findmnt -no TARGET,SOURCE,FSTYPE /var/lib/openobserve df -h --output=size,target /var/lib/openobserve

/var/lib/openobserve /dev/nvme1n1 ext4 Size Mounted on 20G /var/lib/openobserve

SSH accepts keys only. Passwords, root logins and X11 forwarding are switched off in /etc/ssh/sshd_config.d/00-sac.conf, which sorts before every other SSH setting. Check the effective values:

sudo sshd -T | grep -E '^(passwordauthentication|kbdinteractiveauthentication|permitrootlogin|x11forwarding|pubkeyauthentication) '

permitrootlogin no pubkeyauthentication yes passwordauthentication no kbdinteractiveauthentication no x11forwarding no

The addresses and user names are in the root-only file /root/openobserve-vector-credentials.txt. Print them:

sudo grep -E '^(OPENOBSERVE_URL|OPENOBSERVE_USERNAME|INGEST_URL|OTLP_URL|INGEST_USERNAME|OPENOBSERVE_INGEST_USER)=' /root/openobserve-vector-credentials.txt

OPENOBSERVE_URL=https://ec2-<public-ip-dashed>.compute-1.amazonaws.com/ OPENOBSERVE_USERNAME=admin@openobserve.local INGEST_URL=https://ec2-<public-ip-dashed>.compute-1.amazonaws.com/ingest/ OTLP_URL=https://ec2-<public-ip-dashed>.compute-1.amazonaws.com/otlp/ INGEST_USERNAME=ingest OPENOBSERVE_INGEST_USER=default

The same file holds three secrets, all unique to this instance:

• OPENOBSERVE_PASSWORD: the web interface and API root login
• INGEST_PASSWORD: the login your other machines use for the ingest endpoints
• OPENOBSERVE_INGEST_TOKEN: the token that Vector on this instance uses

Show one with, for example, sudo grep '^OPENOBSERVE_PASSWORD=' /root/openobserve-vector-credentials.txt, and store all three in your password manager.

The address uses the instance's public DNS name from EC2 instance metadata (IMDSv2), or its public or private IPv4 address when it has no public name. The certificate names that host plus the instance's public and private IP addresses:

openssl x509 -in /etc/ssl/sac/openobserve-vector.crt -noout -subject -ext subjectAltName -enddate

subject=CN = ec2-<public-ip-dashed>.compute-1.amazonaws.com, O = sac X509v3 Subject Alternative Name: DNS:ec2-<public-ip-dashed>.compute-1.amazonaws.com, IP Address:<public-ip>, IP Address:<private-ip>, IP Address:127.0.0.1, DNS:localhost notAfter=<date about 27 months after first boot>

The commands below confirm the following behavior:

• Port 80 redirects to HTTPS.
• /healthz answers without a login.
• The API refuses a missing or wrong password and accepts yours.
• Vector's token is refused for reading.
• The ingest endpoint refuses a missing or wrong login.
• A request for a host name the instance does not serve is dropped.

The commands read the secrets into shell variables and hand them to curl through a file descriptor, so they never appear in the process list or your shell history:

C=/root/openobserve-vector-credentials.txt
H=$(sudo sed -n 's#^OPENOBSERVE_URL=https://\([^/]*\)/.*#\1#p' $C)
U=$(sudo sed -n 's/^OPENOBSERVE_USERNAME=//p' $C);
PW=$(sudo sed -n 's/^OPENOBSERVE_PASSWORD=//p' $C)
TU=$(sudo sed -n 's/^OPENOBSERVE_INGEST_USER=//p' $C);
TK=$(sudo sed -n 's/^OPENOBSERVE_INGEST_TOKEN=//p' $C) CA=/etc/ssl/sac/openobserve-vector.crt; R=(--cacert $CA --resolve "$H:443:127.0.0.1") curl -s -o /dev/null -w 'http:// -> %{http_code} %{redirect_url}\n' --resolve "$H:80:127.0.0.1" "http://$H/" curl -s -w ' <- healthz, HTTP %{http_code}\n' "${R[@]}" "https://$H/healthz" curl -s -o /dev/null -w 'no login -> HTTP %{http_code}\n' "${R[@]}" "https://$H/api/default/streams" curl -s -o /dev/null -w 'wrong password -> HTTP %{http_code}\n' "${R[@]}" -K <(printf 'user = "%s:%s"\n' "$U" not-the-password) "https://$H/api/default/streams" curl -s -o /dev/null -w 'your password -> HTTP %{http_code}\n' "${R[@]}" -K <(printf 'user = "%s:%s"\n' "$U" "$PW") "https://$H/api/default/streams" curl -s -o /dev/null -w 'token, read -> HTTP %{http_code}\n' "${R[@]}" -K <(printf 'user = "%s:%s"\n' "$TU" "$TK") "https://$H/api/default/streams" curl -s -o /dev/null -w 'ingest, no login -> HTTP %{http_code}\n' "${R[@]}" -H 'Content-Type: application/json' -d '[{"message":"x"}]' "https://$H/ingest/" curl -s -o /dev/null -w 'ingest, wrong -> HTTP %{http_code}\n' "${R[@]}" -K <(printf 'user = "ingest:%s"\n' not-the-password) -H 'Content-Type: application/json' -d '[{"message":"x"}]' "https://$H/ingest/" curl -s -o /dev/null -w 'metrics -> HTTP %{http_code}\n' "${R[@]}" "https://$H/metrics" curl -sk -o /dev/null -w 'other host -> HTTP %{http_code} (000 = connection dropped)\n' -H 'Host: evil.example' https://127.0.0.1/ || true

http:// -> 301 https://ec2-<public-ip-dashed>.compute-1.amazonaws.com/ {"status":"ok"} <- healthz, HTTP 200 no login -> HTTP 401 wrong password -> HTTP 401 your password -> HTTP 200 token, read -> HTTP 401 ingest, no login -> HTTP 401 ingest, wrong -> HTTP 401 metrics -> HTTP 404 other host -> HTTP 000 (000 = connection dropped)

The --resolve option sends the request to this instance while keeping the real host name, so the certificate check passes. From another machine, drop --resolve and use the address directly.

The image ships /usr/local/sbin/openobserve-vector-e2e.sh, the same check that first boot runs. It writes a uniquely tagged line to the system journal and waits until Vector has delivered that exact line to OpenObserve's host_logs stream. Run it, then list the streams Vector has created:

sudo /usr/local/sbin/openobserve-vector-e2e.sh C=/root/openobserve-vector-credentials.txt
H=$(sudo sed -n 's#^OPENOBSERVE_URL=https://\([^/]*\)/.*#\1#p' $C)
U=$(sudo sed -n 's/^OPENOBSERVE_USERNAME=//p' $C);
PW=$(sudo sed -n 's/^OPENOBSERVE_PASSWORD=//p' $C)
curl -s --cacert /etc/ssl/sac/openobserve-vector.crt --resolve "$H:443:127.0.0.1" -K <(printf 'user = "%s:%s"\n' "$U" "$PW") \
  "https://$H/api/default/streams" | jq -r '[.list[].name | select(startswith("host_"))] | sort | join(" ")'

E2E_PIPELINE_OK tag=sac-e2e-<random> path=journald->vector->openobserve/host_logs host_logs host_metrics

Each host_logs record is kept narrow and readable. It holds message, service (the systemd unit), severity, host, command, pid, transport and the timestamp.

OpenObserve's own routine request logging is filtered out before it is shipped, so the instance does not fill its store with logs about itself.

Open the OPENOBSERVE_URL address from Step 4 in your browser. The certificate is self-signed and made on your instance, so your browser asks you to accept it once. Step 11 shows how to use your own name and certificate.

Sign in with the email admin@openobserve.local and your OPENOBSERVE_PASSWORD.

Open Logs and select the host_logs stream to see this instance's journal as Vector collected it. Click a field on the left to add it to the table, or type a filter such as service='nginx.service' and choose Run query.

Open Dashboards and select Host Overview (Vector). First boot provisions it with log volume, memory, load average and filesystem usage from the host_logs and host_metrics streams.

Your other machines send to the two authenticated HTTPS endpoints with the INGEST_USERNAME and INGEST_PASSWORD from Step 4. JSON posted to /ingest/ lands in the remote_logs stream.

The commands below send one event the way a remote machine would, then find it with an SQL search:

C=/root/openobserve-vector-credentials.txt
H=$(sudo sed -n 's#^OPENOBSERVE_URL=https://\([^/]*\)/.*#\1#p' $C)
U=$(sudo sed -n 's/^OPENOBSERVE_USERNAME=//p' $C);
PW=$(sudo sed -n 's/^OPENOBSERVE_PASSWORD=//p' $C)
IU=$(sudo sed -n 's/^INGEST_USERNAME=//p' $C);
IP=$(sudo sed -n 's/^INGEST_PASSWORD=//p' $C) R=(--cacert /etc/ssl/sac/openobserve-vector.crt --resolve "$H:443:127.0.0.1") curl -s -o /dev/null -w 'ingest -> HTTP %{http_code}\n' "${R[@]}" -K <(printf 'user = "%s:%s"\n' "$IU" "$IP") \
  -H 'Content-Type: application/json' -d '[{"message":"order placed","service":"checkout","level":"info"}]' "https://$H/ingest/" sleep 8
NOW=$(date +%s%6N);
FROM=$((NOW - 3600000000)) curl -s "${R[@]}" -K <(printf 'user = "%s:%s"\n' "$U" "$PW") -H 'Content-Type: application/json' \
  -d "{\"query\":{\"sql\":\"SELECT message, service, level FROM \\\"remote_logs\\\" WHERE service = 'checkout'\",\"start_time\":$FROM,\"end_time\":$NOW,\"from\":0,\"size\":1}}" \
  "https://$H/api/default/_search" | jq -c '.hits[0] | {message, service, level}'

ingest -> HTTP 200 {"message":"order placed","service":"checkout","level":"info"}

Search times are in microseconds since the epoch.

OpenTelemetry. Send OTLP logs over HTTP to https://<your-host>/otlp/v1/logs with the same login. They land in the otlp_logs stream. Vector's OTLP receiver takes the OTLP protobuf encoding, which is what OpenTelemetry SDKs and the Collector send by default (http/protobuf). For example, configure an OpenTelemetry Collector exporter like this:

exporters: otlphttp/sac: logs_endpoint: https://<your-host>/otlp/v1/logs encoding: proto headers: Authorization: Basic <base64 of ingest:INGEST_PASSWORD> tls: ca_file: /etc/otel/openobserve-vector.crt
# or insecure_skip_verify: true while you test

Other agents. Fluent Bit, Vector or any shipper that can POST JSON over HTTPS with Basic auth can use /ingest/. OTLP over gRPC (port 4317) is not exposed, so use OTLP over HTTPS as above.

Copy /etc/ssl/sac/openobserve-vector.crt to your machines so they can verify the self-signed certificate, or install a certificate from your own authority (Step 11). Open port 443 in the security group to their addresses only.

OpenObserve's own ingest APIs are also served at https://<your-host>/api/default/... (JSON, Elasticsearch bulk, OTLP). They take an OpenObserve login rather than the ingest password. For agents that use them, create a separate user in IAM > Users instead of sharing the root login.

OpenObserve and nginx cannot reach the EC2 instance metadata service. Vector can, so its AWS sinks can use the instance's IAM role. Every secret is in a root-only file. Check the deny lists and the file permissions:

for u in openobserve nginx vector; do printf '%-12s IPAddressDeny=%s\n' "$u" "$(systemctl show -p IPAddressDeny --value $u)"; done
sudo stat -c '%a %U:%G %n' /root/openobserve-vector-credentials.txt /etc/openobserve/root-user.env /etc/openobserve/ingest-user.env \
  /etc/nginx/ingest-cred.env /etc/nginx/.htpasswd-ingest /etc/vector/vector.yaml /etc/ssl/sac/openobserve-vector.key cat /etc/openobserve/web-url.env

openobserve IPAddressDeny=169.254.169.254/32 168.63.129.16/32 fd00:ec2::254/128 nginx IPAddressDeny=169.254.169.254/32 168.63.129.16/32 fd00:ec2::254/128 vector IPAddressDeny= 600 root:root /root/openobserve-vector-credentials.txt 600 root:root /etc/openobserve/root-user.env 600 root:root /etc/openobserve/ingest-user.env 600 root:root /etc/nginx/ingest-cred.env 640 root:www-data /etc/nginx/.htpasswd-ingest 640 root:vector /etc/vector/vector.yaml 600 root:root /etc/ssl/sac/openobserve-vector.key ZO_WEB_URL=https://ec2-<public-ip-dashed>.compute-1.amazonaws.com

ZO_WEB_URL is the address OpenObserve uses in links, for example in alert messages. It comes from the configured host name, never from a request.

nginx stores the ingest password only as a bcrypt hash. Vector re-checks the same login on the requests that nginx passes to it.

The pipeline Vector runs is /etc/vector/vector.yaml (owner root, group vector, mode 0640). First boot renders it from the template /usr/share/sac/vector.yaml.tmpl with this instance's ingestion token and ingest password. Edit the rendered file, validate it and restart Vector:

sudo nano /etc/vector/vector.yaml
sudo vector validate --no-environment /etc/vector/vector.yaml
sudo systemctl restart vector

Typical changes are to add Vector sources (files, syslog, Kafka), add host metric collectors or add a second sink.

Vector uses the instance's IAM role through IMDSv2, and no other service on the image can reach IMDS. For S3, CloudWatch Logs or Kinesis sinks, attach an IAM role to the instance and never put access keys in the file.

Keep every Vector listener on 127.0.0.1 and publish new endpoints through nginx with authentication, as /ingest/ is.

The served host name comes from configuration, never from a request. To serve the appliance under your own DNS name, point the name at the instance and run the helper. It records the name, makes a new self-signed certificate for it, updates ZO_WEB_URL and /root/openobserve-vector-credentials.txt, reloads nginx and restarts OpenObserve:

sudo openobserve-vector-set-hostname logs.example.com

To use a certificate from your own certificate authority instead, put the full chain in /etc/ssl/sac/openobserve-vector.crt and the private key in /etc/ssl/sac/openobserve-vector.key (owner root, mode 0600). Then run sudo nginx -t && sudo systemctl reload nginx. Running openobserve-vector-set-hostname again replaces them with a self-signed pair.

To change the passwords and the token:

• Root password: sign in, open IAM > Users, edit admin@openobserve.local and set a new password. Then update /root/openobserve-vector-credentials.txt. OpenObserve uses the value in /etc/openobserve/root-user.env only when it creates the root user on an empty store.
• Ingest password: run sudo htpasswd -B /etc/nginx/.htpasswd-ingest ingest, which prompts for the new password. Put the same password in the password: line of the remote_http source in /etc/vector/vector.yaml. Then run sudo systemctl restart vector && sudo systemctl reload nginx, and update /etc/nginx/ingest-cred.env and the credentials file.
• Vector's ingestion token: create a new token under Data sources > Manage Tokens. Put it in the four password: lines of the oo_* sinks in /etc/vector/vector.yaml, restart Vector, then disable the old token.

• Storage. All data is under /var/lib/openobserve on its own EBS volume. That covers ingested logs, metrics and traces, plus the metadata database with users, dashboards and alerts. Grow the volume with Modify volume in the EC2 console, then run sudo resize2fs <device>. Find the device with findmnt /var/lib/openobserve.
• Backups. Back up with EBS snapshots of the data volume. For a consistent copy, stop Vector and OpenObserve first (sudo systemctl stop vector openobserve) or snapshot both volumes together.
• Retention. Data is kept for 3,650 days by default. So that the volume does not fill, set a shorter retention per stream under Data > Streams. For the whole instance, set ZO_COMPACT_DATA_RETENTION_DAYS=<days> in /etc/openobserve/openobserve.env, then run sudo systemctl restart openobserve.
• Operations. Manage the services with systemctl status openobserve vector nginx and read their logs with journalctl -u openobserve -u vector -u nginx. The vector top command shows live throughput through Vector's loopback API. Unattended upgrades keep Ubuntu and nginx patched. Check the listing for newer OpenObserve and Vector versions.

• Restrict ports 443 and 80 to your team's networks and your machines' networks. No OpenObserve or Vector port is ever exposed.
• Give each team its own OpenObserve user with the least access it needs, instead of sharing the root login. Give your machines the ingest login, which can only send data.
• SSH accepts key logins as ubuntu only, with no passwords, no root login and no X11 forwarding. Use sudo for administration.
• The AMI requires IMDSv2, so launch with a metadata hop limit of 1. nginx logs the real client address of every request and ignores X-Forwarded-For headers sent by clients.
• The web interface shows an offer for OpenObserve's commercial edition. This image runs the open-source edition and works without it.