OpenSearch 3.9.0 Fluent Bit 5.1.4

In the AWS Marketplace listing, choose Continue to Subscribe, then Continue to Configuration. Pick OpenSearch 3.9.0 + Fluent Bit 5.1.4 on Ubuntu 24.04 as the software version, choose your Region, and select Launch through EC2.

Choose m5.large or larger, your key pair and a security group as described above. OpenSearch is a JVM service and Dashboards is a Node service, and the image sizes the OpenSearch heap from the memory it finds at boot. Increase the root volume on the Configure storage step if you expect to keep a lot of log history. Under Advanced details, keep Metadata version at V2 only.

To launch with the AWS CLI instead, replace the values in angle brackets:

aws ec2 run-instances --region <region> --image-id <ami-id-from-the-listing> \
  --instance-type m5.large --key-name <your-key-pair> \
  --security-group-ids <your-security-group> --subnet-id <your-subnet> \
  --metadata-options HttpTokens=required,HttpEndpoint=enabled \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=opensearch-logs}]'

Version SSH user

OpenSearch 3.9.0 + Fluent Bit 5.1.4 on Ubuntu 24.04 ubuntu

ssh -i <your-key.pem> ubuntu@<public-ip>

The login message shows the Dashboards address and the location of the credentials.

First boot creates the certificates, passwords and keystore. It starts OpenSearch on an empty data directory, applies this instance's security configuration, installs the single-node index template, starts Dashboards, Fluent Bit and nginx, and creates the Dashboards index pattern. This takes a few minutes on a 2 vCPU instance. When first boot has finished, run:

systemctl is-active opensearch opensearch-dashboards fluent-bit nginx
sudo test -f /var/lib/sac/data/opensearch-fluent-bit-firstboot.done && echo FIRSTBOOT_DONE cat /etc/sac/opensearch-fluent-bit-versions.txt

active active active active FIRSTBOOT_DONE opensearch=3.9.0 opensearch-dashboards=3.9.0 fluent-bit=5.1.4

If FIRSTBOOT_DONE is not printed yet, follow progress with sudo journalctl -u opensearch-fluent-bit-firstboot -f.

Only SSH, nginx and the authenticated forward input listen on public addresses. OpenSearch, Dashboards and the Fluent Bit monitor listen on loopback:

sudo ss -Hlntp | awk '{print $4, $6}' | grep -v '127.0.0.5' | sed -E 's/,pid=[0-9]+,fd=[0-9]+//g' | sort

0.0.0.0:22 users:(("sshd"),("systemd")) 0.0.0.0:24224 users:(("fluent-bit")) 0.0.0.0:443 users:(("nginx"),("nginx"),("nginx")) 0.0.0.0:80 users:(("nginx"),("nginx"),("nginx")) 127.0.0.1:2020 users:(("fluent-bit")) 127.0.0.1:5601 users:(("node")) [::]:22 users:(("sshd"),("systemd")) [::]:443 users:(("nginx"),("nginx"),("nginx")) [::]:80 users:(("nginx"),("nginx"),("nginx")) [::ffff:127.0.0.1]:9200 users:(("java")) [::ffff:127.0.0.1]:9300 users:(("java"))

SSH accepts keys only. Passwords, root logins and X11 forwarding are switched off in /etc/ssh/sshd_config.d/00-sac.conf, which sorts before every other SSH setting. Check the effective values:

sudo sshd -T | grep -E '^(passwordauthentication|kbdinteractiveauthentication|permitrootlogin|x11forwarding|pubkeyauthentication) '

permitrootlogin no pubkeyauthentication yes passwordauthentication no kbdinteractiveauthentication no x11forwarding no

First boot writes everything it generated to the root-only file /root/opensearch-fluent-bit-credentials.txt (mode 0600). List the entries the file holds without printing any secret:

sudo grep -oE '^[a-z_]+=' /root/opensearch-fluent-bit-credentials.txt
sudo stat -c '%a %U:%G' /root/opensearch-fluent-bit-credentials.txt
sudo awk -F= '/^dashboards_url=/{print $2}' /root/opensearch-fluent-bit-credentials.txt

dashboards_url= admin_username= admin_password= opensearch_url= opensearch_ca= forward_host= forward_port= forward_shared_key= forward_ca= dashboards_server_password= fluentbit_writer_password= 600 root:root https://<your-public-dns-name>/

When you are ready to sign in, show the admin password:

sudo awk -F= '/^admin_password=/{print $2}' /root/opensearch-fluent-bit-credentials.txt

Confirm that the certificate authority is unique to this instance and that the published demo login is refused:

openssl x509 -in /etc/sac/opensearch-root-ca.pem -noout -subject curl -s -o /dev/null -w 'HTTP %{http_code}\n' --cacert /etc/sac/opensearch-root-ca.pem -u 'admin:admin' https://127.0.0.1:9200/

subject=CN = sac-opensearch-ca-05b72e4e3ee00e5c, OU = sac-appliance, O = sac HTTP 401

The 16 hex characters after sac-opensearch-ca- are random and different on every instance.

Browse to the dashboards_url from the credentials file (https://<your-public-dns-name>/). Your instance's own authority issued the certificate, so your browser warns that it is not signed by a public authority. Step 10 shows how to trust it or use your own name. Sign in as admin with the admin_password.

Open Discover. The sac-logs-* index pattern is already the default, so the histogram and the record list show what Fluent Bit has collected from this instance since first boot.

Every record collected on the instance carries an appliance field and the instance's hostname. Use them to narrow a search with Dashboards Query Language, for example:

appliance:"sac-opensearch-log-stack" and log:"payment-service"

To see every field OpenSearch has mapped, open Dashboards Management, then Index patterns. The sac-logs-* pattern lists the fields, with @timestamp as the time field.

Fluent Bit collects anything written to a .log file directly under /var/log/sac/sac-logs/. It rescans the directory every five seconds, so new files are picked up without a restart:

echo "$(date -Is) payment-service order failed id=42 reason=card_declined" | sudo tee -a /var/log/sac/sac-logs/app.log

Search for the line in Discover, or from the command line as shown in Step 8.

The image ships a probe that writes a uniquely tagged line into the watched directory and then reads that exact line back from OpenSearch. Run it:

sudo /usr/local/sbin/opensearch-fluent-bit-e2e.sh

E2E: emitted probe token SACE2E32a40df594d8237e665a8386 into /var/log/sac/sac-logs/sac-e2e-probe.log E2E PASS: token SACE2E32a40df594d8237e665a8386 emitted by Fluent Bit and retrieved from OpenSearch after 2 poll(s) {"log":"2026-10-21T06:49:00+00:00 sac end-to-end pipeline probe token=SACE2E32a40df594d8237e665a8386","hostname":"ip-172-31-87-177","appliance":"sac-opensearch-log-stack","@timestamp":"2026-10-21T06:49:00.501Z"}

The token differs on every run. If the probe cannot read its own line back, it exits with a non-zero status and reports where delivery stopped.

The REST API listens on loopback only. Run these commands on the instance, or forward the port first with ssh -L 9200:127.0.0.1:9200 ubuntu@<public-ip>. The examples read the password into a shell variable and pass it to curl on standard input, so it never appears on a command line:

PW=$(sudo awk -F= '/^admin_password=/{print $2}' /root/opensearch-fluent-bit-credentials.txt) printf 'user = "admin:%s"\n' "$PW" | curl -s -K - --cacert /etc/sac/opensearch-root-ca.pem \
  'https://127.0.0.1:9200/_cluster/health' | jq -r '"status=\(.status) nodes=\(.number_of_nodes)"' printf 'user = "admin:%s"\n' "$PW" | curl -s -K - --cacert /etc/sac/opensearch-root-ca.pem \
  'https://127.0.0.1:9200/_cat/indices/sac-logs-*?h=health,index,pri,rep' printf 'user = "admin:%s"\n' "$PW" | curl -s -K - --cacert /etc/sac/opensearch-root-ca.pem \
  -H 'Content-Type: application/json' 'https://127.0.0.1:9200/sac-logs-*/_count' \
  -d '{"query":{"match_phrase":{"log":"payment-service"}}}' | jq '.count'

status=green nodes=1 green sac-logs-2026.10.21 1 0 5

The cluster is green because first boot installs an index template that gives the daily sac-logs-* indices one shard and no replica, which suits a single node.

Remote Fluent Bit (or Fluentd) agents send to TCP 24224 with the forward protocol over TLS and must present this instance's shared key. The instance refuses a sender that does not use TLS or does not present the key.

• In the instance's security group, allow TCP 24224 from the private addresses of the sending machines only.
• Copy the certificate authority /etc/sac/opensearch-root-ca.pem to each sender (it is public material) and read forward_shared_key from the credentials file.
• Add an output like this to each sender's Fluent Bit configuration, using the instance's private address or DNS name:

[OUTPUT] Name forward Match * Host <this-instance-private-dns-name> Port 24224 Shared_Key <forward_shared_key> Self_Hostname <sender-name> tls On tls.verify On tls.ca_file /etc/fluent-bit/sac-opensearch-ca.pem

The forward certificate is issued for the names in Step 10, so connect with one of those names or set tls.vhost. Forwarded records are indexed into the same sac-logs-* indices and keep the sender's own fields.

First boot configures the instance to serve its public DNS name, public IP and private IP. To use your own DNS name, point the name at the instance and run:

sudo opensearch-set-hostname logs.example.com

The command issues new web and forward certificates for that name from the instance's own authority, updates nginx and the credentials file, and reloads the services. To stop browser warnings, import /etc/sac/opensearch-root-ca.pem into your browser or operating system trust store. Alternatively, replace /etc/ssl/opensearch-web/web.crt and web.key with a certificate from your own authority and run sudo systemctl reload nginx.