SecureAnyCloud
Deployment guide

Fluent Bit

Overview

Fluent Bit is the fast, lightweight, vendor-neutral log and metrics processor and forwarder from the Fluent project (a CNCF graduated project). It collects data from many sources, parses and filters it, and routes it to dozens of destinations, all from a tiny memory footprint.

The SecureAnyCloud image installs Fluent Bit from the official Fluent Bit apt repository as /opt/fluent-bit/bin/fluent-bit. It ships a ready-to-run sample pipeline and enables the built-in HTTP monitoring server, so you can check health and metrics from the moment the VM boots. This is a headless service: the HTTP monitoring server binds to 127.0.0.1 only, so no inbound application port is needed beyond SSH.

What you get:

  • Fluent Bit 5.x (Apache-2.0) from the official Fluent Bit apt repository
  • The package fluent-bit.service systemd unit, enabled and running on boot
  • A sample pipeline at /etc/fluent-bit/fluent-bit.conf: a cpu input, a record_modifier filter, and two outputs (a rotating file under /var/log/fluent-bit/ and stdout)
  • The built-in HTTP monitoring server on 127.0.0.1:2020 (/api/v1/health, /api/v1/metrics, /api/v1/uptime)
  • 24/7 SecureAnyCloud support

Prerequisites

You need an active Azure subscription, an SSH key pair, and a VNet with a subnet in the target region. Standard_B2s (2 vCPU / 4 GiB RAM) is a good starting point.

On the network security group (NSG), allow inbound 22/tcp from your management network. No inbound application ports are required.

Step 1 - Deploy from the Azure Marketplace

Sign in to the Azure Portal, choose Create a resource, search the Marketplace for Fluent Bit by SecureAnyCloud, and select Create. On the Basics tab, pick your subscription, resource group, region and size. Under Administrator account, choose SSH public key and paste your key. Under Inbound port rules, allow SSH (22). Then select Review + create and Create.

Step 2 - Deploy from the Azure CLI

# Accept the Marketplace terms once per subscription (required before the first CLI deploy)
az vm image terms accept --urn sac1647283583153:fluent-bit-ubuntu-24-04:default:latest
az vm create \
  --resource-group <your-rg> \
  --name fluent-bit \
  --image sac1647283583153:fluent-bit-ubuntu-24-04:default:latest \
  --size Standard_B2s \
  --admin-username azureuser \
  --ssh-key-values ~/.ssh/id_ed25519.pub \
  --vnet-name <your-vnet> --subnet <your-subnet> \
  --public-ip-sku Standard

Step 3 - Connect to your VM

ssh azureuser@<vm-public-ip>

Step 4 - Confirm Fluent Bit is installed and running

Check the version and that the service is active:

fluent-bit --version systemctl is-active fluent-bit

fluent-bit --version reports Fluent Bit v5.x, and systemctl is-active fluent-bit prints active.

Step 5 - Review the sample pipeline

The ready-to-run pipeline lives at /etc/fluent-bit/fluent-bit.conf. It enables the HTTP monitoring server, reads CPU metrics every second (a guaranteed source of records), stamps each record with the hostname, and fans the stream out to a file and to stdout:

cat /etc/fluent-bit/fluent-bit.conf

Step 6 - Check the HTTP monitoring server health

Fluent Bit's built-in HTTP server exposes a liveness endpoint that returns 200 ok when the engine is healthy:

curl -s http://127.0.0.1:2020/api/v1/health

It prints ok.

Step 7 - Inspect live pipeline metrics

The /api/v1/metrics endpoint returns JSON with per-plugin record and byte counters. Watch the input, filter and output counters climb as the pipeline processes records:

curl -s http://127.0.0.1:2020/api/v1/metrics | head -c 600

The input object shows the cpu plugin's records count, and the output object shows the file and stdout plugins each processing those records.

Step 8 - Watch records flow through the pipeline

The file output writes every record to /var/log/fluent-bit/records.log. Tail it to see the live stream. Each line is a CPU sample stamped with your hostname and the sac-sample pipeline tag:

tail -n 5 /var/log/fluent-bit/records.log

Step 9 - Customise the pipeline

Edit /etc/fluent-bit/fluent-bit.conf to point Fluent Bit at your own sources and destinations, then restart the service. A few common building blocks:

# Tail application log files [INPUT] Name tail Path /var/log/myapp/*.log Tag myapp
# Forward to Elasticsearch / OpenSearch [OUTPUT] Name es Match * Host my-search-host Port 9200
# Forward to Loki, Kafka, S3, Azure Blob, HTTP, and many more

After editing, apply your changes:

sudo systemctl restart fluent-bit

Fluent Bit ships dozens of input, parser, filter and output plugins (tail, systemd, syslog, HTTP, Prometheus, OpenTelemetry, Elasticsearch, Loki, Kafka, S3, Azure, and more). See the Fluent Bit documentation for the full catalogue.

Maintenance

  • Service: systemd manages the service. Use systemctl status fluent-bit and journalctl -u fluent-bit.
  • Configuration: The files are /etc/fluent-bit/fluent-bit.conf and /etc/fluent-bit/parsers.conf. Restart the service after edits.
  • Monitoring: The HTTP server on 127.0.0.1:2020 exposes /api/v1/health, /api/v1/metrics and /api/v1/uptime. Scrape /api/v1/metrics/prometheus for Prometheus.
  • Upgrades: The package is held at the shipped version. To upgrade, run sudo apt-mark unhold fluent-bit && sudo apt-get update && sudo apt-get install --only-upgrade fluent-bit.
  • Security patches: unattended-upgrades remains enabled, so the OS continues to receive security updates automatically.

Support

SecureAnyCloud provides 24/7 expert support for this image. Contact sales@secureanycloud.com.

Fluent Bit is a trademark of the Fluent project, hosted by the Cloud Native Computing Foundation. SecureAnyCloud is not affiliated with or endorsed by the Fluent project or the CNCF.

Deploy on Azure

Launch Fluent Bit on Ubuntu 24.04 LTS on Azure with 24/7 support from SecureAnyCloud.

View this image

Need Help?

Our support team is available 24/7.

sales@secureanycloud.com

Deploy on Azure