SecureAnyCloud
Deployment guide

Fluentd

Overview

Fluentd is the open source unified logging layer from the Fluent project, a CNCF graduated project. It collects data from many sources, structures it as JSON, filters and buffers it, and routes it to dozens of destinations through a rich plugin ecosystem.

This SecureAnyCloud image installs Fluentd from the official Fluentd fluent-package LTS apt repository as /opt/fluent/bin/fluentd. It ships a ready-to-run sample pipeline and enables the built-in monitor_agent HTTP endpoint, so you can inspect health and per-plugin metrics from the moment the VM boots.

What you get:

  • Fluentd 1.16 (fluent-package 5 LTS, Apache-2.0) from the official Fluentd apt repository
  • The package fluentd.service systemd unit, enabled and running on boot
  • A sample pipeline at /etc/fluent/fluentd.conf: a dummy input, a record_transformer filter, and a copy output that fans out to a rotating file under /var/log/fluentd/ and to stdout
  • The built-in monitor_agent HTTP endpoint on 127.0.0.1:24220 (/api/plugins.json, /api/config.json)
  • 24/7 SecureAnyCloud support

This is a headless service. The monitor_agent HTTP endpoint binds to 127.0.0.1 only, so no inbound application port is needed beyond SSH.

Prerequisites

You need an active Azure subscription, an SSH key pair, and a VNet with a subnet in the target region.

Standard_B2s (2 vCPU / 4 GiB RAM) is a good starting size.

Configure the network security group (NSG) to allow inbound 22/tcp from your management network. No inbound application ports are required.

Step 1 - Deploy from the Azure Marketplace

Sign in to the Azure Portal, choose Create a resource, search the Marketplace for Fluentd by SecureAnyCloud, and select Create.

On the Basics tab, pick your subscription, resource group, region and size. Under Administrator account, choose SSH public key and paste your key. Under Inbound port rules, allow SSH (22).

Select Review + create, then Create.

Step 2 - Deploy from the Azure CLI

# Accept the Marketplace terms once per subscription (required before the first CLI deploy)
az vm image terms accept --urn sac1647283583153:fluentd-ubuntu-24-04:default:latest
az vm create \
  --resource-group <your-rg> \
  --name fluentd \
  --image sac1647283583153:fluentd-ubuntu-24-04:default:latest \
  --size Standard_B2s \
  --admin-username azureuser \
  --ssh-key-values ~/.ssh/id_ed25519.pub \
  --vnet-name <your-vnet> --subnet <your-subnet> \
  --public-ip-sku Standard

Step 3 - Connect to your VM

Connect over SSH with the administrator account you created at deployment:

ssh azureuser@<vm-public-ip>

Step 4 - Confirm Fluentd is installed and running

Check the version and confirm that the service is active:

fluentd --version systemctl is-active fluentd

fluentd --version reports fluent-package 5.0.x fluentd 1.16.x, and systemctl is-active fluentd prints active.

Step 5 - Review the sample pipeline

The ready-to-run pipeline lives at /etc/fluent/fluentd.conf. Print it and read through it:

  • It enables the monitor_agent HTTP endpoint.
  • The dummy input generates a sample record every second, so there is always a source of records.
  • The record_transformer filter stamps each record with the hostname and a pipeline tag.
  • The copy output fans the matched stream out to a file and to stdout.
cat /etc/fluent/fluentd.conf

Step 6 - Inspect live plugin metrics

Fluentd's built-in monitor_agent exposes /api/plugins.json. It returns 200 with JSON that lists every configured plugin and its per-plugin counters. Query it from the VM and pretty-print the result with jq:

curl -s http://127.0.0.1:24220/api/plugins.json | jq '.plugins[] | {type, plugin_id}'

The output lists the monitor_agent source, the dummy input, the record_transformer filter, and the copy, file and stdout output plugins.

Step 7 - Watch records flow through the pipeline

The file output writes every record to /var/log/fluentd/records.<date>.log. Tail the file to see the live stream. Each line is a JSON record stamped with your hostname and the sac-sample pipeline tag:

tail -n 5 /var/log/fluentd/records.*.log

Step 8 - Customize the pipeline

Edit /etc/fluent/fluentd.conf to point Fluentd at your own sources and destinations, then restart the service. These are a few common building blocks:

# Tail application log files <source> @type tail path /var/log/myapp/*.log pos_file /var/log/fluentd/myapp.pos tag myapp <parse> @type none </parse> </source>
# Forward to Elasticsearch / OpenSearch <match myapp.**> @type elasticsearch host my-search-host port 9200 </match>
# Forward to Loki, Kafka, S3, Azure, HTTP, and many more

After you edit the file, apply your changes:

sudo systemctl restart fluentd

Fluentd has a large plugin ecosystem, including tail, syslog, http, forward, Elasticsearch, Loki, Kafka, S3 and Azure. See the Fluentd documentation for the full plugin catalog. Install extra plugins with sudo /opt/fluent/bin/fluent-gem install <plugin>.

Maintenance

  • Service: systemd manages Fluentd. Use systemctl status fluentd and journalctl -u fluentd.
  • Configuration: edit /etc/fluent/fluentd.conf, then restart the service.
  • Monitoring: the monitor_agent HTTP endpoint on 127.0.0.1:24220 exposes /api/plugins.json and /api/config.json for health and per-plugin metrics.
  • Plugins: install Ruby gem plugins with sudo /opt/fluent/bin/fluent-gem install <plugin>.
  • Security patches: unattended-upgrades stays enabled, so the OS keeps receiving security updates automatically. The fluent-package package stays updatable through apt.

Support

SecureAnyCloud provides 24/7 expert support for this image. Contact us at sales@secureanycloud.com.

Fluentd is a trademark of the Fluent project, hosted by the Cloud Native Computing Foundation. SecureAnyCloud is not affiliated with or endorsed by the Fluent project or the CNCF.

Deploy on Azure

Launch Fluentd on Ubuntu 24.04 LTS on Azure with 24/7 support from SecureAnyCloud.

View this image

Need Help?

Our support team is available 24/7.

sales@secureanycloud.com

Deploy on Azure