Deployment guide

Apache Druid

Overview

Apache Druid is an open-source, real-time analytics database built for fast slice-and-dice queries on event and time-series data at scale. It powers sub-second OLAP queries over streaming and batch ingestion through a column-oriented store, a built-in indexing service and a rich web console. This SecureAnyCloud image installs Apache Druid 37.0.0 on OpenJDK 17 as a single-server deployment on Azure.

What you get:

Apache Druid 37.0.0 running on OpenJDK 17, installed at /opt/druid
A complete single-server deployment through bin/start-druid (auto memory-sizing) with the bundled micro-quickstart profile: embedded Apache ZooKeeper, an embedded Derby metadata store and local-disk deep storage
The full coordinator, overlord, broker, historical, middle-manager and router stack under systemd
The Druid web console, which has no authentication of its own, served behind an nginx reverse proxy with HTTP basic auth on port 80 (every Druid service binds loopback 127.0.0.1)
A per-VM console password for user admin, generated on the first boot of every VM and stored in a root-only file
A dedicated Azure data disk at /var/lib/druid holding deep-storage segments, the Derby metadata store, ZooKeeper data and indexing task logs, separate from the OS disk and re-provisioned with every VM
druid.service and nginx.service as systemd units, enabled and active
24/7 SecureAnyCloud support

Prerequisites

Before you deploy this image, make sure you have:

An active Azure subscription, an SSH key pair, and a VNet and subnet in the target region.
A VM size of Standard_B4ms (4 vCPU / 16 GiB RAM) or larger. This is the recommended minimum, because Druid runs several JVM services on one VM and needs the memory. Scale up for heavier ingestion and query loads.
NSG inbound rules that allow 22/tcp from your management network and 80/tcp from the networks that need the Druid console. Front the proxy with TLS for public exposure (see Enabling a trusted certificate).

Step 1 - Deploy from the Azure Marketplace

Sign in to the Azure Portal, choose Create a resource, search the Marketplace for Apache Druid by SecureAnyCloud, and select Create.

On Basics, pick your subscription, resource group, region and size (Standard_B4ms or larger). Under Administrator account, choose SSH public key and paste your key. Under Inbound port rules, allow SSH (22) and HTTP (80).

Review the dedicated data disk on the Disks tab, then choose Review + create and Create.

Step 2 - Deploy from the Azure CLI

# Accept the Marketplace terms once per subscription (required before the first CLI deploy)
az vm image terms accept --urn sac1647283583153:apache-druid-ubuntu-24-04:default:latest
az vm create \
  --resource-group <your-rg> \
  --name druid \
  --image sac1647283583153:apache-druid-ubuntu-24-04:default:latest \
  --size Standard_B4ms \
  --admin-username azureuser \
  --ssh-key-values ~/.ssh/id_ed25519.pub \
  --vnet-name <your-vnet> --subnet <your-subnet> \
  --public-ip-sku Standard
az vm open-port --resource-group <your-rg> --name druid --port 80 --priority 1010

Step 3 - Connect to your VM

ssh azureuser@<vm-public-ip>

Step 4 - Confirm the services are running

Druid runs several JVM services and takes one to three minutes to start fully on first boot. Once it is up, both systemd units report active. The unauthenticated nginx health endpoint answers 200, and Druid's own health endpoint also answers 200:

systemctl is-active druid.service nginx.service curl -s -o /dev/null -w 'nginx health: %{http_code}\n' http://127.0.0.1/health

Both services report active, and GET /health returns 200.

Step 5 - Retrieve your console login

The console username is admin. The password is generated uniquely on the first boot of your VM and written to a root-only file:

sudo cat /root/druid-credentials.txt

The username is admin. The password is the DRUID_ADMIN_PASSWORD value, and DRUID_URL is the address to open in your browser.

Step 6 - Sign in to the Druid web console

Browse to http://<vm-public-ip>/. When the browser prompts for HTTP basic authentication, sign in with admin and the password from Step 5. You land on the Druid unified console, where you can run SQL queries, manage ingestion, browse datasources and inspect the services.

You can confirm authentication from the shell. The console has no native authentication, so nginx enforces it. A request with no credentials and a request with the wrong password are both rejected with 401, while the per-VM password is accepted:

PASS=$(sudo grep '^DRUID_ADMIN_PASSWORD=' /root/druid-credentials.txt | cut -d= -f2-)
echo "no credentials -> $(curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1/)"
echo "wrong password -> $(curl -s -o /dev/null -w '%{http_code}' -u 'admin:wrong' http://127.0.0.1/)"
echo "correct password -> $(curl -sL -o /dev/null -w '%{http_code}' -u "admin:$PASS" http://127.0.0.1/)"

The first two print 401, and the correct password prints 200.

Step 7 - Query the Druid status and cluster APIs

Druid exposes a rich REST API through the same authenticated proxy. Read the build version and the live cluster topology:

PASS=$(sudo grep '^DRUID_ADMIN_PASSWORD=' /root/druid-credentials.txt | cut -d= -f2-)
curl -s -u "admin:$PASS" http://127.0.0.1/status | python3 -c 'import sys,json; d=json.load(sys.stdin); print("Druid version:", d["version"]); print("modules loaded:", len(d["modules"]))'
curl -s -u "admin:$PASS" http://127.0.0.1/druid/coordinator/v1/loadstatus

The status endpoint reports Druid version: 37.0.0 with its loaded extension modules, and the coordinator loadstatus returns a JSON object ({} on a fresh cluster with no segments yet to load).

Step 8 - Ingest your first data

From the console, choose Load data, then Start a new spec. Select a sample input (for example the bundled Wikipedia edits example) or point Druid at your own batch file or streaming source.

Walk through parse, transform, partition and tune, then choose Submit to submit the ingestion spec. When the task completes, the new datasource appears under Datasources. You can query it immediately from the Query view with Druid SQL.

Step 9 - Confirm persistence on the data disk

All of Druid's persistent state lives on the dedicated Azure data disk at /var/lib/druid. Your segments, metadata, ZooKeeper data and task logs survive reboots and are included in any image you capture:

findmnt /var/lib/druid ls /var/lib/druid/var

findmnt shows /var/lib/druid mounted from the data disk, and var holds the druid (deep storage and the Derby metadata store), zk (ZooKeeper data), sv (supervised service state) and tmp subdirectories.

Enabling a trusted certificate

The image serves the console over plain HTTP behind nginx basic auth. For production, terminate TLS at the nginx proxy with a real domain pointed at the VM's public IP. Install certbot and request a certificate, replacing the domain with yours. certbot configures nginx to serve HTTPS on port 443:

sudo apt-get update && sudo apt-get install -y certbot python3-certbot-nginx
sudo certbot --nginx -d your-domain.example.com

After certbot completes, the Druid console is reachable at https://your-domain.example.com/ with a browser-trusted certificate. The basic-auth prompt is unchanged.

Backup and maintenance

Your Druid segments, metadata, ZooKeeper data and task logs live on the data disk at /var/lib/druid. Snapshot that disk to back up your cluster state and ingested data. The Druid binaries and single-server configuration are at /opt/druid/current.

Keep the OS patched with sudo apt update && sudo apt upgrade. The image already applies all available Ubuntu security updates at build time.

The service restarts cleanly with sudo systemctl restart druid. Allow one to three minutes for the cluster to come back up.

To change the console password, regenerate the nginx basic-auth entry with sudo htpasswd -B /etc/nginx/druid.htpasswd admin and reload nginx.

Support

This image is backed by 24/7 SecureAnyCloud support. Contact us by email or chat for help with ingestion specs, datasources, Druid SQL, scaling out to a clustered deployment and TLS.

All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.

Deploy on Azure

Launch Apache Druid on Ubuntu 24.04 LTS on Azure with 24/7 support from SecureAnyCloud.

View this image

Need Help?

Our support team is available 24/7.

sales@secureanycloud.com

Deploy on Azure

Overview

This image runs Apache Druid, a real-time analytics database designed for fast aggregations and interactive, low-latency queries over large event and time-series datasets. It is deployed as a single-server bundle. The coordinator, overlord, broker, historical, middleManager and router processes run together, alongside a bundled Apache ZooKeeper and an embedded metadata store. A systemd service starts them on boot and restarts them on failure.

What you get:

Druid installed under /opt/druid and running as a dedicated unprivileged druid system account.
A dedicated, independently resizable EBS data volume mounted at /var/lib/druid. Druid's entire persistent working tree lives there: the deep-storage segments, the metadata store, the ZooKeeper data and the indexing task logs. Your ingested data survives instance replacement and is captured into the image layout.
The Druid unified web console, served on port 80 by an nginx reverse proxy. Every Druid process binds only to loopback, so the single public entry point is nginx, which enforces HTTP basic authentication in front of the console.
A console password generated on the first boot of every deployed instance, so two instances launched from the same AMI never share a password. It is written to /root/druid-credentials.txt with mode 0600, so only the root user can read it.
JVM memory tuned to the instance size on every boot. A boot-time wrapper reads the instance's total memory, reserves headroom for the operating system, the bundled ZooKeeper and the page cache, and hands the remainder to Druid's automatic memory sizer. The same image fits the recommended m5.large and scales up on larger instances without reconfiguration.

Prerequisites

Before you deploy this image, make sure you have:

An Amazon Web Services account where you can launch EC2 instances
IAM permissions to launch instances, create security groups, and subscribe to AWS Marketplace products
An EC2 key pair in the target Region for SSH access to the instance
A VPC and subnet in the target Region, with a security group allowing inbound port 22 from your management network and port 80 for the web console
The AWS CLI (version 2) installed locally, if you plan to deploy from the command line

Launch the Instance from the AWS Marketplace

Sign in to the AWS Management Console, open the EC2 service, and choose Launch instance. Under Application and OS Images, choose AWS Marketplace AMIs and search for Apache Druid. Select the SecureAnyCloud listing and choose Select, then choose Continue on the subscription summary.

Pick an instance type of m5.large or larger. Druid is a JVM analytics workload, so size the instance for your data volume and query concurrency. Choose your EC2 key pair under Key pair (login).

Under Network settings, select your VPC and subnet. Create or select a security group that opens port 22 from your management network and port 80 for the web console. Leave the root volume at the default size or larger, and keep the attached data volume that holds Druid's segments and metadata.

Choose Launch instance. First-boot initialisation runs immediately. After the instance state becomes Running and the status checks pass, Druid takes one to three minutes to bring up every process and become healthy.

Launch the Instance from the AWS CLI

The following block launches an instance from the SecureAnyCloud Apache Druid Marketplace AMI into an existing subnet and security group. Replace <ami-id> with the AMI ID shown on the Marketplace listing and <key-name> with your EC2 key pair name. Replace <subnet-id> with your subnet ID and <security-group-id> with a security group that opens ports 22 and 80 as described above.

aws ec2 run-instances \
  --image-id <ami-id> \
  --instance-type m5.large \
  --key-name <key-name> \
  --subnet-id <subnet-id> \
  --security-group-ids <security-group-id> \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=apache-druid}]'

When the instance reaches the Running state and its status checks pass, note its public IP address or DNS name. You can find them in the EC2 console or with aws ec2 describe-instances.

Connect to Your Instance

Connect over SSH with your key pair and the login user for your operating system variant.

OS variant SSH login user

Ubuntu 24.04 ubuntu

ssh -i <key-name>.pem ubuntu@<public-ip>

Retrieve the Console Password

The Druid console password is unique to your instance and is generated on first boot. Read it as root:

sudo cat /root/druid-credentials.txt

The file lists the console URL, the administrator user (admin) and the generated password. It is mode 0600 and owned by root, so only the root user can read it.

Every command below that needs the password reads it straight from this file into a shell variable, so you never have to copy or paste it.

Confirm the Services Are Running

Druid and nginx run as systemd services. Confirm that both are active:

systemctl is-active druid.service nginx.service

nginx serves an unauthenticated health endpoint on port 80, which is ideal for load-balancer health checks. Confirm that it answers:

curl -fsS http://127.0.0.1/health

Verify the Authenticated Console

The console itself requires the per-instance password. First, confirm that an unauthenticated request is rejected:

curl -s -o /dev/null -w 'unauthenticated console: HTTP %{http_code}\n' http://127.0.0.1/

You should see HTTP 401. Now read the password from the credentials file into a shell variable, without printing it, and confirm that the authenticated console returns HTTP 200:

PW=$(sudo awk -F= '/^DRUID_ADMIN_PASSWORD=/{print $2}' /root/druid-credentials.txt)
curl -sL -o /dev/null -w 'authenticated console: HTTP %{http_code}\n' -u "admin:$PW" http://127.0.0.1/

Open the Web Console

Browse to http://<public-ip>/ and sign in as user admin with the password from Step 4. The Druid unified console opens on the Datasources view, which lists every datasource with its availability, segment count, total rows and size.

The Query tab gives you an interactive SQL editor. You can run an aggregation over your data and get ranked results in well under a second.

The Tasks view shows your ingestion history and the status of every batch and streaming task.

The Services view lists every Druid process (coordinator, overlord, broker, historical, middleManager and router), so you can confirm that the whole single-server bundle is healthy.

Run a SQL Query from the Command Line

Druid exposes a SQL-over-HTTP endpoint. Run a simple connectivity query that needs no data. The command reads the password from the credentials file, so it is never printed:

PW=$(sudo awk -F= '/^DRUID_ADMIN_PASSWORD=/{print $2}' /root/druid-credentials.txt)
curl -s -u "admin:$PW" -H 'Content-Type: application/json' -XPOST http://127.0.0.1/druid/v2/sql -d '{"query":"SELECT 1 + 1 AS two"}'

The response is [{"two":2}], which confirms that the broker and the SQL engine are answering through the authenticated proxy.

Load the Wikipedia Sample Dataset

Druid ships a sample Wikipedia edit-stream dataset and a batch ingestion spec. Submitting the spec to the overlord creates a wikipedia datasource that you can query. Run the following on the instance, replacing ADMIN_PASSWORD with the password from Step 4:

D=/opt/druid/current sed "s#\"baseDir\" : \"quickstart/tutorial/\"#\"baseDir\" : \"$D/quickstart/tutorial/\"#" \
  "$D/quickstart/tutorial/wikipedia-index.json" > /tmp/wikipedia-index.json curl -s -u "admin:ADMIN_PASSWORD" -H 'Content-Type: application/json' \
  -XPOST http://<public-ip>/druid/indexer/v1/task -d @/tmp/wikipedia-index.json

Watch the task reach SUCCESS in the console Tasks view. A few seconds later its segment is loaded, and the wikipedia datasource appears on the Datasources view. You can then query it, for example:

SELECT page, COUNT(*) AS edits, SUM(added) AS chars_added FROM wikipedia GROUP BY page ORDER BY edits DESC LIMIT 12

To ingest your own batch or streaming data through a guided flow, use the console's Load data wizard instead.

Data Persistence and the Dedicated Data Volume

Druid's entire persistent working tree lives on a dedicated EBS data volume mounted at /var/lib/druid, separate from the operating-system disk. Confirm the mount and its free space:

findmnt /var/lib/druid df -h /var/lib/druid

The segments, the metadata store, the ZooKeeper data and the task logs all live on this volume. You can resize it independently for capacity and IOPS, and your ingested data is decoupled from the instance's root disk. Back it up with Amazon EBS snapshots.

Security Model

Per-instance credentials. The console password is generated on the first boot of every instance and stored in /root/druid-credentials.txt (mode 0600). No shared or default password ships in the image.
Loopback isolation. Every Druid process and the bundled ZooKeeper bind only to 127.0.0.1. Nothing Druid exposes is reachable off the instance except through nginx.
Authenticated front door. nginx on port 80 is the single public entry point, and it enforces HTTP basic authentication in front of the console. Only the unauthenticated /health endpoint is open, for load-balancer checks.
Network hardening. Restrict port 80 to trusted sources in your security group, and deploy in a private VPC subnet where possible. The dedicated data volume can use AWS EBS encryption at rest.

Maintenance

Restart Druid: run sudo systemctl restart druid.service. It can take one to three minutes to become healthy again.
JVM memory: memory is recomputed from the instance's total RAM on every boot, so resizing the instance automatically re-tunes Druid on the next restart. No manual JVM configuration is required.
Upgrades: keep the operating system current with sudo apt-get update && sudo apt-get upgrade.

Support

This image is maintained by SecureAnyCloud with 24/7 technical support by email and live chat. Contact sales@secureanycloud.com for deployment and instance-sizing guidance, ingestion and data-source configuration, query tuning, JVM administration, and any issues, including refund requests.

Deploy on AWS

Launch Apache Druid on AWS with 24/7 support from SecureAnyCloud.

View on Marketplace

Need Help?

Our support team is available 24/7.

sales@secureanycloud.com

Deploy on AWS