Deployment guide

BlazingMQ 0.95.20

Overview

BlazingMQ is an open source message queueing system from Bloomberg. Its high-performance C++ broker (bmqbrkr) gives applications durable queues, either persistent or in-memory, with priority (work-queue), fan-out and broadcast delivery, consumer acknowledgements (confirms), message properties and subscriptions. Applications connect with the BlazingMQ C++ or Java SDK, and the bmqtool command line client publishes and consumes for testing and operations.

This SecureAnyCloud AMI runs BlazingMQ 0.95.20 on Ubuntu 24.04 on Amazon EC2. It is one software version of the SecureAnyCloud streaming and messaging listing on AWS Marketplace, which you select when you subscribe and launch.

What you get:

  • Built for you. BlazingMQ publishes no prebuilt broker. This image ships bmqbrkr and bmqtool compiled from the BlazingMQ 0.95.20 source release, against the exact BDE 4.39.0.0 and NTF 2.6.12 libraries that upstream pins for that release. Every source archive is verified by its SHA-256 digest. No compiler or build tree is left on the image.
  • Reachable from the instance only. Out of the box, BlazingMQ listens on every network interface. Here the broker listens on 127.0.0.1:30114 only, set in its own configuration. A check after every start stops the broker if it ever listens anywhere else. You use the broker from applications on the instance, or from your workstation through an SSH tunnel (Step 7). The security group needs nothing but SSH.
  • Authentication required. Out of the box, BlazingMQ lets every client in without credentials. Here the broker requires BASIC authentication and refuses anonymous clients when the session is negotiated.
  • A credential generated on your instance. At first boot, each instance creates its own broker password (160 bits of randomness) for the user bmquser. The password is never in the image and never on a command line, and only root can read the files that hold it. Rotate it with one command (Step 10).
  • A credential that stays out of the logs. Upstream BlazingMQ writes its whole configuration file, credential included, to its log at every start, and its BROKERCONFIG DUMP admin command returns it. This image carries a small SecureAnyCloud source patch (shipped as source, Step 12). The patch logs only the configuration directory and redacts every credential in BROKERCONFIG DUMP.
  • Fail closed. The broker refuses to start until first boot has generated the credential. A configuration check refuses any configuration that would let anonymous clients in or listen beyond 127.0.0.1.
  • Hardened defaults. The broker runs as the unprivileged blazingmq user in a systemd sandbox (no capabilities, a read-only system, writes only to its own storage and log directories). It can reach nothing but 127.0.0.1, and both BlazingMQ services are blocked from the instance metadata service. The AMI requires IMDSv2.
  • Fully patched Ubuntu 24.04 LTS at build time, with unattended security upgrades switched on.
  • On the image: BlazingMQ 0.95.20 bmqbrkr and bmqtool (statically linked with BDE 4.39.0.0 and NTF 2.6.12), Python 3 and OpenSSL from Ubuntu 24.04 (the first-boot helpers), cloud-init and the AWS Systems Manager agent.
  • Defaults: default user ubuntu, x86_64 architecture, recommended instance type m5.large.

Prerequisites

  • An AWS account subscribed to the listing.
  • An EC2 key pair.
  • A VPC subnet with a route to the internet, or a private route you use for management.
  • A security group that allows TCP 22 (SSH) from your admin network. This is the only port to open. The broker port 30114 is never reachable from other machines, whatever the security group says.
  • Enough root volume for your queues. Message storage lives on the root volume under /var/lib/blazingmq/storage. The AMI's root volume is 8 GiB, so for production, launch with a larger root volume sized for your queues. The root partition grows to the volume size at first boot.
1

Launch the instance

In the AWS Marketplace listing, choose Continue to Subscribe, then Continue to Configuration. Select BlazingMQ 0.95.20 on Ubuntu 24.04 as the software version, pick your Region, and choose Launch through EC2.

Choose m5.large, your key pair, the security group described above and the root volume size you need.

To launch with the AWS CLI instead, replace the values in angle brackets:

aws ec2 run-instances --region <region> --image-id <ami-id-from-the-listing> \
  --instance-type m5.large --key-name <your-key-pair> \
  --security-group-ids <your-security-group> --subnet-id <your-subnet> \
  --block-device-mappings 'DeviceName=/dev/sda1,Ebs={VolumeSize=50,VolumeType=gp3}' \
  --metadata-options HttpTokens=required,HttpEndpoint=enabled \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=blazingmq}]'

First boot takes a few seconds once the instance is running. The boot service generates the credential and renders the broker configuration, then the broker starts.

2

Connect to your instance

On BlazingMQ 0.95.20 on Ubuntu 24.04, the SSH user is ubuntu. Connect with the key pair you chose at launch:

ssh -i <your-key.pem> ubuntu@<instance-public-dns>
3

Check the services

systemctl is-active blazingmq-firstboot blazingmq
sudo test -f /var/lib/sac/data/blazingmq-firstboot.done && echo FIRSTBOOT_DONE

Both units should be active, and the first-boot marker should be present:

active active FIRSTBOOT_DONE

The broker listens on the loopback address only. SSH is the only service that other machines can reach:

sudo ss -lntp | awk 'NR>1 {print $4, $6}' | sed -E 's/,pid=[0-9]+,fd=[0-9]+//g' | sort
0.0.0.0:22 users:(("sshd"),("systemd")) 127.0.0.1:30114 users:(("bmqbrkr")) 127.0.0.53%lo:53 users:(("systemd-resolve")) 127.0.0.54:53 users:(("systemd-resolve")) [::]:22 users:(("sshd"),("systemd"))
4

Your credential

First boot writes two root-only files:

  • /root/blazingmq-credentials.txt holds the broker address, the mechanism, the user and the password as KEY=VALUE lines.
  • /root/blazingmq-bmqtool.json is a bmqtool profile that holds the same broker address and credential, so bmqtool never needs the password on its command line.

List the keys without printing any value:

sudo ls -l /root/blazingmq-credentials.txt /root/blazingmq-bmqtool.json | awk '{print $1, $3, $4, $NF}'
sudo grep -oE '^[A-Z_]+=' /root/blazingmq-credentials.txt
-rw------- root root /root/blazingmq-bmqtool.json -rw------- root root /root/blazingmq-credentials.txt BMQ_BROKER_URI= BMQ_AUTHN_MECHANISM= BMQ_USER= BMQ_PASSWORD= <set-at-first-boot>=

BMQ_BROKER_URI is tcp://127.0.0.1:30114, BMQ_AUTHN_MECHANISM is BASIC and BMQ_USER is bmquser.

To read the password when you configure an application, use sudo grep '^BMQ_PASSWORD=' /root/blazingmq-credentials.txt | cut -d= -f2-. Paste the password straight into that application's own secret store, and avoid leaving it in your shell history.

BlazingMQ 0.95.20 has no per-user permissions. Every authenticated client may publish, consume and run the broker's admin commands. Treat this credential as an administrator credential and give it only to applications you trust.

5

Run the self-check

The image ships a self-check that proves the security defaults on your own instance, with your own credential. It verifies each of these:

  • The listener is on the loopback address only.
  • An authenticated client can publish and consume.
  • An anonymous client and a wrong password are both refused.
  • Instance metadata is blocked inside the broker.
  • The credential is redacted from the admin dump and is absent from every log and the journal.

The self-check takes about two minutes:

sudo blazingmq-selfcheck
ok: first boot done; blazingmq and blazingmq-firstboot healthy; credential files root-only (broker config 0640 root:blazingmq) ok: bmqbrkr listens on 127.0.0.1:30114 only; refused on this host's 1 other address(es) ok: authenticated publish -> consume round trip on bmq://bmq.test.persistent.priority/sac-selfcheck (CIMG1793271533) ok: anonymous client refused ('Anonymous credential is disallowed'); wrong password refused ('Invalid username or password for user 'bmquser''); nothing they sent was queued ok: metadata 000 (v4 and v4-mapped) inside the broker's cgroup, loopback 30114 open there; control 200 from this shell ok: sac patch: BROKERCONFIG DUMP answers <redacted>; the start-up log names only the config directory ok: credential absent from /var/log, /var/lib/blazingmq and the journal (OK 2 value(s) absent from 16724 entries / 16968830 export bytes (export + json-decoded; ANSI control FOUND)); no ANSI colour ok: no ordering cycle in this boot SELFCHECK OK
6

Publish and consume with bmqtool

bmqtool reads the broker address and the credential from the profile file through --profile. Publish three messages to a persistent priority queue and wait for the broker's acknowledgements:

sudo bmqtool --profile /root/blazingmq-bmqtool.json --mode auto \
  --queueuri bmq://bmq.test.persistent.priority/guide-demo --queueflags write,ack \
  --eventscount 3 --eventsize 1 --messagepattern hello --shutdownGrace 3 2>&1 | grep -a -E '^ *ACK|Final stats'
ACK | 3| 3| 108 B Final stats: produced 3 messages [27 B] in 3 events [180 B] (Protocol: 85%)

Consume the messages, print each one and confirm it. A confirmed message is removed from the queue:

sudo bmqtool --profile /root/blazingmq-bmqtool.json --mode auto \
  --queueuri bmq://bmq.test.persistent.priority/guide-demo --queueflags read \
  --dumpmsg --confirmmsg --shutdownGrace 5 2>&1 | grep -a -E '^ *0:|CONFIRM'
0: 68656C6C 6F303030 30303030 3030 |hello000000000 | 0: 68656C6C 6F303030 30303030 3031 |hello000000001 | 0: 68656C6C 6F303030 30303030 3032 |hello000000002 | CONFIRM| 3| 1| 84 B

bmqtool --mode cli (the default mode) opens an interactive prompt where you can start, open uri="bmq://..." flags="write,ack", post uri="bmq://..." payload=["text"] and close. bmqtool --help lists every option.

An unprivileged user can run bmqtool with a copy of the profile that user owns:

sudo install -o <app-user> -g <app-user> -m 0600 /root/blazingmq-bmqtool.json /home/<app-user>/blazingmq-bmqtool.json
7

Connect from your workstation (SSH tunnel)

The broker is reachable from the instance only. To use it from a workstation, forward a local port to the instance's loopback address over SSH. Then point your client at tcp://127.0.0.1:30114 on your workstation:

ssh -i <your-key.pem> -N -L 30114:127.0.0.1:30114 ubuntu@<instance-public-dns>

The tunnel carries the BlazingMQ protocol inside SSH, so the credential and the messages are encrypted between your workstation and the instance.

For applications on other EC2 instances, run the same tunnel from each client instance (for example as a systemd unit with autossh), or run the application on this instance.

8

Client SDKs with the credential from a file

Every SDK example below reads the credential from a file that the application owns, never from the command line or the source code. Copy /root/blazingmq-credentials.txt to that file with sudo install -o <app-user> -m 0600 ....

Java (blazingmq-sdk-java, Maven com.bloomberg.bmq:bmq-sdk:0.0.18, the first release with authentication support). The credentials file is KEY=VALUE lines, so java.util.Properties reads it as is:

Properties creds = new Properties(); try (FileInputStream in = new FileInputStream("/etc/myapp/blazingmq.properties")) { creds.load(in); } final byte[] authn = (creds.getProperty("BMQ_USER") + ":" + creds.getProperty("BMQ_PASSWORD")) .getBytes(StandardCharsets.UTF_8); SessionOptions options = SessionOptions.builder() .setBrokerUri(URI.create(creds.getProperty("BMQ_BROKER_URI"))) .setAuthnCredentialCb(() -> AuthnCredential.builder() .setMechanism("BASIC") .setData(authn) .build()) .build(); AbstractSession session = new Session(options, event -> {}); session.start(Duration.ofSeconds(15)); Queue queue = session.getQueue(new Uri("bmq://bmq.test.persistent.priority/java-demo"), QueueFlags.setWriter(0), event -> {}, null, null); queue.open(QueueOptions.createDefault(), Duration.ofSeconds(15)); queue.post(queue.createPutMessage(ByteBuffer.wrap("hello from Java".getBytes(StandardCharsets.UTF_8))));

The SDK logs Sending authentication message (mechanism: BASIC) when the session starts, and never logs the credential. It re-authenticates on its own before the broker's 10-minute authentication lifetime ends.

C++ (the BlazingMQ C++ SDK libbmq from the same 0.95.20 source release). Build your application on a build host with upstream's bin/build-ubuntu.sh, because this image carries no compiler. Give the session a credential callback that reads the file:

#include <bmqa_session.h> #include <bmqt_authncredential.h> #include <bmqt_sessionoptions.h> static bsl::optional<bmqt::AuthnCredential> loadCredential() { bsl::ifstream in("/etc/myapp/blazingmq.properties"); bsl::string line, user, password; while (bsl::getline(in, line)) { if (line.rfind("BMQ_USER=", 0) == 0) user = line.substr(9); if (line.rfind("BMQ_PASSWORD=", 0) == 0) password = line.substr(13); } const bsl::string data = user + ":" + password; return bmqt::AuthnCredential("BASIC", bsl::vector<char>(data.begin(), data.end())); } bmqt::SessionOptions options; options.setBrokerUri("tcp://127.0.0.1:30114").setAuthnCredentialCb(&loadCredential); bmqa::Session session(options); session.start();

Python: the BlazingMQ Python SDK (blazingmq on PyPI, 1.2.2) has no authentication support yet. It cannot connect to a broker that requires authentication, and this image requires it. Use the Java or C++ SDK, or bmqtool, until a Python SDK release adds authentication.

9

Domains and queues

Queues live in domains. A queue URI is bmq://<domain>/<queue-name>, and queues are created on first use inside an existing domain. The image ships upstream's seven ready-to-use domains, one per delivery mode and storage type:

sudo ls /etc/blazingmq/domains
bmq.test.mem.broadcast.json bmq.test.mem.fanout.json bmq.test.mem.priority.json bmq.test.persistent.fanout.json bmq.test.persistent.fanout.sc.json bmq.test.persistent.priority.json bmq.test.persistent.priority.sc.json
  • priority is a work queue that delivers each message to one consumer.
  • fanout delivers every message to each named application (appIds).
  • broadcast delivers to every connected consumer without storing messages.
  • persistent domains keep messages on disk under /var/lib/blazingmq/storage, and mem domains keep them in memory.

To add your own domain, copy one of these files to /etc/blazingmq/domains/<your.domain>.json (owner root, group blazingmq, mode 0640). Edit its limits (maxQueues, storage.domainLimits, storage.queueLimits, maxDeliveryAttempts, messageTtl), then restart the broker:

sudo install -o root -g blazingmq -m 0640 /etc/blazingmq/domains/bmq.test.persistent.priority.json /etc/blazingmq/domains/orders.priority.json
sudo nano /etc/blazingmq/domains/orders.priority.json
sudo systemctl restart blazingmq

The broker's own settings are rendered at first boot into /etc/blazingmq/bmqbrkrcfg.json (it holds the credential). The single-node cluster with its storage limits is in /etc/blazingmq/clusters.json. The BlazingMQ configuration documentation describes every field.

10

Admin commands and credential rotation

The broker accepts admin commands on its control pipe. blazingmq-ctl sends a command as the broker's own user, and the answer goes to the broker's log:

sudo blazingmq-ctl HELP
sudo journalctl -u blazingmq -n 150 -o cat --no-pager | grep -a -E "^ +(HELP|BROKERCONFIG DUMP|DOMAINS DOMAIN <name> INFOS)$"
blazingmq-ctl: sent 'HELP' (the answer is in: journalctl -u blazingmq.service -n 200) HELP BROKERCONFIG DUMP DOMAINS DOMAIN <name> INFOS

To replace the credential, for example after an application that held it is retired, generate a new one. The broker restarts and the old password is refused at once. Update every application's copy afterward:

sudo blazingmq-rotate-credential
ROTATED: the new credential is in /root/blazingmq-credentials.txt and /root/blazingmq-bmqtool.json; the old one is refused
11

Instance metadata blocked for the broker

Both BlazingMQ services run with a kernel filter that blocks the instance metadata service (169.254.169.254, its IPv6 form and its IPv4-mapped form). The broker may talk to the loopback address only:

systemctl show -p IPAddressAllow -p IPAddressDeny blazingmq.service grep -E '^IPAddress' /etc/systemd/system/blazingmq.service
IPAddressAllow=127.0.0.0/8 ::1/128 IPAddressDeny=::/0 0.0.0.0/0 IPAddressAllow=localhost IPAddressDeny=any IPAddressDeny=169.254.169.254/32 168.63.129.16/32 fd00:ec2::254/128 ::ffff:169.254.169.254/128 ::ffff:168.63.129.16/128

Your own tools on the instance still reach the metadata service through IMDSv2 as usual.

12

Provenance and the SecureAnyCloud patch

The source patch, its SHA-256 sum and a README ship in /opt/blazingmq/sac-patches/. Every source digest and the binaries' digests are in /usr/share/doc/blazingmq/PROVENANCE:

cd /opt/blazingmq/sac-patches && sha256sum -c SHA256SUMS grep -E '^ (blazingmq|bde |ntf-core)' /usr/share/doc/blazingmq/PROVENANCE | awk '{print $1, $2}'
0001-sac-no-credentials-in-logs.patch: OK blazingmq v0.95.20 bde 4.39.0.0 ntf-core 2.6.12

bmqbrkr --version prints upstream's fixed development version 99.99.99, because BlazingMQ's build never stamps the release number into the binary. Upstream's own container image behaves the same way. PROVENANCE is the record of what was built.

Logs and troubleshooting

  • Broker log: sudo journalctl -u blazingmq (console) and /var/log/blazingmq/logs.* (files, rotated at 256 MiB, kept 10 days).
  • First boot: sudo journalctl -u blazingmq-firstboot.
  • Broker does not start: the journal names the reason. blazingmq-config-gate: REFUSED: ... means the configuration would allow anonymous clients, has no credential or listens beyond 127.0.0.1.
  • Storage: /var/lib/blazingmq/storage, one partition of up to 256 MiB of data and 64 MiB of journal by default (clusters.json, partitionConfig).

Licenses

BlazingMQ, BDE and NTF are licensed under the Apache License 2.0. The license texts, upstream's NOTICES.txt and the third-party notices for binary distribution are in /usr/share/doc/blazingmq/.

Support

SecureAnyCloud provides support for this image by email at sales@secureanycloud.com.

BlazingMQ is a trademark of Bloomberg L.P. SecureAnyCloud is not affiliated with or endorsed by Bloomberg.

Deploy on AWS

Launch BlazingMQ 0.95.20 on Ubuntu 24.04 on AWS with 24/7 support from SecureAnyCloud.

View on Marketplace

Need Help?

Our support team is available 24/7.

sales@secureanycloud.com

Deploy on AWS