Mosquitto 2

Check that the service is running and that the broker listens on port 1883:

sudo systemctl status mosquitto.service --no-pager sudo ss -tln | grep 1883

Read the per-VM password from the credentials file:

sudo cat /stage/scripts/mosquitto-credentials.log

Subscribe in one terminal and publish from another to confirm that the broker accepts the login and delivers messages:

PASS=$(sudo grep '^MOSQUITTO_PASSWORD=' /stage/scripts/mosquitto-credentials.log | cut -d= -f2-)
# Terminal A: subscribe to all topics mosquitto_sub -h localhost -u sac -P "${PASS}" -t '#' -v
# Terminal B: publish mosquitto_pub -h localhost -u sac -P "${PASS}" -t sensors/temp -m 22.5 mosquitto_pub -h localhost -u sac -P "${PASS}" -t devices/01/state -m online

These are the files that make up the installation:

Component Path

Binary /usr/sbin/mosquitto

Main config /etc/mosquitto/mosquitto.conf

SecureAnyCloud overrides /etc/mosquitto/conf.d/sac.conf

Password file /etc/mosquitto/passwd (mode 0640 root:mosquitto)

Persistence /var/lib/mosquitto/

Log /var/log/mosquitto/mosquitto.log

Credentials /stage/scripts/mosquitto-credentials.log

• Restrict the NSG so that port 1883 only reaches IoT clients.
• Enable TLS on port 8883 with certificate and CA files in conf.d/.
• Use per-client ACLs through acl_file in conf.d/.
• Patch monthly with apt-get update && apt-get upgrade && reboot.

In the AWS Marketplace listing, choose Continue to Subscribe, then Continue to Configuration. Pick Mosquitto 2 MQTT Broker on Ubuntu 24.04 as the software version and select your Region. Then choose Launch through EC2.

Choose m5.large, or a larger type for many thousands of connections or high message rates. Select your key pair and a security group as described above.

To launch with the AWS CLI instead, replace the values in angle brackets:

aws ec2 run-instances --region <region> --image-id <ami-id-from-the-listing> \
  --instance-type m5.large --key-name <your-key-pair> \
  --security-group-ids <your-security-group> --subnet-id <your-subnet> \
  --metadata-options HttpTokens=required,HttpEndpoint=enabled \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=mosquitto-broker}]'

The image requires IMDSv2 (session tokens).

First boot takes under a minute after the instance is running. The boot service reads the instance's EC2 names and creates the logins and the certificate. Then the broker starts and the guard checks it.

Version SSH user

Mosquitto 2 MQTT Broker on Ubuntu 24.04 ubuntu

ssh -i <your-key.pem> ubuntu@<public-ip>

The login banner shows where the logins and the certificate are stored.

Confirm that the three services are active and that the broker listens only on the expected ports:

systemctl is-active mosquitto-metadata-names mosquitto-firstboot mosquitto
sudo ss -tlnp | grep -E ':(22|80|443|1883|8080|8081|8883|9001) '
sudo ss -ulnp | grep mosquitto || echo 'no UDP socket'

active active active LISTEN 0 100 0.0.0.0:8883 0.0.0.0:* users:(("mosquitto",pid=1047,fd=5)) LISTEN 0 4096 0.0.0.0:22 0.0.0.0:* users:(("sshd",pid=1189,fd=3),("systemd",pid=1,fd=59)) LISTEN 0 100 [::]:8883 [::]:* users:(("mosquitto",pid=1047,fd=6)) LISTEN 0 4096 [::]:22 [::]:* users:(("sshd",pid=1189,fd=4),("systemd",pid=1,fd=60)) no UDP socket

Each service has one job:

• mosquitto-metadata-names reads this instance's EC2 names for the certificate.
• mosquitto-firstboot prepares the logins and the certificate.
• mosquitto is the broker. It listens on 8883 (IPv4 and IPv6) and nothing else.

Everything generated for your instance is in /root/mosquitto-credentials.txt, which only root can read. View the whole file with sudo cat /root/mosquitto-credentials.txt. To show only the lines that are not secret, run:

sudo grep -E '^MOSQUITTO_(HOST|MQTTS_URL|PORT|PRIVATE_IP|ADMIN_USER|APP_USER|APP_TOPICS|TLS_CA_FILE|TLS_CERT_SHA256|TLS_PUBKEY_SHA256)=' /root/mosquitto-credentials.txt

MOSQUITTO_HOST=44.204.185.170 MOSQUITTO_MQTTS_URL=mqtts://44.204.185.170:8883 MOSQUITTO_PORT=8883 MOSQUITTO_PRIVATE_IP=172.31.89.225 MOSQUITTO_ADMIN_USER=sac-admin MOSQUITTO_APP_USER=sac-app MOSQUITTO_APP_TOPICS=app/# MOSQUITTO_TLS_CA_FILE=/etc/mosquitto/tls/mosquitto.crt MOSQUITTO_TLS_CERT_SHA256=EB:D0:A2:74:C4:B0:EB:34:11:A8:CD:67:13:BB:33:AD:B2:7F:3E:68:4E:D7:87:40:FF:CD:98:E3:8F:58:D3:9D MOSQUITTO_TLS_PUBKEY_SHA256=6de06a0c9fda4b9eedf6690a35fbeed31b1f687beb5cf159e3f8373b727a86cc

The file also holds MOSQUITTO_ADMIN_PASSWORD and MOSQUITTO_APP_PASSWORD. Give devices and applications the sac-app login or a scoped user of their own (Step 8), never the administrator's.

Keep passwords in files with mode 0600, not in shell history or on command lines. Every user on a machine can see a command line through ps.

The broker's certificate is self-signed and made on your instance. Inspect it:

sudo openssl x509 -in /etc/mosquitto/tls/mosquitto.crt -noout -subject -enddate -ext subjectAltName,basicConstraints
sudo openssl x509 -in /etc/mosquitto/tls/mosquitto.crt -noout -fingerprint -sha256

subject=CN = mosquitto-broker, O = sac notAfter=Feb 10 06:45:22 2029 GMT X509v3 Subject Alternative Name: IP Address:44.204.185.170, DNS:ec2-44-204-185-170.compute-1.amazonaws.com, IP Address:172.31.89.225, DNS:ip-172-31-89-225.ec2.internal, IP Address:127.0.0.1, DNS:localhost X509v3 Basic Constraints: critical CA:FALSE sha256 Fingerprint=EB:D0:A2:74:C4:B0:EB:34:11:A8:CD:67:13:BB:33:AD:B2:7F:3E:68:4E:D7:87:40:FF:CD:98:E3:8F:58:D3:9D

CA:FALSE means it is an end-entity certificate that cannot sign anything else. The SHA-256 fingerprint is the MOSQUITTO_TLS_CERT_SHA256 value in your credentials file.

Copy the certificate to each client machine. The certificate is public, and the private key never leaves the instance:

client-host$ ssh -i <your-key.pem> ubuntu@<public-ip> 'sudo cat /etc/mosquitto/tls/mosquitto.crt' > mosquitto.crt client-host$ openssl x509 -in mosquitto.crt -noout -fingerprint -sha256

Compare the fingerprint with MOSQUITTO_TLS_CERT_SHA256 before you trust the file. Then give your client only this file as its CA file, with host name checking on. Use --cafile for mosquitto_pub and mosquitto_sub, or tls_set(ca_certs=...) in Paho.

Do not add the file to the operating system's trust store. The client should trust this one broker, not every program on the machine. Your client then refuses any other server, including one in the middle that presents a different certificate.

The certificate covers the instance's public and private IPv4 addresses and EC2 DNS names, localhost and 127.0.0.1. An EC2 stop and start gives the instance a new public IP.

At the next boot, the certificate is re-issued for the new names with the same key. Clients that pin the public key (MOSQUITTO_TLS_PUBKEY_SHA256) keep working. Clients that pin the certificate file need the new copy.

mosquitto-probe is a small MQTT client. It reads passwords from a file, or from the credentials file with --identity, and trusts only the broker's certificate. Run these checks:

sudo mosquitto-probe connect --identity app
sudo mosquitto-probe roundtrip --identity app --topic app/guide-test
sudo mosquitto-probe acl --identity app --topic orders/guide-test || true
sudo mosquitto-probe connect --anonymous || true

CONNACK-OK proto=4 tls=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 ROUNDTRIP OK topic=app/guide-test qos=1 bytes=43 proto=4 tls=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 ACL-DENIED sub=0x01 pub=ack leaked=0 control=delivered REFUSED rc=5 not-authorized proto=4

The application login connects, subscribes to app/guide-test, publishes a message there and receives it back.

Outside app/#, the broker silently drops what the login publishes and delivers nothing to its subscription. MQTT does not tell the client. The acl check proves this with sac-admin as the observer.

A client without a login is refused with CONNACK code 5 (not authorized). mosquitto-probe exits non-zero on a refusal, hence || true.

On the client machine, install the Mosquitto command-line clients (sudo apt-get install mosquitto-clients on Ubuntu or Debian, brew install mosquitto on macOS). Copy the certificate as in Step 5.

Then put the connection settings in an options file that only you can read. mosquitto_pub and mosquitto_sub read their default options, one per line, from mosquitto_pub and mosquitto_sub files in the directory named by XDG_CONFIG_HOME (normally ~/.config). The password is therefore never on a command line. Keep the files in a directory of their own to have one set per broker and login:

client-host$ umask 077; mkdir -p ~/mqtt-app client-host$ printf -- '-h <public-ip>\n-p 8883\n--cafile %s/mosquitto.crt\n-u sac-app\n' "$PWD" > ~/mqtt-app/mosquitto_pub client-host$ ssh -i <your-key.pem> ubuntu@<public-ip> "sudo sed -n 's/^MOSQUITTO_APP_PASSWORD=/-P /p' /root/mosquitto-credentials.txt" >> ~/mqtt-app/mosquitto_pub client-host$ cp ~/mqtt-app/mosquitto_pub ~/mqtt-app/mosquitto_sub

Subscribe in one terminal and publish from another:

client-host$ XDG_CONFIG_HOME=~/mqtt-app mosquitto_sub -t 'app/demo/#' -q 1 -v client-host$ XDG_CONFIG_HOME=~/mqtt-app mosquitto_pub -t app/demo/hello -q 1 -m 'hello from mosquitto_pub'

app/demo/hello hello from mosquitto_pub

Use a name the certificate covers as -h: the public or private IP, an EC2 DNS name, or your own DNS name once you have added it (Step 12). This works with the 2.0 clients in Ubuntu 24.04 and the 2.1 clients on the instance. Version 2.1 can also take a file with -o <file>.

If you subscribe to a topic outside app/# with this login, you receive nothing, and anything you publish there is dropped.

You can do the same from Python with Eclipse Paho (sudo apt-get install python3-paho-mqtt, or pip install paho-mqtt):

# paho_demo.py - publish and receive through your sac Mosquitto broker over MQTT/TLS (paho-mqtt 1.6 or 2.x).
# Usage:
python3 paho_demo.py <broker-host> <mosquitto.crt> <user> <password-file> <topic>
# The password is read from a file, never the command line; TLS trusts ONLY the broker's own certificate. import ssl import sys import threading import paho.mqtt.client as mqtt host, cafile, user, pwfile, topic = sys.argv[1:6] with open(pwfile) as f: password = f.read().strip() got = threading.Event() try: c = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2) except AttributeError: # paho-mqtt 1.x c = mqtt.Client() c.username_pw_set(user, password) c.tls_set(ca_certs=cafile, tls_version=ssl.PROTOCOL_TLS_CLIENT) # verify against this one certificate, host name checked c.on_connect = lambda cl, *a: cl.subscribe(topic, qos=1) c.on_subscribe = lambda cl, *a: cl.publish(topic, b"hello from paho", qos=1) c.on_message = lambda cl, u, m: (print("received:", m.topic, m.payload.decode()), got.set()) c.connect(host, 8883) c.loop_start() ok = got.wait(15) c.loop_stop() c.disconnect() sys.exit(0 if ok else 1)

client-host$ ssh -i <your-key.pem> ubuntu@<public-ip> "sudo sed -n 's/^MOSQUITTO_APP_PASSWORD=//p' /root/mosquitto-credentials.txt" > app.password client-host$ python3 paho_demo.py <public-ip> mosquitto.crt sac-app app.password app/paho-demo received: app/paho-demo hello from paho

The URL form is mqtts://<public-ip>:8883. Some clients accept the login inside the URL (mqtts://sac-app:<password>@...). If you use that form, build the URL inside your program from the password file. Do not write it in a configuration file or on a command line.

Give each device type or application its own login, limited to its own topics, rather than sharing one login. mosquitto-user reads the new password from standard input and writes its PBKDF2-SHA512 hash. It adds the user's ACL block, checks both files with the configuration gate and reloads the broker. Connected clients stay connected:

umask 077 openssl rand -hex 16 > ~/sensors.password
sudo mosquitto-user add sensors --topic 'sensors/#' < ~/sensors.password
sudo mosquitto-user generate dashboard --read 'sensors/#'
sudo mosquitto-user list

mosquitto-user: sensors set mosquitto-user: dashboard added; its password is in /root/mosquitto-user-dashboard.password (root only, 0600) sac-admin topic readwrite #; topic read $SYS/# sac-app topic readwrite app/# sensors topic readwrite sensors/# dashboard topic read sensors/#

--topic grants read and write on a topic filter. --read grants read only. Give either option as often as you need.

Giving these options for an existing user replaces its topic access. Without them, an existing user keeps its access and only its password changes. generate creates a random password in /root/mosquitto-user-<name>.password (root only). Passwords must be at least 16 characters.

Try the new logins:

sudo mosquitto-probe roundtrip --user sensors --password-file ~/sensors.password --topic sensors/17/temperature
sudo mosquitto-probe acl --user sensors --password-file ~/sensors.password --topic app/guide-test --control-topic sensors/acl-check || true

ROUNDTRIP OK topic=sensors/17/temperature qos=1 bytes=43 proto=4 tls=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 ACL-DENIED sub=0x01 pub=ack leaked=0 control=delivered

Hand ~/sensors.password and /root/mosquitto-user-dashboard.password to your applications' secret store, then delete the local copies with shred -u. To remove the example users, run:

sudo mosquitto-user delete sensors
sudo mosquitto-user delete dashboard
sudo shred -u /root/mosquitto-user-dashboard.password; shred -u ~/sensors.password

If you manage passwords yourself, use mosquitto_passwd with the password on standard input. Never use -b, which puts the password on the command line. Pass -H sha512-pbkdf2, because Mosquitto 2.1's default argon2id hash cannot be made with Ubuntu 24.04's OpenSSL 3.0 and fails with Error: Unable to hash password. Then run sudo systemctl reload mosquitto.

mosquitto.service runs mosquitto-config-gate before every start and every reload, and mosquitto-guard after every start. You can check both, and the whole instance, at any time:

sudo mosquitto-config-gate --unit && echo gate-ok
sudo mosquitto-selfcheck

mosquitto-config-gate: OK (unit): listener 8883 TLS (tlsv1.2 minimum), anonymous off, 2 users in the password file, 2 ACL user blocks gate-ok ok: units active: mosquitto-metadata-names, mosquitto-firstboot, mosquitto ok: OK: listener 8883 TLS (tlsv1.2 minimum), anonymous off, 2 users in the password file, 2 ACL user blocks ok: OK (check): listening 0.0.0.0:8883 [::]:8883, no UDP; anonymous and a random password refused over TLS (rc=5 rc=5 rc=0x87 rc=0x87) ok: listening set: 22 (ssh) and 8883 (MQTT over TLS, IPv4 + IPv6) only; no UDP from the broker ok: certificate served on 8883 = /etc/mosquitto/tls/mosquitto.crt = credentials file (SHA-256 EB:D0:A2:74:C4:B0:EB:34...); covers 172.31.89.225 127.0.0.1 (endpoint 44.204.185.170) ok: MQTT over TLS publish/subscribe round trips (QoS 1): sac-app via 127.0.0.1 and 172.31.89.225 (3.1.1 and 5.0); sac-admin on ops/ (TLSv1.3 cipher=TLS_AES_256_GCM_SHA384) ok: refusals: anonymous, wrong passwords and an unknown user (MQTT 3.1.1 CONNACK 5 / 5.0 reason 0x87 not authorized; the broker does not say which part was wrong); sac-app outside app/# receives and leaks nothing; plain MQTT on 8883 not answered; 1883 8080 8081 9001 8884 443 80 closed on 172.31.89.225 ok: instance metadata blocked inside the broker's cgroup (000) while loopback is open; no outbound connection (no bridge, no telemetry) SELFCHECK OK

The gate refuses any edit to /etc/mosquitto/mosquitto.conf or the ACL that allows anonymous clients, adds a plain-text or second listener, a WebSockets listener, a bridge or a plugin, or removes the TLS certificate, the password file or the ACL.

On a start, the broker does not start. On systemctl reload, the running broker keeps its previous settings. journalctl -u mosquitto shows the reason. If the running broker ever listens anywhere but 8883 or accepts an anonymous login, the guard stops it.

mosquitto-selfcheck prints one ok: line per area and ends with SELFCHECK OK.

Package updates: libcjson1

Every Ubuntu 24.04 package on the image is fully patched at build time, with one known exception. libcjson1 (cJSON 1.7.17, from Ubuntu's universe archive) has a security update that Ubuntu publishes only in Ubuntu Pro's esm-apps channel (1.7.17-1ubuntu0.1~esm3). Mosquitto's packages require the library, so it cannot be removed.

Mosquitto uses cJSON for its plugins and its control API. This image runs neither, and the configuration gate refuses both. sudo pro security-status lists the library as the only pending esm-apps update.

If you attach the instance to Ubuntu Pro (sudo pro attach, then sudo apt-get update && sudo apt-get upgrade), the fix installs like any other update. Each new release of this image checks again whether the fix has reached Ubuntu's main archive.

Mosquitto is often run with a plain-text listener on 1883, and sometimes with anonymous access. User names, passwords and every message then cross the network in clear text, so anyone on the path of a cloud instance can read and inject messages.

This image configures exactly one listener, MQTT over TLS on 8883. It does so in Mosquitto's own configuration rather than relying on a firewall:

• 1883 (MQTT): not configured. Every maintained MQTT client supports TLS on 8883.
• WebSockets (often 8080, 8081 or 9001): not configured. A browser application should reach MQTT through your own back end. Alternatively, add a WebSockets listener behind TLS yourself once you have reviewed it; the configuration gate refuses one by default.
• Anonymous access: refused on every listener (allow_anonymous false).

From another machine, a connection to 1883, 8080, 8081, 9001, 8884, 443 or 80 is refused even if your security group allows it. Plain MQTT sent to 8883 gets no answer.

The EC2 instance metadata service (169.254.169.254) hands out the instance's IAM role credentials to any process that asks, so a network service must not be able to reach it. mosquitto and mosquitto-firstboot run with the metadata addresses blocked by systemd (IPAddressDeny).

Only mosquitto-metadata-names, which reads the instance's names for the certificate, may reach the metadata service. It may reach nothing else. Inspect the settings:

for u in mosquitto mosquitto-firstboot mosquitto-metadata-names; do printf '%-26s deny=%s allow=%s\n' "$u" "$(systemctl show -p IPAddressDeny --value "$u")" "$(systemctl show -p IPAddressAllow --value "$u")"; done

mosquitto deny=::ffff:169.254.169.254/128 ::ffff:168.63.129.16/128 168.63.129.16/32 169.254.169.254/32 fd00:ec2::254/128 allow= mosquitto-firstboot deny=169.254.169.254/32 168.63.129.16/32 ::ffff:169.254.169.254/128 fd00:ec2::254/128 ::ffff:168.63.129.16/128 allow= mosquitto-metadata-names deny=::/0 0.0.0.0/0 allow=169.254.169.254/32 fd00:ec2::254/128

mosquitto-selfcheck proves the block from inside the broker's own control group, where a request to the metadata service gets no answer.

Point a DNS name at the instance. An Elastic IP keeps the address across stop and start. Then add the name to the certificate:

sudo mosquitto-set-hostname <dns-name>

This re-issues the self-signed certificate with the same key to cover the name, updates the endpoint in /root/mosquitto-credentials.txt and restarts the broker. Copy the new mosquitto.crt to clients that pin the certificate file.

To use a certificate from your CA (your own, or a public one such as Let's Encrypt), copy the chain (server certificate first) and its unencrypted PEM key to the instance. Then run:

sudo mosquitto-install-cert <certificate-chain.pem> <private-key.pem> <dns-name>

Port 8883 then serves your certificate, and clients trust your CA instead of pinning this instance. The boot-time renewal never replaces an installed certificate. sudo mosquitto-set-hostname <name> goes back to a new self-signed certificate.