Semaphore UI 2.19.17

In the AWS Marketplace listing, choose Continue to Subscribe, then Continue to Configuration. Pick Semaphore UI 2.19.17 on Ubuntu 24.04 as the software version and select your Region, then choose Launch through EC2. Choose m5.large (or larger if you run many tasks at once), your key pair and a security group as described above.

To launch with the AWS CLI instead, replace the values in angle brackets:

aws ec2 run-instances --region <region> --image-id <ami-id-from-the-listing> \
  --instance-type m5.large --key-name <your-key-pair> \
  --security-group-ids <your-security-group> --subnet-id <your-subnet> \
  --metadata-options HttpTokens=required,HttpEndpoint=enabled \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=semaphore}]'

First boot takes under a minute after the instance is running.

For Semaphore UI 2.19.17 on Ubuntu 24.04, the SSH user is ubuntu:

ssh -i <your-key.pem> ubuntu@<public-ip>

First boot writes the configuration, creates the datastore and the administrator, and mints the certificate. nginx and Semaphore start only after first boot has finished:

systemctl is-active nginx semaphore
sudo test -f /var/lib/sac/data/semaphore-firstboot.done && echo FIRSTBOOT_DONE

Both services are active, and the first-boot marker is present:

active active FIRSTBOOT_DONE

Only SSH and nginx listen on network ports. Semaphore listens on the loopback interface only:

sudo ss -lntp | awk 'NR>1 && $4 !~ /^127\.0\.0\.5[34]/ {split($6, p, "\""); print $4, p[2]}' | sort

0.0.0.0:22 sshd 0.0.0.0:443 nginx 0.0.0.0:80 nginx 127.0.0.1:3000 semaphore [::]:22 sshd [::]:443 nginx [::]:80 nginx

To check the versions of Semaphore and the tools it runs, use:

semaphore version ansible --version | head -1 tofu version | head -1 git --version

2.19.17-f8cefe6-1792391823 ansible [core 2.21.6] OpenTofu v1.13.9 git version 2.43.0

sudo cat /root/semaphore-credentials.txt

Only root can read the file:

# Semaphore UI 2.19.17 - generated on first boot by semaphore-firstboot.service
# These credentials are unique to this instance; none was baked into the image. Keep this file secret. SEMAPHORE_URL=https://203.0.113.10/ SEMAPHORE_HOST=203.0.113.10 SEMAPHORE_ADMIN_USER=admin SEMAPHORE_ADMIN_PASSWORD=<SEMAPHORE_ADMIN_PASSWORD>
#
# Open SEMAPHORE_URL in a browser (self-signed certificate minted on this instance's first boot; port 80 redirects to # https) and sign in with the user and password above. Then change the password (your avatar > Edit user). # The access-key encryption key in /etc/semaphore/config.json is also unique to this instance: it encrypts every SSH # key and secret you store in Semaphore. Back it up with the datastore /var/lib/semaphore/db/semaphore.db. # Tasks run as the OS user semaphore (no sudo, no instance-metadata access). # Logs:
journalctl -u semaphore Change the name it serves as:
sudo semaphore-set-hostname <your-dns-name>
# User guide: https://secureanycloud.com/stacks/semaphore-ui-2-19-17/

SEMAPHORE_URL uses the instance's public IPv4 address until you set your own name (Step 8). If the instance has no public address, it uses the private one.

Open SEMAPHORE_URL in your browser. The certificate is self-signed, so your browser asks you to confirm it the first time (or install your own, Step 8). Sign in as admin with the password from Step 4.

Change the password straight away. Open the menu under your name at the bottom left, choose Edit Account and set a new password.

A new instance has exactly one user. Under Users (in the same menu), add an account for each person. Tick the Admin box only for the people who manage Semaphore itself.

A new instance has no projects. Choose New project, give it a name, and then complete the following sections in this order:

• Key Store: add the SSH private key your managed hosts accept (type SSH Key), and a None key for public Git repositories. Semaphore encrypts every key with this instance's own access key encryption key.
• Repositories: add the Git URL of your playbooks and the branch, with the key Semaphore should use to clone it.
• Inventory: add your hosts, as a static inventory or a file in the repository, with the SSH key to log in to them.
• Variable Groups: add extra variables and environment variables for the runs.
• Task Templates: add one template per playbook, tying the playbook to an inventory, a repository and a variable group.

Then choose Run on a template. Each run appears under Dashboard, History with its status, who started it and how long it took. Open a run to see its full Ansible log.

Tasks run on this instance as the user semaphore, which cannot use sudo. Point your inventories at the hosts you manage over SSH rather than at localhost. To use become on those hosts, add the sudo password as a key of type Login with password and pick it as the inventory's Become key.

OpenTofu templates take a workspace inventory instead of a host list. Under Inventory, choose New inventory, select the type OpenTofu Workspace, and enter the workspace name (default for the default workspace).

Create the template with the app OpenTofu Code, your repository and that inventory. A run plans first and waits for you to confirm the plan, unless the template has Auto approve ticked.

Keep OpenTofu state in a remote backend (for example an S3 bucket) declared in your configuration. A plan run by Semaphore cannot read this instance's IAM role, because tasks are blocked from the instance metadata service. Give OpenTofu its cloud credentials through a variable group or the Key Store.

Point a DNS record at the instance (an Elastic IP keeps the address stable), then set the name that Semaphore is served as. The command mints a new self-signed certificate for the name and points nginx at it. It also sets Semaphore's web address and the allowed origin, updates /root/semaphore-credentials.txt, and keeps the name across reboots:

sudo semaphore-set-hostname <your-domain>

To use a certificate from a public or internal certificate authority instead, copy it over the generated pair and reload nginx. Running semaphore-set-hostname again replaces your certificate with a self-signed one.

sudo install -m 0644 <your-domain>.crt /etc/nginx/ssl/semaphore.crt
sudo install -m 0600 <your-domain>.key /etc/nginx/ssl/semaphore.key
sudo nginx -t && sudo systemctl reload nginx

Until you set a name, the image follows the instance's address. If you stop and start the instance and it gets a new public address, the next boot moves Semaphore and its certificate to that address.

nginx serves only Semaphore's pages and API. The configuration, the datastore and dotfiles answer 404. The API needs a signed-in session, plain HTTP redirects to HTTPS, a request for a name the instance does not serve gets no answer, and a sign-in from another origin is refused. Verify this with:

H=$(sudo grep '^SEMAPHORE_HOST=' /root/semaphore-credentials.txt | cut -d= -f2-) for p in /auth/login /api/ping /healthz /api/projects /api/users /api/runners /etc/semaphore/config.json /db/semaphore.db /.git/config; do curl -sk -o /dev/null -w "%{http_code} $p\n" --resolve "$H:443:127.0.0.1" "https://$H$p" done curl -s -o /dev/null -w 'http:// -> %{http_code}\n' --resolve "$H:80:127.0.0.1" "http://$H/" curl -sk -o /dev/null -w 'unknown host name -> %{http_code}\n' --resolve "other.example:443:127.0.0.1" "https://other.example/" || true curl -sk -o /dev/null -w 'sign in from another origin -> %{http_code}\n' --resolve "$H:443:127.0.0.1" \
  -H 'Origin: https://other.example' -H 'Content-Type: application/json' -d '{"auth":"admin","password":"x"}' "https://$H/api/auth/login"

200 /auth/login 200 /api/ping 200 /healthz 401 /api/projects 401 /api/users 401 /api/runners 404 /etc/semaphore/config.json 404 /db/semaphore.db 404 /.git/config http:// -> 301 unknown host name -> 000 sign in from another origin -> 403

The credential check signs in through the API with your password. It confirms that a wrong password and admin/admin are refused and that remote runner registration is shut:

sudo semaphore-cred-roundtrip

OK https 301 -> https://203.0.113.10/; certificate verifies for 203.0.113.10; admin 204 (cookie Secure HttpOnly SameSite=Lax, /api/user admin); wrong and admin/admin 401, empty refused; foreign Origin 403; anonymous API 401; runner registration shut (400)

Every service runs with the instance metadata service blocked. Semaphore, with every task it starts, runs as semaphore with no privileges:

for u in nginx semaphore semaphore-firstboot semaphore-host-follow; do echo "$u $(systemctl show -p IPAddressDeny --value $u)"; done systemctl show semaphore -p User -p NoNewPrivileges -p ProtectSystem -p CapabilityBoundingSet
sudo -l -U semaphore | tail -1

nginx fd00:ec2::254/128 168.63.129.16/32 169.254.169.254/32 semaphore fd00:ec2::254/128 168.63.129.16/32 169.254.169.254/32 semaphore-firstboot fd00:ec2::254/128 168.63.129.16/32 169.254.169.254/32 semaphore-host-follow fd00:ec2::254/128 168.63.129.16/32 169.254.169.254/32 CapabilityBoundingSet= User=semaphore ProtectSystem=strict NoNewPrivileges=yes User semaphore is not allowed to run
sudo on ip-172-31-90-141.

A playbook that tries to reach the metadata service from inside a task fails with a timeout.

The configuration is readable only by Semaphore, and the datastore only by the semaphore user:

sudo stat -c '%U:%G %a %n' /etc/semaphore/config.json /var/lib/semaphore /var/lib/semaphore/db/semaphore.db

root:semaphore 640 /etc/semaphore/config.json root:semaphore 750 /var/lib/semaphore semaphore:semaphore 640 /var/lib/semaphore/db/semaphore.db

Everything Semaphore stores is in its SQLite datastore /var/lib/semaphore/db/semaphore.db. The keys and secrets in it are encrypted with the access_key_encryption key in /etc/semaphore/config.json, so back the two up together. A datastore without its configuration cannot decrypt your Key Store.

Both files are on the root volume, so the simplest backup is an EBS snapshot of it. For a consistent copy that you can keep elsewhere, run:

sudo sqlite3 /var/lib/semaphore/db/semaphore.db ".backup '<backup-dir>/semaphore-$(date +%F).db'"
sudo install -m 0600 /etc/semaphore/config.json <backup-dir>/semaphore-config-$(date +%F).json

Operating system security updates install automatically. Run sudo apt-get update && sudo apt-get upgrade to apply everything else. nginx and Semaphore restart cleanly with all of their data.

ansible-core lives in its own Python environment, which apt does not update. To move to a newer ansible-core 2.21 release, or to add Python libraries your playbooks need, run:

sudo /opt/semaphore-ansible/bin/python -m pip install --upgrade 'ansible-core>=2.21,<2.22'

Install Ansible collections from a collections/requirements.yml file in your playbook repository. Semaphore installs them at the start of each run. New Semaphore releases arrive as new versions of this listing, and support can help you move your datastore to one.

Key locations on the instance:

• Semaphore: /usr/bin/semaphore (the community build, owned by root)
• Configuration: /etc/semaphore/config.json (root:semaphore, 0640)
• Datastore: /var/lib/semaphore/db/semaphore.db
• Working files and clones: /var/lib/semaphore/tmp
• ansible-core: /opt/semaphore-ansible (linked into /usr/local/bin)
• OpenTofu: /usr/local/bin/tofu
• Logs: journalctl -u semaphore, /var/log/nginx
• nginx site: /etc/nginx/sites-available/sac-semaphore
• Certificate and key: /etc/nginx/ssl/semaphore.crt, /etc/nginx/ssl/semaphore.key
• Helpers: /usr/local/sbin/semaphore-set-hostname, /usr/local/sbin/semaphore-cred-roundtrip
• Credentials: /root/semaphore-credentials.txt (mode 0600)
• Source and licences: /usr/share/doc/semaphore-sac/

• Allow port 22 from your admin network only, and ports 80 and 443 only from the networks your users are on.
• Use your own DNS name and a certificate your browsers trust (Step 8).
• Change the admin password as soon as you sign in, give each person their own account, and keep the Admin role for the few people who manage Semaphore.
• Give Semaphore SSH keys that are limited to what its playbooks need, and use a separate key per environment.
• Do not give this instance an IAM role that your playbooks should not have. Tasks cannot read the role, but other software you add might.
• Keep the operating system patched, and snapshot the root volume regularly.