OPAL 0.9.9

In the AWS Marketplace listing, choose Continue to Subscribe, then Continue to Configuration. Pick OPAL 0.9.9 on Ubuntu 24.04 as the software version, select your Region, and choose Launch through EC2. Choose your instance type, key pair, subnet and a security group as described above.

To launch with the AWS CLI instead, replace the values in angle brackets:

aws ec2 create-security-group --region <region> --vpc-id <your-vpc> \
  --group-name opal --description "OPAL: SSH from admins, HTTPS from publishers and OPAL clients"
aws ec2 authorize-security-group-ingress --region <region> --group-id <sg-id> --ip-permissions \
  'IpProtocol=tcp,FromPort=22,ToPort=22,IpRanges=[{CidrIp=<your-admin-cidr>}]' \
  'IpProtocol=tcp,FromPort=443,ToPort=443,IpRanges=[{CidrIp=<your-client-cidr>}]'
aws ec2 run-instances --region <region> --image-id <ami-id-from-the-listing> \
  --instance-type t3.medium --key-name <your-key-pair> \
  --security-group-ids <sg-id> --subnet-id <your-subnet> \
  --metadata-options HttpTokens=required,HttpEndpoint=enabled \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=opal}]'

First boot takes well under a minute. It names the server after the instance's public DNS name, or its private DNS name when the instance has no public one.

To use your own name from the first boot, add user data that writes the name before first boot runs, for example #!/bin/sh followed by mkdir -p /etc/sac && echo opal.example.com > /etc/sac/opal-domain.preset. You can also change the name later with opal-set-domain (Step 10).

For OPAL 0.9.9 on Ubuntu 24.04, the SSH user is ubuntu.

ssh -i <your-key.pem> ubuntu@<public-ip>

First boot generates the secrets and the certificate and seeds the policy repository. It then starts OPA, the OPAL server, the OPAL client and nginx:

systemctl is-active opa opal-server opal-client nginx opal-guard
sudo test -f /var/lib/sac/data/opal-firstboot.done && echo FIRSTBOOT_DONE

active active active active active FIRSTBOOT_DONE

Only SSH and nginx listen on the network. OPA (8181), the OPAL server (7002) and the OPAL client API (7000) are on loopback:

sudo ss -tlnH | awk '{print $4}' | sort -u

0.0.0.0:22 0.0.0.0:443 0.0.0.0:80 127.0.0.1:7000 127.0.0.1:7002 127.0.0.1:8181 127.0.0.53%lo:53 127.0.0.54:53 [::]:22 [::]:443 [::]:80

The self-test exchanges the master token for a JWT over HTTPS, publishes a data update, checks that OPA's decision follows the update, and then removes the update:

sudo opal-selftest

OPAL_SELFTEST_OK

The root-only file /root/opal-credentials.txt holds the server's URLs, the certificate fingerprint and the master token (OPAL_AUTH_MASTER_TOKEN, 64 hexadecimal characters). Show everything except the token:

sudo grep -v '^OPAL_AUTH_MASTER_TOKEN' /root/opal-credentials.txt

# OPAL 0.9.9 - written on first boot by opal-firstboot.service. Unique to this instance. Keep it secret. OPAL_SERVER_URL=https://ec2-203-0-113-10.compute-1.amazonaws.com OPAL_JWKS_URL=https://ec2-203-0-113-10.compute-1.amazonaws.com/.well-known/jwks.json OPAL_TLS_SHA256=AB:CD:...:EF
#
# The master token only exchanges for JWTs at POST /token ({"type": "client" | "datasource"}); every other OPAL server
# route needs a JWT this instance signed. Or mint one here without the master token:
sudo opal-token client 365 > client.jwt
# OPA (the policy engine, 127.0.0.1:8181) is NOT exposed; query it on this instance as root, e.g.
#
sudo curl -s http://127.0.0.1:8181/v1/data/sac/authz/allow -d '{"input":{"user":"alice","action":"read"}}'
# Policies: the git repository /var/lib/opal/policy-repo.git (edit /var/lib/opal/policy-work as the opal user).
# The certificate is self-signed and unique to this instance: compare its SHA-256 fingerprint (above) with the one your
# client shows, or replace it with a CA-issued certificate for your own name.
# Your own name (point DNS at this instance first):
sudo opal-set-domain opal.example.com
# Self-test:
sudo opal-selftest User guide: https://secureanycloud.com/stacks/opal-0-9-9/

The master token is the key to the whole server, because anyone who holds it can mint JWTs that publish data. Keep it on the instance and hand out JWTs instead (Step 6). The commands in Step 6 read the token from the file without ever printing it.

The certificate is self-signed for this instance and is not a certificate authority. Compare its fingerprint with OPAL_TLS_SHA256 above before you trust it on another host.

The curl command below fetches the public signing keys through nginx on this instance. The --connect-to option sends the request to 127.0.0.1, so it works however your security group and DNS are set up, and the certificate must still match the server's name:

openssl x509 -in /etc/ssl/opal/opal.crt -noout -fingerprint -sha256 curl -sS --cacert /etc/ssl/opal/opal.crt --connect-to ::127.0.0.1:443 <OPAL_JWKS_URL>

sha256 Fingerprint=AB:CD:...:EF {"keys":[{"kty":"RSA","kid":"...","alg":"RS256","n":"...","e":"AQAB"}]}

The image ships an example policy, sac/authz.rego. It denies by default, and it allows a user an action when the data lists that action for that user. With no data published yet, alice may not read:

sudo curl -s http://127.0.0.1:8181/v1/data/sac/authz/allow -d '{"input":{"user":"alice","action":"read"}}'; echo

{"result":false}

Your applications ask the same question of the OPA next to them, either on this instance or on any host that runs an OPAL client (Step 9). OPA is never reachable from the network, and on this instance only root and the OPAL services can connect to it:

curl -s -o /dev/null -m 5 -w '%{http_code}\n' http://127.0.0.1:8181/v1/data || echo "refused for user $(id -un)"

000 refused for user ubuntu

Publishers use a datasource JWT, and OPAL clients use a client JWT. Ask the server for these JWTs over HTTPS with the master token. The commands below keep every token in a file that only you can read, never on a command line.

On the instance, --connect-to ::127.0.0.1:443 sends each request to nginx on this machine under the server's public name. The request therefore does not depend on your security group or on how the name resolves inside the VPC. In the first minutes after launch, the name can still resolve to the public address:

umask 077
HOST=$(cat /etc/sac/opal-domain)
sudo sed -n 's/^OPAL_AUTH_MASTER_TOKEN=/Authorization: Bearer /p' /root/opal-credentials.txt > ~/opal-master.h curl -s --cacert /etc/ssl/opal/opal.crt --connect-to ::127.0.0.1:443 -H @"$HOME/opal-master.h" -H 'Content-Type: application/json' \
  -d '{"type":"datasource"}' "https://$HOST/token" \
  | python3 -c 'import json,sys; print("Authorization: Bearer " + json.load(sys.stdin)["token"])' > ~/opal-ds.h curl -s --cacert /etc/ssl/opal/opal.crt --connect-to ::127.0.0.1:443 -H @"$HOME/opal-master.h" -H 'Content-Type: application/json' \
  -d '{"type":"client","ttl":2592000}' "https://$HOST/token" \
  | python3 -c 'import json,sys; print("Authorization: Bearer " + json.load(sys.stdin)["token"])' > ~/opal-client.h wc -c ~/opal-ds.h ~/opal-client.h test -s ~/opal-ds.h && test -s ~/opal-client.h && echo "both JWTs saved"

784 /home/ubuntu/opal-ds.h 779 /home/ubuntu/opal-client.h 1563 total both JWTs saved

The response of POST /token is {"token": "<JWT>", "type": "bearer", "details": {...}}. The details field names the peer type and the expiry. The default expiry is one year, and ttl is in seconds.

On the instance itself, you can also mint a JWT without the master token: sudo opal-token client 365 > client.jwt. Use datasource in place of client for a datasource JWT. The last argument is the lifetime in days.

Grant alice the read action by publishing a data update to the OPAL server. The server sends the update to every subscribed client, and each client writes it into OPA:

HOST=$(cat /etc/sac/opal-domain)
curl -s --cacert /etc/ssl/opal/opal.crt --connect-to ::127.0.0.1:443 -H @"$HOME/opal-ds.h" -H 'Content-Type: application/json' \
  -d '{"entries":[{"url":"","data":{"actions":["read"]},"dst_path":"/sac/users/alice","topics":["policy_data"],"save_method":"PUT"}],"reason":"grant alice read"}' \
  "https://$HOST/data/config"; echo sleep 3
sudo curl -s http://127.0.0.1:8181/v1/data/sac/authz/allow -d '{"input":{"user":"alice","action":"read"}}'; echo
sudo curl -s http://127.0.0.1:8181/v1/data/sac/authz/allow -d '{"input":{"user":"alice","action":"write"}}'; echo
sudo curl -s http://127.0.0.1:8181/v1/data/sac/users; echo
sudo curl -s http://127.0.0.1:8181/v1/data/sac/users/alice | grep -q read && echo "alice's grant is in OPA"

{"status":"ok"} {"result":true} {"result":false} {"result":{"alice":{"actions":["read"]}}} alice's grant is in OPA

An entry can carry its data inline, as in this example. It can also carry a url that the client fetches the data from, such as an HTTPS API of yours on a public address, with config for headers. The client fetches with GET only and refuses private and loopback addresses (Step 12).

A client JWT can read but cannot publish, and any request without a valid JWT is refused:

HOST=$(cat /etc/sac/opal-domain)
curl -s -o /dev/null -w 'client JWT publishing: %{http_code}\n' --cacert /etc/ssl/opal/opal.crt --connect-to ::127.0.0.1:443 -H @"$HOME/opal-client.h" \
  -H 'Content-Type: application/json' -d '{"entries":[],"reason":"x"}' "https://$HOST/data/config"
curl -s -o /dev/null -w 'no token: %{http_code}\n' --cacert /etc/ssl/opal/opal.crt --connect-to ::127.0.0.1:443 \
  -H 'Content-Type: application/json' -d '{"entries":[],"reason":"x"}' "https://$HOST/data/config"
curl -s -o /dev/null -w 'wrong token: %{http_code}\n' --cacert /etc/ssl/opal/opal.crt --connect-to ::127.0.0.1:443 -H 'Authorization: Bearer not-a-real-token' \
  "https://$HOST/policy?path=sac"
curl -s -o /dev/null -w 'OPA through nginx: %{http_code}\n' --cacert /etc/ssl/opal/opal.crt --connect-to ::127.0.0.1:443 "https://$HOST/v1/data"

client JWT publishing: 401 no token: 401 wrong token: 401 OPA through nginx: 404

The OPAL server tracks the main branch of /var/lib/opal/policy-repo.git and checks it every 10 seconds. Work in the checkout /var/lib/opal/policy-work as the opal user. Add a rule that lets a user granted * take any action, commit and push the rule, and then grant bob *:

sudo -u opal -H tee -a /var/lib/opal/policy-work/sac/authz.rego > /dev/null <<'EOF'
# A user granted "*" may take any action. allow if { "*" in data.sac.users[input.user].actions } EOF
sudo -u opal -H git -C /var/lib/opal/policy-work commit -qam "Allow users granted * any action"
sudo -u opal -H git -C /var/lib/opal/policy-work push -q
HOST=$(cat /etc/sac/opal-domain)
curl -s -o /dev/null -w 'grant bob *: %{http_code}\n' --cacert /etc/ssl/opal/opal.crt --connect-to ::127.0.0.1:443 -H @"$HOME/opal-ds.h" -H 'Content-Type: application/json' \
  -d '{"entries":[{"url":"","data":{"actions":["*"]},"dst_path":"/sac/users/bob","topics":["policy_data"],"save_method":"PUT"}],"reason":"grant bob everything"}' \
  "https://$HOST/data/config" for i in $(seq 1 30); do r=$(sudo curl -s http://127.0.0.1:8181/v1/data/sac/authz/allow -d '{"input":{"user":"bob","action":"delete"}}'); [ "$r" = '{"result":true}' ] && break; sleep 2; done; echo "bob delete: $r"; [ "$r" = '{"result":true}' ]

grant bob *: 200 bob delete: {"result":true}

The new rule reaches OPA a few seconds after the push, with no restart. The README in the repository repeats these commands.

Everything the server offers is on HTTPS port 443, so any host that your security group allows can publish data and run an OPAL client. Copy the certificate to that host and compare its fingerprint with OPAL_TLS_SHA256. Give the host a JWT (Step 6) rather than the master token. From the other host, run the following with the server's name in place of <opal-host>:

openssl x509 -in opal.crt -noout -fingerprint -sha256 curl -s --cacert opal.crt https://<opal-host>/healthcheck; echo curl -s -o /dev/null -w 'policy with a client JWT: %{http_code}\n' --cacert opal.crt -H @opal-client.h "https://<opal-host>/policy?path=sac"

sha256 Fingerprint=AB:CD:...:EF {"status":"ok"} policy with a client JWT: 200

An OPAL client on another host needs these settings. The client can be the upstream opal-client 0.9.9 package or the permitio/opal-client container, with its own OPA next to it:

OPAL_SERVER_URL=https://<opal-host> OPAL_CLIENT_TOKEN=<a client JWT from Step 6> OPAL_DATA_TOPICS=policy_data OPAL_POLICY_STORE_URL=http://127.0.0.1:8181
# only while the server keeps its self-signed certificate (Step 10 replaces it): OPAL_CLIENT_SELF_SIGNED_CERTIFICATES_ALLOWED=true OPAL_CLIENT_SSL_CONTEXT_TRUSTED_CA_FILE=/path/to/opal.crt

The client subscribes to the server's websocket at wss://<opal-host>/ws, which refuses connections without a client JWT. It downloads the policy bundle and then receives every policy commit and data update, exactly as the client on this instance does. In our test, a remote client had this instance's policy within 4 seconds of starting, and a data update published here arrived about a second later.

The OPAL client ignores OPAL_CLIENT_SSL_CONTEXT_TRUSTED_CA_FILE unless OPAL_CLIENT_SELF_SIGNED_CERTIFICATES_ALLOWED=true is also set. With both settings, it trusts exactly the certificate in that file. Upstream recommends a certificate from a public certificate authority for production, which needs neither setting (Step 10).

Point a DNS name at the instance, and use an Elastic IP to keep the address stable. Then make that name the server's name. The opal-set-domain command renders nginx for the new name, creates a new self-signed certificate for it, and updates the URLs in /root/opal-credentials.txt:

sudo opal-set-domain opal.example.com

To use a certificate from a public certificate authority instead, run opal-set-domain first. Then install your certificate and key as /etc/ssl/opal/opal.crt and /etc/ssl/opal/opal.key (root only), and run sudo systemctl reload nginx. Clients then no longer need the --cacert or trusted CA file setting.

To track your own repository instead of the local one, set OPAL_POLICY_REPO_URL in /etc/opal/server.env. If the branch is not main, also set OPAL_POLICY_REPO_MAIN_BRANCH.

The server keeps its working clone in /var/lib/opal/regoclone and refuses to start on a clone of a different repository. Remove the clone while the server is stopped:

sudo systemctl stop opal-server sudoedit /etc/opal/server.env
# OPAL_POLICY_REPO_URL=https://github.com/<your-org>/<your-policies>.git
sudo -u opal rm -rf /var/lib/opal/regoclone/opal_repo_clone
sudo systemctl start opal-server

The server clones the new repository, and every client loads its policies within seconds. The clients reconnect by themselves after the restart. Only .rego and .json files are loaded. The instance metadata block stays in force for every URL that the server or the client fetches.

The image does not expose OPAL's /webhook route to the network, so the server polls the repository every OPAL_POLICY_REPO_POLLING_INTERVAL seconds (10 by default here).

Metadata service. Every OPAL service, OPA, nginx and the first-boot service are blocked from the instance metadata service, so a data source pointed at it simply fails:

systemctl show -p IPAddressDeny opa opal-server opal-client nginx

IPAddressDeny=fd00:ec2::254/128 169.254.169.254/32 168.63.129.16/32 ::ffff:169.254.169.254/128 IPAddressDeny=fd00:ec2::254/128 169.254.169.254/32 168.63.129.16/32 ::ffff:169.254.169.254/128 IPAddressDeny=0.0.0.0/8 168.63.129.16/32 fc00::/7 ::ffff:0.0.0.0/104 fe80::/10 10.0.0.0/8 169.254.0.0/16 ::ffff:100.64.0.0/106 ::ffff:172.16.0.0/108 100.64.0.0/10 ::ffff:169.254.0.0/112 172.16.0.0/12 192.168.0.0/16 ::ffff:192.168.0.0/112 ::ffff:10.0.0.0/104 IPAddressDeny=fd00:ec2::254/128 169.254.169.254/32 168.63.129.16/32 ::ffff:169.254.169.254/128

The third entry is the OPAL client. It also drops every private, link-local and unique-local range, which cover the metadata addresses, so systemd folds them in. Loopback stays open so the client can reach OPA and the OPAL server. The order of the entries varies between boots.

Owner guard. opal-guard.service loads the firewall rule that keeps other local users away from OPA and the OPAL loopback ports. OPA, the OPAL server and the client require this rule. If the rule is removed, they stop rather than run unguarded:

systemctl is-active opal-guard;
sudo opal-selftest

active OPAL_SELFTEST_OK

Data source URLs. The OPAL client fetches every data source URL through SecureAnyCloud's restricted fetcher (/usr/local/lib/sac/opal/pylib/sac_opal_fetch.py, set by OPAL_FETCH_PROVIDER_MODULES in /etc/opal/client.env). The client refuses to start with any other fetcher.

The fetcher sends GET requests only, with no body and no redirects, to http:// or https:// URLs whose every address is a public internet address. It refuses loopback, private (RFC 1918), CGNAT, link-local, IPv6 unique-local and documentation addresses, every address in this instance's VPC, and an entry that names another fetcher. It checks the address of a DNS name when it connects, and it connects only to the address it checked.

The client's service is also firewalled from private addresses (systemctl show -p IPAddressDeny opal-client). Each refusal is logged without the URL's query string. To load data that lives on a private address, publish it inline (Step 7) from a host that can read it.

This example tries to write a policy into OPA and to read a private address, and both are refused:

HOST=$(cat /etc/sac/opal-domain)
SINCE=$(date '+%Y-%m-%d %H:%M:%S')
curl -s --cacert /etc/ssl/opal/opal.crt --connect-to ::127.0.0.1:443 -H @"$HOME/opal-ds.h" -H 'Content-Type: application/json' \
  -d '{"entries":[{"url":"http://127.0.0.1:8181/v1/policies/demo","config":{"method":"put","data":"package demo"},"dst_path":"/sac/demo","topics":["policy_data"],"save_method":"PUT"},{"url":"http://10.0.0.1/users.json","dst_path":"/sac/demo","topics":["policy_data"],"save_method":"PUT"}],"reason":"refused sources"}' \
  "https://$HOST/data/config"; echo for i in $(seq 1 40); do [ "$(sudo journalctl -u opal-client --since "$SINCE" -o cat | grep -c 'sac-fetch REFUSED')" -ge 2 ] && break; sleep 1; done
sudo journalctl -u opal-client --since "$SINCE" -o cat | grep -o 'sac-fetch REFUSED.*' echo "OPA policy demo: $(sudo curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:8181/v1/policies/demo)"
sudo journalctl -u opal-client --since "$SINCE" -o cat | grep -q 'REFUSED rfc1918: http://10.0.0.1/users.json' && echo "both sources refused"

{"status":"ok"} sac-fetch REFUSED method: http://127.0.0.1:8181/v1/policies/demo sac-fetch REFUSED rfc1918: http://10.0.0.1/users.json OPA policy demo: 404 both sources refused

Logs. View the logs with sudo journalctl -u opal-server -u opal-client -u opa. A refused request is logged as Authentication failed with 401, without the token it carried. nginx keeps its access log in /var/log/nginx and logs errors only at crit, because request lines can carry tokens.

Tokens. Hand out client and datasource JWTs, never the master token. Give each JWT a lifetime that suits its holder (ttl in Step 6, or the day count of opal-token). Delete the header files when you are done: shred -u ~/opal-master.h ~/opal-ds.h ~/opal-client.h.

Security group. Allow TCP 443 only from the hosts that publish data or run OPAL clients, and allow TCP 22 only from your administrators.

Updates. Unattended security upgrades stay on for Ubuntu, nginx and git. OPAL and OPA are pinned, and the exact versions and hashes are in /usr/share/doc/sac-opal (requirements.lock.txt, pip-freeze.txt, opa/BUILD-INFO). New OPAL and OPA releases arrive as new versions of this listing.