Afanasy 3.4.1

Launch the Afanasy 3.4.1 on Ubuntu 24.04 by SecureAnyCloud image from the Azure Marketplace on a Standard_B2s or larger VM. In the Network Security Group, open inbound ports 22 (SSH), 80 (HTTP) and 443 (HTTPS). The image provisions the 30 GiB data disk that holds the afserver store automatically.

When the VM is running, connect over SSH as azureuser with the SSH key you selected at launch:

ssh azureuser@<public-ip>

The Afanasy server and nginx start automatically on boot. Confirm both are active:

systemctl is-active afserver nginx

Each command should print active. For the full status view:

systemctl --no-pager status afserver nginx | grep -E 'afserver.service|nginx.service|Active:' | head

HTTP Basic auth protects the Web Monitor with a username and password that are unique to your VM and generated on first boot. Read them from the credentials file:

sudo cat /var/lib/sac/secrets/sac-credentials.txt

The AFANASY_WEB_USER and AFANASY_WEB_PASSWORD values are your Web Monitor credential. Keep the password secret, because it grants full access to the render farm dashboard.

The /healthz endpoint is unauthenticated and confirms that the server is up, so you can safely use it for a load balancer or availability probe. The server's own HTTP port 51000 is reachable only from inside the VM:

curl -ks https://127.0.0.1/healthz echo
curl -s -o /dev/null -w 'afserver HTTP %{http_code}\n' http://127.0.0.1:51000/

Both should report HTTP 200. nginx answers /healthz, and afserver answers on its own port on the loopback interface.

The Web Monitor requires the per-VM credential. A request that authenticates with it returns HTTP 200. A request with no credential, or the wrong one, is rejected by nginx with HTTP 401, because the image ships no default login:

WEB_USER=$(sudo grep '^AFANASY_WEB_USER=' /var/lib/sac/secrets/sac-credentials.txt | cut -d= -f2-)
WEB_PASS=$(sudo grep '^AFANASY_WEB_PASSWORD=' /var/lib/sac/secrets/sac-credentials.txt | cut -d= -f2-)
echo "authenticated: $(curl -ks -o /dev/null -w '%{http_code}' -u "$WEB_USER:$WEB_PASS" https://127.0.0.1/)"
echo "no credential: $(curl -ks -o /dev/null -w '%{http_code}' https://127.0.0.1/)"
echo "wrong password: $(curl -ks -o /dev/null -w '%{http_code}' -u "$WEB_USER:wrong" https://127.0.0.1/)"

The authenticated request returns 200. The other two return 401.

Browse to https://<public-ip>/ and accept the per-VM self-signed certificate. Your browser prompts for the Basic auth credential. Enter the AFANASY_WEB_USER and AFANASY_WEB_PASSWORD from Step 3. The Afanasy Web Monitor opens on its Jobs view, which shows the live server and its built-in system jobs (post_commands, wake-on-lan, events) that keep the farm running:

Select Farm in the top navigation to see the render pool tree. On a fresh appliance the farm is empty. The root pool / is present and ready, and you add render nodes to it. The left-hand panel gives you the full pool controls: create pools, free or heal hosts, and manage power and tickets.

Open the Users view to see the users the server knows about, with their job and error-solving settings.

Open the Monitors view to see the connected Web Monitors. Your live dashboard session appears here, which confirms that it is registered with the server.

The appliance ships the coordinator only, so you connect your own render machines to it. On each render machine, install CGRU, point it at this server's address and start afrender so that it registers with the farm.

The server port 51000 is closed at the Network Security Group by default. Open it, or a private subset of it, to the machines that will run afrender. Treat it as a trusted-network port rather than a public one, because the Basic auth that protects the Web Monitor does not front the render protocol.

As nodes come online, they appear in the Farm view and start taking tasks from any jobs you submit.

The afserver store holds jobs, the render-node registry and the server's working state. It lives on the dedicated Azure data disk mounted at /var/lib/cgru. Confirm that the store is on the data disk:

mountpoint /var/lib/cgru ls -la /var/lib/cgru/store

The store directory is /var/lib/cgru/store. Your render farm state is independent of the OS disk, and you can resize the volume on its own.

In the AWS Marketplace listing, choose Continue to Subscribe, then Continue to Configuration. Pick Afanasy 3.4.1 on Ubuntu 24.04 as the software version and select your Region, then choose Launch through EC2. Choose m5.large, your key pair and a security group as described in Prerequisites. The AMI includes the farm data volume, so leave its block device mapping in place.

With the AWS CLI (replace the values in angle brackets):

aws ec2 run-instances --region <region> --image-id <ami-id-from-the-listing> \
  --instance-type m5.large --key-name <your-key-pair> \
  --security-group-ids <your-security-group> --subnet-id <your-subnet> \
  --metadata-options HttpTokens=required,HttpEndpoint=enabled \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=afanasy}]'

For Afanasy 3.4.1 on Ubuntu 24.04, the SSH user is ubuntu.

ssh -i <your-key.pem> ubuntu@<public-ip>

On first boot, the instance generates the certificate and the login, and then afserver and nginx start:

systemctl is-active cgru-firstboot afserver nginx
sudo test -f /var/lib/sac/data/cgru-firstboot.done && echo FIRSTBOOT_DONE

All three units are active and the first boot marker is present:

active active active FIRSTBOOT_DONE

nginx is the only service on public addresses (80 and 443). afserver listens on loopback:

sudo ss -lntp | awk 'NR>1 {print $4, $6}' | sed -E 's/pid=[0-9]+,fd=[0-9]+/.../g' | sort

0.0.0.0:22 users:(("sshd",...),("systemd",...)) 0.0.0.0:443 users:(("nginx",...),("nginx",...),("nginx",...)) 0.0.0.0:80 users:(("nginx",...),("nginx",...),("nginx",...)) 127.0.0.1:51000 users:(("afserver",...)) 127.0.0.53%lo:53 users:(("systemd-resolve",...)) 127.0.0.54:53 users:(("systemd-resolve",...)) [::]:22 users:(("sshd",...),("systemd",...)) [::]:443 users:(("nginx",...),("nginx",...),("nginx",...)) [::]:80 users:(("nginx",...),("nginx",...),("nginx",...))

sudo cat /var/lib/sac/secrets/sac-credentials.txt

The file is readable by root only:

# Afanasy 3.4.1 - generated on first boot by cgru-firstboot.service
# The Web Monitor login is unique to this instance; none was baked into the image. Keep this file secret. AFANASY_URL=https://ec2-<public-ip-dashed>.<region>.compute.amazonaws.com/ AFANASY_WEB_USER=afadmin AFANASY_WEB_PASSWORD=<AFANASY_WEB_PASSWORD>
# afserver listens on 127.0.0.1:51000 only (render nodes: see the guide, "Connect render nodes").
# Change the name the Web Monitor is served as:
sudo afanasy-set-hostname <your-dns-name>
# User guide: https://secureanycloud.com/stacks/afanasy-3-4-1/

The URL uses the instance's public DNS name from the EC2 instance metadata. The certificate also names the public and private IP addresses, so https://<public-ip>/ works too. To use your own DNS name, see Step 9. To share the file, for example in a support request, print it with the password masked:

sudo sed -E 's/^(AFANASY_WEB_PASSWORD=).*/\1********/' /var/lib/sac/secrets/sac-credentials.txt | grep '^AFANASY_'

Port 80 redirects to HTTPS, the health check is open for load balancers, and a host name the instance does not serve gets no answer:

H=$(sudo grep '^AFANASY_URL=' /var/lib/sac/secrets/sac-credentials.txt | sed -E 's#^AFANASY_URL=https://([^/]+)/.*#\1#')
curl -s -o /dev/null -w 'http:// -> %{http_code} %{redirect_url}\n' --resolve "$H:80:127.0.0.1" "http://$H/"
curl -sk -o /dev/null -w 'healthz -> HTTP %{http_code}\n' --resolve "$H:443:127.0.0.1" "https://$H/healthz"
curl -sk -o /dev/null -w 'unknown host -> HTTP %{http_code}\n' -H 'Host: other.example' https://127.0.0.1/ || true

http:// -> 301 https://ec2-<public-ip-dashed>.<region>.compute.amazonaws.com/ healthz -> HTTP 200 unknown host -> HTTP 000

The Web Monitor refuses a request with no password or a wrong one, and accepts this instance's login. The password is passed to curl on standard input, so it never appears in the process list or your shell history:

sudo bash -c ' F=/var/lib/sac/secrets/sac-credentials.txt P=$(grep "^AFANASY_WEB_PASSWORD=" $F | cut -d= -f2-) H=$(grep "^AFANASY_URL=" $F | sed -E "s#^AFANASY_URL=https://([^/]+)/.*#\1#") R="--resolve $H:443:127.0.0.1" curl -sk $R -o /dev/null -w "no password -> HTTP %{http_code}\n" "https://$H/" printf "user = \"afadmin:wrong\"\n" | curl -K - -sk $R -o /dev/null -w "wrong password -> HTTP %{http_code}\n" "https://$H/" printf "user = \"afadmin:%s\"\n" "$P" | curl -K - -sk $R -o /dev/null -w "per-instance login -> HTTP %{http_code}\n" "https://$H/" '

no password -> HTTP 401 wrong password -> HTTP 401 per-instance login -> HTTP 200

The same login protects afserver's JSON API, which the Web Monitor and your scripts use. This lists the jobs on the farm; a new farm holds only afserver's own system job, afanasy:

sudo bash -c ' F=/var/lib/sac/secrets/sac-credentials.txt P=$(grep "^AFANASY_WEB_PASSWORD=" $F | cut -d= -f2-) H=$(grep "^AFANASY_URL=" $F | sed -E "s#^AFANASY_URL=https://([^/]+)/.*#\1#") Q="{\"get\":{\"type\":\"jobs\"}}" printf "user = \"afadmin:%s\"\n" "$P" | curl -K - -sk --resolve "$H:443:127.0.0.1" -X POST \
  -H "AFANASY: ${#Q}" -H "Origin: https://$H" --data-binary "$Q" "https://$H/" | jq -r ".jobs[].name" '

afanasy

Open AFANASY_URL in your browser and sign in as afadmin with the password from Step 4. The certificate is self-signed, so your browser asks you to confirm it the first time. To install your own certificate, see Step 9.

Jobs lists every job with its state, owner, priority and frame range. The afanasy job is afserver's own system job, which runs post commands, wake-on-LAN and events. In this example, demo-render is a 24-frame job waiting for render nodes. The buttons on the left start, pause, stop, restart and delete the selected job:

Double-click a job to open its tasks. Each frame is a task with its state, error count and starts; SKIP, RESTART and CONTINUE act on the selected tasks:

Farm shows the render pools and the render nodes connected to them, with controls to add pools, pause nodes, mark them busy (NIMBY), eject tasks and run services. A new farm has only the root pool / until render nodes connect (Step 8):

Users lists everyone who has submitted jobs, with their priority, error handling and how their jobs are ordered:

Monitors lists the connected Web Monitor sessions. Every browser session arrives from 127.0.0.1 because nginx is the only path to afserver; the real client addresses are in /var/log/nginx/access.log:

A note on Afanasy's own permissions: the Web Monitor's VISOR and GOD modes, which let a user manage other users' jobs and the farm, are switches in the browser, not passwords checked by the server. Treat everyone who has the Web Monitor login as a farm administrator.

Artists normally submit from their applications with CGRU's submitters or the af Python module on a workstation that can reach afserver (Step 8). You can also submit through the authenticated API. This request sends a small two-frame test job, lists it and deletes it again. With no render nodes connected, its tasks wait in the queue:

sudo bash -c ' F=/var/lib/sac/secrets/sac-credentials.txt P=$(grep "^AFANASY_WEB_PASSWORD=" $F | cut -d= -f2-) H=$(grep "^AFANASY_URL=" $F | sed -E "s#^AFANASY_URL=https://([^/]+)/.*#\1#") api() { printf "user = \"afadmin:%s\"\n" "$P" | curl -K - -sk --resolve "$H:443:127.0.0.1" -X POST \
  -H "AFANASY: ${#1}" -H "Origin: https://$H" --data-binary "$1" "https://$H/"; } J="{\"job\":{\"name\":\"hello-farm\",\"user_name\":\"afadmin\",\"host_name\":\"guide\",\"blocks\":[{\"name\":\"frames\",\"service\":\"generic\",\"capacity\":1000,\"flags\":1,\"frame_first\":1,\"frame_last\":2,\"frames_per_task\":1,\"frames_inc\":1,\"command\":\"echo frame @#@\"}]}}" ID=$(api "$J" | jq -r ".id") echo "submitted job id $ID" api "{\"get\":{\"type\":\"jobs\"}}" | jq -r ".jobs[] | select(.name==\"hello-farm\") | \"\(.name): \(.blocks[0].tasks_num) tasks\"" api "{\"action\":{\"type\":\"jobs\",\"ids\":[$ID],\"user_name\":\"afadmin\",\"host_name\":\"guide\",\"operation\":{\"type\":\"delete\"}}}" | jq -r ".info.text" '

submitted job id 2 hello-farm: 2 tasks Action processed.

Render nodes, and workstations that submit with CGRU's own tools, connect straight to afserver on TCP 51000. That protocol has no authentication, so admit only machines you control and only from inside your VPC.

• Find the instance's private address:

jq -r '.ds["meta-data"]["local-ipv4"]' /run/cloud-init/instance-data.json

• Make afserver listen on that address. Edit /etc/default/afserver and set AF_SERVER_BIND_ADDRESS to the private address, then restart afserver:

sudo sed -i 's/^AF_SERVER_BIND_ADDRESS=.*/AF_SERVER_BIND_ADDRESS=<private-ip>/' /etc/default/afserver
sudo systemctl restart afserver
sudo ss -lntp | grep afserver

• Allow TCP 51000 into this instance's security group only from the render nodes' security group (never from 0.0.0.0/0):

aws ec2 authorize-security-group-ingress --group-id <afanasy-server-sg> \
  --protocol tcp --port 51000 --source-group <render-nodes-sg>

• On each render node, install CGRU 3.4.1 (the same release as the server), point it at the server in <cgru-location>/config.json and start afrender:

{"cgru_config": {"af_servername": "<private-ip>", "af_serverport": 51000}} <cgru-location>/start/AFANASY/_afrender.sh

The node appears in Farm within a few seconds and starts taking tasks. Render nodes run job commands as the user that runs afrender, so run it as a dedicated unprivileged user on each node.

To close the server again, set AF_SERVER_BIND_ADDRESS=127.0.0.1, restart afserver and remove the security group rule. afserver refuses to start if the address is empty or not an IPv4 address.

Point a DNS record at the instance (an Elastic IP keeps the address stable), then tell the instance its name. This command regenerates the self-signed certificate for that name, updates the credentials file and reloads nginx:

sudo afanasy-set-hostname farm.your-domain.example

To use a certificate from a public CA instead, replace /etc/nginx/ssl/afanasy.crt (full chain) and /etc/nginx/ssl/afanasy.key (mode 0600) and run sudo systemctl reload nginx.

afserver listens on loopback and cannot start without the first boot login; afserver, nginx and the first boot service are blocked from the instance metadata service:

grep '^AF_SERVER_BIND_ADDRESS=' /etc/default/afserver systemctl show afserver -p Requires --value systemctl show afserver nginx cgru-firstboot -p IPAddressDeny --value

AF_SERVER_BIND_ADDRESS=127.0.0.1 -.mount var-lib-cgru.mount cgru-firstboot.service sysinit.target system.slice 169.254.169.254/32 168.63.129.16/32 fd00:ec2::254/128 169.254.169.254/32 168.63.129.16/32 fd00:ec2::254/128 169.254.169.254/32 168.63.129.16/32 fd00:ec2::254/128

afserver runs unprivileged in a read-only system with only the farm data writable, and there is no compiler on the image:

systemctl show afserver -p User -p ProtectSystem -p NoNewPrivileges -p ReadWritePaths command -v gcc g++ make cmake || echo "no compiler toolchain on the image"

User=afanasy ReadWritePaths=/var/lib/cgru ProtectSystem=strict NoNewPrivileges=yes no compiler toolchain on the image