Arroyo

Search the Azure Marketplace for Arroyo, choose the plan, and create the VM.

In the networking step, attach an NSG that allows inbound TCP 22 (SSH) and TCP 80 (the console) from your client networks only. For production, put a TLS reverse proxy in front of port 80.

RG="arroyo-prod"; LOCATION="eastus"; VM_NAME="arroyo-01" IMAGE_URN="sac1647283583153:arroyo-ubuntu-24-04:default:latest"
# Accept the Marketplace terms once per subscription (required before the first CLI deploy)
az vm image terms accept --urn sac1647283583153:arroyo-ubuntu-24-04:default:latest
az vm create \
  --resource-group "$RG" --name "$VM_NAME" --location "$LOCATION" \
  --image "$IMAGE_URN" --size Standard_B2s \
  --admin-username azureuser --generate-ssh-keys \
  --public-ip-sku Standard
# then open TCP 80 to your own client network only:
az vm open-port --resource-group "$RG" --name "$VM_NAME" --port 80 --priority 900

ssh azureuser@<vm-ip>

arroyo.service, nginx.service and arroyo-firstboot.service all start automatically on first boot.

Run the following to confirm that the services are active, first boot has completed, and the listeners are bound as expected.

sudo systemctl is-active arroyo nginx arroyo-firstboot
sudo test -f /var/lib/sac/data/arroyo-firstboot.done && echo FIRSTBOOT_DONE
sudo ss -tlnp | grep -E ':80 |:5115 |:5114 '

Expected output: all three services are active, nginx is bound to public port 80, and the Arroyo services are bound to loopback only.

active active active FIRSTBOOT_DONE LISTEN 0 1024 127.0.0.1:5115 0.0.0.0:* users:(("arroyo",pid=3659,fd=14)) LISTEN 0 1024 127.0.0.1:5114 0.0.0.0:* users:(("arroyo",pid=3659,fd=12)) LISTEN 0 511 0.0.0.0:80 0.0.0.0:* users:(("nginx",pid=3670,fd=5),...) LISTEN 0 511 [::]:80 [::]:* users:(("nginx",pid=3670,fd=6),...)

The console has no default password. On first boot, the VM generates a unique console password and a unique engine key and writes them to a root-only note. The nginx login gate requires that password.

Read the note, then check that the gate rejects an unauthenticated request and accepts the per-VM login:

sudo cat /root/arroyo-info.txt curl -s -o /dev/null -w 'no credentials -> HTTP %{http_code}\n' http://127.0.0.1/api/v1/pipelines
P=$(sudo grep '^ARROYO_ADMIN_PASSWORD=' /root/arroyo-info.txt | cut -d= -f2-)
curl -s -o /dev/null -w 'with the per-VM login -> HTTP %{http_code}\n' -u "admin:$P" http://127.0.0.1/api/v1/pipelines

The note contains the console username, password, engine key and URL, all unique to this machine and never baked into the image. An unauthenticated request is rejected with 401, and the per-VM login returns 200:

ARROYO_ADMIN_USERNAME=admin ARROYO_ADMIN_PASSWORD=<ARROYO_ADMIN_PASSWORD> ARROYO_API_KEY=<ARROYO_API_KEY> ARROYO_URL=http://<vm-ip>/ no credentials -> HTTP 401 with the per-VM login -> HTTP 200

The ARROYO_URL line shows the address the VM sees for itself. Browse to the console at your VM's public IP address instead.

Arroyo has no login of its own, so this image adds two layers. nginx is the browser gate. The engine also requires a bearer key, which nginx presents for you, so even a process on the VM cannot reach the engine directly without the key.

Call the engine directly, first without the key and then with it:

K=$(sudo grep '^ARROYO_API_KEY=' /root/arroyo-info.txt | cut -d= -f2-)
curl -s -o /dev/null -w 'direct to the engine, no key -> HTTP %{http_code}\n' http://127.0.0.1:5115/api/v1/pipelines
curl -s -o /dev/null -w 'direct to the engine, with key -> HTTP %{http_code}\n' -H "Authorization: Bearer $K" http://127.0.0.1:5115/api/v1/pipelines

Expected output: the engine rejects the request without the key and accepts the one with it.

direct to the engine, no key -> HTTP 401 direct to the engine, with key -> HTTP 200

You need the engine key only if you call the REST API directly instead of through the console or the proxy.

Open the console in a browser at your VM's public IP address. When the nginx login prompt appears, enter the username admin and the password from Step 5.

You land on the Pipelines view, which lists every pipeline on this cluster with its state, runtime and task count. A freshly deployed machine starts with an empty list.

Choose Create Pipeline to open the SQL editor. Arroyo pipelines are SQL: you declare a source table with a connector, then write a query over it.

Paste the following statement. It reads Arroyo's built-in demo stream and transforms each event:

CREATE TABLE impulse_src WITH ( connector = 'impulse', event_rate = '20' ); SELECT counter, counter * 10 + 7 AS transformed FROM impulse_src;

The editor validates and plans the query as you type and shows a live preview of the results, so you can check the output before you start the pipeline. The impulse connector supplies its own schema, so the CREATE TABLE declares no columns.

Give the pipeline a name and start it. The job detail view shows the pipeline as a graph of operators, with live throughput, watermark and checkpoint metrics for each one.

You can watch events flow through the source, the projection and the sink.

Everything in the console is also available over the REST API through the same login. Run the following to create a pipeline and check that its job is scheduled:

P=$(sudo grep '^ARROYO_ADMIN_PASSWORD=' /root/arroyo-info.txt | cut -d= -f2-) Q='CREATE TABLE impulse_src WITH (connector = '"'"'impulse'"'"', event_rate = '"'"'20'"'"'); SELECT counter, counter * 10 + 7 AS transformed FROM impulse_src;'
PID=$(curl -s -u "admin:$P" -H 'Content-Type: application/json' \
  --data "$(jq -nc --arg q "$Q" '{name:"guide-demo", query:$q, parallelism:1}')" \
  http://127.0.0.1/api/v1/pipelines | jq -r .id)
echo "pipeline: $PID" curl -s -u "admin:$P" "http://127.0.0.1/api/v1/pipelines/$PID/jobs" | jq -r '.data[0] | "job \(.id) state \(.state)"'

Expected output: the pipeline is created and its job is scheduled.

pipeline: pl_XXXXXXXXXX job job_XXXXXXXXXX state Created

Remove the demo pipeline when you are finished with it. A running pipeline cannot be deleted, so stop it first and wait for the job to report Stopped.

P=$(sudo grep '^ARROYO_ADMIN_PASSWORD=' /root/arroyo-info.txt | cut -d= -f2-)
PID=$(curl -s -u "admin:$P" http://127.0.0.1/api/v1/pipelines | jq -r '.data[]|select(.name=="guide-demo")|.id' | head -1)
curl -s -o /dev/null -u "admin:$P" -X PATCH -H 'Content-Type: application/json' \
  --data '{"stop":"immediate"}' "http://127.0.0.1/api/v1/pipelines/$PID" for i in $(seq 1 20); do
S=$(curl -s -u "admin:$P" "http://127.0.0.1/api/v1/pipelines/$PID/jobs" | jq -r '.data[0].state') [ "$S" = "Stopped" ] && break; sleep 3 done curl -s -o /dev/null -w 'delete -> HTTP %{http_code}\n' -u "admin:$P" -X DELETE "http://127.0.0.1/api/v1/pipelines/$PID"

Expected output:

delete -> HTTP 200

The impulse connector is a demo source. In production, you point Arroyo at your own streams. You configure a connector in the same CREATE TABLE statement, as in this Kafka topic example:

CREATE TABLE events ( user_id TEXT, amount DOUBLE, event_time TIMESTAMP ) WITH ( connector = 'kafka', bootstrap_servers = 'broker.example.com:9092', topic = 'events', type = 'source', format = 'json', 'source.offset' = 'latest' ); SELECT user_id, sum(amount) AS total FROM events GROUP BY user_id, tumble(interval '1 minute');

To browse the available connectors, open Connections in the console, then choose Create Connection. Each connector card shows whether it can act as a source, a sink or both.

Creating a connection profile lets you configure and test a connector once before you reference it from SQL.

SQL pipelines need no compiler toolchain, so the image ships without one and starts fast and fully offline. User-defined functions (UDFs) are compiled from Rust, so they do require one.

To enable UDFs, allow the compiler service to install its toolchain and restart Arroyo:

sudo sed -i 's/^install-clang = false/install-clang = true/; s/^install-rustc = false/install-rustc = true/' /etc/arroyo/config.toml
sudo systemctl restart arroyo

The first restart after you enable this downloads the toolchain, so allow several minutes and outbound internet access. Compiling UDFs is CPU- and memory-intensive. If you rely on them, use a VM size of at least 4 vCPU and 16 GB, such as Standard_D4s_v3.

The same two settings live in the template at /usr/share/arroyo/config.toml.tmpl. Change them there too if you want the change to survive a re-render.

Everything the image installs lives in a handful of predictable paths. List them and print the installed version:

/usr/local/bin/arroyo --version ls -1 /usr/local/bin/arroyo /etc/arroyo/config.toml /usr/share/arroyo/config.toml.tmpl \
  /etc/systemd/system/arroyo.service /usr/local/sbin/arroyo-firstboot.sh \
  /etc/nginx/sites-available/sac-arroyo

Expected output: the pinned version and the installed components.

arroyo 0.15.0 /etc/arroyo/config.toml /etc/nginx/sites-available/sac-arroyo /etc/systemd/system/arroyo.service /usr/local/bin/arroyo /usr/local/sbin/arroyo-firstboot.sh /usr/share/arroyo/config.toml.tmpl

Check the service status, restart it, or read its recent logs:

sudo systemctl status arroyo --no-pager sudo systemctl restart arroyo
sudo journalctl -u arroyo -n 50 --no-pager

Arroyo is a simple systemd service with Restart=always, so it comes back automatically after a crash or a reboot.

Pipeline definitions live in the SQLite metadata store at /var/lib/arroyo/config.sqlite, and pipeline state lives in /var/lib/arroyo/checkpoints. Back up /var/lib/arroyo to preserve both.

nginx enforces the console login with HTTP Basic Auth, using the file /etc/nginx/arroyo.htpasswd. To set a new password, rewrite that file with htpasswd (bcrypt) and reload nginx:

NEWPASS='<new-password>'
sudo htpasswd -bB /etc/nginx/arroyo.htpasswd admin "$NEWPASS"
sudo systemctl reload nginx

If you rely on /root/arroyo-info.txt, update it to match, and store the new password in your own secret manager. The engine key is separate and does not change when you change the console password.

For production, terminate TLS in front of port 80. The reverse proxy is already in place, so you only need to add a certificate with certbot for your own domain name:

sudo apt-get update && sudo apt-get install -y certbot python3-certbot-nginx
sudo certbot --nginx -d your-domain.example.com

certbot edits the nginx server block to listen on 443 with your certificate and can redirect HTTP to HTTPS. Once TLS is in place, the console login is protected in transit.

• Restrict inbound TCP 80 (and 443) to your own client networks with an NSG.
• Add TLS (Step 16) so the login and your pipeline definitions are encrypted in transit.
• Change the console password (Step 15) and keep /root/arroyo-info.txt readable only by root.
• Keep every Arroyo service bound to loopback. Never expose ports 5114 to 5118 directly.
• Treat the engine key as a secret. It grants full API access if the proxy is bypassed.
• Keep the OS patched. Unattended security upgrades are enabled by default.

Arroyo is dual-licensed under the Apache License 2.0 or the MIT license, at your option. This is a repackaged open-source software product with additional charges for SecureAnyCloud support services.

All product and company names are trademarks or registered trademarks of their respective holders. Their use does not imply any affiliation with or endorsement by them.

Deploy on Azure

Find Arroyo on Ubuntu 24.04 LTS on the Azure Marketplace and launch it in minutes.

Need help?

SecureAnyCloud provides 24/7 support with a 24h response SLA. Visit secureanycloud.com or contact support through the Azure Marketplace listing.

Deploy on Azure

Launch Arroyo on Ubuntu 24.04 LTS on Azure with 24/7 support from SecureAnyCloud.

View this image

Need Help?

Our support team is available 24/7.

sales@secureanycloud.com