Deployment guide

Feldera 0.378.0

Overview

Feldera is an incremental compute engine. You declare tables and views in standard SQL, and Feldera compiles the program into a running pipeline that keeps every view correct as rows are inserted, updated and deleted. It does only the work each change requires instead of recomputing from scratch.

Data arrives through HTTP or connectors (Kafka, Debezium, PostgreSQL, Delta Lake, S3, files, URLs and more), and results leave the same ways. You can also query materialized views ad hoc. Rust user-defined functions (UDFs) extend the SQL.

This guide covers the Feldera 0.378.0 on Ubuntu 24.04 AMI from SecureAnyCloud on Amazon EC2. It is one version of the SecureAnyCloud streaming and messaging listing on AWS Marketplace, and you select it as the software version when you subscribe and launch.

The image runs the MIT-licensed Feldera open-source edition the way upstream ships it. It uses the official all-in-one pipeline-manager container (REST API, Web Console, the SQL-to-Rust compiler and the runner that executes compiled pipelines) with an official PostgreSQL container as its metadata store. Both are pinned by digest and started by systemd.

What you get:

  • Nothing reachable off the instance except SSH and HTTPS. Upstream publishes the console on every interface with no login. Here Docker publishes the pipeline-manager to 127.0.0.1:8080 only, PostgreSQL has no network at all (the pipeline-manager reaches it through a unix socket), and compiled pipelines listen inside the container only. nginx is the only service on a network address.
  • The console and REST API over HTTPS only, behind a login or an API key. nginx serves them on port 443 (port 80 only redirects). The console uses HTTP Basic authentication for the user admin. Scripts, the Python SDK and the fda CLI send this instance's API key as a bearer token. Both are generated on your instance at first boot, the password is stored only as a bcrypt hash, and failed logins are rate-limited. Requests for host names the instance does not serve are dropped. (Feldera's open-source edition has no login of its own without an external OIDC provider, so nginx is the authentication layer.)
  • Guarded streaming and query requests. Another web page cannot use your browser's saved login to run ad-hoc SQL, stream a pipeline's output or logs, or trigger profiles and support bundles. Those requests are refused unless they come from the console page itself (or carry the API key). Other sites' Origin headers, cross-site WebSocket upgrades, CORS preflights and ; path parameters (plain or encoded as %3B) are refused too.
  • Compressed request bodies refused at the front door. Feldera 0.378.0 is built on actix-web 4.11.0 / actix-http 3.12.1, whose request-body decompression has no size limit (RUSTSEC-2026-0447, -0448 and -0455; no Feldera release carries a fix yet). No Feldera client compresses what it sends, so nginx answers 415 to any request whose Content-Encoding is not identity, before the body is read and even with a valid login. nginx is also the only HTTP parser a client reaches: it re-frames request bodies, forwards no request trailers and opens a fresh connection to Feldera for every request.
  • Sandboxed containers. Signing in lets you compile and run SQL pipelines and Rust code on the instance. The pipeline-manager runs as an unprivileged user with no Linux capabilities, no privilege escalation, a read-only root file system, Docker's default seccomp profile, no Docker socket and no host network. It cannot read the credentials file, the TLS key, the login hash or the PostgreSQL password. Pipelines cannot open connections to the instance metadata service (so they can never read the instance's IAM credentials) or to the instance's own services.
  • No default secret. The login, the API key, the PostgreSQL password and the TLS private key are generated on your instance at first boot. The image ships none of them, and PostgreSQL starts with an empty database.
  • HTTPS from the first boot. The certificate is generated on your instance for its own names and addresses. It is an end-entity certificate: it can only vouch for this instance and can never sign other certificates. Install your own CA-issued certificate with one command (Step 14).
  • Telemetry off. No analytics key is configured, the start gate refuses one, and the page's Content Security Policy blocks the console's tracking pixel. nginx hands Feldera the real client address and the configured host name (never values a client sends), SSH accepts keys only, and Ubuntu 24.04 LTS is fully patched at build time with unattended security upgrades switched on.

What is on the image: Feldera 0.378.0 (the official images.feldera.com/feldera/pipeline-manager:0.378.0 image, pinned by digest), PostgreSQL 16.15 (the official postgres:16.15-alpine3.24 image, pinned by digest), the fda CLI 0.378.0 (from the same Feldera image), Docker CE (from Docker's signed repository), nginx 1.24 (from Ubuntu 24.04), cloud-init and the AWS Systems Manager agent.

Default user: ubuntu. Architecture: x86_64. Recommended instance type: m5.large (choose more memory for large pipelines). Root volume: 30 GiB.

Prerequisites

  • An AWS account subscribed to the listing, an EC2 key pair and a VPC subnet with a route to the internet (or a private route you use for management).
  • A security group that allows TCP 22 (SSH) and TCP 443 (the console and API) from your admin network, plus TCP 80 if you want plain HTTP requests redirected to HTTPS. There is nothing else to open: the pipeline-manager listens on the loopback address only and PostgreSQL has no network.
  • Outbound access for your pipelines. Pipelines open outbound connections to your sources and sinks (Kafka brokers, databases, object storage, URLs), so allow that outbound traffic.
1

Launch the instance

In the AWS Marketplace listing, choose Continue to Subscribe, then Continue to Configuration. Pick Feldera 0.378.0 on Ubuntu 24.04 as the software version, select your Region, and choose Launch through EC2.

Choose m5.large (or larger), your key pair and a security group as described above.

To launch with the AWS CLI instead, replace the values in angle brackets:

aws ec2 run-instances --region <region> --image-id <ami-id-from-the-listing> \
  --instance-type m5.large --key-name <your-key-pair> \
  --security-group-ids <your-security-group> --subnet-id <your-subnet> \
  --metadata-options HttpTokens=required,HttpEndpoint=enabled \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=feldera}]'

First boot takes about two minutes after the instance is running. The boot service generates the login, the API key, the PostgreSQL password and the certificate. PostgreSQL initializes its empty database, and the pipeline-manager prepares its state volume from the image's precompiled Rust dependencies. Then nginx opens.

2

Connect to your instance

Connect over SSH as the default user for this version:

Version SSH user

Feldera 0.378.0 on Ubuntu 24.04 ubuntu

ssh -i <your-key.pem> ubuntu@<public-ip>
3

Check the services and the listening set

Check the services:

systemctl is-active docker sac-feldera-firewall feldera-metadata-names feldera-firstboot feldera-postgres feldera nginx
sudo test -f /var/lib/sac/data/feldera-firstboot.done && echo FIRSTBOOT_DONE
active active active active active active active FIRSTBOOT_DONE

Only SSH and nginx listen on network addresses. Docker publishes the pipeline-manager to 127.0.0.1:8080 and nothing else. Confirm the listening set:

sudo ss -lntp | awk 'NR>1 {print $4, $6}' | grep -v '127.0.0.5' | sed -E 's/,pid=[0-9]+,fd=[0-9]+//g' | sort
sudo docker port feldera-manager
sudo docker port feldera-postgres | wc -l
0.0.0.0:22 users:(("sshd"),("systemd")) 0.0.0.0:443 users:(("nginx"),("nginx"),("nginx")) 0.0.0.0:80 users:(("nginx"),("nginx"),("nginx")) 127.0.0.1:8080 users:(("docker-proxy")) [::]:22 users:(("sshd"),("systemd")) [::]:443 users:(("nginx"),("nginx"),("nginx")) [::]:80 users:(("nginx"),("nginx"),("nginx")) 8080/tcp -> 127.0.0.1:8080 0
4

Get the console URL, login, API key and certificate fingerprint

Read the credentials file. It is readable by root only, and it holds the console URL, the admin user and password, the API key and the SHA-256 fingerprint of this instance's certificate:

sudo cat /root/feldera-credentials.txt
FELDERA_URL=https://<public-ip>/ FELDERA_HOST=<public-ip> FELDERA_UI_USER=admin FELDERA_UI_PASSWORD=<FELDERA_UI_PASSWORD> FELDERA_API_KEY=<set-at-first-boot>:<your-api-key> FELDERA_TLS_CERT_SHA256=<the certificate's SHA-256 fingerprint, AA:BB:...> FELDERA_TLS_CERT_FILE=/etc/sac/feldera-ui.pem

The file also carries comment lines with the same security note and the commands used in this guide.

To show only the URL, the user and the fingerprint (without the password or the key), run:

sudo grep -E '^FELDERA_(URL|UI_USER|TLS_CERT_SHA256)=' /root/feldera-credentials.txt
FELDERA_URL=https://44.204.156.48/ FELDERA_UI_USER=admin FELDERA_TLS_CERT_SHA256=6F:7D:44:91:92:C2:2F:86:7E:2E:A3:49:FA:5F:58:F6:F1:AD:32:D1:D1:2A:B0:38:5C:2E:B2:95:40:CA:B9:16

The host is the instance's public IP address from the EC2 instance metadata (the private address when the instance has no public one). The certificate also names the public DNS name, the private IP address and the private DNS name.

If you stop and start the instance without an Elastic IP, the boot service picks up the new address and issues a new certificate for it (unless you installed your own, Step 14).

To read the certificate's names and constraints from the instance, run:

openssl x509 -in /etc/sac/feldera-ui.pem -noout -ext subjectAltName,basicConstraints,keyUsage,extendedKeyUsage
X509v3 Subject Alternative Name: IP Address:44.204.156.48, DNS:ec2-44-204-156-48.compute-1.amazonaws.com, IP Address:172.31.89.197, DNS:ip-172-31-89-197.ec2.internal, IP Address:127.0.0.1, DNS:localhost X509v3 Basic Constraints: critical CA:FALSE X509v3 Key Usage: critical Digital Signature, Key Encipherment X509v3 Extended Key Usage: TLS Web Server Authentication
5

Open the Web Console

Browse to the FELDERA_URL from Step 4. The certificate is self-signed and unique to this instance, so the browser warns about it the first time.

Check the certificate before you continue. Open the browser's certificate viewer for the page and compare its SHA-256 fingerprint with FELDERA_TLS_CERT_SHA256 from the credentials file. Continue only if they match, and only for this site. Never add this certificate to an operating-system or browser trust store. For a certificate browsers trust without a warning, install a CA-issued one (Step 14).

The browser then asks for a user name and password. Sign in as admin with the password from the credentials file. To print just the password on the instance, run:

sudo awk -F= '$1=="FELDERA_UI_PASSWORD" {print $2}' /root/feldera-credentials.txt

The home page lists your pipelines and the bundled use cases and tutorials (screenshot at the top of this guide).

Select New Pipeline to open the SQL editor. Write tables and views, then press Start to compile the program to a native pipeline and run it. Use the Runtime, Ad-Hoc Queries, Change Stream, Health and Logs tabs under the editor to watch it:

6

The REST API

Everything the console does goes through Feldera's REST API (/v0/..., described at /v0/... in the Feldera API reference). Through nginx it accepts either the console login or the API key:

  • /root/feldera-api.curl holds the API key as a curl header line, so the key never appears on a command line.
  • /root/feldera-curl.conf holds the console login the same way.

On the instance, the examples pin the instance's public name to 127.0.0.1 with --resolve, so they work whatever the security group allows. They also check the server against this instance's certificate:

H=$(sudo sed -n 's/^FELDERA_HOST=//p' /root/feldera-credentials.txt)
curl -s --resolve "$H:443:127.0.0.1" --cacert /etc/sac/feldera-ui.pem https://$H/healthz
sudo curl -s -K /root/feldera-api.curl --resolve "$H:443:127.0.0.1" --cacert /etc/sac/feldera-ui.pem https://$H/v0/config \
  | python3 -c 'import json,sys; c=json.load(sys.stdin); print(c["version"], c["edition"])' curl -s -o /dev/null -w 'without credentials: %{http_code}\n' --resolve "$H:443:127.0.0.1" --cacert /etc/sac/feldera-ui.pem https://$H/v0/pipelines curl -s -o /dev/null -w 'direct to 127.0.0.1:8080 as ubuntu: %{http_code}\n' -m 5 http://127.0.0.1:8080/v0/pipelines || true
ok 0.378.0 Open source without credentials: 401 direct to 127.0.0.1:8080 as ubuntu: 000

127.0.0.1:8080 itself has no login (Feldera's own), so the instance refuses connections to it from every local user except root and nginx.

From your workstation, copy the certificate (Step 4), check its fingerprint, and keep the API key in a file only you can read. Then call the API the same way:

scp -i <your-key.pem> ubuntu@<public-ip>:/etc/sac/feldera-ui.pem feldera.pem openssl x509 -in feldera.pem -noout -fingerprint -sha256
# must match FELDERA_TLS_CERT_SHA256 ( umask 077; printf 'header = "Authorization: Bearer %s"\n' "<your-api-key>" > feldera-api.curl )
curl -K feldera-api.curl --cacert feldera.pem https://<public-ip>/v0/pipelines

Requests made with the console login (rather than the API key) to the per-pipeline read routes, /v0/metrics and /v0/cluster must carry the browser's same-origin signal. In such scripts, add -H 'Sec-Fetch-Site: same-origin'.

7

Create and run a SQL pipeline

This pipeline keeps a running total of orders per customer. Create it through the API. The console's New Pipeline does the same.

mkdir -p ~/feldera-demo && cd ~/feldera-demo cat > orders.sql <<'SQL' CREATE TABLE orders (id BIGINT NOT NULL PRIMARY KEY, customer VARCHAR NOT NULL, amount DECIMAL(10,2) NOT NULL) WITH ('materialized' = 'true'); CREATE MATERIALIZED VIEW customer_totals AS SELECT customer, COUNT(*) AS orders, SUM(amount) AS total FROM orders GROUP BY customer; SQL python3 -c 'import json; print(json.dumps({"name": "orders", "program_code": open("orders.sql").read(), "runtime_config": {"workers": 2}}))' > pipeline.json
H=$(sudo sed -n 's/^FELDERA_HOST=//p' /root/feldera-credentials.txt)
sudo curl -s -K /root/feldera-api.curl --resolve "$H:443:127.0.0.1" --cacert /etc/sac/feldera-ui.pem \
  -X PUT -H 'Content-Type: application/json' --data-binary @pipeline.json https://$H/v0/pipelines/orders \
  | python3 -c 'import json,sys; p=json.load(sys.stdin); print(p["name"], p["program_status"])'
orders Pending

Feldera compiles the SQL to Rust and the Rust to a native binary in the background. A small program takes about a minute.

On a new instance, the first compile takes a few minutes longer, because the instance's disk loads its blocks from the AMI snapshot on first read. A first-boot service, feldera-prewarm, reads Feldera's compiler files in the background during the first minutes to speed this up.

Wait for the compile, then start the pipeline:

H=$(sudo sed -n 's/^FELDERA_HOST=//p' /root/feldera-credentials.txt) api() {
sudo curl -s -K /root/feldera-api.curl --resolve "$H:443:127.0.0.1" --cacert /etc/sac/feldera-ui.pem "$@"; } for i in $(seq 1 180); do
S=$(api https://$H/v0/pipelines/orders | python3 -c 'import json,sys; print(json.load(sys.stdin)["program_status"])'); [ "$S" = Success ] && break; sleep 5; done echo "program: $S"
program: Success
H=$(sudo sed -n 's/^FELDERA_HOST=//p' /root/feldera-credentials.txt) api() {
sudo curl -s -K /root/feldera-api.curl --resolve "$H:443:127.0.0.1" --cacert /etc/sac/feldera-ui.pem "$@"; } api -X POST -o /dev/null -w 'start: %{http_code}\n' https://$H/v0/pipelines/orders/start for i in $(seq 1 60); do
D=$(api https://$H/v0/pipelines/orders | python3 -c 'import json,sys; print(json.load(sys.stdin)["deployment_status"])'); [ "$D" = Running ] && break; sleep 2; done echo "pipeline: $D"
start: 202 pipeline: Running

Push rows in over HTTP (JSON, CSV, Avro and Parquet are supported), then query the view. Upserts and deletes update the totals incrementally.

cd ~/feldera-demo
H=$(sudo sed -n 's/^FELDERA_HOST=//p' /root/feldera-credentials.txt) api() {
sudo curl -s -K /root/feldera-api.curl --resolve "$H:443:127.0.0.1" --cacert /etc/sac/feldera-ui.pem "$@"; } printf '%s\n' '{"insert": {"id": 1, "customer": "acme", "amount": 120.50}}' '{"insert": {"id": 2, "customer": "acme", "amount": 79.50}}' \
  '{"insert": {"id": 3, "customer": "globex", "amount": 42.00}}' > batch.json api -X POST -H 'Content-Type: application/json' --data-binary @batch.json -o /dev/null -w 'ingress: %{http_code}\n' \
  "https://$H/v0/pipelines/orders/ingress/orders?format=json&update_format=insert_delete" printf '%s\n' '{"delete": {"id": 2, "customer": "acme", "amount": 79.50}}' > delete.json api -X POST -H 'Content-Type: application/json' --data-binary @delete.json -o /dev/null -w 'delete: %{http_code}\n' \
  "https://$H/v0/pipelines/orders/ingress/orders?format=json&update_format=insert_delete" sleep 2 api -G --data-urlencode 'sql=SELECT customer, orders, total FROM customer_totals ORDER BY customer' --data-urlencode 'format=text' \
  https://$H/v0/pipelines/orders/query; echo
ingress: 200 delete: 200 +----------+--------+--------+ | customer | orders | total | +----------+--------+--------+ | acme | 1 | 120.50 | | globex | 1 | 42.00 | +----------+--------+--------+

To follow a view's changes as they happen, send POST /v0/pipelines/orders/egress/customer_totals?format=json. It streams every insert and delete. The console's Change Stream tab shows the same.

8

The fda CLI on the instance

fda is Feldera's command-line client (the same release, copied from the Feldera image). It reads the API key from a file with --oidc-token-file (it sends the file's contents as the bearer token). It checks the server against this instance's certificate with --tls-cert. No secret appears on the command line:

F() {
sudo fda --host https://127.0.0.1 --tls-cert /etc/sac/feldera-ui.pem --oidc-token-file /root/feldera-api-key "$@"; } F --version F pipelines F query orders "SELECT customer, total FROM customer_totals ORDER BY customer"
fda 0.378.0 ╭────────┬─────────╮ │ name │ status │ ├────────┼─────────┤ │ orders │ Running │ ╰────────┴─────────╯ +----------+--------+ | customer | total | +----------+--------+ | acme | 120.50 | | globex | 42.00 | +----------+--------+

fda can also create pipelines from SQL files (fda create <name> program.sql), start and stop them, stream changes (fda listen), tail logs and open an interactive SQL shell (fda shell <name>).

To use it from your workstation, download fda for your platform from the Feldera 0.378.0 release and point it at https://<public-ip> with the same three options.

9

The Python SDK

The Feldera Python SDK (pip install feldera==0.378.0) talks to the same API. Run it from your workstation, with the certificate copied as in Step 6 and the API key in a file only you can read:

python3 -m venv feldera-venv && . feldera-venv/bin/activate && pip install feldera==0.378.0 ( umask 077; printf '%s\n' "<your-api-key>" > feldera-api-key )
from feldera import FelderaClient, PipelineBuilder client = FelderaClient("https://<public-ip>", api_key=open("feldera-api-key").read().strip(),
# the key never appears in your code requests_verify="feldera.pem")
# trust this instance's certificate only print([p.name for p in client.pipelines()]) sql = """ CREATE TABLE readings (sensor VARCHAR NOT NULL, value DOUBLE NOT NULL) WITH ('materialized' = 'true'); CREATE MATERIALIZED VIEW sensor_avg AS SELECT sensor, AVG(value) AS avg_value FROM readings GROUP BY sensor; """ pipeline = PipelineBuilder(client, name="sensors", sql=sql).create_or_replace() pipeline.start() pipeline.input_json("readings", [{"sensor": "s1", "value": 20.5}, {"sensor": "s1", "value": 21.5}]) print(list(pipeline.query("SELECT * FROM sensor_avg"))) pipeline.stop(force=True)

requests_verify pins this instance's certificate for the SDK only. Never point it at a system trust store.

10

Connectors and the security note

Tables and views read and write external systems through connectors declared in SQL. This example declares a table fed from a Kafka topic and a view written back to another one:

CREATE TABLE clicks (user_id BIGINT, url VARCHAR, ts TIMESTAMP) WITH ( 'connectors' = '[{"transport": {"name": "kafka_input", "config": {"bootstrap.servers": "<broker>:9092", "topics": ["clicks"]}}, "format": {"name": "json", "config": {"update_format": "raw"}}}]'); CREATE MATERIALIZED VIEW clicks_per_user WITH ( 'connectors' = '[{"transport": {"name": "kafka_output", "config": {"bootstrap.servers": "<broker>:9092", "topic": "clicks-per-user"}}, "format": {"name": "json"}}]') AS SELECT user_id, COUNT(*) AS clicks FROM clicks GROUP BY user_id;

Other connectors cover Debezium and PostgreSQL CDC, Delta Lake and Iceberg, S3 and other object stores, files, HTTP URLs, Redis, NATS and more (see the connector reference). For a TLS Kafka broker with a self-signed certificate, pin that certificate in the connector's own ssl.ca.pem setting (the PEM text), never in a system trust store.

Security note. A pipeline is native code compiled from your SQL, Rust UDFs are arbitrary Rust, and connectors open connections to any address the instance can reach. Anyone with the console login or the API key can execute code on this instance and make it connect to systems in your network. Share both only with people you would give a shell to. The image confines that code (Step 13), but it is still code you run.

For Rust UDFs, declare CREATE FUNCTION <name>(...) RETURNS ...; in the SQL and put the implementation in the pipeline's UDF tab (udf_rust in the API, plus crate dependencies in udf_toml). See the Rust UDFs reference. Feldera fetches crates your UDFs add from crates.io at compile time, and the image's precompiled dependencies cover Feldera's own.

11

Where pipelines and data live

The pipeline-manager keeps pipeline definitions, program versions and runtime state in PostgreSQL. It keeps compiled programs, checkpoints and pipeline storage in its own state volume. Both are Docker volumes that start empty on every new instance (the state volume is prepared from the image's precompiled Rust dependency cache at first start). List them:

sudo docker volume ls --format '{{.Name}}' | sort
sudo docker inspect --format '{{range .Mounts}}{{.Name}} -> {{.Destination}}{{"\n"}}{{end}}' feldera-manager | sed '/^$/d'
sudo du -sh /var/lib/docker/volumes/feldera-pgdata /var/lib/docker/volumes/feldera-home | awk '{print $2, $1}' | sed 's#/var/lib/docker/volumes/##'
feldera-home feldera-pgdata feldera-pgsock feldera-pgsock -> /run/postgresql feldera-home -> /home/ubuntu/.feldera feldera-pgdata 48M feldera-home 6.0G

For large pipeline state, grow the root volume (EBS Modify volume, then sudo growpart /dev/nvme0n1 1 && sudo resize2fs /dev/nvme0n1p1) or move Docker's data to a dedicated EBS volume.

PostgreSQL has no network. To inspect it, use sudo docker exec -it feldera-postgres psql -U postgres -d feldera.

12

Security checks

Run feldera-selfcheck to re-verify the whole appliance with this instance's credentials. It checks:

  • The services and container hardening, the exact listening set, the network guards and telemetry off.
  • The HTTPS front door: login and API key accepted; anonymous, wrong-password and wrong-key requests refused; unknown host names dropped; cross-origin requests refused; compressed request bodies refused with 415; and ; and %3B path parameters refused with 400.
  • The loopback owner rule: only root and nginx reach the pipeline-manager on 127.0.0.1:8080, and other local users, including ubuntu, are refused on 127.0.0.1, ::1 and the container bridge.
  • The certificate.
  • A real pipeline with a Rust UDF that probes its own sandbox from inside. The instance metadata service and the host's own services time out while a loopback control connects, and no secret file is visible.

The check takes about two minutes and deletes its pipeline afterwards:

sudo feldera-selfcheck
SELFCHECK OK: Feldera 0.378.0 open-source (telemetry keys empty); services active; containers uid 1000 / 70, read-only, no capabilities, no Docker socket; listening 22, 80 (301), 443, 127.0.0.1:8080 only; login + API key accepted, anonymous / wrong password / wrong key 401, unknown Host dropped, cross-origin 403, compressed request bodies 415, path parameters 400; loopback API owner-fenced (root + nginx only; nobody/ubuntu refused on 127.0.0.1, ::1 and the bridge); pipeline sac_selfcheck_3418 compiled a Rust UDF and ran; metadata service TIMEOUT and host services TIMEOUT from inside the pipeline (loopback control OPEN), no secret file visible; certificate 6F:7D:44:91:92:C2:2F:86...
13

The container sandbox and the metadata block

The pipeline-manager and PostgreSQL containers run unprivileged, with no capabilities, no privilege escalation and a read-only root file system:

for c in feldera-manager feldera-postgres; do
sudo docker inspect --format "$c: user={{.Config.User}} readonly={{.HostConfig.ReadonlyRootfs}} privileged={{.HostConfig.Privileged}} network={{.HostConfig.NetworkMode}} capdrop={{.HostConfig.CapDrop}} secopt={{.HostConfig.SecurityOpt}}" $c; done
feldera-manager: user=1000:1000 readonly=true privileged=false network=feldera-net capdrop=[ALL] secopt=[no-new-privileges] feldera-postgres: user=70:70 readonly=true privileged=false network=none capdrop=[ALL] secopt=[no-new-privileges]

Containers may not reach the instance metadata service (169.254.169.254, and 168.63.129.16) or open connections to this instance's own services. The host services that do not need the metadata service are denied it by systemd IPAddressDeny rules. Only feldera-metadata-names, which reads the instance's names for the certificate, may reach it:

sudo iptables -S DOCKER-USER | grep DROP
sudo iptables -S SAC-FELDERA-IN for u in feldera feldera-postgres nginx docker feldera-metadata-names; do printf '%s: deny=%s allow=%s\n' $u "$(systemctl show -p IPAddressDeny --value $u.service | tr ' ' '\n' | sort | tr '\n' ' ' | sed 's/ $//')" "$(systemctl show -p IPAddressAllow --value $u.service)"; done
-A DOCKER-USER -d 168.63.129.16/32 -j DROP -A DOCKER-USER -d 169.254.169.254/32 -j DROP -N SAC-FELDERA-IN -A SAC-FELDERA-IN -i br-feldera -m conntrack --ctstate NEW -j DROP feldera: deny=168.63.129.16/32 169.254.169.254/32 ::ffff:168.63.129.16/128 ::ffff:169.254.169.254/128 fd00:ec2::254/128 allow= feldera-postgres: deny=168.63.129.16/32 169.254.169.254/32 ::ffff:168.63.129.16/128 ::ffff:169.254.169.254/128 fd00:ec2::254/128 allow= nginx: deny=168.63.129.16/32 169.254.169.254/32 ::ffff:168.63.129.16/128 ::ffff:169.254.169.254/128 fd00:ec2::254/128 allow= docker: deny=168.63.129.16/32 169.254.169.254/32 ::ffff:168.63.129.16/128 ::ffff:169.254.169.254/128 fd00:ec2::254/128 allow= feldera-metadata-names: deny=0.0.0.0/0 ::/0 allow=fd00:ec2::254/128 169.254.169.254/32

Pipelines can still reach your sources and sinks on the network and the internet. Only the metadata addresses and the host itself are blocked. Connectors that would normally use the instance's IAM role (S3, Delta Lake on S3, MSK IAM) need explicit credentials in their configuration.

14

Your own DNS name and a CA-issued certificate

To serve the console under your own DNS name, point the name at the instance (an Elastic IP is recommended) and set it. The command below writes the name into nginx and issues a new self-signed certificate that covers it:

sudo feldera-set-hostname feldera.example.com

To install a certificate from your CA (or Let's Encrypt), copy the full chain (server certificate first) and the unencrypted private key to the instance, then run:

sudo feldera-install-cert /path/to/fullchain.pem /path/to/privkey.pem feldera.example.com

The boot-time renewal never replaces a certificate installed this way. sudo feldera-set-hostname <name> goes back to a self-signed certificate. Both commands update the URL and fingerprint in /root/feldera-credentials.txt.

15

Change the console password or the API key

The password is stored as a bcrypt hash in /etc/nginx/feldera-htpasswd. To set a new one without putting it on a command line, let htpasswd prompt for it. Then update the curl config and the credentials file:

sudo htpasswd -B -C 10 /etc/nginx/feldera-htpasswd admin
sudo nano /root/feldera-curl.conf /root/feldera-credentials.txt
sudo systemctl reload nginx

To rotate the API key, write a new 64-character hex key (for example from openssl rand -hex 32) after apikey: in /etc/nginx/feldera-api-key.conf, /root/feldera-api.curl, /root/feldera-api-key and the credentials file with an editor. Then run sudo systemctl reload nginx.

16

Logs, versions and file locations

List the container versions and the listening sockets:

sudo docker exec feldera-manager /home/ubuntu/feldera/build/pipeline-manager --version 2>/dev/null | tail -1 sudo docker exec feldera-postgres postgres --version journalctl -u feldera -b --no-pager -o cat | grep -E '^Version:|starting service' | sed -E 's/^\[manager\] [0-9TZ:.-]+ +//' | awk 'n < 2 {print; n++}'
Pipeline manager CLI 0.378.0 postgres (PostgreSQL) 16.15 INFO actix_server::server: starting service: "actix-web-service-0.0.0.0:8085", workers: 2, listening on: 0.0.0.0:8085 INFO actix_server::server: starting service: "actix-web-service-0.0.0.0:8089", workers: 2, listening on: 0.0.0.0:8089

The two 0.0.0.0 listeners are inside the pipeline-manager container's own network namespace (its compiler and runner services). On the instance itself, only 127.0.0.1:8080 is published, and only root and nginx may connect to it (Step 3).

Path Contents

/etc/systemd/system/feldera.service, feldera-postgres.service the two containers (images pinned by digest)

/etc/feldera/manager.env, /etc/feldera/postgres/ pipeline-manager environment (telemetry off), PostgreSQL access rules (no secrets)

/etc/feldera/secrets/postgres-password this instance's PostgreSQL superuser password (only the PostgreSQL container reads it)

Docker volumes feldera-pgdata, feldera-home metadata database; compiled programs and pipeline storage

/etc/nginx/sites-available/sac-feldera the nginx front door

/root/feldera-credentials.txt, /root/feldera-curl.conf, /root/feldera-api.curl, /root/feldera-api-key login, API key, curl configs (root only)

/usr/share/doc/feldera Feldera's license and the image provenance

The pipeline-manager logs to the journal (journalctl -u feldera), PostgreSQL logs to journalctl -u feldera-postgres, and nginx logs to /var/log/nginx/. A pipeline's own log is on its Logs tab and at /v0/pipelines/<name>/logs.

17

Backup and maintenance

  • Backups. Feldera can checkpoint pipeline state and resume from it. Back up the PostgreSQL metadata with sudo docker exec feldera-postgres pg_dump -U postgres feldera > feldera-metadata.sql. Keep your SQL programs and UDFs in version control, because pipelines are re-created from them.
  • Updates. Ubuntu security updates install automatically (unattended-upgrades). The Feldera and PostgreSQL images are pinned by digest. To move to a newer Feldera release, launch the newer version from the listing and re-create your pipelines there.
  • Sizing. Compiling uses CPU and memory, and each running pipeline keeps its state in memory and storage. m5.large suits development and small pipelines. Use larger instances for large state or high input rates.
18

Security recommendations

  • Limit TCP 443 and 22 in the security group to your admin network. The console login and the API key grant code execution on the instance (Step 10), so share them only with people you would give a shell to.
  • Serve the console under your own DNS name with a certificate from a trusted CA (Step 14). Until then, check the self-signed certificate's fingerprint (Step 4) before you sign in, and pin it in clients (--cacert, --tls-cert, requests_verify).
  • Keep the pipeline-manager on the loopback address. Do not add other published ports to feldera.service or remove the network guards. The services refuse to start without them.
  • Give connectors credentials of their own, scoped to the data they need. The instance's IAM role is deliberately unreachable from pipelines.
  • HTTP Basic authentication has no sign-out. Close the browser to end a session on a shared computer.

Support and licensing

SecureAnyCloud provides 24/7 technical support for this image by email and live chat: sales@secureanycloud.com. The Feldera open-source edition is licensed under the MIT License (the license text is on the instance in /usr/share/doc/feldera/LICENSE; Feldera Enterprise features are not part of this image), PostgreSQL under the PostgreSQL License, nginx under the 2-clause BSD license and Docker Engine under the Apache License 2.0. Feldera is a trademark of Feldera, Inc. This image is provided by SecureAnyCloud and is not affiliated with or endorsed by Feldera, Inc.

Deploy on AWS

Launch Feldera 0.378.0 on Ubuntu 24.04 on AWS with 24/7 support from SecureAnyCloud.

View on Marketplace

Need Help?

Our support team is available 24/7.

sales@secureanycloud.com

Deploy on AWS