Prefect

Sign in to the AWS Management Console, open the EC2 service, and select Launch instance. Under Application and OS Images, choose AWS Marketplace AMIs and search for Prefect. Select the SecureAnyCloud listing and choose Select, then Continue on the subscription summary.

Pick an instance type of t3.medium or larger. Choose your EC2 key pair under Key pair (login). Under Network settings, select your VPC and subnet, and create or select a security group that opens port 22 from your management network and port 80 for the Prefect server and UI. Leave the root volume at the default size or larger.

Select Launch instance. First-boot initialisation takes a short time after the instance state becomes Running and the status checks pass. This is when the per-instance admin password is generated and the server runs its first database migrations.

The following block launches an instance from the SecureAnyCloud Prefect Marketplace AMI into an existing subnet and security group. Replace <ami-id> with the AMI ID shown on the Marketplace listing. Replace <key-name> with your EC2 key pair name and <subnet-id> with your subnet ID. Replace <security-group-id> with a security group that opens ports 22 and 80 as described above.

aws ec2 run-instances \
  --image-id <ami-id> \
  --instance-type t3.medium \
  --key-name <key-name> \
  --subnet-id <subnet-id> \
  --security-group-ids <security-group-id> \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=prefect}]'

When the instance reaches the Running state and its status checks pass, note its public IP address or DNS name. You can find it in the EC2 console or with aws ec2 describe-instances.

Connect over SSH using your key pair and the login user for your operating system variant. For Ubuntu 24.04, the login user is ubuntu:

ssh -i <key-name>.pem ubuntu@<public-ip>

The Prefect UI admin password is unique to your instance. It was generated on first boot. Read it as root:

sudo cat /root/prefect-credentials.txt

The file lists the UI URL, the admin user (admin) and the generated password, along with a ready-to-paste PREFECT_API_URL for connecting workers and clients. Keep this password in a safe place.

The Prefect server and UI are served on port 80 by nginx behind HTTP Basic authentication. In a browser, go to:

http://<instance-public-ip>/

Sign in as admin with the password from the credentials file when prompted. The dashboard gives you a live overview of your flow runs, task runs and work pools, updated in real time over the proxied WebSocket connection.

Open Runs in the left navigation to see every flow run and task run. Filter by state, flow, deployment, work pool or tag, and drill into the logs and timeline of an individual run.

Open Work Pools to create and manage the pools that workers poll for scheduled deployment runs. Pools let you route work to the infrastructure you choose.

Over SSH, confirm that the server and the nginx proxy are active, check the API health endpoint, and confirm the listening ports:

sudo systemctl is-active prefect nginx curl -s http://127.0.0.1:4200/api/health
sudo ss -tlnp | grep -E ':(80|4200) '

You should see both services reported as active and the health endpoint returning true. The server listens on 127.0.0.1:4200 (loopback only), and nginx listens on port 80.

Confirm the installed Prefect version with the bundled virtual environment:

/opt/prefect/venv/bin/prefect version

Prefect turns plain Python functions into orchestrated flows and tasks. Create a file etl.py on the instance and run it with the bundled virtual environment, so that the run is recorded against the local server. The following example defines a flow with three tasks and runs it three times:

cat > /tmp/etl.py <<'PYEOF' from prefect import flow, task @task def extract(): return [1, 2, 3, 4, 5] @task def transform(data): return [x * 10 for x in data] @task def load(data): return sum(data) @flow(name="example-etl-pipeline") def etl_pipeline(): raw = extract() clean = transform(raw) return load(clean) if __name__ == "__main__": print(etl_pipeline()) PYEOF PREFECT_API_URL=http://127.0.0.1:4200/api /opt/prefect/venv/bin/python /tmp/etl.py

Each run appears immediately in the Runs view of the UI with its state, logs and timeline. To add retries, caching and concurrency limits to a task, add the matching arguments to the @task decorator. No extra infrastructure is needed.

To submit work from another machine or to run a worker, point the Prefect client at this server through the authenticating proxy. The credentials file prints the exact URL. It has the form below, where <password> is the generated admin password and <instance-public-ip> is this instance's public address:

prefect config set PREFECT_API_URL=http://admin:<password>@<instance-public-ip>/api

A worker started against a work pool polls the server for scheduled deployment runs and executes them on its own infrastructure. Create a work pool from the Work Pools view in the UI, or with the bundled CLI on the instance. Then start a worker against it from wherever your code should run.

Your flows often need extra libraries. Install them into the bundled virtual environment, so that the server process and flows run with /opt/prefect/venv/bin/python can import them. The virtual environment lives at /opt/prefect/venv and is owned by the prefect service account. Confirm that you are using the bundled pip:

/opt/prefect/venv/bin/pip --version

Then install whatever your flows need, for example sudo /opt/prefect/venv/bin/pip install pandas requests. The virtual environment lives on the dedicated data volume, so your installed packages persist independently of the operating system disk.

The server data directory, PREFECT_HOME, lives on a dedicated EBS volume mounted at /var/lib/prefect. This keeps the embedded SQLite metadata database, the server logs and the storage off the operating system disk, and lets you resize or snapshot the volume independently. Confirm the mount:

df -h /var/lib/prefect

To grow the data store, expand the EBS volume in the AWS console. Then grow the filesystem on the instance with sudo resize2fs on the underlying device. The directory is owned by the prefect service account.

nginx serves the Prefect server and UI over plain HTTP on port 80. For production use, place them behind TLS.

Obtain a certificate for your domain. For example, use a managed certificate on an Application Load Balancer in front of the instance, or install Certbot on the instance for your-domain.

Then configure nginx to listen on 443 with your certificate and proxy to 127.0.0.1:4200, exactly as the bundled site does for port 80. Keep the HTTP Basic authentication and the WebSocket upgrade headers in place. Restrict the security group so that ports 80 and 443 are reachable only from the networks that need the server and UI.

Back up the server by snapshotting the /var/lib/prefect EBS volume. The snapshot captures the embedded SQLite metadata database, logs and storage.

Apply operating system security updates with sudo apt-get update && sudo apt-get upgrade, and reboot when a new kernel is installed. The Prefect server and nginx start automatically on boot. The server runs its database migrations automatically when a new Prefect version introduces a schema change.