DuckDB

• Open the AWS Marketplace and search for "DuckDB SecureAnyCloud".
• Select Continue to Subscribe, accept the terms, then select Continue to Configuration.
• Choose your Region and Software Version.
• Select Continue to Launch.
• Choose Launch through EC2 for full control over instance configuration.
• Select your instance type (t3.large is recommended).
• Configure storage with at least 20 GB of gp3. Increase it for large datasets.
• Configure your Security Group with one inbound rule: port 22, TCP, source your IP address, for SSH access.
• Select your EC2 key pair and launch the instance.

Wait until the instance is running and has passed both status checks (2/2). Then connect over SSH:

ssh -i your-key.pem ec2-user@<public-ip-address>

To switch to the root user, run:

sudo su -

The DuckDB executable is at /apps/duckdb/duckdb. Start the interactive CLI:

/apps/duckdb/duckdb

At the DuckDB prompt you can run SQL queries directly.

Use an in-memory database (the default):

SELECT 'Hello from DuckDB on AWS!' AS message;

Create a persistent database:

/apps/duckdb/duckdb /apps/duckdb/mydata.db

Sign in to the AWS Management Console, open the EC2 service and select Launch instance. Under Application and OS Images, choose AWS Marketplace AMIs and search for DuckDB. Select the SecureAnyCloud listing, choose Select, then choose Continue on the subscription summary.

Pick an instance type of m5.large or larger. Analytical queries hold working sets in memory, so DuckDB benefits from the memory-to-vCPU ratio of the m-class families. Choose your EC2 key pair under Key pair (login).

Under Network settings, select your VPC and subnet. Create or select a security group that allows inbound port 22 from your management network and inbound port 80 from the trusted networks that will reach JupyterLab. Do not open port 80 to the public internet: JupyterLab executes arbitrary Python, and the basic-authentication password is the only barrier. Leave the root volume at the default size or larger.

Select Launch instance. First-boot initialisation takes approximately one minute after the instance state becomes Running and the status checks pass.

Run the following command to launch an instance from the SecureAnyCloud DuckDB Marketplace AMI into an existing subnet and security group. Replace <ami-id> with the AMI ID shown on the Marketplace listing and <key-name> with your EC2 key pair name. Replace <subnet-id> with your subnet ID and <security-group-id> with a security group that opens ports 22 and 80 as described above.

aws ec2 run-instances \
  --image-id <ami-id> \
  --instance-type m5.large \
  --key-name <key-name> \
  --subnet-id <subnet-id> \
  --security-group-ids <security-group-id> \
  --block-device-mappings '[{"DeviceName":"/dev/sda1","Ebs":{"VolumeSize":30,"VolumeType":"gp3"}}]' \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=duckdb-01}]'

On success, the command prints a JSON document. Note the instance ID. Once the instance is running, retrieve its public address with aws ec2 describe-instances --instance-ids <instance-id> --query "Reservations[].Instances[].PublicIpAddress" --output text.

Connect over SSH with the key pair you selected and the public IP address from step 2. For the DuckDB 1.5 on Ubuntu 24.04 AMI, the SSH login user is ubuntu.

The first-boot service runs before the SSH daemon is ready, so the credentials file is always in place when you log in for the first time.

ssh <login-user>@<public-ip>
sudo cat /stage/scripts/duckdb-credentials.log

The plain text file contains the JupyterLab URL, the basic-authentication username (duckdb) and the basic-authentication password. Copy these values to a password manager or an encrypted vault. Do not commit them to source control.

Each command block in this guide that talks to JupyterLab through nginx starts by reading the password from the credentials file into a PASSWORD shell variable, so every block is self-contained:

PASSWORD="$(sudo awk -F= '/^DUCKDB_UI_PASSWORD=/ {print $2}' /stage/scripts/duckdb-credentials.log)" echo "duckdb basic-auth password length: ${#PASSWORD}"

The image includes an unauthenticated health endpoint at /healthz on port 80, served by nginx itself. JupyterLab is served at / behind basic authentication. Check both:

curl -s -o /dev/null -w "healthz: %{http_code}\n" http://127.0.0.1/healthz
curl -s -o /dev/null -w "no auth: %{http_code}\n" http://127.0.0.1/ PASSWORD="$(sudo awk -F= '/^DUCKDB_UI_PASSWORD=/ {print $2}' /stage/scripts/duckdb-credentials.log)"
curl -s -L -o /dev/null -w "with auth: %{http_code}\n" -u "duckdb:${PASSWORD}" http://127.0.0.1/

Expect this output:

healthz: 200 no auth: 401 with auth: 200

The 401 response without credentials confirms that nginx enforces basic authentication on the JupyterLab path. The 200 response with the correct credentials confirms that the request reaches JupyterLab on 127.0.0.1:8888 through the reverse proxy.

Browse to http://<public-ip>/. When your browser prompts for HTTP basic authentication, enter duckdb as the username and the password you retrieved in step 3.

JupyterLab opens on the launcher. The file browser on the left shows the bundled starter notebook 01-duckdb-quickstart.ipynb. Double-click the notebook to open it.

The notebook has one connection cell followed by three analytical query cells. Each query runs against the bundled parquet sample.

From the menu, select Run, then Run All Cells. The notebook runs the connection cell, prints v1.5.2, then runs three queries:

• Query 1 counts the rows in the parquet file. The bundled January 2024 yellow-taxi sample has 2,964,624 rows.
• Query 2 groups by passenger count and computes the average fare and trip distance.
• Query 3 groups by hour of day and computes the trip volume.

Query 2 shows that 1-passenger trips dominate the dataset, at around 2.2 million of the 2.96 million trips. They have an average fare close to $17.50 and an average trip distance close to 3.1 miles. Query 1 shows DuckDB scanning the entire parquet file in under one second on m5.large.

The DuckDB command line client is on the PATH for every Linux user. You can run analytical queries from an SSH session without opening JupyterLab.

duckdb --version

v1.5.2 (Variegata) 8a5851971f

Open a session against the persistent database that the notebook uses. The CLI shows a prompt where you can run SQL directly:

duckdb /opt/duckdb/samples/main.duckdb

Inside the CLI, run a sample query:

SELECT COUNT(*) FROM '/opt/duckdb/samples/nyc_yellow_taxi_2024_01.parquet';

Or pass a SQL statement non-interactively with the -c option:

duckdb -c "SELECT COUNT(*) AS trips FROM '/opt/duckdb/samples/nyc_yellow_taxi_2024_01.parquet';"

┌─────────┐ │ trips │ │ int64 │ ├─────────┤ │ 2964624 │ └─────────┘

DuckDB offers a rich SQL dialect, window functions and JSON support. The read_parquet, read_csv and read_json table functions query files directly, without loading them first.

The image contains these components:

• Operating system: Ubuntu 24.04 LTS, from the AWS Ubuntu base AMI, fully patched at build time
• DuckDB CLI 1.5.2: from the github.com/duckdb/duckdb releases (SHA256 pinned)
• DuckDB Python client 1.5.2: from PyPI, installed in /opt/duckdb-venv
• Python 3.12: Ubuntu noble distribution package
• JupyterLab 4.5: from PyPI, installed in /opt/duckdb-venv
• pandas 2.3: from PyPI
• PyArrow 24: from PyPI
• nginx 1.24: Ubuntu noble distribution package

The Python virtual environment at /opt/duckdb-venv is owned by root. The JupyterLab service runs as the duckdb system user, which has no shell. Notebooks, sample data and persistent databases live on the dedicated EBS data volume at /opt/duckdb.

DuckDB can query CSV, parquet and JSON files directly on Amazon S3 without copying them locally. The image pre-installs the httpfs extension into the persistent database, so first boot has no outbound dependencies.

To query a parquet file in your own S3 bucket, load the extension, then point a SELECT at an s3:// URL:

LOAD httpfs; CREATE SECRET ( TYPE S3, PROVIDER credential_chain ); SELECT COUNT(*) FROM 's3://my-bucket/path/to/file.parquet';

If you launch the EC2 instance with an IAM instance profile that grants s3:GetObject on the target prefix, the credential_chain provider picks up temporary credentials from the EC2 metadata service. For long-lived static credentials, use the KEY and SECRET parameters of CREATE SECRET.

JupyterLab serves notebooks from /opt/duckdb/notebooks, which is on the dedicated EBS data volume. Notebooks you save from the UI land in that directory and survive instance restarts. A samples/ subdirectory that you create for parquet files or CSV exports also lives on the data volume.

The starter notebook exists in two places: the original at /opt/duckdb/samples/01-duckdb-quickstart.ipynb and an editable copy at /opt/duckdb/notebooks/01-duckdb-quickstart.ipynb. Edits to the copy do not change the original.

DuckDB writes persistent databases to disk at every CHECKPOINT and on a clean shutdown. The JupyterLab systemd unit receives a clean SIGTERM on stop, so a normal systemctl stop jupyterlab.service or an instance reboot does not corrupt open databases.

Check service status:

sudo systemctl status jupyterlab.service nginx.service --no-pager

Restart the JupyterLab service, for example after you change a Python dependency:

sudo systemctl restart jupyterlab.service

The DuckDB data tier lives on the EBS volume mounted at /opt/duckdb. The simplest backup is an EBS snapshot of that volume. Create it from your management environment with the AWS CLI:

aws ec2 create-snapshot \
  --volume-id <volume-id> \
  --description "DuckDB data backup $(date -u +%Y%m%d)" \
  --tag-specifications 'ResourceType=snapshot,Tags=[{Key=Product,Value=duckdb-aws},{Key=Type,Value=data}]'

Snapshots are incremental, so later backups store only the changed blocks. To automate rotation, create a snapshot lifecycle policy under Data Lifecycle Manager in the EC2 console. For per-database backups, use the DuckDB EXPORT DATABASE statement to write a parquet directory that you can copy to S3.

The image serves JupyterLab over plain HTTP on port 80, because the right place to terminate TLS depends on your environment. The recommended pattern is an AWS Application Load Balancer in front of the instance. The load balancer terminates TLS with an ACM certificate, and its target group forwards plain HTTP to port 80 on the instance.

Alternatively, run a reverse proxy on the same host. The image does not include certbot, but you can install it and run it against a domain you own. Replace <your-domain> with your real domain:

sudo apt-get update
sudo apt-get install -y certbot python3-certbot-nginx
sudo certbot --nginx -d <your-domain> --redirect --non-interactive --agree-tos -m <your-email>

When certbot rewrites the nginx configuration, it preserves the basic-authentication block. JupyterLab stays password-protected over HTTPS.

The image generates the per-instance password on first boot. To rotate it later, regenerate the htpasswd entry:

NEW_PASSWORD="$(openssl rand -hex 16)"
sudo htpasswd -bc /etc/nginx/.duckdb-htpasswd duckdb "${NEW_PASSWORD}"
sudo chown root:www-data /etc/nginx/.duckdb-htpasswd
sudo chmod 0640 /etc/nginx/.duckdb-htpasswd
sudo systemctl reload nginx echo "New JupyterLab password: ${NEW_PASSWORD}"

To keep the stored record in sync, also update /stage/scripts/duckdb-credentials.log.

The image ships with Ubuntu unattended-upgrades enabled, so security updates install automatically. To apply outstanding updates immediately, run:

sudo apt-get update
sudo apt-get -y upgrade

A kernel update may require a reboot. Schedule reboots through your normal change-control process.