lakeFS

Search the Azure Marketplace for lakeFS, select the SecureAnyCloud publisher, and click Create.

Configure the Network Security Group to allow TCP 80 (web UI and API) from your team networks and TCP 22 for administration. Add TCP 443 if you plan to terminate TLS with your own certificate.

RG="lakefs-prod"; LOCATION="eastus"; VM_NAME="lakefs1" IMAGE_URN="sac1647283583153:lakefs-ubuntu-24-04:default:latest" SSH_KEY="$(cat ~/.ssh/id_rsa.pub)" az group create --name "$RG" --location "$LOCATION"
# Accept the Marketplace terms once per subscription (required before the first CLI deploy)
az vm image terms accept --urn sac1647283583153:lakefs-ubuntu-24-04:default:latest
az vm create \
  --resource-group "$RG" --name "$VM_NAME" \
  --image "$IMAGE_URN" \
  --size Standard_B2s \
  --admin-username azureuser \
  --ssh-key-values "$SSH_KEY" \
  --public-ip-sku Standard
# Open the web UI, API and admin ports on the VM NSG:
az vm open-port --resource-group "$RG" --name "$VM_NAME" --port 80 --priority 100
az vm open-port --resource-group "$RG" --name "$VM_NAME" --port 22 --priority 110

On first boot, a one-time service sets up lakeFS for your VM. It generates the per-instance encryption key and runs lakefs setup to create the per-instance admin access key and secret. It then starts the lakeFS server and nginx, verifies that the admin authenticates while anonymous access is rejected, and writes the credentials to /root/lakefs-credentials.txt.

This usually completes within a minute. Retrieve the credentials over SSH:

ssh azureuser@<vm-public-ip>
sudo cat /root/lakefs-credentials.txt

systemctl is-active lakefs nginx /opt/lakefs/lakefs --version ss -tlnp | grep -E ':8000|:80'

The lakeFS server binds to loopback 127.0.0.1:8000 only, and nginx fronts it on port 80. This keeps the API server off the public interface, so nginx is the single entry point.

Browse to http://<vm-public-ip>/ and sign in with the Access Key ID and Secret Access Key from /root/lakefs-credentials.txt. These credentials are unique to your VM.

After you sign in, the repositories list opens. From there you can create repositories and open existing ones. Each repository has a default branch and a storage namespace on the local block store.

The server configuration lives at /etc/lakefs/config.yaml. It selects the local block store for the versioned data and the embedded local key-value store for the metadata, both under /var/lib/lakefs on the dedicated data disk.

The auth.encrypt.secret_key shown in the file is not the real key. The real per-instance key is injected from the data disk at first boot through an environment variable. It is never written into this file or shipped in the image.

sudo grep -vE '^\s*#' /etc/lakefs/config.yaml | sed 's/secret_key:.*/secret_key: <redacted>/' findmnt /var/lib/lakefs

Configure lakectl once with your credentials and the server endpoint. Then work with your data as you would with Git: create a repository, upload objects and commit them. Branch off to experiment in isolation, and merge back when ready.

# Configure lakectl (paste the access key, secret and http://<vm-public-ip>/ endpoint): lakectl config
# Create a repository on the local block store: lakectl repo create lakefs://sample-data-repo local://sample-data-repo --default-branch main
# Commit an object on main:
echo "id,name,amount" > transactions.csv lakectl fs upload -s transactions.csv lakefs://sample-data-repo/main/datasets/transactions.csv lakectl commit lakefs://sample-data-repo/main -m "Add transactions dataset"
# Branch off to experiment without touching main, then merge back: lakectl branch create lakefs://sample-data-repo/experiment --source lakefs://sample-data-repo/main lakectl merge lakefs://sample-data-repo/experiment lakefs://sample-data-repo/main

The same operations are available in the web UI. Open a repository to browse its objects on any branch, and open the Commits tab to see the full history.

The admin credentials in /root/lakefs-credentials.txt are generated per VM on first boot, and only root can read the file. Every API and UI request must authenticate, and an unauthenticated call is rejected with HTTP 401. The health endpoints answer without authentication so that a load balancer can probe them.

# Anonymous API access is rejected:
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8000/api/v1/repositories
# 401
# Health endpoints answer 200:
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1/healthz
# 200
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8000/_health
# 200

Terminate TLS. Put your own certificate on nginx and serve the UI and API over HTTPS on port 443. lakeFS works over plain HTTP on a trusted network, but always terminate TLS before you expose it to the internet.

Point the block store at cloud storage. For real data lakes, set blockstore.type to azure (Azure Blob Storage) or s3 in /etc/lakefs/config.yaml. lakeFS then versions the data already in your object storage instead of the local disk.

Back the metadata with PostgreSQL. For high availability and larger teams, switch database.type to postgres and point it at a managed PostgreSQL instance. Several lakeFS nodes can then share state.

Rotate and manage credentials. Create additional users and access keys from the Administration area of the web UI, and keep the root credentials file secured.

Keep the OS patched. Unattended security upgrades remain enabled on the image.