Deployment guide

Mage AI

Overview

Mage AI is an open-source data pipeline tool for transforming and integrating data. It is a modern, developer-friendly alternative to Apache Airflow. Instead of writing DAGs by hand, you build pipelines from composable blocks (data loaders, transformers and exporters) in a visual, notebook-style editor with live output previews. You then schedule them with built-in triggers.

The SecureAnyCloud image installs Mage AI 0.9.79 into a Python virtualenv at /opt/mage/venv and runs it in standalone mode as a dedicated mage system user. The web UI is bound to loopback behind an nginx reverse proxy on TCP 80 with HTTP Basic auth. The Mage project and all pipeline data persist on a dedicated Azure data disk, and every VM generates its own unique web password on first boot.

What you get:

  • Mage AI 0.9.79 in a Python virtualenv at /opt/mage/venv, run by the mage system user
  • The visual pipeline editor and scheduler UI, fronted by nginx on :80
  • nginx HTTP Basic auth (Mage's web UI has no built-in authentication by default) with a per-VM password in a root-only file
  • WebSocket pass-through in nginx, so the live editor and pipeline run output stream correctly
  • A dedicated Azure data disk at /var/lib/mage that holds the Mage project and all pipeline metadata, separate from the OS disk and re-provisioned with every VM
  • mage.service and nginx.service as systemd units, enabled and active
  • 24/7 SecureAnyCloud support

Prerequisites

  • An active Azure subscription
  • An SSH key pair
  • A VNet and subnet in the target region

Standard_B2ms (2 vCPU, 8 GiB RAM) is a good starting point. Scale up for heavier pipelines and concurrent runs. Allow 22/tcp from your management network and 80/tcp for the web UI in the NSG inbound rules. Front the UI with TLS for public exposure (see Enabling HTTPS).

Step 1 - Deploy from the Azure Marketplace

Sign in to the Azure Portal, choose Create a resource, search the Marketplace for Mage AI by SecureAnyCloud and select Create.

On Basics, pick your subscription, resource group, region and size. Under Administrator account, choose SSH public key and paste your key. Under Inbound port rules, allow SSH (22) and HTTP (80).

Review the dedicated data disk on the Disks tab, then choose Review + create and Create.

Step 2 - Deploy from the Azure CLI

az vm create \
  --resource-group <your-rg> \
  --name mage-ai \
  --image <marketplace-image-urn> \
  --size Standard_B2ms \
  --admin-username azureuser \
  --ssh-key-values ~/.ssh/id_ed25519.pub \
  --vnet-name <your-vnet> --subnet <your-subnet> \
  --public-ip-sku Standard
az vm open-port --resource-group <your-rg> --name mage-ai --port 80 --priority 1010

Step 3 - Connect to your VM

ssh azureuser@<vm-public-ip>

Step 4 - Confirm the services are running

systemctl is-active mage.service nginx.service

Both services report active. Mage starts its web server and scheduler in standalone mode and is ready to build pipelines.

Step 5 - Retrieve your web password

Your VM generates a unique admin password on first boot and writes it to a root-only file:

sudo cat /root/mage-ai-credentials.txt

The file contains mage.admin.user (admin), mage.admin.pass and the URL for the web UI. Store the password somewhere safe.

Step 6 - Check the health endpoint

nginx serves an unauthenticated health endpoint for load balancers and probes:

curl -s http://localhost/health

It returns ok.

Step 7 - Open the visual pipeline editor

Browse to http://<vm-public-ip>/ and sign in with the browser's HTTP Basic auth prompt as admin with the password from Step 5. Mage opens on the Pipelines dashboard. It lists every pipeline in the project and lets you create a new one with + New.

Opening a pipeline shows the notebook-style editor. Each block (loader, transformer or exporter) is a cell with its own code and live output. The block tree on the right shows how the blocks are wired together:

A data loader or transformer block holds Python (or SQL or R) code with an @data_loader or @transformer decorator. Run the block to preview its output inline before you wire it into the pipeline:

The Triggers and Runs views schedule the pipeline and list every run with its status, so you can monitor and re-run executions:

Step 8 - Confirm the API behind the auth wall

The Mage web server answers behind the same Basic auth. From the VM, a request without credentials is rejected, and a request with the per-VM password is accepted:

curl -s -o /dev/null -w 'no-auth: %{http_code}\n' http://localhost/
curl -s -o /dev/null -w 'with-auth: %{http_code}\n' -u admin:<MAGE_ADMIN_PASSWORD> http://localhost/

The first prints 401 and the second prints 200. This proves that nginx enforces authentication in front of Mage's auth-less web server.

Step 9 - Create your first pipeline

  • In the UI, choose + New > Standard (batch) to scaffold a pipeline.
  • Choose + Data loader > Python > API/Base template to add a loader block.
  • Edit the block's @data_loader function and choose the block's run button to preview the output.
  • Add a Transformer block downstream and wire it in.

Mage saves the pipeline definition to the project on the data disk automatically.

Step 10 - Inspect the project on the data disk

The Mage project (pipelines, blocks and metadata.yaml) lives under the data directory. List it as the mage user:

sudo ls /var/lib/mage/default_repo

You see metadata.yaml, pipelines/, data_loaders/, transformers/ and data_exporters/. This is the project Mage reads and writes as you build.

Step 11 - Confirm data lives on the dedicated disk

The Mage project and all pipeline data are stored on a dedicated Azure data disk. They survive OS changes, and you can resize the disk independently:

findmnt /var/lib/mage

The mount is backed by a separate Azure data disk that is captured into the image and re-provisioned on every VM.

Enabling HTTPS

The nginx reverse proxy terminates plain HTTP on port 80. For public exposure, put a certificate in front of it.

The simplest approach is to add a DNS name for the VM and use the companion SecureAnyCloud nginx-ssl-certbot image as a TLS reverse proxy. Alternatively, install certbot and extend the existing nginx site with a listen 443 ssl; server block and your certificate paths.

Keep Mage itself bound to loopback, so the only public surface is the authenticated, TLS-terminated proxy. The existing nginx site already passes the WebSocket upgrade headers Mage needs, so the live editor keeps working over TLS.

Maintenance

  • Configuration: the listener host, port and project path are set in /etc/default/mage. Edit it and run sudo systemctl restart mage to apply changes.
  • Backups: snapshot the /var/lib/mage data disk, or copy the default_repo project tree to Azure Blob Storage.
  • Upgrades: activate the virtualenv (source /opt/mage/venv/bin/activate), run pip install --upgrade mage-ai, then run sudo systemctl restart mage.
  • Security patches: unattended-upgrades stays enabled, so the OS keeps receiving security updates automatically.

Support

SecureAnyCloud provides 24/7 expert support for this image. Contact sales@secureanycloud.com.

Mage is a trademark of Mage Technologies, Inc. This image is produced by SecureAnyCloud and is not affiliated with or endorsed by the Mage project. Mage AI is distributed under the Apache License 2.0.

Deploy on Azure

Launch Mage AI on Ubuntu 24.04 LTS on Azure with 24/7 support from SecureAnyCloud.

View on Marketplace

Need Help?

Our support team is available 24/7.

sales@secureanycloud.com

Deploy on Azure