Ollama

Search the Azure Marketplace for Ollama, select the SecureAnyCloud publisher, and select Create.

Configure the network security group (NSG) with these rules, allowed from your client networks: TCP 22 (admin), TCP 443 (model API over HTTPS) and TCP 80 (HTTP-to-HTTPS redirect and health probe).

RG="ollama-prod"; LOCATION="eastus"; VM_NAME="ollama-01" IMAGE_URN="sac1647283583153:ollama-ubuntu-24-04:default:latest" SSH_KEY="$(cat ~/.ssh/id_rsa.pub)" az group create --name "$RG" --location "$LOCATION" az network vnet create -g "$RG" --name ollama-vnet --address-prefix 10.100.0.0/16 --subnet-name ollama-subnet --subnet-prefix 10.100.1.0/24 az network nsg create -g "$RG" --name ollama-nsg az network nsg rule create -g "$RG" --nsg-name ollama-nsg --name allow-ssh --priority 100 \
  --source-address-prefixes "<your-mgmt-cidr>" --destination-port-ranges 22 --access Allow --protocol Tcp az network nsg rule create -g "$RG" --nsg-name ollama-nsg --name allow-https --priority 110 \
  --destination-port-ranges 443 --access Allow --protocol Tcp az network nsg rule create -g "$RG" --nsg-name ollama-nsg --name allow-http --priority 120 \
  --destination-port-ranges 80 --access Allow --protocol Tcp
# Accept the Marketplace terms once per subscription (required before the first CLI deploy)
az vm image terms accept --urn sac1647283583153:ollama-ubuntu-24-04:default:latest
az vm create -g "$RG" --name "$VM_NAME" --image "$IMAGE_URN" \
  --size Standard_NC4as_T4_v3 --storage-sku StandardSSD_LRS \
  --admin-username azureuser --ssh-key-values "$SSH_KEY" \
  --vnet-name ollama-vnet --subnet ollama-subnet --nsg ollama-nsg --public-ip-sku Standard

First boot takes a few minutes. On its first boot, the VM finishes its per-VM setup in the background by generating its own credentials and starting the services. If you connect as soon as the VM shows as running, wait for that setup to complete before you continue:

bash sudo cloud-init status --wait

The command prints status: done when the VM is ready. The credentials and services described below exist only after setup completes.

The per-VM admin credential is generated on first boot and stored in a root-only file. Connect over SSH and read it:

ssh azureuser@<public-ip>
sudo cat /root/ollama-credentials.txt

The file contains the API URL, the username (admin), the per-VM password and quick-start commands. Store the password somewhere safe. It is unique to this VM.

Confirm that both services are active and that Ollama is bound to loopback only. In the listener output, 127.0.0.1:11434 (Ollama) is loopback only, while nginx listens publicly on :443 and :80:

systemctl status ollama nginx --no-pager ss -tln | grep -E ':11434|:443|:80 '

nginx serves GET /api/version without authentication, which suits load balancer and Azure health probes. The endpoint returns the Ollama version as JSON:

curl -s --cacert /etc/nginx/tls/ollama.crt https://127.0.0.1/api/version; echo
curl -k -o /dev/null -w 'healthz: HTTP %{http_code}\n' https://127.0.0.1/healthz

Use the per-VM credential to list models through the API. Then confirm that an anonymous request, with no credential, is rejected with HTTP 401. The self-signed certificate lives at /etc/nginx/tls/ollama.crt, so pass it to curl with --cacert:

PASS=$(sudo grep '^OLLAMA_PASSWORD=' /root/ollama-credentials.txt | cut -d= -f2-)
echo "# Authenticated model list:"
sudo curl -s -u "admin:$PASS" --cacert /etc/nginx/tls/ollama.crt https://127.0.0.1/api/tags | jq '.models[].name' echo "# Anonymous request (no credential) must be rejected:" curl -s -o /dev/null -w 'anonymous /api/tags -> HTTP %{http_code}\n' --cacert /etc/nginx/tls/ollama.crt https://127.0.0.1/api/tags

The authenticated request returns the model list as JSON. The anonymous request returns HTTP 401, because this API allows no anonymous access.

Send a prompt to the pre-pulled model over the authenticated API. The /api/generate endpoint returns a completion. The OpenAI-compatible /v1/chat/completions endpoint accepts the same requests that any OpenAI SDK sends.

PASS=$(sudo grep '^OLLAMA_PASSWORD=' /root/ollama-credentials.txt | cut -d= -f2-) sudo curl -s -u "admin:$PASS" --cacert /etc/nginx/tls/ollama.crt https://127.0.0.1/api/generate \
  -d '{"model":"llama3.2:1b","prompt":"In one sentence, what is Ollama?","stream":false}' | jq -r '.response'

The model returns generated text. On a CPU-only VM this takes a few seconds. On a GPU VM the response is near instant.

Issue the certificate for your public address

On first boot, the image writes the address that Azure reports into the TLS certificate, the credentials file and the login banner. With a Standard SKU public IP (the Azure default, and the --public-ip-sku Standard in Step 2), the current image records the VM's private address (for example 10.100.1.4). Clients that trust ollama.crt then refuse the certificate at the public IP (curl: (60) SSL: no alternative certificate subject name matches).

The current image also passes the address you call on to Ollama. Ollama answers HTTP 403 to any request not addressed to the VM itself, so every call to https://<public-ip>/ or a DNS name is refused, even with the right password. Check the address:

sudo grep '^OLLAMA_URL=' /root/ollama-credentials.txt

If the check shows a private 10. address, issue a new certificate for the public IP on the VM, and stop nginx from passing the public address to Ollama. Run the sed on the sac-ollama site line whatever the check shows. Azure's instance metadata service reports the public IP to the VM itself, and nothing leaves the VM:

PUBLIC_IP=$(curl -s -H Metadata:true 'http://169.254.169.254/metadata/loadbalancer?api-version=2020-10-01' | grep -o '"frontendIpAddress":"[^"]*"' | head -1 | cut -d'"' -f4)
PRIV=$(hostname -I | awk '{print $1}'); echo "public $PUBLIC_IP, private $PRIV"
sudo openssl req -x509 -nodes -days 3650 -newkey rsa:2048 \
  -keyout /etc/nginx/tls/ollama.key -out /etc/nginx/tls/ollama.crt \
  -subj "/C=GB/O=sac/CN=$PUBLIC_IP" \
  -addext "subjectAltName=IP:127.0.0.1,DNS:localhost,DNS:$(hostname -s),IP:$PRIV,IP:$PUBLIC_IP" sudo chmod 0600 /etc/nginx/tls/ollama.key
sudo sed -i "s/\b$PRIV\b/$PUBLIC_IP/g" /root/ollama-credentials.txt /etc/update-motd.d/90-ollama
sudo sed -i -E 's/(proxy_set_header +Host +)\$http_host;/\1localhost;/' /etc/nginx/sites-available/sac-ollama
sudo systemctl restart nginx

For a DNS name, set PUBLIC_IP to the name and write DNS:$PUBLIC_IP in place of IP:$PUBLIC_IP. Your password does not change. Copy the new /etc/nginx/tls/ollama.crt to your clients as ollama.crt.

Ollama can serve any model from its library. From a client, pull a larger model into your VM over the authenticated API. Replace <public-ip> and <password> with the values from /root/ollama-credentials.txt:

curl -u admin:<password> --cacert ollama.crt https://<public-ip>/api/pull -d '{"name":"llama3.2:3b"}'

Inspect the models that are already in the local store on the VM:

sudo -u ollama OLLAMA_HOST=127.0.0.1:11434 ollama show llama3.2:1b
sudo -u ollama OLLAMA_HOST=127.0.0.1:11434 ollama list

The image runs on CPU by default and auto-offloads to the GPU once the NVIDIA driver is present. On a GPU VM, follow these steps.

Add the NVIDIA GPU Driver Extension to the VM. In the Portal, open the VM, go to Extensions and add NVIDIA GPU Driver Extension. Alternatively, run az vm extension set --name NvidiaGpuDriverLinux --publisher Microsoft.HpcCompute .... Reboot the VM when the extension completes.

Confirm the driver: nvidia-smi lists the GPU.

Restart Ollama with sudo systemctl restart ollama, then run a completion. Confirm the offload with sudo -u ollama OLLAMA_HOST=127.0.0.1:11434 ollama ps. PROCESSOR shows the GPU, and size_vram is greater than zero.

No image changes are needed. The same Ollama binary detects and uses the GPU.

Point any OpenAI SDK at https://<public-ip>/v1 with basic auth. Example with curl:

PASS=$(sudo grep '^OLLAMA_PASSWORD=' /root/ollama-credentials.txt | cut -d= -f2-) sudo curl -s -u "admin:$PASS" --cacert /etc/nginx/tls/ollama.crt https://127.0.0.1/v1/chat/completions \
  -d '{"model":"llama3.2:1b","messages":[{"role":"user","content":"Say hello in French"}]}' | jq -r '.choices[0].message.content'

From Python (LangChain, LlamaIndex or the OpenAI SDK), set base_url="https://<public-ip>/v1" and api_key="ollama". Also pass the admin and password Basic authentication pair, together with your CA-trusted or replaced certificate.

The self-signed certificate is a convenience for getting started. For production, front Ollama with your own DNS name and a CA-signed certificate.

Point a DNS record (for example ollama.example.com) at the VM public IP.

Replace /etc/nginx/tls/ollama.crt and /etc/nginx/tls/ollama.key with your CA-signed certificate and key, for example from Let's Encrypt or your internal CA. Then run sudo nginx -t && sudo systemctl reload nginx.

Clients can then call https://ollama.example.com/ with no certificate trust step.

• Runtime: Ollama (pinned at build, with a best-effort update to the latest release on first boot)
• Server binding: 127.0.0.1:11434 (loopback only, no authentication at this layer)
• Front proxy: nginx (TLS on :443, htpasswd Basic auth, :80 -> 301 https)
• Model store: /var/lib/ollama/models on the OS disk (llama3.2:1b pre-pulled)
• Credential: per-VM admin password, generated on first boot, in /root/ollama-credentials.txt (0600)
• TLS certificate: self-signed, regenerated per VM on first boot (SAN = public IP + hostname), in /etc/nginx/tls/
• API: REST (/api/*) and OpenAI-compatible (/v1/*)
• Operating system: Ubuntu 24.04 LTS (patched at build time)
• License: MIT (Ollama)

sudo systemctl status ollama --no-pager sudo journalctl -u ollama --no-pager -n 20
sudo systemctl reload nginx

To restart the runtime after a configuration change, run sudo systemctl restart ollama.

Model weights live under /var/lib/ollama/models and persist across service restarts and VM reboots. For large model libraries, attach an Azure data disk and mount it at /var/lib/ollama/models. To do so, stop ollama, move the data, update the mount and restart the service.

Restrict the NSG. Allow TCP 443 (and 22 for admin) only from the client networks that need the API. Do not expose the model API to the public internet unless you intend to.

Rotate the credential. To change the admin password, run sudo htpasswd -B /etc/nginx/.ollama.htpasswd admin, then sudo systemctl reload nginx. Update /root/ollama-credentials.txt to match.

Add more users. Create additional accounts with sudo htpasswd -B /etc/nginx/.ollama.htpasswd <username>, then reload nginx.

Use a trusted certificate. For production, install one (Step 11) so that clients validate the endpoint without a manual trust step.

Keep the OS patched. Unattended security upgrades remain enabled on the running VM.

Ollama is distributed under the MIT License. This SecureAnyCloud image bundles the unmodified upstream Ollama release. SecureAnyCloud provides the packaging, hardening, per-VM credential and TLS automation, and 24/7 support with a guaranteed 24-hour response SLA.

Deploy on Azure

Find Ollama on Ubuntu 24.04 on the Azure Marketplace, published by SecureAnyCloud. Deploy it from the Portal or the Azure CLI as shown above.

Need Help?

Email sales@secureanycloud.com for deployment help, configuration questions or licensing enquiries.

Deploy on Azure

Launch Ollama on Ubuntu 24.04 LTS on Azure with 24/7 support from SecureAnyCloud.

View this image

Need Help?

Our support team is available 24/7.

sales@secureanycloud.com