Grafana CE with Prometheus

Open Marketplace in the Azure Portal and search for Grafana CE with Prometheus. Select the SecureAnyCloud publisher entry and click Create.

On the Networking tab, attach a network security group that allows inbound TCP 22 from your management IP range and TCP 3000 from your trusted client networks.

Do not expose port 3000 to the public internet without TLS. Grafana listens on plain HTTP by design, so terminate TLS at a reverse proxy. Ports 9090 (Prometheus) and 9100 (node_exporter) are loopback-only and never need an NSG rule.

Click Review + create and wait for validation. Then click Create. Deployment takes around two minutes.

RG="monitoring-prod" LOCATION="eastus" VM_NAME="monitor-01" ADMIN_USER="azureuser" GALLERY_IMAGE_ID="/subscriptions/<sub-id>/resourceGroups/azure-sac/providers/Microsoft.Compute/galleries/sacGallery/images/grafanacewithprometheus-ubuntu-24-04/versions/<version>" SSH_KEY="$(cat ~/.ssh/id_rsa.pub)" az group create --name "$RG" --location "$LOCATION" az network vnet create \
  --resource-group "$RG" \
  --name monitor-vnet --address-prefix 10.94.0.0/16 \
  --subnet-name monitor-subnet --subnet-prefix 10.94.1.0/24 az network nsg create --resource-group "$RG" --name monitor-nsg az network nsg rule create \
  --resource-group "$RG" --nsg-name monitor-nsg \
  --name allow-ssh --priority 100 \
  --source-address-prefixes "<your-mgmt-cidr>" \
  --destination-port-ranges 22 --access Allow --protocol Tcp az network nsg rule create \
  --resource-group "$RG" --nsg-name monitor-nsg \
  --name allow-grafana --priority 110 \
  --source-address-prefixes "<your-mgmt-cidr>" \
  --destination-port-ranges 3000 --access Allow --protocol Tcp az vm create \
  --resource-group "$RG" --name "$VM_NAME" \
  --image "$GALLERY_IMAGE_ID" \
  --size Standard_B2s --storage-sku StandardSSD_LRS \
  --admin-username "$ADMIN_USER" --ssh-key-values "$SSH_KEY" \
  --vnet-name monitor-vnet --subnet monitor-subnet --nsg monitor-nsg \
  --public-ip-sku Standard

Connect to the VM:

ssh azureuser@<vm-ip>

On first boot, node_exporter and Prometheus start automatically. Then grafana-firstboot.service rotates the Grafana admin credentials and starts Grafana.

Confirm all three services are active:

sudo systemctl is-active grafana-server prometheus node_exporter

Expect three lines, each reading active. Next, confirm the listeners. Grafana is on 3000 (all interfaces), and Prometheus and node_exporter are on loopback only:

sudo ss -tlnp | grep -E ':(3000|9090|9100) '

Check the first-boot sentinel and Grafana health:

sudo test -f /var/lib/sac/data/grafana-firstboot.done && echo FIRSTBOOT_DONE curl -s http://localhost:3000/api/health

You should see FIRSTBOOT_DONE and {"database": "ok", "version": "12.4.x", "commit": "..."}.

Prometheus has no authentication and binds to loopback only, so query it directly from the VM. Both the node (node_exporter) and prometheus (self) targets should report health=up:

curl -s http://127.0.0.1:9090/api/v1/targets | jq -r '.data.activeTargets[] | "job=\(.labels.job) instance=\(.labels.instance) health=\(.health)"'

Confirm that node_exporter is exporting host metrics. The count is non-zero:

curl -s http://127.0.0.1:9100/metrics | grep -c '^node_'

sudo cat /stage/scripts/grafana-credentials.log

You will see:

ADMIN_USER=admin ADMIN_PASSWORD=<ADMIN_PASSWORD> GRAFANA_PORT=3000 GRAFANA_URL=http://<vm-ip>:3000

Store the password in your secret store. The file is mode 0600 and readable by root only.

Prove that metrics flow all the way through. This query runs up through Grafana's provisioned Prometheus datasource proxy and returns a live value for every target. Both series report 1:

curl -s -u "admin:<ADMIN_PASSWORD>" "http://localhost:3000/api/datasources/proxy/uid/sac-prometheus/api/v1/query?query=up" | jq '.data.result[]? | {job: .metric.job, instance: .metric.instance, value: .value[1]}'

Open http://<vm-ip>:3000 in your browser. The Grafana sign-in page appears:

Sign in with admin and the password from Step 6. The default admin/admin login has been rotated away and is rejected.

Grafana ships with a Node Metrics (SecureAnyCloud) dashboard already provisioned under the SecureAnyCloud folder. Open it from Dashboards and live host metrics render immediately: CPU busy, memory used, root filesystem used, system load, CPU usage by mode and network traffic.

The dashboard is defined at /var/lib/grafana/dashboards/node-metrics.json and reloads automatically. Edit it in the UI, or drop additional dashboard JSON files into that directory.

Open Connections > Data sources. The Prometheus datasource is already configured as the default and points at the loopback Prometheus on http://127.0.0.1:9090.

You do not need to add or configure it. It is provisioned from /etc/grafana/provisioning/datasources/sac-prometheus.yaml.

Use Explore to run ad-hoc PromQL against the Prometheus datasource. For example, node_load1 graphs the 1-minute load average of the host.

Try these starter queries:

• 100 - (avg(rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) for CPU busy %
• node_memory_MemAvailable_bytes for available memory
• rate(node_network_receive_bytes_total[5m]) for network throughput

To monitor additional hosts, install node_exporter on them. Then add a scrape job to /etc/prometheus/prometheus.yml:

sudo cat /etc/prometheus/prometheus.yml

Add a job under scrape_configs. For example:

- job_name: 'remote-nodes' static_configs: - targets: ['10.0.1.20:9100', '10.0.1.21:9100']

Validate the config, then reload Prometheus without a restart:

sudo -u prometheus /usr/local/bin/promtool check config /etc/prometheus/prometheus.yml curl -s -X POST http://127.0.0.1:9090/-/reload && echo reloaded

The key files and directories are:

• Grafana server binary: /usr/sbin/grafana-server
• Grafana CLI: /usr/sbin/grafana-cli
• Grafana config: /etc/grafana/grafana.ini
• Provisioned datasource: /etc/grafana/provisioning/datasources/sac-prometheus.yaml
• Provisioned dashboards: /var/lib/grafana/dashboards/
• Prometheus binary: /usr/local/bin/prometheus
• Prometheus config: /etc/prometheus/prometheus.yml
• Prometheus data: /var/lib/prometheus/data/
• node_exporter binary: /usr/local/bin/node_exporter
• Credentials file: /stage/scripts/grafana-credentials.log
• First-boot sentinel: /var/lib/sac/data/grafana-firstboot.done

Inspect installed versions:

dpkg-query -W -f='grafana ${Version}\n' grafana /usr/local/bin/prometheus --version 2>&1 | head -1 /usr/local/bin/node_exporter --version 2>&1 | head -1

Status:

sudo systemctl status grafana-server.service --no-pager

Stop, start or restart each service independently:

sudo systemctl restart prometheus.service
sudo systemctl restart node_exporter.service
sudo systemctl restart grafana-server.service

View logs:

sudo journalctl -u prometheus.service --no-pager -n 50
sudo tail -n 50 /var/log/grafana/grafana.log

Reset the Grafana admin password if you lose it:

sudo grafana-cli admin reset-admin-password '<new-password>'

Grafana listens on plain HTTP by design. For production, terminate TLS at a reverse proxy. Use Nginx or Caddy on the same VM, proxying https://:443 to http://localhost:3000. Alternatively, use an Azure Application Gateway with TLS termination that routes to the VM private IP on port 3000.

Restrict the NSG so that port 3000 is reachable only from the proxy. The SecureAnyCloud nginx-ssl-certbot-ubuntu-24-04 image is a purpose-built companion reverse proxy.

Prometheus and node_exporter are loopback-only, so they need no additional network hardening. When you need the Prometheus UI, reach it over an SSH tunnel:

ssh -L 9090:127.0.0.1:9090 azureuser@<vm-ip>

Cannot reach Grafana on port 3000

• Confirm the service: sudo systemctl status grafana-server.service
• Confirm the listener: sudo ss -tln | grep 3000
• Check the log: sudo tail -n 50 /var/log/grafana/grafana.log
• Confirm that the NSG allows TCP 3000 from your client source IP

Dashboard shows "No data"

• Confirm that Prometheus targets are up: curl -s http://127.0.0.1:9090/api/v1/targets | jq '.data.activeTargets[].health'
• Confirm node_exporter: sudo systemctl status node_exporter.service
• Widen the dashboard time range (top right) to the last 15 minutes

Grafana did not rotate the admin password

• Check first boot: sudo journalctl -u grafana-firstboot.service --no-pager
• Read the per-VM password in /stage/scripts/grafana-credentials.log with sudo cat

Prometheus fails to start

• Check the journal: sudo journalctl -u prometheus.service --no-pager -n 100
• Validate the config: sudo -u prometheus /usr/local/bin/promtool check config /etc/prometheus/prometheus.yml
• Check disk space: df -h /var/lib/prometheus

• Change the admin password again from the UI after first login (Profile > Change Password).
• Create per-user accounts under Administration > Users. Never share the admin login.
• Restrict port 22 to your management IP ranges only.
• Restrict port 3000 to trusted client networks, and pair it with TLS as in Step 15.
• Leave Prometheus and node_exporter on loopback. Surface metrics through Grafana rather than exposing 9090 or 9100.
• Enable Grafana alerting and route alerts to PagerDuty, Opsgenie, Slack or email.
• Shred the credentials file once the password is in your secret store: sudo shred -u /stage/scripts/grafana-credentials.log

Grafana Community Edition is licensed under the GNU Affero General Public License version 3 (AGPLv3). Prometheus and node_exporter are licensed under the Apache License 2.0. There is no per-user, per-dashboard, or per-server fee. Grafana is a registered trademark of Grafana Labs; Prometheus is a trademark of The Linux Foundation.

SecureAnyCloud provides commercial support for this image separately from the upstream projects.

Email: sales@secureanycloud.com

Website: secureanycloud.com

Support hours: 24/7 with guaranteed 24 hour response SLA

Deploy on Azure

Launch Grafana CE with Prometheus on Ubuntu 24.04 with 24/7 support from SecureAnyCloud.

View on Marketplace

Need Help?

Our support team is available 24/7.

sales@secureanycloud.com

Deploy on Azure

Launch Grafana CE with Prometheus on Ubuntu 24.04 LTS on Azure with 24/7 support from SecureAnyCloud.

View this image

Need Help?

Our support team is available 24/7.

sales@secureanycloud.com

Find Grafana CE with Prometheus on the AWS Marketplace, subscribe, and launch an instance.

In the launch wizard, attach a security group that allows inbound TCP 22 (SSH) and TCP 3000 (the Grafana UI) from your client networks only. Prometheus and node_exporter are bound to loopback and need no inbound rules of their own.

AMI_ID="<marketplace-ami-id>"
# from the Marketplace listing, in your region KEY_NAME="my-key" SUBNET_ID="subnet-xxxxxxxx" SG_ID="sg-xxxxxxxx"
# allows 22 + 3000 from your networks only aws ec2 run-instances \
  --image-id "$AMI_ID" \
  --instance-type m5.large \
  --key-name "$KEY_NAME" \
  --subnet-id "$SUBNET_ID" \
  --security-group-ids "$SG_ID" \
  --associate-public-ip-address \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=grafana-prometheus}]'

First boot generates a random admin password for this instance alone and writes it to a root-only file. Read the file:

sudo cat /stage/scripts/grafana-credentials.log

The file records the username, the generated password and the URL to open:

# Grafana CE with Prometheus - Per-VM Credentials
# ADMIN_USER=admin ADMIN_PASSWORD=<ADMIN_PASSWORD> GRAFANA_PORT=3000 GRAFANA_URL=http://<instance-public-ip>:3000

All three units should report active:

systemctl is-active grafana-server prometheus node_exporter

active active active

Confirm the versions that shipped in this image:

/usr/local/bin/prometheus --version 2>&1 | head -1 /usr/local/bin/node_exporter --version 2>&1 | head -1 dpkg-query -W -f='grafana ${Version}\n' grafana

prometheus, version 3.13.1 (branch: HEAD, revision: 73ff57ce2b8161059ac7fe5188f03f1c3d22b29a) node_exporter, version 1.12.1 (branch: HEAD, revision: 6044da783597cc3b57aef7580ddcdcff58a4ee99) grafana 12.4.6

Grafana's own health endpoint should report its database as ok:

curl -s http://127.0.0.1:3000/api/health

{ "database": "ok", "version": "12.4.6", "commit": "16fbc24965a6dbeb7ed7facafb090062214ba33e" }

Only the authenticated Grafana UI is exposed. Prometheus and node_exporter listen on 127.0.0.1 and are not reachable from the network. Check the listeners:

sudo ss -tlnp | grep -E '3000|9090|9100' | sed 's/users:.*//'

LISTEN 0 4096 127.0.0.1:9090 0.0.0.0:* LISTEN 0 4096 127.0.0.1:9100 0.0.0.0:* LISTEN 0 4096 *:3000 *:*

Only port 3000 is bound to all interfaces. To reach the Prometheus expression browser from your workstation, use an SSH tunnel instead of opening the port:

ssh -i my-key.pem -L 9090:127.0.0.1:9090 ubuntu@<public-ip>

Then browse to http://localhost:9090 on your own machine.

Prometheus is preconfigured to scrape node_exporter and itself. Both targets should report up:

curl -s 'http://127.0.0.1:9090/api/v1/targets?state=active' | jq -r '.data.activeTargets[] | "\(.labels.job)\t\(.scrapeUrl)\t\(.health)"'

node http://127.0.0.1:9100/metrics up prometheus http://127.0.0.1:9090/metrics up

The Prometheus tsdb and Grafana's state are each on their own EBS volume. Both are mounted by filesystem UUID, so the layout survives reboots and relaunches:

df -h /var/lib/prometheus /var/lib/grafana

Filesystem Size Used Avail Use% Mounted on /dev/nvme1n1 49G 360K 47G 1% /var/lib/prometheus /dev/nvme2n1 20G 49M 19G 1% /var/lib/grafana

Grow either volume in the EC2 console, then extend the filesystem with resize2fs. The OS disk is not affected.

Open http://<instance-public-ip>:3000/ in your browser. Sign in as admin with the password from Step 3.

Go to Dashboards and open Node Metrics (SecureAnyCloud) in the SecureAnyCloud folder. It renders live host metrics immediately: CPU busy, memory used, root filesystem used, load average, CPU usage by mode and network throughput.

Go to Connections > Data sources > Prometheus. The datasource is already provisioned, set as the default, and pointed at the loopback Prometheus on http://127.0.0.1:9090.

Grafana proxies queries to Prometheus on the server side. Run a PromQL query through Grafana's datasource proxy. A live value proves the whole chain: your credential, Grafana's authentication, the proxy and Prometheus itself.

PW=$(sudo grep '^ADMIN_PASSWORD=' /stage/scripts/grafana-credentials.log | cut -d= -f2-)
curl -s -u "admin:$PW" 'http://127.0.0.1:3000/api/datasources/proxy/uid/sac-prometheus/api/v1/query?query=node_load1' | jq -r '.data.result[0].value[1]'

The query returns a numeric load average, for example:

0.89

No shared credential ships in this image. The well-known admin/admin default is rejected, while your per-instance password is accepted:

PW=$(sudo grep '^ADMIN_PASSWORD=' /stage/scripts/grafana-credentials.log | cut -d= -f2-)
echo "default admin/admin -> $(curl -s -o /dev/null -w '%{http_code}' -u 'admin:admin' http://127.0.0.1:3000/api/user)"
echo "per-instance password -> $(curl -s -o /dev/null -w '%{http_code}' -u "admin:$PW" http://127.0.0.1:3000/api/user)"

default admin/admin -> 401 per-instance password -> 200

Prometheus reads its configuration from /etc/prometheus/prometheus.yml. Add a job under scrape_configs for any application /metrics endpoint, another host's node_exporter, or any community exporter:

- job_name: 'my-application' static_configs: - targets: ['10.0.1.25:8080']

Validate the file, then hot-reload Prometheus by sending it SIGHUP. This rereads the configuration in place, so the tsdb stays open and no metrics are dropped:

sudo promtool check config /etc/prometheus/prometheus.yml
sudo systemctl kill -s HUP prometheus.service

Checking /etc/prometheus/prometheus.yml SUCCESS: /etc/prometheus/prometheus.yml is valid prometheus config file syntax

Confirm the reload landed in the journal:

sudo journalctl -u prometheus -n 3 --no-pager | grep -i "loading configuration\|Completed loading"

New targets appear in Grafana immediately and can be charted with PromQL.

sudo systemctl status grafana-server
sudo systemctl restart grafana-server
sudo journalctl -u grafana-server -n 50 --no-pager sudo journalctl -u prometheus -n 50 --no-pager

Grafana's own log lives at /var/log/grafana/grafana.log. Prometheus and node_exporter log to the journal.

Grafana has a large library of ready-made dashboards. In the UI, choose Dashboards > New > Import and enter a dashboard ID from grafana.com. Select the Prometheus datasource when prompted.

Dashboard 1860 (Node Exporter Full) is a comprehensive host-metrics dashboard that works with the node_exporter in this image.

Grafana serves plain HTTP on port 3000 by default. For production, put your own domain and a TLS certificate in front of it.

Either terminate TLS at an AWS Application Load Balancer and forward to port 3000, or install a reverse proxy on the instance and point it at 127.0.0.1:3000. If you terminate TLS upstream, set root_url in /etc/grafana/grafana.ini to your public HTTPS URL, so that generated links and redirects are correct.

• Restrict TCP 3000 and TCP 22 in your security group to known client networks only. Never leave them open to the whole internet.
• Leave Prometheus and node_exporter bound to loopback, and reach them over an SSH tunnel as in Step 5.
• Change the generated admin password to one your team manages. Add individual Grafana users instead of sharing the admin account.
• Put TLS in front of the UI before you expose it beyond a trusted network (Step 16).
• Keep the OS patched. Unattended security upgrades are enabled by default.
• Take EBS snapshots of the Prometheus and Grafana volumes on a schedule that matches your retention needs.

Grafana is a trademark of Raintank, Inc. dba Grafana Labs and is distributed under the GNU Affero General Public License v3.0. Prometheus and node_exporter are Cloud Native Computing Foundation projects distributed under the Apache License 2.0. All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.

Deploy on AWS

Find Grafana CE with Prometheus on the AWS Marketplace to launch this image in your own account.

Need Help?

SecureAnyCloud provides 24/7 technical support for this image. Support covers deployment, upgrades, adding scrape targets and exporters, dashboard and alert configuration, and fronting the UI with your own domain and TLS.

Email sales@secureanycloud.com with your instance ID and a description of the issue.

Deploy on AWS

Launch Grafana CE with Prometheus with 24/7 support from SecureAnyCloud.

View on Marketplace

Need Help?

Our support team is available 24/7.

sales@secureanycloud.com