Gigapipe qryn

• Open the Azure Marketplace and search for Gigapipe (qryn) on Ubuntu 24.04 LTS by SecureAnyCloud.
• Select Create, then choose your subscription, resource group and region.
• Pick a VM size of Standard_B2ms or larger.
• Under Administrator account, select SSH public key and paste your public key.
• Under Inbound port rules, allow SSH (22) and HTTPS (443).
• Select Review + create, then Create.

az group create --name gigapipe-rg --location eastus

az vm image terms accept --urn sac1647283583153:qryn-ubuntu-24-04:default:latest
az vm create \
  --resource-group gigapipe-rg \
  --name gigapipe-vm \
  --image sac1647283583153:qryn-ubuntu-24-04:default:latest \
  --size Standard_B2ms \
  --admin-username azureuser \
  --generate-ssh-keys \
  --public-ip-sku Standard
az vm open-port --resource-group gigapipe-rg --name gigapipe-vm --port 443 --priority 1010

ssh azureuser@<vm-ip>

Gigapipe runs as a systemd service on loopback behind nginx, with ClickHouse on the same VM and the OpenTelemetry Collector feeding it. Two one-shot units run at boot. qryn-tls generates the per-VM TLS certificate before nginx starts. qryn-firstboot runs on first boot only: it generates the secrets, opens the gate that lets Gigapipe start, and runs the self-test.

systemctl is-active clickhouse-server nginx gigapipe otelcol-contrib

Expected output:

active active active active

After a successful run, the one-shot units report active (exited). The first-boot unit then disables itself so it can never run twice:

systemctl --no-pager --plain --no-legend list-units 'qryn*' 'gigapipe*'

Expected output:

gigapipe.service loaded active running Gigapipe (qryn) polyglot observability backend (loopback :3100) (sac) qryn-firstboot.service loaded active exited Gigapipe (qryn) first-boot setup (per-VM secrets, schema, self-test) (sac) qryn-tls.service loaded active exited Mint the per-VM self-signed TLS certificate for the Gigapipe (qryn) front door (sac)

Gigapipe and ClickHouse listen only on the loopback interface, and nginx is the single public listener on ports 80 (which redirects to HTTPS) and 443. Instead of grepping for the ports you expect, list every socket that is bound to something other than loopback, so anything unexpected shows up:

sudo ss -ltn | awk 'NR>1{print $4}' | grep -vE '^(127\.|\[::1\])' | sort -u

Expected output:

0.0.0.0:22 0.0.0.0:443 0.0.0.0:80 [::]:22 [::]:443 [::]:80

The only entries are 22 (SSH) and 80 and 443 (nginx), on both IPv4 and IPv6. Everything else the image runs, Gigapipe on 127.0.0.1:3100 and ClickHouse on 127.0.0.1:8123, 127.0.0.1:9000 and 127.0.0.1:9009, is bound to loopback and is not reachable from outside the VM, and the collector opens no listening port at all. Because this command lists the full set rather than a filtered subset, run it again after you install anything else on the VM.

The front door answers over HTTPS, and only when you supply the credential. The next command reads the per-instance password from the root-only credentials file, then probes the readiness endpoint with the credential, without it, and over plain HTTP:

QP=$(sudo awk -F= '/^QRYN_PASSWORD=/{print $2}' /root/qryn-credentials.txt)
curl -sk -o /dev/null -w 'https /ready with credential -> %{http_code}\n' -u "admin:${QP}" https://127.0.0.1/ready
curl -sk -o /dev/null -w 'https /ready without credential -> %{http_code}\n' https://127.0.0.1/ready
curl -s -o /dev/null -w 'http / -> %{http_code}\n' http://127.0.0.1/

Expected output:

https /ready with credential -> 200 https /ready without credential -> 401 http / -> 301

Every VM generates its own secrets on first boot and writes them to a root-only file. Read it as root:

sudo cat /root/qryn-credentials.txt

The file contains the following values:

• QRYN_USER / QRYN_PASSWORD: the HTTP basic authentication credential for the explorer and every API
• QRYN_URL: the HTTPS URL of your instance
• CLICKHOUSE_ADMIN_USER / CLICKHOUSE_ADMIN_PASSWORD: the ClickHouse default user, for administering the database from the VM
• CLICKHOUSE_APP_USER / CLICKHOUSE_APP_PASSWORD: the gigapipe ClickHouse user the service connects with, limited to the gigapipe database
• CLICKHOUSE_DB: the database that holds every log, metric, trace and profile

Both nginx (/etc/nginx/qryn.htpasswd) and Gigapipe itself (QRYN_LOGIN and QRYN_PASSWORD in /etc/gigapipe/gigapipe.env) enforce the same basic authentication credential, and the collector uses it too (/etc/otelcol-contrib/otelcol-contrib.conf). Keep the file safe. Step 12 explains how to rotate the credential.

No usable secret is baked into the image. Everything security-sensitive is generated uniquely on the first boot of every VM, before Gigapipe is allowed to start:

• the ClickHouse default (administrator) password, so the database is never without one
• the ClickHouse gigapipe application password, for a user that only has rights on the gigapipe database
• the 24-character basic authentication password for the explorer and every ingest and query API
• a per-instance self-signed TLS certificate
• the ClickHouse schema itself, created by Gigapipe's first start on your VM, so the database holds only your data

Gigapipe and ClickHouse bind to loopback and the collector opens no port. As a result, nginx on 443 is the only network-facing surface, and it asks for the credential on every path except the unauthenticated /healthz liveness probe.

Verify the whole model end to end with the bundled self-test. It checks that:

• the front door works over TLS, and requests without a credential or with a wrong password are refused both at nginx and on loopback
• the explorer is served
• a log line pushed through the Loki API is found again with LogQL, and a metric pushed through the OTLP API is found again with PromQL
• the VM's own journal and host metrics from the collector are queryable
• ClickHouse refuses a passwordless default login and accepts the per-VM credentials
• the exposed port set is exactly 22, 80 and 443

Run the self-test:

sudo /usr/local/bin/qryn-selftest

Expected output:

OK qryn self-test passed: healthz 200, :80 301, unauthenticated 401 (edge + loopback), wrong password 401, /ready 200 + explorer UI served, Loki push -> LogQL round-trip OK (sac-qryn-selftest-...), OTLP metric -> PromQL round-trip OK (...), collector journald + host metrics queryable, ClickHouse default refuses no-password + per-VM creds OK + schema present (29 tables), exposed ports exactly {22,80,443}, gigapipe/ClickHouse loopback-only, collector listens nowhere, no swap

ClickHouse refuses a login without a password, even from the VM itself, and accepts the per-instance one:

clickhouse-client -q 'SELECT 1' >/dev/null 2>&1 && echo 'no password -> accepted' || echo 'no password -> refused'
CP=$(sudo awk -F= '/^CLICKHOUSE_ADMIN_PASSWORD=/{print $2}' /root/qryn-credentials.txt) clickhouse-client --password "${CP}" -q "SELECT concat('per VM password -> ClickHouse ', version())"

Expected output:

no password -> refused per VM password -> ClickHouse 26.8.2.7

Open your instance in a browser:

https://<vm-ip>/

The TLS certificate is self-signed and unique to the VM, so your browser warns once. Accept the warning to continue, or install a CA-issued certificate (see Step 12) to remove it. The browser then asks for the basic authentication credential. Sign in as QRYN_USER with QRYN_PASSWORD from your credentials file. After you sign in, nginx sets an HttpOnly session cookie for the explorer, so its queries stay authenticated for the rest of the browser session.

You land on the explorer's search view with the Logs data source selected. The data sources point at the same origin you signed in to, so nothing needs configuring. The collector has been shipping this VM's journal and host metrics since first boot, so the label browser is already populated.

Enter a LogQL stream selector and press Show Results. {service_name="journald"} returns the VM's own systemd journal. Every line carries the unit, syslog_identifier, level, host_name and service_name labels that the collector attached. For example, {service_name="journald", unit="ssh.service"} shows SSH logins, {service_name="journald", level=~"WARN|ERROR"} shows anything that went wrong, and the query below narrows the journal to the appliance's own service units.

Select Labels to open the label browser. It lists every label the collector attached and their values, so you can build a stream selector by clicking instead of typing.

Switch the data source to Metrics and enter a PromQL expression. The collector scrapes host metrics every 15 seconds, so system_memory_usage_bytes{state="used"}, avg(system_cpu_utilization{state!="idle"}), system_filesystem_usage_bytes{mountpoint="/", state="used"} and rate(system_network_io_bytes_total[5m]) all chart immediately.

The Data Sources settings page (/#/datasources) shows the Logs, Metrics and Traces sources all pointing at the address you signed in on, which is your own instance. You do not need to enter anything there, because the explorer authenticates through the session your browser established when it answered the credential prompt. Use this page if you want to point the explorer at a second Gigapipe instance or add request headers.

Every API is available from the VM at https://127.0.0.1/ and from your workstation at https://<vm-ip>/ (the QRYN_URL value from your credentials file). Always supply the basic authentication credential and, until you install a CA-issued certificate, use curl -k. The Loki label API lists what the collector has produced so far:

QP=$(sudo awk -F= '/^QRYN_PASSWORD=/{print $2}' /root/qryn-credentials.txt)
curl -sk -u "admin:${QP}" https://127.0.0.1/loki/api/v1/labels | jq -c .data
curl -sk -u "admin:${QP}" https://127.0.0.1/loki/api/v1/label/level/values | jq -c .data

Expected output:

["host_name","level","os_type","service_name","syslog_identifier","unit","job","source"] ["INFO","WARN"]

Run a LogQL range query to return the VM's own SSH log lines. Gigapipe requires both start and end (in nanoseconds) on range queries:

QP=$(sudo awk -F= '/^QRYN_PASSWORD=/{print $2}' /root/qryn-credentials.txt)
curl -sk -u "admin:${QP}" -G https://127.0.0.1/loki/api/v1/query_range \
  --data-urlencode 'query={service_name="journald", unit="ssh.service"}' \
  --data-urlencode "start=$(( $(date +%s) - 3600 ))000000000" \
  --data-urlencode "end=$(( $(date +%s) + 60 ))000000000" \
  --data-urlencode 'limit=2' | jq -r '.data.result[].values[][1]'

Expected output:

pam_unix(sshd:session): session opened for user azureuser(uid=1000) by azureuser(uid=0) Accepted publickey for azureuser from 203.0.113.10 port 49497 ssh2: RSA SHA256:...

Run a PromQL instant query to return the latest host metric sample. The job label is hostmetrics for everything the collector scrapes:

QP=$(sudo awk -F= '/^QRYN_PASSWORD=/{print $2}' /root/qryn-credentials.txt)
curl -sk -u "admin:${QP}" -G https://127.0.0.1/api/v1/query \
  --data-urlencode 'query=system_memory_usage_bytes{state="used"}' \
  | jq -c '.data.result[0] | {name: .metric.__name__, job: .metric.job, state: .metric.state, bytes: .value[1]}'
curl -sk -u "admin:${QP}" -G https://127.0.0.1/api/v1/label/__name__/values \
  --data-urlencode 'match[]={job="hostmetrics"}' | jq '.data | length'

Expected output:

{"name":"system_memory_usage_bytes","job":"hostmetrics","state":"used","bytes":"1381797888"} 25

Any client that speaks Loki, Prometheus remote write, OTLP, InfluxDB line protocol, Datadog or Elastic can send data to this instance. The simplest demonstration pushes one log line through the Loki API and reads it back with LogQL:

QP=$(sudo awk -F= '/^QRYN_PASSWORD=/{print $2}' /root/qryn-credentials.txt)
NOW=$(date +%s%N)
curl -sk -o /dev/null -w 'push -> %{http_code}\n' -u "admin:${QP}" -H 'Content-Type: application/json' \
  -d "{\"streams\":[{\"stream\":{\"job\":\"guide-example\",\"app\":\"hello\"},\"values\":[[\"${NOW}\",\"hello from the deployment guide\"]]}]}" \
  https://127.0.0.1/loki/api/v1/push sleep 2
curl -sk -u "admin:${QP}" -G https://127.0.0.1/loki/api/v1/query_range \
  --data-urlencode 'query={job="guide-example"}' \
  --data-urlencode "start=$(( $(date +%s) - 600 ))000000000" \
  --data-urlencode "end=$(( $(date +%s) + 60 ))000000000" \
  --data-urlencode 'limit=5' | jq -r '.data.result[].values[][1]'

Expected output:

push -> 204 hello from the deployment guide

Metrics work the same way. This command pushes a gauge as OTLP JSON to /v1/metrics and queries it back with PromQL. The service.name attribute becomes the job label:

QP=$(sudo awk -F= '/^QRYN_PASSWORD=/{print $2}' /root/qryn-credentials.txt)
NOW=$(date +%s%N)
curl -sk -o /dev/null -w 'push -> %{http_code}\n' -u "admin:${QP}" -H 'Content-Type: application/json' \
  -d "{\"resourceMetrics\":[{\"resource\":{\"attributes\":[{\"key\":\"service.name\",\"value\":{\"stringValue\":\"guide-example\"}}]},\"scopeMetrics\":[{\"metrics\":[{\"name\":\"guide_example_gauge\",\"gauge\":{\"dataPoints\":[{\"asDouble\":42,\"timeUnixNano\":\"${NOW}\"}]}}]}]}]}" \
  https://127.0.0.1/v1/metrics sleep 2
curl -sk -u "admin:${QP}" -G https://127.0.0.1/api/v1/query --data-urlencode 'query=guide_example_gauge' \
  | jq -c '.data.result[0] | {metric: .metric, value: .value[1]}'

Expected output:

push -> 200 {"metric":{"__name__":"guide_example_gauge","job":"guide-example","service_name":"guide-example"},"value":"42"}

Point your agents and applications at the following endpoints, all under https://<vm-ip> and all requiring the basic authentication credential:

Data Send to Compatible with

Logs /loki/api/v1/push Promtail, Grafana Alloy, Fluent Bit, Vector and any Loki client

Logs, metrics, traces, profiles /v1/logs, /v1/metrics, /v1/traces, /v1development/profiles OpenTelemetry Collector and SDKs (OTLP over HTTP, protobuf or JSON)

Metrics /api/v1/prom/remote/write Prometheus remote_write, the OpenTelemetry prometheusremotewrite exporter

Metrics /influx/api/v2/write Telegraf and InfluxDB line protocol clients

Traces /tempo/api/push and /api/v2/spans Tempo and Zipkin clients

Logs and metrics /api/v2/logs, /api/v2/series Datadog agent formats

Logs /_bulk Elastic bulk clients

To point an OpenTelemetry Collector at the instance, add an otlphttp exporter with basic authentication:

extensions: basicauth/gigapipe: client_auth: username: admin password: <QRYN_PASSWORD> exporters: otlphttp/gigapipe: endpoint: https://<vm-ip> auth: authenticator: basicauth/gigapipe tls: insecure_skip_verify: true
# until you install a CA issued certificate service: extensions: [basicauth/gigapipe] pipelines: logs: { receivers: [otlp], exporters: [otlphttp/gigapipe] } metrics: { receivers: [otlp], exporters: [otlphttp/gigapipe] } traces: { receivers: [otlp], exporters: [otlphttp/gigapipe] }

For Prometheus, add a remote_write block:

remote_write: - url: https://<vm-ip>/api/v1/prom/remote/write basic_auth: username: admin password: <QRYN_PASSWORD> tls_config: insecure_skip_verify: true
# until you install a CA issued certificate

The OTLP over gRPC receiver is multiplexed on Gigapipe's loopback port and is not exposed by this image. Use OTLP over HTTP through nginx, which every OpenTelemetry exporter supports. If you need gRPC from other hosts, add a location with grpc_pass http://127.0.0.1:3100; to the nginx site.

Grafana is not included in this image. If you run Grafana elsewhere, add this instance as a Loki data source (URL https://<vm-ip>), a Prometheus data source (same URL) and a Tempo data source (same URL). Enable Basic auth on each, enter the QRYN_USER / QRYN_PASSWORD credential, and enable Skip TLS verify until you install a CA-issued certificate.

No plugin is needed. Gigapipe implements the native Loki, Prometheus and Tempo query APIs, so Explore, dashboards, alerting and log-to-trace correlation work as they would against the originals.

Gigapipe reads its configuration from /etc/gigapipe/gigapipe.env, which is generated on first boot. You can add any variable from the upstream configuration reference. The most important ones are shown below with secrets redacted:

sudo sed -E 's/^(CLICKHOUSE_AUTH|QRYN_PASSWORD)=.*/\1=********/' /etc/gigapipe/gigapipe.env | grep -vE '^#|^$'

Expected output:

CLICKHOUSE_SERVER=127.0.0.1 CLICKHOUSE_PORT=9000 CLICKHOUSE_DB=gigapipe CLICKHOUSE_AUTH=******** HOST=127.0.0.1 PORT=3100 QRYN_LOGIN=admin QRYN_PASSWORD=<set-at-first-boot> SAMPLES_DAYS=7 LOG_LEVEL=info

• SAMPLES_DAYS: retention in days for logs, metrics and traces (7 as shipped, applied as a ClickHouse TTL). Raise it once you have sized the disk.
• LOG_LEVEL: debug, info, warn or error.
• BULK_MAX_AGE_MS and BULK_MAX_SIZE_BYTES: how long and how much data Gigapipe batches before flushing to ClickHouse.
• QRYN_RULER_ENABLED=true: enables the recording rules engine and its /api/v1/rules endpoints.
• LOG_DRILLDOWN=true: enables log pattern detection.

After you edit the file, restart the service:

$
sudo systemctl restart gigapipe

ClickHouse stores all data under /var/lib/clickhouse. Its settings live in /etc/clickhouse-server/config.d/sac.xml (loopback only, telemetry off, memory capped at 60 percent of RAM), and its two users are defined in /etc/clickhouse-server/users.d/sac-users.xml (SHA-256 password hashes, loopback only). To see how much space each table uses, query ClickHouse with the administrator credential:

CP=$(sudo awk -F= '/^CLICKHOUSE_ADMIN_PASSWORD=/{print $2}' /root/qryn-credentials.txt) clickhouse-client --password "${CP}" -q "SELECT count() AS tables FROM system.tables WHERE database='gigapipe'" clickhouse-client --password "${CP}" -q "SELECT table, formatReadableSize(sum(bytes_on_disk)) AS size, sum(rows) AS rows FROM system.parts WHERE database='gigapipe' AND active GROUP BY table ORDER BY sum(bytes_on_disk) DESC LIMIT 3 FORMAT PrettyCompactNoEscapes"

Expected output:

29 ┌─table───────┬─size──────┬─rows─┐ 1. │ metrics_15s │ 41.35 KiB │ 1746 │ 2. │ samples_v3 │ 40.42 KiB │ 2606 │ 3. │ time_series │ 20.75 KiB │ 194 │ └─────────────┴───────────┴──────┘

The collector's pipeline is defined in /etc/otelcol-contrib/config.yaml. To ship more data from this VM, add receivers there, such as a filelog receiver for application log files. To receive telemetry from other hosts, send it straight to Gigapipe's endpoints above instead of through the local collector.

The following table lists the key paths.

Path Purpose

/etc/gigapipe/gigapipe.env Gigapipe configuration: ClickHouse connection, basic authentication credential, retention

/usr/local/bin/gigapipe The Gigapipe 5.4.4 release binary

/opt/qryn-view/dist The explorer UI bundle served by nginx

/root/qryn-credentials.txt Per-VM credentials and URL, root-only

/var/lib/clickhouse ClickHouse data (every log, metric, trace and profile)

/etc/clickhouse-server/config.d/sac.xml ClickHouse server settings: loopback only, no telemetry, memory cap

/etc/clickhouse-server/users.d/sac-users.xml The default and gigapipe ClickHouse users

/etc/apt/preferences.d/clickhouse-pin Pins ClickHouse to the verified version; edit it to upgrade deliberately

/etc/otelcol-contrib/config.yaml The collector pipeline (journald and host metrics into Gigapipe)

/etc/otelcol-contrib/otelcol-contrib.conf The collector's environment, including its copy of the credential

/etc/nginx/sites-available/sac-qryn The TLS and basic authentication front door

/etc/nginx/qryn.htpasswd The basic authentication user database

/etc/nginx/conf.d/sac-qryn-session.conf The explorer session token map, root-only, rewritten by qryn-set-password

/usr/local/sbin/qryn-set-password Rotates the basic authentication credential everywhere at once

/etc/nginx/tls/qryn.crt and .key The per-VM TLS certificate and key

/usr/local/bin/qryn-selftest End-to-end appliance self-test

/var/log/sac/sac-firstboot.log First boot log

Confirm the installed versions at any time. The Gigapipe binary is the official release, unchanged since the image was built:

sha256sum /usr/local/bin/gigapipe dpkg-query -W -f='${Package} ${Version}\n' clickhouse-server otelcol-contrib

Expected output:

39ec6718d17a3153da593f215e91cf69bb3d5ec07d9887ff1e8118b91f154f69 /usr/local/bin/gigapipe clickhouse-server 26.8.2.7 otelcol-contrib 0.160.0

The basic authentication credential is enforced by nginx and by Gigapipe, used by the collector, recorded in the credentials file, and tied to the explorer session token in nginx. One helper updates all of them together and restarts the services. The password must be at least 12 characters and contain no colon, quotes or spaces:

$
sudo qryn-set-password '<new-password>'

Every client, browser and collector then needs the new password. Existing explorer sessions are signed out because the session token is regenerated with the new password.

The per-VM certificate is self-signed. For a long-lived deployment, point a DNS name at the VM and obtain a CA-issued certificate for it. Install the certificate and key over /etc/nginx/tls/qryn.crt and /etc/nginx/tls/qryn.key, then reload nginx:

$
sudo systemctl reload nginx

The qryn-tls unit generates a certificate only when none exists, so a certificate you install is never overwritten.