Pathling
Overview
Pathling is an open source FHIR analytics server from CSIRO's Australian e-Health Research Centre. It exposes a standard HL7 FHIR R4 REST endpoint backed by an embedded, single-node Apache Spark engine. Clinical resources such as Patient, Observation and Condition are stored in a local Delta warehouse and queried through the FHIR API.
What sets Pathling apart is analytics. SQL on FHIR ViewDefinitions reshape FHIR data into flat, tabular datasets, so analysts can run population-level analytics without standing up a separate Spark cluster. This guide covers deploying and using Pathling on Ubuntu 24.04 on Azure with the SecureAnyCloud image from the Azure Marketplace.
Pathling itself ships with no authentication, and its write, import and admin surfaces mutate clinical data. The image therefore puts nginx in front of the server and bakes in no credential: each VM generates its own on first boot.
What you get:
- Pathling Server 2.0.1 (FHIR R4 / 4.0.1) with embedded Apache Spark, running under systemd as
pathling.service - The official pinned
ghcr.io/aehrc/pathling:2.0.1container as a boot-managed systemd service, with the Spark warehouse on the local disk and nginx in front of it - The full FHIR REST surface (create, read, search and bulk
$import), plus theViewDefinition/$runoperation that materialises a view as CSV or NDJSON - An nginx reverse proxy on port 80 that enforces per-VM HTTP Basic Auth on all data, write, import and admin paths, with
GET /fhir/metadatapublic - A public, read-only capability statement at GET /fhir/metadata, which serves as a health and discovery surface
- A local Delta warehouse for FHIR resources, empty on arrival and initialised on first container start
- A per-VM access credential generated on first boot and written to the root-only file /root/pathling-credentials.txt. The exposed surface accepts requests only after the credential exists.
- A container that binds host loopback only
- A
ufwfirewall (default deny inbound, allow 22 + 80), which allows just SSH (22) and HTTP (80), and no swap in the image
Deploying the VM
Launch the image from the Azure Marketplace as you would any other VM. The recommended size is Standard_B2ms (2 vCPU / 8 GiB). Pathling embeds a Spark JVM, so it needs more memory than a typical web app. The container's heap is capped at 4 GiB (JAVA_TOOL_OPTIONS=-Xmx4g) to fit this size with headroom.
Larger workloads benefit from a bigger size. If you move up, raise the heap cap in /etc/pathling-appliance/pathling.env in proportion.
Pathling is reached over TCP 80. The Azure network security group (NSG) is your first line of defence: open port 80 only to the client addresses or subnets that need it, never to the whole internet. The per-VM Basic Auth credential is the second layer.
Basic Auth sends credentials with each request, so front the VM with TLS (an Azure Application Gateway or your own reverse proxy) before exposing it beyond a trusted network.
First boot
The first time the VM boots, pathling-firstboot.service runs once and then disables itself. It performs these steps:
- Resolves the VM's public IP for the info file.
- Generates a unique HTTP Basic Auth password for the
pathlinguser. - Writes the password to the nginx credential file
/etc/nginx/pathling.htpasswd. - Writes
/root/pathling-credentials.txt(mode 600, owned by root). - Writes a message-of-the-day pointer.
Only after first boot has written the credential do pathling.service and nginx.service start. The exposed surface is therefore never up without the credential in place. You do not need to do anything.
Retrieving your per-VM credentials
The password is unique to your VM. It is written in plain text only in this one root-only file:
sudo cat /root/pathling-credentials.txtExpected output (your host and password will differ):
# Pathling FHIR analytics server — generated on first boot by pathling-firstboot.service.
# Access the server through nginx on the URL below. All data read/write, the
# $import operation, SQL-on-FHIR analytics, and the /admin console require these
# per-VM HTTP Basic Auth credentials. GET <url>/fhir/metadata is public. PATHLING_URL=http://20.30.40.50 PATHLING_USER=pathling PATHLING_PASSWORD=<unique to your VM>Copy the password into your secret store now. To rotate it, regenerate the nginx credential with sudo htpasswd -B /etc/nginx/pathling.htpasswd pathling. Then reload nginx with sudo systemctl reload nginx and update the credentials file to match.
Checking service health
Three services make up the appliance: Docker (which runs the Pathling container), Pathling itself and nginx. Confirm that all three are active:
sudo systemctl is-active docker pathling nginxExpected output:
active active activeThe capability statement is public, so no credential is needed, and it is the recommended readiness probe. It names the FHIR version the server speaks and the Pathling build:
curl -s http://127.0.0.1/fhir/metadata | jq '{resourceType, fhirVersion, software}'{ "resourceType": "CapabilityStatement", "fhirVersion": "4.0.1", "software": { "name": "Pathling Server", "version": "2.0.1+78a3f75" } }A fhirVersion of 4.0.1 is FHIR R4. The software block confirms the pinned Pathling Server release.
The security model
The per-VM credential protects everything except the capability statement. You can see the model in action. An unauthenticated data read is rejected:
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1/fhir/Patient?_summary=count401The same request with the per-VM credential succeeds. The next command reads the credential straight from the root-only file, so it works on your VM without you typing the password:
sudo bash -c 'U=$(grep ^PATHLING_USER= /root/pathling-credentials.txt | cut -d= -f2); P=$(grep ^PATHLING_PASSWORD= /root/pathling-credentials.txt | cut -d= -f2); curl -s -u "$U:$P" "http://127.0.0.1/fhir/Patient?_summary=count" | jq "{resourceType, total}"'{ "resourceType": "Bundle", "total": 2 }From your workstation, substitute your VM's public address and the password from the credentials file: curl -u pathling:<your-per-vm-password> http://<your-vm-public-ip>/fhir/Patient?_summary=count.
Loading and querying FHIR data
Create a resource with a standard FHIR POST. The server assigns an id and returns 201 Created with a Location header. Run:
sudo bash -c 'U=$(grep ^PATHLING_USER= /root/pathling-credentials.txt | cut -d= -f2); P=$(grep ^PATHLING_PASSWORD= /root/pathling-credentials.txt | cut -d= -f2); curl -s -D - -o /dev/null -u "$U:$P" -X POST http://127.0.0.1/fhir/Patient -H "Content-Type: application/fhir+json" --data "{\"resourceType\":\"Patient\",\"gender\":\"female\",\"name\":[{\"family\":\"Demo\"}]}" | grep -iE "^HTTP|^location"'HTTP/1.1 201 Location: http://127.0.0.1/fhir/Patient/f9ca32c1-9fa7-4f7e-86d4-3eba965fe662Query the resource back with a standard FHIR search. _summary=count returns just the total in the Bundle:
sudo bash -c 'U=$(grep ^PATHLING_USER= /root/pathling-credentials.txt | cut -d= -f2); P=$(grep ^PATHLING_PASSWORD= /root/pathling-credentials.txt | cut -d= -f2); curl -s -u "$U:$P" "http://127.0.0.1/fhir/Patient?gender=female&_summary=count" | jq "{total}"'{ "total": 2 }The full FHIR REST surface is available:
GET /fhir/Patient/{id}reads a resource.PUTupdates a resource, andDELETEremoves it.- The asynchronous bulk
POST /fhir/$import(with headerPrefer: respond-async) loads NDJSON from the server's staging area, which suits larger datasets.
Running analytics with SQL on FHIR
Analytics is the reason to run Pathling rather than a plain FHIR store. A ViewDefinition (from the SQL on FHIR v2 specification) describes how to flatten a FHIR resource into columns. The $run operation executes it over Spark and returns a tabular result as CSV or NDJSON.
This view flattens Patient into an id column and a gender column, and requests CSV output:
sudo bash -c 'U=$(grep ^PATHLING_USER= /root/pathling-credentials.txt | cut -d= -f2); P=$(grep ^PATHLING_PASSWORD= /root/pathling-credentials.txt | cut -d= -f2); VIEW="{\"resourceType\":\"Parameters\",\"parameter\":[{\"name\":\"viewResource\",\"resource\":{\"resourceType\":\"ViewDefinition\",\"name\":\"pb\",\"status\":\"active\",\"resource\":\"Patient\",\"select\":[{\"column\":[{\"name\":\"id\",\"path\":\"getResourceKey()\"},{\"name\":\"gender\",\"path\":\"gender\"}]}]}},{\"name\":\"_format\",\"valueString\":\"text/csv\"}]}"; curl -s -u "$U:$P" -X POST "http://127.0.0.1/fhir/ViewDefinition/\$run" -H "Content-Type: application/fhir+json" -H "Accept: text/csv" --data "$VIEW"'id,gender Patient/f9ca32c1-9fa7-4f7e-86d4-3eba965fe662,female Patient/aebbb420-7743-47d8-9a60-2ab7fbb1d8b1,female Patient/93162a59-9891-4724-a623-250b943279f4,maleEach select.column entry is a FHIRPath expression. Views can reach into nested elements, unnest repeating fields with forEach, and pull coded values out of CodeableConcept.
Because $run executes on Spark, the same view definition scales from a handful of resources to millions. You can point the CSV or NDJSON output straight at an analytics notebook, a data warehouse load or a BI tool.
The admin console
Pathling also serves a browsable admin console at /admin/, where you can import data, browse resources and manage ViewDefinitions interactively. It sits behind the same per-VM credential.
Open http://<your-vm-public-ip>/admin/ in a browser and authenticate with user pathling and the password from your credentials file.
Where things live
/var/lib/pathling/warehouse: Delta warehouse holding the FHIR resources
/var/lib/pathling/staging: staging area for NDJSON $import sources
/etc/pathling-appliance/pathling.env: container configuration (warehouse path, Spark, heap cap)
/etc/nginx/pathling.htpasswd: per-VM HTTP Basic Auth credential (nginx)
/root/pathling-credentials.txt: per-VM credentials, mode 600, root only
/etc/systemd/system/pathling.service: systemd unit running the container
To change the heap cap or Spark settings, edit /etc/pathling-appliance/pathling.env. Then restart the container with sudo systemctl restart pathling.
Security checklist
- Restrict the NSG. Allow port 80 only from the subnets or addresses that need it.
- Terminate TLS in front of the VM. Basic Auth sends the credential with every request. Place an Azure Application Gateway or your own TLS reverse proxy in front of the VM before exposing it beyond a trusted network.
- Rotate the per-VM credential into your secret store and treat
/root/pathling-credentials.txtas the sensitive file it is. Rotate withsudo htpasswd -B /etc/nginx/pathling.htpasswd pathling, then runsudo systemctl reload nginx. - Keep the firewall in place. The image ships
ufwwith default deny inbound and only 22 + 80 allowed. Widen it only deliberately. - Leave
unattended-upgradesenabled so the base system keeps receiving security updates.
Troubleshooting
The server does not respond. Check the three services and the container's own log:
sudo systemctl status pathling --no-pagersudo docker logs pathling 2>&1 | tail -n 40Pathling starts a Spark JVM, so allow a minute or so after boot before GET /fhir/metadata returns 200.
No credentials file. pathling-firstboot.service writes the file on the first boot only. Check the first-boot log:
sudo tail -n 30 /var/log/sac/sac-firstboot.logA request returns 401. Every path except GET /fhir/metadata requires the per-VM credential. Confirm that you are sending user pathling with the password from /root/pathling-credentials.txt. If you changed the password, also confirm that you reloaded nginx (sudo systemctl reload nginx).
Out of memory under load. The Spark heap is capped at 4 GiB to fit Standard_B2ms. For heavier analytics, move to a larger size and raise JAVA_TOOL_OPTIONS=-Xmx in /etc/pathling-appliance/pathling.env in proportion. Then run sudo systemctl restart pathling.
Further reading
- Pathling documentation: https://pathling.csiro.au/docs
- Pathling source and releases: https://github.com/aehrc/pathling
- SQL on FHIR v2 specification: https://sql-on-fhir.org
- HL7 FHIR R4 specification: https://hl7.org/fhir/R4/
Support
This image is published by SecureAnyCloud. For help with the image itself, contact sales@secureanycloud.com.
Deploy on Azure
Launch Pathling on Ubuntu 24.04 on Azure with 24/7 support from SecureAnyCloud.
View this image
Need Help?
Our support team is available 24/7.
sales@secureanycloud.com