DKAN

Prerequisites

Before you deploy, make sure you have:

• An active Azure subscription, an SSH key, and a VNet with a subnet.
• A VM size of Standard_B2s (2 vCPU, 4 GB RAM) or larger. Standard_B2s is the recommended size and is sufficient for a catalog of a few hundred datasets. Move to a D2s or D4s size for large datastore imports, many concurrent API consumers, or heavy harvesting.
• Inbound TCP 80 allowed from the addresses that should reach the portal, and TCP 22 allowed for administration.

Step 1 - Deploy from the Azure Marketplace

In the Azure portal, choose Create a resource, search for DKAN Data Portal on Ubuntu 24.04 by SecureAnyCloud, and select Create. Pick your subscription, resource group and region. Choose the Standard_B2s size, and supply your SSH public key for the azureuser account. Allow inbound ports 22 and 80.

Step 2 - Deploy from the Azure CLI

Accept the Marketplace terms, then create the VM:

# Accept the Marketplace terms once per subscription (required before the first CLI deploy)
az vm image terms accept --urn sac1647283583153:dkan-ubuntu-24-04:default:latest
az vm create \
  --resource-group my-resource-group \
  --name my-dkan \
  --image sac1647283583153:dkan-ubuntu-24-04:default:latest \
  --size Standard_B2s \
  --admin-username azureuser \
  --generate-ssh-keys \
  --public-ip-sku Standard

Step 3 - Connect to your VM

Connect with SSH as azureuser:

ssh azureuser@<vm-public-ip>

Step 4 - Confirm the services are running

Check that both services that back DKAN report active. The cron timer also reports active, because it is armed and waiting. The portal should answer HTTP 200:

systemctl is-active apache2 mysql dkan-cron.timer ss -tlnH | awk '{printf "%-22s %s\n", $4, $1}' | sort -u curl -s -o /dev/null -w 'portal -> HTTP %{http_code}\n' http://127.0.0.1/ curl -s -o /dev/null -w 'catalog -> HTTP %{http_code}\n' http://127.0.0.1/dataset/search curl -s -o /dev/null -w 'API root -> HTTP %{http_code}\n' http://127.0.0.1/api/1

Expected: three active lines, then the listening sockets, then three HTTP 200 lines. MySQL listens on 127.0.0.1:3306 only, so the database is never exposed to the network. Port 80 is the only listener DKAN itself publishes, and port 22 is your SSH administration path.

How the first boot protects an unconfigured portal

dkan-firstboot.service is a one-shot unit that runs on the first boot of each VM. It generates the per-VM secrets, provisions the application database, installs Drupal, enables DKAN and writes the credentials file at mode 0600. Only then does it create a bootstrap-ready marker at /var/lib/sac/data/dkan-bootstrap-ready and restart Apache.

Apache carries ConditionPathExists on that marker, so the public listener is physically unable to answer until first boot has finished. There is no window in which an unconfigured portal is reachable. You can see the gate in the unit itself:

systemctl cat apache2.service | grep -A1 '^# /etc/systemd/system/apache2.service.d/sac-bootstrap-gate.conf'
systemctl is-active dkan-firstboot.service
sudo test -f /var/lib/sac/data/dkan-bootstrap-ready && echo "bootstrap-ready marker present — first boot completed"

On the very first boot, the one-shot reports active. It is active (exited), which is what a completed one-shot with RemainAfterExit=yes reports. After a later reboot, the unit is skipped by its own condition and reports inactive.

Both states are correct. The sentinel at /var/lib/sac/data/dkan-firstboot.done guarantees that the unit runs exactly once.

Step 5 - Retrieve your administrator credentials

Every secret this appliance uses is generated on your VM at first boot: a fresh MySQL root password, a fresh application database password, a fresh Drupal administrator password, and a fresh Drupal hash_salt.

The hash_salt matters more than it sounds. A salt shared between deployments would let a session cookie or a one-time login token minted on one VM be replayed against another, so SecureAnyCloud never bakes one into the image. Check the credentials file and list the variables it sets:

sudo stat -c '%n mode=%a owner=%U:%G' /stage/scripts/dkan-credentials.log
sudo grep -oE '^[A-Z_]+=' /stage/scripts/dkan-credentials.log | sed 's/=$/ = (set, unique per VM)/'

The file is mode 0600 and owned by root, so only root can read it. To see the values, run:

sudo cat /stage/scripts/dkan-credentials.log

It contains the portal, catalog, administration and API URLs, the administrator username (SecureAnyCloud) and password, and the database credentials.

You can prove from the VM's own shell that the database credentials authenticate and that a wrong password is rejected:

DB_USER=$(sudo awk -F= '$1=="DKAN_DB_USER"{print $2}' /stage/scripts/dkan-credentials.log)
DB_PASS=$(sudo awk -F= '$1=="DKAN_DB_PASSWORD"{sub(/^[^=]*=/,"");print}' /stage/scripts/dkan-credentials.log)
OK=$(mysql -h 127.0.0.1 -u "$DB_USER" -p"$DB_PASS" -D dkan -N -B -e 'SELECT 1' </dev/null 2>/dev/null || true)
BAD=$(mysql -h 127.0.0.1 -u "$DB_USER" -p'definitely-not-the-password' -D dkan -N -B -e 'SELECT 1' </dev/null 2>/dev/null || true)
echo "correct password -> ${OK:-rejected}"
echo "wrong password -> ${BAD:-rejected}"

Expected: correct password -> 1 and wrong password -> rejected.

Step 6 - Run the built-in self test

The image ships a self test that checks four things that no unauthenticated probe can. It checks that the per-VM database password authenticates and a wrong one is refused. It checks that the per-VM administrator password authenticates through the real Drupal login form. It checks that an authenticated administration page actually renders. It also checks that a wrong administrator password does not produce a session.

Run the self test:

sudo /usr/local/sbin/dkan-selftest.sh

Expected: four descriptive lines followed by DKAN_SELFTEST_OK.

The authenticated-page check is the one worth understanding. A signed-out request to a healthy portal succeeds even when a session-bearing response is too large for the web server to buffer. Only fetching an admin page with a session proves that the administration UI works.

Step 7 - Confirm the pinned versions and file modes

Check the pinned package versions, the PHP version and the file modes:

php -r '$l=json_decode(file_get_contents("/var/www/dkan/composer.lock"),true); foreach(array_merge($l["packages"],$l["packages-dev"]??[]) as $p){ if(in_array($p["name"],["getdkan/dkan","drupal/core","drush/drush"])) printf("%-18s %s\n",$p["name"],$p["version"]); }'
sudo -u www-data /usr/local/bin/drush status --field=drupal-version
php -v | head -1 sudo stat -c 'settings.php mode=%a owner=%U:%G' /var/www/dkan/docroot/sites/default/settings.php
sudo stat -c 'private dir mode=%a owner=%U:%G' /var/www/dkan/private

Expected: getdkan/dkan 2.23.2, drupal/core 11.4.4, drush/drush 13.7.6, Drupal 11.4.4, PHP 8.3, then settings.php mode=440 owner=root:www-data and private dir mode=750 owner=www-data:www-data.

settings.php holds the database password, so it is mode 0440 owned root:www-data rather than the 0444 Drupal suggests. Apache can still read it, but it is not world-readable, and it is not writable by the web user either.

The private file directory sits outside the document root, so a private file can never be fetched by URL even if a rewrite rule were lost.

Step 8 - Visit the portal

Browse to http://<vm-public-ip>/. The catalog lives at /dataset/search, and the administration UI is under /admin/dkan.

The image does not bake an address into the site configuration. Drupal derives the base URL from the incoming request, so the portal answers correctly on the VM's public address, its private address, or a custom domain you point at it later, with no reconfiguration. You can see that for yourself:

PRIVATE_IP=$(hostname -I | awk '{print $1}')
curl -s -o /dev/null -w 'arbitrary domain -> %{http_code}\n' -H "Host: data.example.com" http://127.0.0.1/user/login
curl -s -o /dev/null -w 'private address -> %{http_code}\n' "http://${PRIVATE_IP}/user/login" grep -c '\$base_url' /var/www/dkan/docroot/sites/default/settings.php || true

Expected: 200 for both, and 0 occurrences of a hard-coded $base_url.

Step 9 - Sign in as the administrator

Browse to http://<vm-public-ip>/user/login, and sign in with the username SecureAnyCloud and the password from the credentials file.

Username: sac Password: <DKAN_ADMIN_PASSWORD>

Step 10 - Load the sample catalog

A brand-new portal is deliberately empty, with no datasets, no accounts other than yours and no cached metadata. DKAN ships a self-contained sample catalog so you can see the whole pipeline work before you publish anything of your own. It harvests from files inside the module on this VM, so it needs no internet access.

Load the sample catalog:

drushw() {
sudo -u www-data /usr/local/bin/drush "$@"; } drushw en sample_content -y >/dev/null 2>&1 drushw dkan:sample-content:create >/dev/null 2>&1 for i in $(seq 1 20); do drushw cron >/dev/null 2>&1 || true
DATASETS=$(curl -s http://127.0.0.1/api/1/metastore/schemas/dataset/items | php -r '$d=json_decode(stream_get_contents(STDIN),true); echo is_array($d)?count($d):0;')
TABLES=$(drushw sql:query "SELECT COUNT(*) FROM information_schema.tables WHERE table_schema='dkan' AND table_name LIKE 'datastore%'" 2>/dev/null | awk '/^[0-9]+$/{v=$1} END{print v+0}')
echo "cron pass ${i}: ${DATASETS:-0} dataset(s) published, ${TABLES:-0} datastore table(s)" [ "${DATASETS:-0}" -ge 1 ] && [ "${TABLES:-0}" -ge 1 ] && break sleep 5 done [ "${DATASETS:-0}" -ge 1 ] || {
echo "no dataset reached the metastore"; exit 1; } [ "${TABLES:-0}" -ge 1 ] || {
echo "no datastore table was created"; exit 1; }
echo "sample catalog loaded: ${DATASETS} dataset(s), ${TABLES} datastore table(s)"

Expected: the dataset and datastore-table counts climb, and the loop stops once both are non-zero, ending with a sample catalog loaded: ... line. Cron is what advances DKAN's harvest and import queues, which is why the image runs it every five minutes from dkan-cron.timer. The loop above just hurries it along.

Two details in that loop are deliberate:

• The table count comes from drush sql:query, which authenticates with the credentials in settings.php. A plain sudo mysql will not work, because first boot rotates the MySQL root password out of the image.
• Drush appends a trailing blank line to its output, so a | tail -1 would read that blank line and hand you an empty string. The awk filter picks the last numeric line instead.

Step 11 - Review the harvest

Choose DKAN → Harvests in the administration menu, or browse to /admin/dkan/harvest. The sample catalog appears as a harvest plan with its extract status and the number of datasets it produced.

This is the same screen you use for a real source. Register another portal's DCAT endpoint as a plan, and DKAN pulls its whole catalog on every cron run.

Step 12 - Review the datasets

Choose DKAN → Datasets, or browse to /admin/dkan/datasets. Every dataset is listed with its data type, publication status and moderation state. Choose + Add new dataset to start the authoring form.

Step 13 - Publish your own dataset

Choose DKAN → Datasets → + Add new dataset. The form is generated from the DCAT metadata schema, so the fields you are asked for are the fields a catalog aggregator expects: title, description, public access level, publication frequency, keywords, publisher, contact point, licence, and one or more distributions.

A distribution is either an uploaded file or a link to one. Upload a CSV, and DKAN loads it into the datastore on the next cron run.

Step 14 - Watch the datastore import

Choose DKAN → Datastore Import Status, or browse to /admin/dkan/datastore/status. Each resource shows its fetch and store stages. done in both columns means the file's contents are in the database and queryable through the API.

Because datastore_mysql_import is enabled, CSV distributions are loaded with MySQL's native LOAD DATA in a single step rather than in row batches. This is what makes multi-megabyte open data files practical on a small VM.

Step 15 - Query the data over the API

This is what separates a data portal from a file share. Pick a tabular distribution and query it, with no download and no parsing:

DIST=$(curl -s 'http://127.0.0.1/api/1/metastore/schemas/dataset/items?show-reference-ids' | php -r '$d=json_decode(stream_get_contents(STDIN),true); foreach($d as $ds){ foreach(($ds["distribution"]??[]) as $x){ if(($x["data"]["format"]??"")==="csv"){ echo $x["identifier"]; exit; } } }')
echo "distribution: ${DIST}" curl -s "http://127.0.0.1/api/1/datastore/query/${DIST}?limit=2" | php -r '$d=json_decode(stream_get_contents(STDIN),true); printf("rows in datastore : %d\ncolumns returned : %d\n", $d["count"], count($d["results"][0])); foreach(array_slice($d["results"],0,2) as $i=>$r){ printf("row %d : %s\n", $i+1, substr(implode(" | ", array_slice(array_values($r),0,5)),0,80)); }' curl -s -G --data-urlencode "query=[SELECT * FROM ${DIST}][LIMIT 1]" http://127.0.0.1/api/1/datastore/sql | php -r '$d=json_decode(stream_get_contents(STDIN),true); printf("SQL endpoint rows : %d\nSQL endpoint cols : %s\n", count($d), substr(implode(", ", array_keys($d[0])),0,70));'

Expected: a row count, a column count, two sample rows, then one row back from the SQL endpoint with its column names.

Two things are easy to get wrong:

• Both APIs are keyed by the distribution identifier, which is the identifier field of an entry in a dataset's distribution array. It is not the identifier nested under %Ref:downloadURL, and it is not the underlying datastore_... MySQL table name. Either of those returns a distribution not found error.
• The SQL endpoint's FROM target is that same distribution identifier, wrapped in DKAN's bracket syntax: [SELECT * FROM <distribution-id>][LIMIT 1].

Only tabular distributions get a datastore. A ZIP or PDF distribution correctly reports No datastore storage found.

Step 16 - The rest of the catalog API

Query the dataset list, the metastore schemas and the OpenAPI specification:

curl -s http://127.0.0.1/api/1/metastore/schemas/dataset/items | php -r '$d=json_decode(stream_get_contents(STDIN),true); printf("datasets published: %d\n", count($d)); foreach(array_slice($d,0,3) as $ds){ printf(" - %s\n", substr($ds["title"]??"",0,60)); }'
curl -s http://127.0.0.1/api/1/metastore/schemas | php -r '$d=json_decode(stream_get_contents(STDIN),true); printf("metastore schemas : %s\n", implode(", ", array_keys($d)));'
curl -s -o /dev/null -w 'OpenAPI spec -> HTTP %{http_code}\n' http://127.0.0.1/api/1

GET /api/1 returns the full OpenAPI description of every endpoint this portal exposes. It is the quickest way to explore the rest of the surface: dataset create and update, harvest plan management, datastore import control, and search.

Removing the sample catalog

When you are finished exploring, remove the sample datasets and turn the module off. Your own datasets are untouched. Run:

sudo -u www-data /usr/local/bin/drush dkan:sample-content:remove
sudo -u www-data /usr/local/bin/drush pmu sample_content -y sudo -u www-data /usr/local/bin/drush cache:rebuild

Administration from the command line

Drush is installed as a wrapper that already points at this project's document root, so it works from any directory:

sudo -u www-data /usr/local/bin/drush status | head -12 sudo -u www-data /usr/local/bin/drush pm:list --status=enabled --filter=dkan --field=name

To reset the administrator password, or to add another administrator, run:

sudo -u www-data /usr/local/bin/drush user:password sac 'your-new-password'
sudo -u www-data /usr/local/bin/drush user:create alice --mail=alice@example.com --password='her-password' sudo -u www-data /usr/local/bin/drush user:role:add administrator alice

Useful DKAN commands:

sudo -u www-data /usr/local/bin/drush dkan:dataset-info <dataset-uuid>
sudo -u www-data /usr/local/bin/drush dkan:datastore:import <resource-identifier>
sudo -u www-data /usr/local/bin/drush dkan:datastore:drop <dataset-uuid>
sudo -u www-data /usr/local/bin/drush cron

Using your own domain and HTTPS

The image serves plain HTTP on port 80, so that it works immediately on any address. For a production portal, point your domain at the VM and terminate TLS. The quickest route is Certbot with the Apache plugin:

sudo apt-get update && sudo apt-get install -y certbot python3-certbot-apache
sudo certbot --apache -d data.example.com

Because no address is baked into the site configuration, the portal starts answering on your domain as soon as DNS resolves. There is nothing to reconfigure in Drupal.

For a production site, also tighten the trusted host pattern. The image ships a permissive pattern so the portal works on any address out of the box. Once you have settled on your domain, edit /var/www/dkan/docroot/sites/default/settings.php (it is mode 0440, so make it writable first) and replace the pattern:

$settings['trusted_host_patterns'] = ['^data\.example\.com$'];

Keeping DKAN and Drupal up to date

DKAN and Drupal core are pinned to the versions verified at build time, so the image you certified is the image that runs. When you are ready, apply upstream updates with Composer:

cd /var/www/dkan
sudo -u www-data composer update drupal/core-recommended --with-all-dependencies sudo -u www-data composer update getdkan/dkan --with-all-dependencies sudo -u www-data /usr/local/bin/drush updatedb -y sudo -u www-data /usr/local/bin/drush cache:rebuild

Always take a backup first (see below), and test updates outside production. Moving from DKAN 2.x to 4.x is a documented migration, not a Composer update. Read upstream's upgrade guide before attempting it.

Backup and maintenance

Back up the database, the uploaded and localized resource files, and the private directory:

cd /var/www/dkan
sudo -u www-data /usr/local/bin/drush sql:dump --result-file=/var/backups/dkan-$(date +%F).sql
sudo tar czf /var/backups/dkan-files-$(date +%F).tar.gz \
  /var/www/dkan/docroot/sites/default/files /var/www/dkan/private

Datastore tables are inside the database dump, so a restored dump brings the imported data back with it.

The operating system receives unattended security upgrades. Check for pending updates with this command:

apt-get -s -o APT::Get::Always-Include-Phased-Updates=true dist-upgrade | grep -c '^Inst ' || true

Sizing and tuning notes

The image is deliberately tuned to fit Standard_B2s (2 vCPU, 4 GB). Apache mpm_prefork is capped at 12 workers. PHP has a 512 MB web limit with no CLI limit, so Composer and datastore imports are not starved. MySQL runs a 256 MB InnoDB buffer pool with performance_schema and the X protocol both off.

A fully loaded portal with the sample catalog imported uses under 1 GB with no swap at all. Azure manages swap on the ephemeral resource disk, and nothing is baked into the OS disk.

If you plan large imports or a busy API, move to a larger VM size first. Then raise MaxRequestWorkers in /etc/apache2/mods-available/mpm_prefork.conf and innodb_buffer_pool_size in /etc/mysql/mysql.conf.d/zz-sac-dkan.cnf.

Licensing

DKAN is distributed under the GNU General Public License, version 2 (GPL-2.0), and Drupal core under GPL-2.0-or-later. The licence texts ship in the image at /var/www/dkan/docroot/modules/contrib/dkan/LICENSE.txt and /var/www/dkan/docroot/core/LICENSE.txt. The bundled select2 JavaScript library is MIT licensed.

SecureAnyCloud charges only for the packaging, maintenance and support of the image. The software itself remains under its open-source licence.

Support

Every SecureAnyCloud image includes 24/7 support. Contact sales@secureanycloud.com with your Azure subscription ID and the VM name.

Deploy on Azure

Launch DKAN Open Data Portal on Ubuntu 24.04 LTS on Azure with 24/7 support from SecureAnyCloud.

View this image

Need Help?

Our support team is available 24/7.

sales@secureanycloud.com