Datashare
Overview
Datashare is an open source, self-hosted search engine for documents, built by the International Consortium of Investigative Journalists (ICIJ), the team behind the Panama Papers and other large cross-border investigations. Point it at a collection of files and it extracts the text (including OCR of scanned pages and images, via Apache Tika and Tesseract), indexes everything into Elasticsearch, and gives you a fast web interface to search the full text and explore your documents. This image runs it on Ubuntu 24.04 on Azure. Datashare's local mode has no login of its own, so the image is locked down by default: the interface is reachable only over HTTPS, with the per-VM credential in /root/datashare-credentials.txt.
What you get:
- Datashare 21.17.0 in local mode (a single analyst, with no external services beyond Elasticsearch), from the official
icij/datasharecontainer pinned by digest. Apache Tika and Tesseract OCR are bundled for text extraction - Elasticsearch 8.19.8 as the search index, bound to loopback. Docker Compose orchestrates it together with the
icij/datasharecontainer. Datashare uses an in-memory task queue and a SQLite metadata store, so there is no Redis and no PostgreSQL to operate - nginx as the only public listener, serving the web interface and REST API over TLS behind a per-VM HTTP Basic Auth gate, with an unauthenticated
/healthzprobe on port 80. The Datashare backend (127.0.0.1:8080) and Elasticsearch (127.0.0.1:9200) are bound to loopback - A per-VM web password and self-signed TLS certificate. On the first boot of every VM, a one-shot service generates them, writes them to
/root/datashare-credentials.txt(mode 0600, root only), and only then starts nginx, so the backend is not served until they exist - A small fictional sample investigation ("
ZephyrMaritime Holdings"), indexed into Elasticsearch at build time, so searching for Zephyr returns real results the first time you open the app. Replace it with your own documents to get started - Unattended security upgrades that keep the host patched on your running VM
Prerequisites
- An active Azure subscription, an SSH public key, and a VNet and subnet in the target region
- Standard_B2ms (2 vCPU, 8 GiB) is the recommended VM size. Elasticsearch and Datashare are both JVM services, so 8 GiB is the sensible floor. Larger document collections benefit from more memory and a larger OS disk.
- Inbound 443/tcp (HTTPS) open to the operators who need the search UI. Port 80 is used only for an unauthenticated health probe and to redirect to HTTPS.
Deploy the image
Create the VM from the SecureAnyCloud Datashare offer. Replace the resource group, image URN and SSH details with your own:
az group create --name datashare-rg --location eastus
# Accept the Marketplace terms once per subscription (required before the first CLI deploy)
az vm image terms accept --urn sac1647283583153:datashare-ubuntu-24-04:default:latest
az vm create \
--resource-group datashare-rg \
--name datashare-vm \
--image sac1647283583153:datashare-ubuntu-24-04:default:latest \
--size Standard_B2ms \
--admin-username azureuser \
--ssh-key-values ~/.ssh/id_rsa.pub \
--public-ip-sku Standard
az vm open-port --resource-group datashare-rg --name datashare-vm --port 443 --priority 1001Wait a minute after first boot so that the VM can generate its per-VM credentials, then sign in.
Confirm the services are running
SSH to the VM and confirm that the Datashare stack and nginx are active. The Datashare backend and Elasticsearch run as containers, and nginx runs on the host as the only public listener.
sudo systemctl is-active datashare.service nginx.service docker.serviceAll three report active. Both containers, Datashare and Elasticsearch, are up, and Elasticsearch reports healthy:
sudo docker compose -f /opt/datashare/docker-compose.yml psCheck the search index
Elasticsearch holds the document index. Confirm that the cluster is healthy and the sample project is populated:
curl -s http://127.0.0.1:9200/_cluster/health | python3 -m json.tool
curl -s "http://127.0.0.1:9200/_cat/indices?v"A single-node cluster reports yellow, because the replica shard has nowhere to go on one node. This is expected and does not affect search.
The local-datashare index holds the shipped sample documents.
Retrieve your per-VM credentials
Every VM writes its own unique web password on first boot. Read it as root:
sudo cat /root/datashare-credentials.txtThe file records the URL, the username (admin) and the generated password. The password shown here is redacted. Yours is a unique 24-character value.
Verify the gate and run a search round trip
nginx fronts the web interface and REST API over TLS behind HTTP Basic Auth. An anonymous request is rejected with 401, and the per-VM credential returns 200.
The full-text search API, which proxies to Elasticsearch, returns the shipped sample documents for a query on Zephyr:
# Anonymous request is rejected by the per-VM Basic Auth gate curl -sk -o /dev/null -w 'anonymous: HTTP %{http_code}\n' https://127.0.0.1/
# The unauthenticated health probe is available for load balancers curl -sk -o /dev/null -w 'healthz: HTTP %{http_code}\n' https://127.0.0.1/healthz
# Authenticated full-text search returns the shipped sample hits
sudo bash -c 'PW=$(grep "^DATASHARE_PASSWORD=" /root/datashare-credentials.txt | cut -d= -f2-); curl -sk -u "admin:$PW" "https://127.0.0.1/api/index/search/local-datashare/_count?q=content:Zephyr"'The anonymous request returns 401, the health probe returns 200, and the authenticated search reports three matching documents.
Sign in to the Datashare web interface
Open https://<your-server-ip>/ in a browser. The certificate is self-signed, so accept the browser warning. For production, place your own domain and a CA-signed certificate in front of nginx.
When prompted, sign in with the username admin and the per-VM password from /root/datashare-credentials.txt.
Datashare opens on the projects view. The pre-indexed sample project ("Default") is listed with its document count and a Search button.
Search your documents
Enter a query, for example Zephyr, to search the full text of every indexed document. Datashare returns each matching document with the search term highlighted in a snippet of its extracted text.
Select a result to open the document. Datashare shows the full extracted text with your search term highlighted, next to tabs for the original view, the metadata and any extracted entities.
Add your own documents
To analyse your own collection, copy your files onto the VM under /var/lib/datashare/data, the Datashare data volume. Then select Tasks > Documents > Add documents in the web interface.
Choose the folder to analyse and whether to run OCR on scanned images and PDFs. Datashare then scans, extracts and indexes the files into Elasticsearch.
How the appliance is secured
- Loopback binding. The Datashare backend (
127.0.0.1:8080) and Elasticsearch (127.0.0.1:9200) are published on loopback only. nginx on port 443 is the sole public listener, so neither the raw backend nor the search index is reachable from the network. - Per-VM credential and certificate. On first boot, a one-shot service generates a unique HTTP Basic Auth password (bcrypt-hashed in
/etc/nginx/.datashare.htpasswd) and regenerates the self-signed TLS certificate, so no two VMs share a secret. The plaintext password is written only to/root/datashare-credentials.txt(0600). - Fail-closed startup. nginx is gated on a bootstrap-ready marker that first boot writes only after the credential and certificate are in place. There is no window in which the search interface is served without a password.
- Production hardening. For internet-facing use, place your own domain and a CA-signed TLS certificate in front of nginx, and restrict inbound 443 to the operators who need access. You can enable Datashare's named-entity (NER) extraction by syncing its NLP models. The core OCR and full-text search work without it.
Support
Every SecureAnyCloud deployment is backed by 24/7 expert support for the image itself. Datashare is an open source project from the ICIJ, so raise product questions and feature requests with the upstream community.
Deploy on Azure
Launch Datashare 21.17.0 on Ubuntu 24.04 LTS on Azure with 24/7 support from SecureAnyCloud.
View this image
Need Help?
Our support team is available 24/7.
sales@secureanycloud.com