Evidently 0.7.23

In the AWS Marketplace listing, choose Continue to Subscribe, then Continue to Configuration. Pick Evidently 0.7.23 on Ubuntu 24.04 as the software version, select your Region, and choose Launch through EC2.

Choose m5.large, your key pair and a security group as described above. Keep the 20 GiB data volume the AMI defines. You can grow it later.

To launch with the AWS CLI instead, replace the values in angle brackets:

aws ec2 run-instances --region <region> --image-id <ami-id-from-the-listing> \
  --instance-type m5.large --key-name <your-key-pair> \
  --security-group-ids <your-security-group> --subnet-id <your-subnet> \
  --metadata-options HttpTokens=required,HttpEndpoint=enabled \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=evidently}]'

First boot takes about a minute. It generates the instance's password and certificate, creates the demo project, verifies the access controls, and only then lets nginx start.

Connect over SSH as the default user for this version:

Version SSH user

Evidently 0.7.23 on Ubuntu 24.04 ubuntu

ssh -i <your-key.pem> ubuntu@<public-ip>

Check the service itself, not just the unit state. The following commands check the services, then the health endpoint and Evidently's version through nginx on HTTPS:

systemctl is-active evidently-egress-guard evidently-firstboot evidently nginx
H=$(sudo sed -n 's#^EVIDENTLY_URL=https://\([^/]*\)/.*#\1#p' /root/evidently-credentials.txt)
PW=$(sudo sed -n 's/^EVIDENTLY_PASSWORD=//p' /root/evidently-credentials.txt) EV="curl -s --cacert /etc/nginx/ssl/evidently.crt --resolve $H:443:127.0.0.1"
C=$($EV -o /dev/null -w '%{http_code}' "https://$H/healthz")
V=$($EV -K <(printf 'user = "admin:%s"\n' "$PW") "https://$H/api/version") echo "GET /healthz -> HTTP $C" echo "GET /api/version -> $V" [ "$C" = 200 ] && echo "$V" | grep -q '"version":"0.7.23"' && echo EVIDENTLY_OK

Expected output:

active active active active GET /healthz -> HTTP 200 GET /api/version -> {"application":"Evidently UI","version":"0.7.23","commit":"74f4243e"} EVIDENTLY_OK

Read the credentials file:

sudo cat /root/evidently-credentials.txt

The file is readable by root only. Your values differ from the example below:

# Evidently - this instance's credentials (sac, minted at first boot). Keep this file secret. EVIDENTLY_URL=https://ec2-<public-ip-dashed>.compute-1.amazonaws.com/ EVIDENTLY_USER=admin EVIDENTLY_PASSWORD=<24 characters generated on your instance> EVIDENTLY_TLS_SHA256_FINGERPRINT=<unique to your instance> # Change the password:
sudo evidently-set-password

EVIDENTLY_URL uses the instance's public DNS name from EC2 instance metadata, and the certificate also names the public and private IP addresses. If you stop and start the instance and its public address changes, the next boot updates the name, the certificate and this file. To serve your own DNS name, see Step 10.

Open EVIDENTLY_URL in your browser. The certificate is self-signed for your instance, so the browser warns once. Compare the fingerprint it shows with EVIDENTLY_TLS_SHA256_FINGERPRINT before you accept it.

The browser then asks for a user name and password. Enter admin and EVIDENTLY_PASSWORD. You land on the project list with the sac-demo project.

Open sac-demo and select Reports. The demo snapshot is listed, timestamped from your instance's first boot.

Open the report. The demo compares a synthetic reference sample with a deliberately shifted current sample. The data drift section shows drift in each column with the statistical test, the drift score and the two distributions. Further down, the data summary shows per-column statistics.

nginx is the only way in. Every request except /healthz needs the password, a cross-site write is refused even with the password, and only nginx and root may open Evidently's own port. Verify this with the following checks:

H=$(sudo sed -n 's#^EVIDENTLY_URL=https://\([^/]*\)/.*#\1#p' /root/evidently-credentials.txt)
PW=$(sudo sed -n 's/^EVIDENTLY_PASSWORD=//p' /root/evidently-credentials.txt) EV="curl -s --cacert /etc/nginx/ssl/evidently.crt --resolve $H:443:127.0.0.1 -o /dev/null -w %{http_code}"
A=$($EV "https://$H/api/projects")
W=$($EV -K <(printf 'user = "admin:%s"\n' "wrong-password-$RANDOM") "https://$H/api/projects")
G=$($EV -K <(printf 'user = "admin:%s"\n' "$PW") "https://$H/api/projects")
X=$($EV -K <(printf 'user = "admin:%s"\n' "$PW") -X POST -H 'Origin: https://evil.example' \
  -H 'Content-Type: application/json' -d '{"name":"cross-site"}' "https://$H/api/projects")
N=$(sudo -u nobody curl -s -o /dev/null -m 4 -w '%{http_code}' http://127.0.0.1:8000/api/version || true)
echo "no password -> HTTP $A, wrong password -> HTTP $W, your password -> HTTP $G" echo "cross site POST with your password -> HTTP $X" echo "another user to Evidently's port 8000 -> HTTP $N"
sudo ss -Htln | awk '{print $4}' | sort -u | tr '\n' ' '; echo [ "$A" = 401 ] && [ "$W" = 401 ] && [ "$G" = 200 ] && [ "$X" = 403 ] && [ "$N" = 000 ] && echo ACCESS_OK

Expected output:

no password -> HTTP 401, wrong password -> HTTP 401, your password -> HTTP 200 cross site POST with your password -> HTTP 403 another user to Evidently's port 8000 -> HTTP 000 0.0.0.0:22 0.0.0.0:443 0.0.0.0:80 127.0.0.1:8000 127.0.0.53%lo:53 127.0.0.54:53 [::]:22 [::]:443 [::]:80 ACCESS_OK

HTTP 000 means the connection was refused.

The cross-site guard looks only at requests that change something, and only when they carry an Origin or Referer for another site. The Evidently SDK and curl send neither, and the dashboard itself sends its own address.

Requests are also limited to 3,000 a minute per client, with a burst of 1,000. An IPv4 client is counted by its address and an IPv6 client by its /64 network. A client over the limit gets HTTP 429.

nginx checks the password with bcrypt, so wrong guesses are slow, and the password is 24 random letters and digits. If the password file is ever missing, unreadable or malformed, nginx answers 401 or 500 and does not start again until it is fixed. There is no state in which the dashboard is served without the password.

Reports are computed with the Evidently Python SDK, which is installed in /opt/evidently/venv, and stored as snapshots in a project. The simplest way to add one from your own pipeline is to send it to the service over HTTPS with the SDK's remote workspace, using the URL and password from Step 4.

On the instance itself, this example creates a project, computes a data drift report over two small dataframes and uploads it:

PW=$(sudo sed -n 's/^EVIDENTLY_PASSWORD=//p' /root/evidently-credentials.txt)
H=$(sudo sed -n 's#^EVIDENTLY_URL=https://\([^/]*\)/.*#\1#p' /root/evidently-credentials.txt)
cat > /tmp/evidently-example.py <<'PYEOF' import os import pandas as pd from evidently import DataDefinition, Dataset, Report from evidently.presets import DataDriftPreset from evidently.ui.workspace import RemoteWorkspace import evidently.legacy.ui.workspace.remote as remote import requests
# The SDK's HTTP client ignores REQUESTS_CA_BUNDLE, so trust the instance's own
# certificate explicitly. With a certificate from a public CA (Step 10), drop this class. class TrustInstanceCert(requests.Session): def __init__(self): super().__init__() self.verify = os.environ["EV_CA"] remote.Session = TrustInstanceCert ws = RemoteWorkspace("https://admin:%s@%s/" % (os.environ["EV_PW"], os.environ["EV_HOST"])) project = ws.create_project("my-model", description="example from the sac guide") ref = pd.DataFrame({"feature_a": range(100), "feature_b": [x % 7 for x in range(100)]}) cur = pd.DataFrame({"feature_a": range(50, 150), "feature_b": [x % 3 for x in range(100)]}) schema = DataDefinition(numerical_columns=["feature_a", "feature_b"]) run = Report([DataDriftPreset()]).run(Dataset.from_pandas(cur, data_definition=schema), Dataset.from_pandas(ref, data_definition=schema)) ws.add_run(project.id, run) print(project.id) PYEOF
P=$(EV_PW="$PW" EV_HOST="$H" EV_CA=/etc/nginx/ssl/evidently.crt DO_NOT_TRACK=1 \
  /opt/evidently/venv/bin/python /tmp/evidently-example.py) rm -f /tmp/evidently-example.py
N=$(curl -s --cacert /etc/nginx/ssl/evidently.crt --resolve $H:443:127.0.0.1 -K <(printf 'user = "admin:%s"\n' "$PW") \
  "https://$H/api/projects/$P/reports" | python3 -c 'import json,sys; print(len(json.load(sys.stdin)))') echo "project my-model ($P) holds $N report(s)" [ "$N" -ge 1 ] && echo SDK_OK

Expected output:

project my-model (01a1c264-c267-77d1-a9fc-f1495f874823) holds 1 report(s) SDK_OK

The project id is different on your instance.

Refresh the dashboard. The new project appears with the report under Reports.

To send reports from another machine, install the same version of the SDK (pip install evidently==0.7.23), point the URL at your instance and trust its certificate (or install your own, Step 10). Add a snapshot on each run of your pipeline to build a monitoring history.

Open a project's Datasets tab to upload a CSV or Parquet file for later evaluation. Evidently 0.7.23 checks upload file names: a name that carries a path (../, an absolute path or a backslash) is refused, so an upload cannot write outside the workspace. nginx accepts uploads up to 200 MB.

Deleting a dataset hides it but does not erase it. Evidently 0.7.23 marks a deleted dataset as deleted and keeps its file in the workspace, under /var/lib/evidently/workspace/<project-id>/datasets/<dataset-id>/. If a dataset held data you must erase, delete the dataset in the dashboard and then remove that directory as root.

Evidently needs the network only for features you configure, for example a remote S3-compatible store. The egress fence refuses every private and VPC address to the evidently account. To allow one, list its CIDR in the allow file and reload the fence:

echo '10.0.5.25/32' | sudo tee -a /etc/sac/evidently-egress-allow
sudo systemctl reload evidently-egress-guard

Metadata, link-local, loopback and multicast ranges are always refused and cannot be listed. A line that is not a CIDR, or a range that is too broad, is rejected and the fence stays as it was.

Point a DNS record at the instance, then set the name Evidently is served as. The command below creates a new self-signed certificate for the name, updates nginx, the cross-site guard and the URL and fingerprint in /root/evidently-credentials.txt, and keeps the name across reboots:

sudo evidently-set-hostname <your-domain>

To use a certificate from your own or a public certificate authority instead, install it over the generated pair and reload nginx. Running evidently-set-hostname again replaces it with a self-signed one.

sudo install -m 0644 <your-domain>.crt /etc/nginx/ssl/evidently.crt
sudo install -m 0600 <your-domain>.key /etc/nginx/ssl/evidently.key
sudo nginx -t && sudo systemctl reload nginx

Run the following command:

sudo evidently-set-password

This generates a new 24-character password, replaces the bcrypt entry nginx checks, updates /root/evidently-credentials.txt and reloads nginx. It does not print the password, so read it from the file. To choose your own (24 to 128 letters and digits), pass it on standard input so it never appears on a command line:

sudo evidently-set-password --stdin < <a-file-holding-your-password>

List the egress fence rules in force. Each rule carries a comment naming its purpose:

sudo nft list table inet sac_evidently_egress | grep -oE 'comment "sac-[a-z0-9]+"' | sort -u

Expected output:

comment "sac-allow" comment "sac-allow6" comment "sac-fence" comment "sac-ipv6" comment "sac-local8000" comment "sac-loopback" comment "sac-metadata" comment "sac-metadata6" comment "sac-multicast" comment "sac-private" comment "sac-reply" comment "sac-vpc"

Next, confirm that the metadata service and your VPC are refused to Evidently, while root on the same instance reaches them (the control). The instance's own SSH port stands in for any service in your VPC:

I=$(sudo curl -s -m 4 -o /dev/null -w '%{http_code}' -X PUT -H 'X-aws-ec2-metadata-token-ttl-seconds: 60' http://169.254.169.254/latest/api/token)
N=$(sudo setpriv --reuid=evidently --regid=evidently --clear-groups curl -s -m 4 -o /dev/null -w '%{http_code}' -X PUT -H 'X-aws-ec2-metadata-token-ttl-seconds: 60' http://169.254.169.254/latest/api/token || true)
P=$(hostname -I | awk '{print $1}')
VR=$(sudo timeout 4 bash -c "exec 3<>/dev/tcp/$P/22" 2>/dev/null && echo OPEN || echo REFUSED)
VN=$(sudo setpriv --reuid=evidently --regid=evidently --clear-groups timeout 4 bash -c "exec 3<>/dev/tcp/$P/22" 2>/dev/null && echo OPEN || echo REFUSED)
echo "metadata service: root HTTP $I, evidently HTTP $N"
echo "own address port 22: root $VR, evidently $VN" [ "$I" = 200 ] && [ "$N" = 000 ] && [ "$VR" = OPEN ] && [ "$VN" = REFUSED ] && echo EGRESS_FENCE_OK

Expected output:

metadata service: root HTTP 200, evidently HTTP 000 own address port 22: root OPEN, evidently REFUSED EGRESS_FENCE_OK

Evidently's server does not fetch URLs you give it. None of its API fields takes a URL, and the only network calls in the server package are its telemetry (switched off) and a demo data download that the service never runs. The fence is there in case that ever changes.

Next, check request hygiene. nginx judges the raw request line before it decodes or normalizes the path, so encoded slashes, dot segments, backslashes and doubled slashes are refused. The --path-as-is option sends the path exactly as written:

H=$(sudo sed -n 's#^EVIDENTLY_URL=https://\([^/]*\)/.*#\1#p' /root/evidently-credentials.txt) EV="curl -s --cacert /etc/nginx/ssl/evidently.crt --resolve $H:443:127.0.0.1 --path-as-is -o /dev/null -w %{http_code}" OK=1 for p in '/healthz/../api/projects' '/api%2fprojects' '/api//projects' '/api/%2e%2e/api/projects' '/api%5cprojects'; do c=$($EV "https://$H$p"); echo "GET $p -> HTTP $c" [ "$c" = 400 ] || OK=0 done c=$($EV "https://$H/healthz"); echo "GET /healthz -> HTTP $c" [ "$OK" = 1 ] && [ "$c" = 200 ] && echo RAW_LINE_OK

Expected output:

GET /healthz/../api/projects -> HTTP 400 GET /api%2fprojects -> HTTP 400 GET /api//projects -> HTTP 400 GET /api/%2e%2e/api/projects -> HTTP 400 GET /api%5cprojects -> HTTP 400 GET /healthz -> HTTP 200 RAW_LINE_OK

Finally, confirm the service hardening. Telemetry is off in the running service, and the service runs as an unprivileged account in a systemd sandbox that is blocked from the metadata service. The password and the TLS key are readable by root only, and the password file nginx checks holds only a bcrypt hash:

sudo cat /proc/$(systemctl show -p MainPID --value evidently)/environ | tr '\0' '\n' | grep -E '^(DO_NOT_TRACK|EVIDENTLY_)' | sort systemctl show evidently -p User -p NoNewPrivileges -p ProtectSystem -p IPAddressDeny
sudo stat -c '%U:%G %a %n' /root/evidently-credentials.txt /etc/nginx/.evidently.htpasswd /etc/nginx/ssl/evidently.key /var/lib/evidently
sudo cut -c1-11 /etc/nginx/.evidently.htpasswd

Expected output:

DO_NOT_TRACK=1 EVIDENTLY_DISABLE_TELEMETRY=1 IPAddressDeny=240.0.0.0/4 224.0.0.0/4 169.254.169.254/32 198.18.0.0/15 fd00:ec2::254/128 192.0.0.0/24 168.63.129.16/32 User=evidently ProtectSystem=strict NoNewPrivileges=yes root:root 600 /root/evidently-credentials.txt root:www-data 640 /etc/nginx/.evidently.htpasswd root:root 600 /etc/nginx/ssl/evidently.key evidently:evidently 750 /var/lib/evidently admin:$2y$1

List the installed versions, build provenance and dependency licenses:

/opt/evidently/venv/bin/python -c 'import importlib.metadata as m; print("evidently", m.version("evidently"), "| pandas", m.version("pandas"))' ls /usr/share/doc/evidently /usr/share/evidently cat /usr/share/doc/evidently/BUILD-PROVENANCE.txt tail -1 /usr/share/evidently/dependency-licences-python.txt find /opt/evidently/venv -type f \( -iname '*.jar' -o -iname '*.war' -o -iname '*.ear' \) | wc -l

Expected output:

evidently 0.7.23 | pandas 2.3.3 /usr/share/doc/evidently: BUILD-PROVENANCE.txt LICENSE requirements.lock.txt /usr/share/evidently: dependency-licences-python.txt evidently 0.7.23 (sac build) wheel: evidently-0.7.23-py3-none-any.whl sha256 dbba5537a644f3401b8a393008cd126e304bcf939a070a7c4399bcc5b97061f6 (PyPI) tag commit: 74f4243e4e20265e2ae61beb5a613f436607a6a3 (Apache-2.0 LICENSE from the tag) dependencies: requirements.lock.txt (sha256 880e6eaa772b0609f96b11f9b3d18b402862ab0eddbe20caacae161444ce640e): uv pip compile evidently==0.7.23 -c 'pandas<3' --python-version 3.12 --python-platform x86_64-manylinux_2_39 --exclude-newer 2026-11-08T00:00:00Z --generate-hashes, installed with uv 0.13.7 pip install --require-hashes --no-deps --only-binary :all: (no compiler on the image) telemetry: off (DO_NOT_TRACK=1, EVIDENTLY_DISABLE_TELEMETRY=1) LICENCE-SCAN OK (80 distributions, unknown=[]) 0

Evidently is licensed under the Apache License 2.0, and its full text ships in /usr/share/doc/evidently/LICENSE. requirements.lock.txt is the exact dependency set, with every package pinned by version and SHA-256. dependency-licences-python.txt lists the license of each installed package, and all are permissive open source licenses (MIT, BSD, Apache 2.0, PSF, ISC, MPL 2.0 and similar).

The dashboard's web interface ships inside the Evidently wheel, built by the Evidently project from its own source. Its JavaScript dependencies are MIT, ISC, BSD and Apache 2.0 licensed. There are no Java archives on the image.

The dependency set was resolved with one constraint, pandas<3, which is inside Evidently's own supported range. pandas 3 removed a function that Evidently 0.7.23 uses, so the dashboard would not start with it.