Chroma 1.5.9

In the AWS Marketplace listing, choose Continue to Subscribe, then Continue to Configuration. Pick Chroma 1.5.9 on Ubuntu 24.04 as the software version and select your Region. Then choose Launch through EC2.

Choose m5.large or larger, your key pair, and a security group as described above. Chroma keeps the indexes of the collections in use in memory, so size the instance's memory for your data.

The AMI brings two EBS volumes: the root volume and a 30 GiB data volume for /var/lib/chroma. Enlarge the data volume at launch if you expect a large dataset.

With the AWS CLI (replace the values in angle brackets):

aws ec2 run-instances --region <region> --image-id <ami-id-from-the-listing> \
  --instance-type m5.large --key-name <your-key-pair> \
  --security-group-ids <your-security-group> --subnet-id <your-subnet> \
  --metadata-options HttpTokens=required,HttpEndpoint=enabled \
  --block-device-mappings 'DeviceName=/dev/sdf,Ebs={VolumeSize=100,VolumeType=gp3}' \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=chroma}]'

Version SSH user

Chroma 1.5.9 on Ubuntu 24.04 ubuntu

ssh -i <your-key.pem> ubuntu@<public-ip>

First boot generates the certificate and the API token, starts Chroma on loopback, opens nginx, and runs an authenticated check before it finishes (under a minute):

systemctl is-active chroma nginx chroma-firstboot chroma-port-guard
sudo test -f /var/lib/sac/data/chroma-firstboot.done && echo FIRSTBOOT_DONE

active active active active FIRSTBOOT_DONE

Chroma listens on 127.0.0.1:8000 only. nginx (80, 443) and SSH (22) are the only services reachable from the network:

sudo ss -lntpH | grep -v '127.0.0.5' | awk '{split($6, p, "\""); print $4, p[2]}' | sort -u

0.0.0.0:22 sshd 0.0.0.0:443 nginx 0.0.0.0:80 nginx 127.0.0.1:8000 chroma [::]:22 sshd [::]:443 nginx [::]:80 nginx

The chroma process is Chroma's Rust server, started by the Python package's chroma run command. From another machine, port 8000 does not answer.

sudo grep -E '^CHROMA_(URL|HOST|PORT|CA_CERT)=' /root/chroma-credentials.txt

CHROMA_URL=https://ec2-<public-ip-dashed>.<region>.compute.amazonaws.com/ CHROMA_HOST=ec2-<public-ip-dashed>.<region>.compute.amazonaws.com CHROMA_PORT=443 CHROMA_CA_CERT=/etc/ssl/chroma/chroma.crt

• CHROMA_URL and CHROMA_HOST use the instance's public DNS name from EC2 instance metadata (the public IP address when the subnet has no public DNS names); the certificate also names the public and private IP addresses. To serve Chroma under your own DNS name, see Step 9.
• The same file holds CHROMA_TOKEN=<CHROMA_TOKEN>, the API token, plus ready-made client settings. Show the whole file with sudo cat /root/chroma-credentials.txt when you need the token. Keep it secret, because it grants full access to every collection.

The certificate is self-signed. Copy /etc/ssl/chroma/chroma.crt to your client machine and pass it to the client (see Step 6), or install your own certificate (Step 9).

The examples read the token from the credentials file and hand it to curl as a header on standard input (-H @-). The token never appears on a command line or in your shell history:

C=/etc/ssl/chroma/chroma.crt; U=https://127.0.0.1/api/v2/version curl -s --cacert $C -o /dev/null -w 'no token: %{http_code}\n' $U curl -s --cacert $C -o /dev/null -w 'wrong token: %{http_code}\n' -H "Authorization: Bearer $(openssl rand -hex 32)" $U curl -s --cacert $C -o /dev/null -w 'example token: %{http_code}\n' -H 'Authorization: Bearer test-token' $U
sudo sed -n 's/^CHROMA_TOKEN=\(.*\)/Authorization: Bearer \1/p' /root/chroma-credentials.txt | curl -s --cacert $C -w ' <- this instance'"'"'s token: %{http_code}\n' -H @- $U curl -s --cacert $C https://127.0.0.1/api/v2/heartbeat; echo

no token: 401 wrong token: 401 example token: 401 "1.0.0" <- this instance's token: 200 {"nanosecond heartbeat":1792947367580985415}

A refused request gets this JSON body from nginx: {"error":"Unauthorized","message":"sac: a valid Chroma token is required (Authorization: Bearer <token> or X-Chroma-Token: <token>)"}.

What is open without the token: only GET (and HEAD) /api/v2/heartbeat. This is Chroma's liveness probe. It returns a timestamp and nothing else, so load balancers and monitors can check the instance.

Every other path and method needs the token, including the API documentation (/docs, /openapi.json), every collection, tenant and database route, /api/v2/version, /api/v2/healthcheck, /api/v2/pre-flight-checks, /api/v2/auth/identity, the reset route and unknown paths. From another machine, use CHROMA_URL instead of https://127.0.0.1.

On your client machine, with the client installed and the instance's certificate copied to chroma.crt, connect with the host and token from Step 4. Chroma's TokenAuthClientProvider sends the token as Authorization: Bearer:

import chromadb from chromadb.config import Settings client = chromadb.HttpClient( host="ec2-<public-ip-dashed>.<region>.compute.amazonaws.com", port=443, ssl=True, settings=Settings( chroma_client_auth_provider="chromadb.auth.token_authn.TokenAuthClientProvider", chroma_client_auth_credentials="<CHROMA_TOKEN>", chroma_server_ssl_verify="chroma.crt", anonymized_telemetry=False, ), ) films = client.create_collection("films", metadata={"hnsw:space": "cosine"}, embedding_function=None) films.add( ids=["carol", "wonder", "pi", "madmax"], embeddings=[[0.9, 0.1, 0.1, 0.0], [0.1, 0.2, 0.3, 0.4], [0.1, 0.8, 0.2, 0.1], [0.2, 0.3, 0.3, 0.4]], documents=["Carol", "Wonder Woman", "Life of Pi", "Mad Max: Fury Road"], metadatas=[{"year": 2015}, {"year": 2017}, {"year": 2012}, {"year": 2015}], ) hits = films.query(query_embeddings=[[0.1, 0.2, 0.3, 0.35]], n_results=2, include=["documents", "distances"]) print(list(zip(hits["documents"][0], [round(d, 4) for d in hits["distances"][0]]))) hits = films.query(query_embeddings=[[0.1, 0.2, 0.3, 0.35]], n_results=1, where={"year": 2015}, include=["documents"]) print(hits["documents"][0])

[('Wonder Woman', 0.0022), ('Mad Max: Fury Road', 0.0185)] ['Mad Max: Fury Road']

embedding_function=None means you supply the embeddings your own model produces (any dimension). This is the usual setup for a server.

If you prefer Chroma to embed text for you, give the collection an embedding function in your client. Embedding functions run in the client, not on this server. Only clients you control should create or read collections, because Chroma's Python client loads the embedding function that a collection's configuration names.

With the wrong token, or without TokenAuthClientProvider, chromadb.HttpClient(...) raises a ValueError carrying nginx's 401 message instead of connecting.

Every operation is also available as JSON over HTTPS under /api/v2/. On the instance:

C=/etc/ssl/chroma/chroma.crt; B=https://127.0.0.1/api/v2/tenants/default_tenant/databases/default_database/collections k() {
sudo sed -n 's/^CHROMA_TOKEN=\(.*\)/Authorization: Bearer \1/p' /root/chroma-credentials.txt | curl -s --cacert "$C" -H @- -H 'Content-Type: application/json' "$@"; echo; }
ID=$(k -X POST -d '{"name":"movies","configuration":{"hnsw":{"space":"l2"}}}' $B | jq -r .id) k -X POST -d '{"ids":["carol","wonder","pi"],"embeddings":[[0.9,0.1,0.1],[0.1,0.2,0.3],[0.1,0.8,0.2]],"documents":["Carol","Wonder Woman","Life of Pi"]}' $B/$ID/add k -X POST -d '{"query_embeddings":[[0.1,0.25,0.3]],"n_results":1,"include":["documents","distances"]}' $B/$ID/query k $B/$ID/count k -X DELETE $B/movies

{} {"ids":[["wonder"]],"embeddings":null,"documents":[["Wonder Woman"]],"uris":null,"metadatas":null,"distances":[[0.0024999997]],"include":["documents","distances"]} 3 {}

The image also ships an end-to-end check that you can run at any time. First boot runs it too. It reads the credentials file itself and talks to Chroma over HTTPS through nginx with the instance's certificate pinned.

The check confirms that port 80 redirects, an unknown host name is dropped, and every route of Chroma's API is refused without the token. It also confirms that wrong, shortened and example tokens are refused. Finally, this instance's token creates a temporary collection whose nearest-neighbor query returns the right vector, and the collection is then deleted:

sudo chroma-verify.sh

port 80 -> 301 https://ec2-<public-ip-dashed>.<region>.compute.amazonaws.com/ ; an unknown Host is dropped on 80 and 443 ; certificate verifies for ec2-<public-ip-dashed>.<region>.compute.amazonaws.com ; GET /api/v2/heartbeat open (POST 403) anonymous: 40 method+route probes (every route of Chroma's API, the API docs, unknown paths) -> 401 from nginx's token gate wrong token (Bearer and X-Chroma-Token), short token, Basic, other header, example tokens (test-token, chr0ma-t0k3n, ...) -> 401 this instance's token: version 1.0.0 (Bearer) + identity (X-Chroma-Token) 200; collection created, 3 vectors added, nearest neighbour 'east', count 3, deleted (Chroma 1.5.9) CHROMA_ROUNDTRIP_OK

The files that matter are locked down:

sudo stat -c '%a %U:%G %n' /root/chroma-credentials.txt /etc/nginx/sac-chroma-auth.conf /etc/ssl/chroma/chroma.key /var/lib/chroma /etc/chroma/config.yaml

600 root:root /root/chroma-credentials.txt 600 root:root /etc/nginx/sac-chroma-auth.conf 600 root:root /etc/ssl/chroma/chroma.key 750 chroma:chroma /var/lib/chroma 644 root:root /etc/chroma/config.yaml

Chroma runs unprivileged in a sandbox that can write only to its data volume. It may exchange packets with loopback addresses only, has telemetry off, and never writes a core dump. nginx cannot reach the instance metadata service either:

systemctl show chroma -p User -p ProtectSystem -p ReadWritePaths -p NoNewPrivileges -p LimitCORE -p IPAddressDeny -p IPAddressAllow -p Environment
systemctl show nginx -p IPAddressDeny

IPAddressAllow=127.0.0.0/8 ::1/128 IPAddressDeny=::/0 0.0.0.0/0 Environment=PYTHONNOUSERSITE=1 HOME=/run/chroma ANONYMIZED_TELEMETRY=False LimitCORE=0 User=chroma ReadWritePaths=/var/lib/chroma ProtectSystem=strict NoNewPrivileges=yes IPAddressDeny=fd00:ec2::254/128 169.254.169.254/32 168.63.129.16/32

Only nginx and root can open Chroma's port on the instance, and the port is dropped on the network interfaces:

curl -s -m 5 -o /dev/null -w "127.0.0.1:8000 as $(id -un) -> %{http_code}\n" http://127.0.0.1:8000/api/v2/heartbeat || true
sudo nft list chain inet sac_chroma input | grep drop

127.0.0.1:8000 as ubuntu -> 000 iifname != "lo" tcp dport 8000 counter packets 0 bytes 0 drop

nginx passes Chroma a fixed host name and the real client address from the connection, never values a client sends. It removes the token headers before proxying:

grep -E 'proxy_set_header (Host|X-Forwarded-For|X-Real-IP|Forwarded|Authorization|X-Chroma-Token) ' /etc/nginx/chroma-proxy-headers.conf

proxy_set_header Host $chroma_host; proxy_set_header Forwarded ""; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header Authorization ""; proxy_set_header X-Chroma-Token "";

The certificate was generated on this instance and names its own addresses:

H=$(sudo sed -n 's/^CHROMA_HOST=//p' /root/chroma-credentials.txt)
echo | openssl s_client -connect 127.0.0.1:443 -servername "$H" 2>/dev/null | openssl x509 -noout -subject -ext subjectAltName

subject=CN = ec2-<public-ip-dashed>.<region>.compute.amazonaws.com, O = sac X509v3 Subject Alternative Name: DNS:ec2-<public-ip-dashed>.<region>.compute.amazonaws.com, IP Address:<public-ip>, IP Address:<private-ip>, DNS:ip-<private-ip-dashed>.ec2.internal, IP Address:127.0.0.1, DNS:localhost

The token reached no log, the journal, shell history, backups, Chroma's data or any file in /etc other than nginx's token file:

sudo chroma-secret-leak-check.sh

LEAKCHECK_OK the token is in no journal, auth.log, syslog, cloud-init or nginx log, Chroma data, shell history, MOTD, /var/backups, /etc (only the nginx token file), /usr/local, /opt/chroma or /tmp

Point a DNS record at the instance, then set the name Chroma is served as. This mints a new self-signed certificate for the name, updates nginx's host rule and the URL in /root/chroma-credentials.txt, reloads nginx, and keeps the name across reboots:

sudo chroma-set-hostname <your-domain>

Without a configured name, the instance re-reads its address from EC2 instance metadata at every boot. A stop and start that changes the public IP address therefore keeps the API reachable.

To use a certificate from your own certificate authority instead, copy it over /etc/ssl/chroma/chroma.crt and chroma.key (key mode 0600) and run sudo systemctl reload nginx. Running chroma-set-hostname again replaces it with a self-signed one.

To generate a new API token (if the old one was ever exposed), run:

sudo chroma-rotate-token

It writes a new token to nginx's token file and to /root/chroma-credentials.txt, reloads nginx, and checks that the old token is refused and the new one works. Update every client that used the old token.

Chroma's anonymized product telemetry is switched off on this image. The Chroma service may exchange packets with loopback addresses only, so it makes no outbound connection at all: no telemetry, no model downloads and no calls to other machines or to the instance metadata service.

Chroma's server needs none of that to store and query embeddings. Chroma's log confirms it at every start:

journalctl -u chroma -o cat --no-pager | grep -m1 -x 'No telemetry is configured.'

No telemetry is configured.

Embedding functions (OpenAI, Cohere, Hugging Face, local models and so on) run in your client. The client calls the model provider and sends Chroma the resulting vectors, so the server's network block does not affect them.

If you need the server itself to connect out (for example, to an OpenTelemetry collector), remove IPAddressDeny=any and IPAddressAllow=localhost from /etc/systemd/system/chroma.service, and take care when you do. The service still refuses an open_telemetry setting in its configuration unless you also change chroma-check-config (Step 12). The instance metadata block (/etc/systemd/system/chroma.service.d/50-sac-imds-deny.conf) should stay.

The data directory is its own EBS volume, mounted by filesystem UUID:

findmnt -n -o SOURCE,FSTYPE,TARGET /var/lib/chroma; df -h /var/lib/chroma | tail -1

/dev/nvme1n1 ext4 /var/lib/chroma /dev/nvme1n1 30G 320K 28G 1% /var/lib/chroma

To grow it, enlarge the volume in the EC2 console (or with aws ec2 modify-volume), then run sudo resize2fs /dev/nvme1n1 on the instance. No restart is needed. If the volume is ever missing, Chroma refuses to start rather than write your data to the root disk.

EBS snapshots of the data volume are the simplest whole-dataset backup. For a consistent copy, stop Chroma (sudo systemctl stop chroma), snapshot the volume, and start it again (sudo systemctl start chroma).

Chroma's server settings are in /etc/chroma/config.yaml:

port: 8000 listen_address: "127.0.0.1" persist_path: "/var/lib/chroma" allow_reset: false

The service runs chroma-check-config before every start. It refuses to start unless Chroma listens on 127.0.0.1:8000 (nginx serves the public port and checks the token), keeps its data on /var/lib/chroma, keeps the reset endpoint off, and has no telemetry exporter or CORS setting. journalctl -u chroma shows the reason.

nginx's site is /etc/nginx/sites-available/sac-chroma (client_max_body_size is 512 MB for large batches). After a change, run sudo systemctl restart chroma or sudo systemctl reload nginx.

/opt/chroma/venv/bin/python -c 'import chromadb; print("chromadb", chromadb.__version__)' nginx -v 2>&1; lsb_release -ds ls /usr/share/doc/sac/chroma/

chromadb 1.5.9 nginx version: nginx/1.24.0 (Ubuntu) Ubuntu 24.04.5 LTS LICENSE RESIDUALS.txt VERSION chromadb_rust_bindings.cyclonedx.json dependency-licences.txt requirements-pinned.txt

VERSION records the release and the checksum of the wheel, requirements-pinned.txt every dependency with its checksum, dependency-licences.txt their licences, and chromadb_rust_bindings.cyclonedx.json the software bill of materials of Chroma's Rust server.

Known advisories. The chromadb package has four published advisories with no fixed release yet (GHSA-f4j7-r4q5-qw2c, GHSA-36p7-vc44-83pf, GHSA-2wm9-hf6c-p5cr, GHSA-xph7-9rjv-w5fr).

All four are in Chroma's older Python server and its Python authorization providers. This image does not run them and cannot run them, because their web framework is not installed. chroma run starts the Rust server, which stores a collection's embedding function settings as data and never executes them. Access is controlled by the single per-instance token in nginx.

Advisories for crates in the Rust server's bill of materials concern code paths it does not use here (outbound TLS and cloud clients and command-line tools, with no outbound connections allowed). RESIDUALS.txt explains each one, and the image build re-checked them against the public advisory databases. SecureAnyCloud moves to the release that fixes them once it is published.