Text Embeddings Inference

Prerequisites

• An AWS account subscribed to this product in the AWS Marketplace
• An EC2 key pair in your target region for SSH access
• A security group that allows inbound TCP 22 (SSH) from your IP and TCP 80 (HTTP) from your users
• A general-purpose instance. m5.large is recommended, because embedding models are light and run comfortably on 2 vCPU. For higher throughput or larger models, use m5.xlarge or m5.2xlarge, or the compute-optimised c5 or c6i family.

Connecting to your instance

OS variant Login user Example

Ubuntu 24.04 ubuntu ssh -i your-key.pem ubuntu@<instance-public-ip>

Step 1 - Launch from the AWS Marketplace console

• Open the product page in the AWS Marketplace. Choose Continue to Subscribe, then Continue to Configuration.
• Select the Text Embeddings Inference 1.9 on Ubuntu 24.04 delivery option and your region, then choose Continue to Launch.
• Choose a general-purpose instance type (m5.large or larger), your VPC and subnet, your key pair and the security group described above, then launch the instance.

Step 2 - Launch from the AWS CLI

aws ec2 run-instances \
  --image-id ami-xxxxxxxxxxxxxxxxx \
  --instance-type m5.large \
  --key-name your-key \
  --security-group-ids sg-xxxxxxxx \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=tei}]'

Step 3 - Connect to your instance

ssh -i your-key.pem ubuntu@<instance-public-ip>

Step 4 - Confirm the services

systemctl is-active docker.service tei.service nginx.service curl -s http://127.0.0.1/health -w ' HTTP %{http_code}\n'

The output looks like this:

active active active HTTP 200

Step 5 - Retrieve your password

sudo cat /root/tei-credentials.txt

# Text Embeddings Inference - generated on first boot by tei-firstboot.service TEI_URL=http://<instance-public-ip>/ TEI_MODEL=BAAI/bge-small-en-v1.5 TEI_USERNAME=admin TEI_PASSWORD=<your-unique-password>

Step 6 - Inspect the served model

The /health endpoint is open. Every other endpoint requires the password, with user admin.

PASS=$(sudo grep '^TEI_PASSWORD=' /root/tei-credentials.txt | cut -d= -f2-)
curl -s -u admin:$PASS http://127.0.0.1/info | jq -c '{model_id, max_input_length}'

{"model_id":"BAAI/bge-small-en-v1.5","max_input_length":512}

Step 7 - Generate embeddings (OpenAI-compatible)

PASS=$(sudo grep '^TEI_PASSWORD=' /root/tei-credentials.txt | cut -d= -f2-)
curl -s -u admin:$PASS http://127.0.0.1/v1/embeddings -H 'Content-Type: application/json' \
  -d '{"model":"bge-small-en-v1.5","input":["the quick brown fox","retrieval augmented generation"]}' \
  | jq -c '{model, count:(.data|length), dim:(.data[0].embedding|length)}'

{"model":"BAAI/bge-small-en-v1.5","count":2,"dim":384}

The native TEI endpoint /embed returns the raw vector array:

PASS=$(sudo grep '^TEI_PASSWORD=' /root/tei-credentials.txt | cut -d= -f2-)
curl -s -u admin:$PASS http://127.0.0.1/embed -H 'Content-Type: application/json' \
  -d '{"inputs":"hello world"}' | jq -c '{dim:(.[0]|length)}'

{"dim":384}

To call the API from your own machine, use the instance's public IP instead of 127.0.0.1. Both /embed (native) and /v1/embeddings (OpenAI-compatible) require HTTP Basic Authentication.

Step 8 - Use the OpenAI SDK

from openai import OpenAI import base64 auth = base64.b64encode(b"admin:<your-unique-password>").decode() client = OpenAI(base_url="http://<instance-public-ip>/v1", api_key="unused", default_headers={"Authorization": "Basic " + auth}) r = client.embeddings.create(model="bge-small-en-v1.5", input=["hello world"]) print(len(r.data[0].embedding))

You can store the vectors in a vector database such as Weaviate or Chroma to build a retrieval-augmented generation pipeline.

Serving a different model

TEI serves the model named in its environment file. Edit the file, then restart the service:

sudo sed -i 's#^MODEL=.*#MODEL=BAAI/bge-large-en-v1.5#' /etc/tei/tei.env
sudo systemctl restart tei.service

TEI downloads the model into /var/lib/tei/data on start. It also serves reranking models, which you query through the /rerank endpoint, and sequence-classification models. Larger models run better on a bigger instance, such as m5.xlarge or a compute-optimised c6i type.

Enabling HTTPS

sudo apt-get update && sudo apt-get install -y certbot python3-certbot-nginx
sudo certbot --nginx -d your-domain.example.com

Certbot edits the nginx site at /etc/nginx/sites-available/sac-tei to add the TLS listener, and it arranges automatic renewal.

Backup and maintenance

• Back up models. Model weights live under /var/lib/tei/data on a dedicated EBS volume. Snapshot that volume to back up downloaded models, or download them again by model name.
• Rotate the password. The password is in the nginx htpasswd file /etc/nginx/.tei.htpasswd. Rotate it with sudo htpasswd /etc/nginx/.tei.htpasswd admin.
• Restart and read logs. Restart with sudo systemctl restart tei.service. Read logs with sudo journalctl -u tei.service and sudo docker logs tei.

Support

SecureAnyCloud provides 24/7 technical support for this image by email and chat. Support covers TEI deployment, model selection, instance sizing, batching and throughput tuning, the OpenAI-compatible API, TLS termination and scaling. Contact details are on the AWS Marketplace listing.

Deploy on AWS

Launch Text Embeddings Inference on AWS with 24/7 support from SecureAnyCloud.

View on Marketplace

Need Help?

Our support team is available 24/7.

sales@secureanycloud.com