AnythingLLM
Overview
AnythingLLM is an all-in-one AI application from Mintplex Labs. It lets you chat with your documents using retrieval-augmented generation (RAG), organise conversations into workspaces, and share access across a team with full multi-user controls. It bundles a built-in vector database (LanceDB) and a SQLite metadata store, and it connects to the large language model (LLM) provider of your choice: OpenAI, Anthropic or a local Ollama.
The SecureAnyCloud image installs AnythingLLM 1.14.1 on Node.js 20 LTS and runs the server and document collector as systemd services behind an nginx reverse proxy on port 80. It stores the database, vector store and uploads on a dedicated Azure data disk, and it generates all application secrets plus a unique admin user on the first boot of every VM.
What you get:
- AnythingLLM 1.14.1 on Node.js 20 LTS, served on port 80 via nginx
- The built-in LanceDB vector store and SQLite database on a dedicated Azure data disk at
/var/lib/anythingllm - Multi-user mode enabled with a unique admin user generated on first boot
- WebSocket-aware reverse proxy and a 1 GB upload limit for document ingestion
anythingllm.service,anythingllm-collector.serviceandnginx.serviceas systemd units, enabled and active- 24/7 SecureAnyCloud support
The image ships AnythingLLM running and ready. The application and admin panel are fully functional out of the box. Chat needs an LLM provider, which you supply on first use (see Step 7). No LLM provider or API key is preset.
Prerequisites
You need an active Azure subscription, an SSH key pair, and a VNet with a subnet in the target region.
Standard_B4ms (4 vCPU / 16 GiB RAM) is a good starting size for an AI workload. Configure the network security group (NSG) to allow inbound 22/tcp from your management network and 80/tcp for the web application. For public exposure, front the application with TLS (see Step 10).
Step 1 - Deploy from the Azure Marketplace
Sign in to the Azure Portal, choose Create a resource, search the Marketplace for AnythingLLM by SecureAnyCloud, and select Create.
On the Basics tab, pick your subscription, resource group, region and size. Under Administrator account, choose SSH public key and paste your key. Under Inbound port rules, allow SSH (22) and HTTP (80).
Review the dedicated data disk on the Disks tab, then select Review + create and Create.
Step 2 - Deploy from the Azure CLI
# Accept the Marketplace terms once per subscription (required before the first CLI deploy)
az vm image terms accept --urn sac1647283583153:anythingllm-ubuntu-24-04:default:latest
az vm create \
--resource-group <your-rg> \
--name anythingllm \
--image sac1647283583153:anythingllm-ubuntu-24-04:default:latest \
--size Standard_B4ms \
--admin-username azureuser \
--ssh-key-values ~/.ssh/id_ed25519.pub \
--vnet-name <your-vnet> --subnet <your-subnet> \
--public-ip-sku Standard
az vm open-port --resource-group <your-rg> --name anythingllm --port 80 --priority 1010Step 3 - Connect to your VM
ssh azureuser@<vm-public-ip>Step 4 - Confirm the services are running
systemctl is-active anythingllm.service anythingllm-collector.service nginx.serviceAll three services report active. On first boot, AnythingLLM generates its secrets, seeds the admin user, enables multi-user mode and starts the server.
Step 5 - Retrieve your admin credentials
The first boot of your VM generates a unique admin password and writes it to a root-only file. Read it as root:
sudo cat /root/anythingllm-credentials.txtThe file contains anythingllm.admin.user (admin) and anythingllm.admin.pass. Store the password in a safe place.
Step 6 - Sign in to AnythingLLM
Open http://<vm-public-ip>/ in your browser and sign in as user admin with the password from Step 5. To check first that the API is healthy, run this command from the shell:
curl -s http://127.0.0.1:3001/api/pingThe server responds {"online":true}.
You can also confirm your admin login programmatically. The multi-user login endpoint returns a session token:
PASS=$(sudo grep '^anythingllm.admin.pass=' /root/anythingllm-credentials.txt | cut -d= -f2-)
curl -s -H 'Content-Type: application/json' \
--data "{\"username\":\"admin\",\"password\":\"${PASS}\"}" \
http://127.0.0.1:3001/api/request-token | head -c 80A successful login returns JSON containing "valid":true and a "token".
Step 7 - Connect your LLM provider
AnythingLLM ships with no LLM provider configured, so you choose one on first use. In the web UI, open Settings (the wrench icon), then AI Providers, then LLM Preference. Pick your provider and supply the credentials:
- OpenAI or Anthropic: paste your API key and select a model.
- Local Ollama: run Ollama on a reachable host and point AnythingLLM at
http://<ollama-host>:11434.
Embeddings for document ingestion use the built-in native embedder by default. Document upload and vector search therefore work without any external provider, and only chat generation needs the LLM you select here.
Step 8 - Create a workspace and chat with your documents
In the UI, select New Workspace, give it a name, and use Upload to add PDFs, Word documents, text files or web pages. AnythingLLM's document collector extracts the text, and the native embedder vectorises it into the LanceDB store. You can then ask questions grounded in your documents in the workspace chat.
Step 9 - Data, database and vector store on the dedicated disk
All durable state lives on the dedicated Azure data disk mounted at /var/lib/anythingllm:
df -h /var/lib/anythingllm
sudo ls /var/lib/anythingllm/storageThe listing shows the anythingllm.db SQLite database, the lancedb vector store, the documents cache and the per-VM .env. Because this is a separate managed disk, you can snapshot it independently, and it survives a VM resize.
Step 10 - Enabling HTTPS
For public exposure, front AnythingLLM with TLS. Point a DNS A record at your VM's public IP, then install a certificate with Certbot and the nginx plugin:
The commands install Certbot and request a certificate for your domain (replace your-domain.example.com). Certbot edits the nginx site in place and reloads it. The packaged systemd timer renews the certificate automatically.
Step 11 - Maintenance
Restart the application or follow its logs:
sudo systemctl restart anythingllm.service
sudo journalctl -u anythingllm.service -n 50 --no-pagerThe collector runs as a separate service:
sudo systemctl restart anythingllm-collector.serviceUbuntu's unattended-upgrades stays enabled in the image, so your VM keeps applying OS security updates automatically.
Screenshots
The following screenshot shows the terminal output you can expect during setup:
The AnythingLLM web application:
Support
SecureAnyCloud publishes this image and provides 24/7 support for it. AnythingLLM is a trademark of Mintplex Labs Inc. SecureAnyCloud is not affiliated with or endorsed by Mintplex Labs. AnythingLLM is distributed under the MIT License.
Deploy on Azure
Launch AnythingLLM on Ubuntu 24.04 LTS on Azure with 24/7 support from SecureAnyCloud.
View this image
Need Help?
Our support team is available 24/7.
sales@secureanycloud.com
Overview
This image runs AnythingLLM, the all-in-one open source application for chatting with your documents using any large language model. AnythingLLM turns any document, resource or piece of content into context that a chatbot can reference during conversation, with a clean workspace based UI, retrieval augmented generation (RAG), AI agents and a built-in vector database.
AnythingLLM is built bare-metal from the official source on Node.js 22 LTS and runs as a dedicated unprivileged anythingllm system account. Two systemd services run on boot and restart on failure: the application server (anythingllm.service, bound to 127.0.0.1:3001) and the document collector (anythingllm-collector.service, bound to 127.0.0.1:8888). All application state, the SQLite database, the LanceDB vector store, the embedder model cache and your uploaded documents, lives at /opt/anythingllm/storage, a dedicated, independently resizable EBS data volume that survives instance replacement.
An nginx reverse proxy publishes the workspace UI on port 80 with WebSocket support for streaming chat, and is the only public entry point: the security group opens only port 80 (and port 22 for SSH), so the application is always reached through the authenticating proxy. On the first boot of every deployed instance the image generates a fresh admin password, seeds it into the database as a bcrypt hash, and enables multi-user mode so the instance requires a login. Two instances launched from the same Amazon Machine Image never share a password. The password is written to /root/anythingllm-aws-credentials.txt with mode 0600 so that only the root user can read it.
The image ships RAG-ready: a built-in local embedding model (Xenova/all-MiniLM-L6-v2) and a built-in local vector database (LanceDB) are already configured, so document upload, chunking, embedding and semantic search all work on a fresh launch with no external service to configure. AnythingLLM is large language model provider agnostic and ships with no LLM key bundled; you choose your provider and supply your key once in Settings to enable chat.
Prerequisites
Before you deploy this image you need:
- An Amazon Web Services account where you can launch EC2 instances
- IAM permissions to launch instances, create security groups, and subscribe to AWS Marketplace products
- An EC2 key pair in the target Region for SSH access to the instance
- A VPC and subnet in the target Region, with a security group allowing inbound port 22 from your management network and port 80 for the workspace UI
- The AWS CLI (version 2) installed locally if you plan to deploy from the command line
- An API key for your preferred large language model provider (for example OpenAI, Anthropic or Azure OpenAI), or a reachable local model server such as Ollama, to enable chat
Launch the Instance from the AWS Marketplace
Sign in to the AWS Management Console, open the EC2 service, and select Launch instance. Under Application and OS Images choose AWS Marketplace AMIs and search for AnythingLLM. Select the SecureAnyCloud listing and choose Select, then Continue on the subscription summary.
Pick an instance type of t3.large or larger. Choose your EC2 key pair under Key pair (login). Under Network settings select your VPC and subnet, and either create or select a security group that opens port 22 from your management network and port 80 for the workspace UI. Leave the root volume at the default size or larger.
Select Launch instance. First boot initialisation takes up to a minute or two after the instance state becomes Running and the status checks pass, while the admin password is generated and the application restarts.
Launch the Instance from the AWS CLI
The following block launches an instance from the SecureAnyCloud AnythingLLM Marketplace AMI into an existing subnet and security group. Replace <ami-id> with the AMI ID shown on the Marketplace listing, <key-name> with your EC2 key pair name, <subnet-id> with your subnet ID, and <security-group-id> with a security group that opens ports 22 and 80 as described above.
aws ec2 run-instances \
--image-id <ami-id> \
--instance-type t3.large \
--key-name <key-name> \
--subnet-id <subnet-id> \
--security-group-ids <security-group-id> \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=anythingllm}]'When the instance reaches the Running state and its status checks pass, note its public IP address or DNS name from the EC2 console or with aws ec2 describe-instances.
Connect to Your Instance
Connect over SSH using your key pair and the login user for your operating system variant.
OS variant SSH login user
Ubuntu 24.04 ubuntu
ssh -i <key-name>.pem ubuntu@<public-ip>Retrieve the Admin Password
The admin password is unique to your instance and was generated on first boot. Read it as root:
sudo cat /root/anythingllm-aws-credentials.txtThe file lists the workspace URL, the admin user (admin) and the generated password. Keep this password somewhere safe.
Sign In to the Workspace UI
The AnythingLLM workspace UI is served on port 80 by nginx. In a browser, go to:
http://<instance-public-ip>/You are prompted to sign in. Enter the user admin and the password from the credentials file. After signing in you land in the AnythingLLM workspace, where you create workspaces, upload documents and chat with them.
A workspace is a container for a set of documents and the conversations grounded in them. Create a new workspace from the sidebar, give it a name, and open it to start a chat thread.
Confirm AnythingLLM Is Running
Over SSH, confirm the application server, the collector and the nginx proxy are active and that the ports are listening:
sudo systemctl is-active anythingllm anythingllm-collector nginxsudo ss -tlnp | grep -E ':(80|3001|8888) 'You should see all three services reported as active, the application server listening on port 3001 and the collector on port 8888 on the instance, and nginx listening on port 80. The security group exposes only port 80 (and port 22 for SSH), so the application is reached exclusively through the authenticating nginx proxy; keep ports 3001 and 8888 closed in your security group.
You can also check the server health endpoint over loopback on the instance:
curl -s http://127.0.0.1:3001/api/pingThis returns {"online":true} when the server is up.
Choose Your LLM Provider
AnythingLLM ships with the local embedder and vector database already configured, but it is provider agnostic for the chat model and ships with no LLM key. To enable chat, open Settings from the sidebar, then LLM Preference. Choose your provider, for example OpenAI, Anthropic, Azure OpenAI, or a local Ollama server, and supply your API key or endpoint. Save the setting.
Your prompts and documents stay on your instance for embedding and retrieval; only the final prompt and retrieved context are sent to the model provider you select.
Upload Documents and Build Your Knowledge Base
Open a workspace and use the document manager to upload content. AnythingLLM accepts PDFs, text, Microsoft Office documents, web pages and more. The document collector extracts the text, the built-in local embedder chunks and embeds it, and the chunks are stored in the bundled LanceDB vector database, all on your instance.
Once a document is embedded into a workspace, ask questions in the chat thread. AnythingLLM retrieves the most relevant chunks and passes them to your chosen LLM, so answers are grounded in your own content.
The Data Volume
All application state lives on a dedicated EBS volume mounted at /opt/anythingllm/storage. This keeps the database, the vector store and your uploaded documents off the operating system disk and lets you resize or snapshot them independently. Confirm the mount with:
df -h /opt/anythingllm/storageTo grow the storage, expand the EBS volume in the AWS console, then grow the filesystem on the instance with sudo resize2fs on the underlying device.
Enable HTTPS
The workspace UI is served over plain HTTP on port 80 by nginx. For production use, place it behind TLS. Obtain a certificate for your domain (for example with a managed certificate on an Application Load Balancer in front of the instance, or with Certbot installed on the instance), then configure nginx to listen on 443 with your certificate and proxy to 127.0.0.1:3001 exactly as the bundled site does for port 80, keeping the WebSocket upgrade headers in place. Restrict the security group so ports 80 and 443 are reachable only from the networks that use the application.
Backup and Maintenance
Back up the application by snapshotting the /opt/anythingllm/storage EBS volume, which captures the database, the vector store and your uploaded documents in one consistent set. Apply operating system security updates with sudo apt-get update && sudo apt-get upgrade and reboot when a new kernel is installed; AnythingLLM and nginx start automatically on boot.
Support
This image is published and supported by SecureAnyCloud. Support covers deployment, workspace and document configuration, embedding and vector database tuning, LLM provider setup, AI agents, API keys, TLS and storage tuning. Contact SecureAnyCloud through the support channel listed on the AWS Marketplace listing.
All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.
Deploy on AWS
Launch AnythingLLM on AWS with 24/7 support from SecureAnyCloud.
View on Marketplace
Need Help?
Our support team is available 24/7.
sales@secureanycloud.com