SecureAnyCloud
Deployment guide

LLM Inference

Overview

This page is the entry point for the SecureAnyCloud Hardened Ollama, LiteLLM & Self-Hosted LLM Server AMIs listing on AWS Marketplace. The listing is a single product that offers several products and releases as separate software versions.

Subscribe once, then choose the version you want from the Software version list at launch. Open that product's guide for the full setup and connection walk-through.

What SecureAnyCloud adds to every version:

  • Every image is built by SecureAnyCloud on a fully patched base, with the latest security updates applied at build time.
  • Each product is CPU-only. No GPU drivers or CUDA libraries ship.
  • Each product generates its administrator password, API keys and secrets uniquely on the first boot of each instance.
  • Each product serves its web interface or API over HTTPS with a per-instance certificate.
  • The services cannot reach the instance metadata service, so a prompt or a configured URL can never fetch the instance's credentials.

Pricing is metered hourly per EC2 instance type and is the same for every version in this listing. The recommended instance type is m5.large.

How to pick a version

  • Open the listing on AWS Marketplace, choose View purchase options, then choose Subscribe. One subscription covers every product and release in the listing, and the free trial starts with it.
  • When the subscription is active, choose Launch your software.
  • Choose One-click launch from AWS Marketplace, or Launch from EC2 Console for full control. In Version, choose the product and release you want. Each version is a separate software version, named <product> <release> on <operating system> - <date>, for example AnythingLLM 1.14.1 on Ubuntu 24.04 - YYYY-MM-DD. The date is when that version was added to the listing.
  • Choose your Region, an instance type (m5.large is recommended), your EC2 key pair and a security group that opens the ports in the product's guide. Then choose Launch.
  • After launch, open the product's guide for first login.

You can run several products from the same subscription. Launch one instance per product, each from its own software version.

Versions in this listing, newest release first:

  • AnythingLLM: 1.14.1 on Ubuntu 24.04
  • Bifrost: 2.3.1 on Ubuntu 24.04
  • big-AGI: 2.0.5 on Ubuntu 24.04
  • GPT-Load: 1.4.12 on Ubuntu 24.04
  • Infinity: 0.0.77 on Ubuntu 24.04
  • LiteLLM: 1.88.0 on Ubuntu 24.04
  • llama.cpp Server: b11669 on Ubuntu 24.04
  • LocalAI: 4.11.0 on Ubuntu 24.04
  • NextChat: 2.16.1 on Ubuntu 24.04
  • Ollama: 0.40.6 on Ubuntu 24.04

Security model

Every version in this listing follows the same model:

  • No default login. Administrator passwords, database passwords and application secrets are generated on the first boot of each instance and written to a root-only file. No two instances share a secret.
  • Only the documented ports. Each guide lists the ports the product needs. Open only those in your security group.
  • Internal services on localhost. Databases and caches listen on 127.0.0.1 only. Never open database ports (3306, 5432, 6379) to the internet.
  • Patched base. The operating system is fully patched when the image is built.
  • HTTPS on first boot. These versions serve HTTPS with a per-instance certificate from first boot: AnythingLLM 1.14.1 on Ubuntu 24.04, Bifrost 2.3.1 on Ubuntu 24.04, GPT-Load 1.4.12 on Ubuntu 24.04, Infinity 0.0.77 on Ubuntu 24.04, LiteLLM 1.88.0 on Ubuntu 24.04, LocalAI 4.11.0 on Ubuntu 24.04, NextChat 2.16.1 on Ubuntu 24.04, Ollama 0.40.6 on Ubuntu 24.04, big-AGI 2.0.5 on Ubuntu 24.04 and llama.cpp Server b11669 on Ubuntu 24.04. The other versions serve HTTP. Their guides show how to add a certificate on the instance, or behind an Application Load Balancer with AWS Certificate Manager.

Choose your product

Each product has its own guide. Find your product below and pick the matching software version at launch. All versions run on Ubuntu 24.04.

  • AnythingLLM: software version AnythingLLM 1.14.1 on Ubuntu 24.04
  • Bifrost: software version Bifrost 2.3.1 on Ubuntu 24.04
  • big-AGI: software version big-AGI 2.0.5 on Ubuntu 24.04
  • GPT-Load: software version GPT-Load 1.4.12 on Ubuntu 24.04
  • Infinity: software version Infinity 0.0.77 on Ubuntu 24.04
  • LiteLLM: software version LiteLLM 1.88.0 on Ubuntu 24.04
  • llama.cpp Server: software version llama.cpp Server b11669 on Ubuntu 24.04
  • LocalAI: software version LocalAI 4.11.0 on Ubuntu 24.04
  • NextChat: software version NextChat 2.16.1 on Ubuntu 24.04
  • Ollama: software version Ollama 0.40.6 on Ubuntu 24.04

How to tell which version you launched

The EC2 console shows the version you chose on the instance's launch record (AMI name and description). Your AWS Marketplace subscription shows it too.

On the instance itself, the login message after SSH names the product and its credentials file. Open the matching product guide for the rest of the walk-through.

Support

SecureAnyCloud provides 24/7 support for every version in this listing through the AWS Marketplace support channel. Raise a ticket in the Marketplace console or email sales@secureanycloud.com.

All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.

Deploy on AWS

Launch Hardened Ollama, LiteLLM & Self Hosted LLM Server AMIs with 24/7 support from SecureAnyCloud.

View this image

Need Help?

Our support team is available 24/7.

sales@secureanycloud.com

Deploy on AWS