whisper.cpp Server 1.9.5

In the AWS Marketplace listing, choose Continue to Subscribe, then Continue to Configuration. Pick whisper.cpp Server 1.9.5 on Ubuntu 24.04 as the software version, choose your Region, then choose Launch through EC2. Choose m5.large (see Performance for larger sizes), your key pair, and a security group as above.

To launch with the AWS CLI instead, replace the values in angle brackets:

aws ec2 run-instances --region <region> --image-id <ami-id-from-the-listing> \
  --instance-type m5.large --key-name <your-key-pair> \
  --security-group-ids <your-security-group> --subnet-id <your-subnet> \
  --metadata-options HttpTokens=required,HttpEndpoint=enabled \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=whisper-cpp}]'

Connect to the whisper.cpp Server 1.9.5 on Ubuntu 24.04 version as the SSH user ubuntu:

ssh -i <your-key.pem> ubuntu@<public-ip>

First boot generates the certificate and the password, then starts the inference engine and nginx. Check that first boot has finished and the services are running:

systemctl is-active whisper-server nginx whisper-cpp-server-firstboot
sudo test -f /var/lib/sac/data/whisper-cpp-server-firstboot.done && echo FIRSTBOOT_DONE

All three units are active and the first-boot marker is present:

active active active FIRSTBOOT_DONE

Only SSH and nginx listen on public addresses. The inference engine listens on loopback:

sudo ss -lntp | awk 'NR>1 {print $4, $6}' | grep -v '127.0.0.5' | sort

0.0.0.0:22 users:(("sshd",...)) 0.0.0.0:443 users:(("nginx",...)) 0.0.0.0:80 users:(("nginx",...)) 127.0.0.1:8080 users:(("whisper-server",...)) [::]:22 users:(("sshd",...)) [::]:443 users:(("nginx",...)) [::]:80 users:(("nginx",...))

sudo cat /root/whisper-cpp-server-credentials.txt

The file is readable by root only and holds the URLs, the user name and this instance's password:

WHISPER_URL=https://ec2-<public-ip-dashed>.<region>.compute.amazonaws.com/ WHISPER_INFERENCE_URL=https://ec2-<public-ip-dashed>.<region>.compute.amazonaws.com/inference WHISPER_USER=whisper WHISPER_PASSWORD=<WHISPER_PASSWORD>

The URL uses the instance's public DNS name from EC2 instance metadata. The certificate also names the public and private IP addresses, so https://<public-ip>/ works too. To use your own DNS name, see Step 9.

The image ships two short clips in /usr/local/share/whisper-cpp-server/samples/. jfk.wav is 11 seconds of President Kennedy's 1961 inaugural address (public domain, from the whisper.cpp source tree). sac-test.wav is a sentence synthesised on the build machine.

Transcribe the recording through nginx with your password, and check the transcript itself, not only the status code. The password is passed to curl on standard input, so it never appears in the process list or your shell history:

sudo bash -c ' F=/root/whisper-cpp-server-credentials.txt P=$(grep "^WHISPER_PASSWORD=" $F | cut -d= -f2-) H=$(grep "^WHISPER_URL=" $F | sed -E "s#^WHISPER_URL=https://([^/]+)/.*#\1#") auth() { printf "user = \"whisper:%s\"\n" "$P"; } T=$(auth | curl -K - -sk --resolve "$H:443:127.0.0.1" \
  -F file=@/usr/local/share/whisper-cpp-server/samples/jfk.wav -F response_format=json \
  "https://$H/inference") echo "$T" case "$T" in *"ask not what your country can"*) echo TRANSCRIPT_OK ;; *) echo "unexpected transcript"; exit 1 ;; esac '

{"text":" And so my fellow Americans, ask not what your country can\n do for you, ask what you can do for your country.\n"} TRANSCRIPT_OK

Without the password, or with a wrong one, nginx refuses the request before it reaches the engine:

H=$(sudo grep '^WHISPER_URL=' /root/whisper-cpp-server-credentials.txt | sed -E 's#^WHISPER_URL=https://([^/]+)/.*#\1#') S=/usr/local/share/whisper-cpp-server/samples/jfk.wav
A=$(curl -sk -o /dev/null -w '%{http_code}' --resolve "$H:443:127.0.0.1" -F file=@$S "https://$H/inference")
W=$(printf 'user = "whisper:not-the-password"\n' | curl -K - -sk -o /dev/null -w '%{http_code}' --resolve "$H:443:127.0.0.1" -F file=@$S "https://$H/inference")
echo "no password: HTTP $A";
echo "wrong password: HTTP $W" [ "$A" = 401 ] && [ "$W" = 401 ] && echo REFUSED_OK

no password: HTTP 401 wrong password: HTTP 401 REFUSED_OK

An HTTP 200 with an empty text would mean the model failed to load, which is why the check above looks at the words.

Open the WHISPER_URL in your browser. The certificate is self-signed, so your browser asks you to confirm it the first time. Alternatively, install your own certificate (Step 9). Without credentials, nginx answers 401.

Sign in as whisper with the password from Step 4. The page documents the API and has an upload form. Choose an audio file, pick a response format and select Submit. The transcript comes back in the browser.

The page also documents a /load endpoint. It is refused on this image (HTTP 403), as Step 8 shows.

From your own machine, use the URL and password from Step 4. The -k flag accepts the self-signed certificate. ffmpeg on the instance converts the input, so MP3, M4A, FLAC, OGG, WAV and the audio track of most video files are accepted:

curl -k -u whisper:<WHISPER_PASSWORD> -F file=@recording.mp3 -F response_format=text https://<public-ip>/inference

Pass response_format to choose the output:

• text: plain transcript text.
• json: {"text": "..."}.
• verbose_json: transcript, language, duration, segments, and per-word timings and probabilities.
• srt: SubRip subtitles.
• vtt: WebVTT subtitles.

To make subtitles for the test recording on the instance, run:

sudo bash -c ' F=/root/whisper-cpp-server-credentials.txt P=$(grep "^WHISPER_PASSWORD=" $F | cut -d= -f2-) H=$(grep "^WHISPER_URL=" $F | sed -E "s#^WHISPER_URL=https://([^/]+)/.*#\1#") OUT=$(printf "user = \"whisper:%s\"\n" "$P" | curl -K - -sk --resolve "$H:443:127.0.0.1" \
  -F file=@/usr/local/share/whisper-cpp-server/samples/jfk.wav -F response_format=srt "https://$H/inference") echo "$OUT" case "$OUT" in *"-->"*"fellow Americans"*) echo SUBTITLES_OK ;; *) echo "no subtitles"; exit 1 ;; esac '

1 00:00:00,000 --> 00:00:06,740 And so my fellow Americans, ask not what your country can 2 00:00:06,740 --> 00:00:11,000 do for you, ask what you can do for your country. SUBTITLES_OK

Uploads are limited to 25 MB each, the common limit for speech-to-text APIs. For a longer recording, either compress it (ffmpeg -i long.wav -ac 1 -b:a 32k long.mp3 fits about 1 hour 45 minutes into 25 MB) or raise client_max_body_size (see Configuration, models and password rotation).

Audio and transcripts are not stored. Each upload is converted in the engine's private temporary directory and deleted when the request finishes.

The model loading endpoint is refused even with the password, and any other path is not found. The status check works without a login, for load balancers, and returns only {"status":"ok"}:

sudo bash -c ' F=/root/whisper-cpp-server-credentials.txt P=$(grep "^WHISPER_PASSWORD=" $F | cut -d= -f2-) H=$(grep "^WHISPER_URL=" $F | sed -E "s#^WHISPER_URL=https://([^/]+)/.*#\1#") R="--resolve $H:443:127.0.0.1" auth() { printf "user = \"whisper:%s\"\n" "$P"; } L=$(auth | curl -K - -sk $R -o /dev/null -w "%{http_code}" -F model=/etc/shadow "https://$H/load") N=$(auth | curl -K - -sk $R -o /dev/null -w "%{http_code}" "https://$H/anything-else") echo "load endpoint: HTTP $L"; echo "other path: HTTP $N" echo "health: $(curl -sk $R "https://$H/health")" [ "$L" = 403 ] && [ "$N" = 404 ] && echo SURFACE_OK '

load endpoint: HTTP 403 other path: HTTP 404 health: {"status":"ok"} SURFACE_OK

An empty or broken upload is answered with an error and never takes the engine down. The engine process keeps the same process ID:

sudo bash -c ' F=/root/whisper-cpp-server-credentials.txt P=$(grep "^WHISPER_PASSWORD=" $F | cut -d= -f2-) H=$(grep "^WHISPER_URL=" $F | sed -E "s#^WHISPER_URL=https://([^/]+)/.*#\1#") PID=$(systemctl show -p MainPID --value whisper-server) E=$(mktemp --suffix=.wav) OUT=$(printf "user = \"whisper:%s\"\n" "$P" | curl -K - -sk --resolve "$H:443:127.0.0.1" \
  -F file=@$E -F response_format=json "https://$H/inference") rm -f $E echo "empty upload: $OUT" [ "$(systemctl show -p MainPID --value whisper-server)" = "$PID" ] || { echo "engine restarted"; exit 1; } echo ENGINE_STILL_RUNNING case "$OUT" in *"\"error\""*) echo REJECTED_OK ;; *) echo "no error returned"; exit 1 ;; esac '

empty upload: {"error":"FFmpeg conversion failed."} ENGINE_STILL_RUNNING REJECTED_OK

A clip that decodes but is too short to hold speech (a few milliseconds) returns HTTP 200 with an empty transcript.

The engine is not reachable on the instance's own network address, only through nginx:

C=$(curl -s -o /dev/null -m 5 -w '%{http_code}' "http://$(hostname -I | awk '{print $1}'):8080/health" || true)
echo "engine on the network address: HTTP $C" [ "$C" = 000 ] && echo LOOPBACK_ONLY

engine on the network address: HTTP 000 LOOPBACK_ONLY

The engine runs unprivileged in a sandbox with a memory ceiling, and neither it nor nginx can reach the instance metadata service:

systemctl show whisper-server -p User -p ProtectSystem -p NoNewPrivileges -p MemoryMax -p IPAddressDeny
systemctl show nginx -p IPAddressDeny

MemoryMax=<75% of the instance memory, in bytes> IPAddressDeny=168.63.129.16/32 fd00:ec2::254/128 169.254.169.254/32 User=whisper ProtectSystem=strict NoNewPrivileges=yes IPAddressDeny=168.63.129.16/32 fd00:ec2::254/128 169.254.169.254/32

nginx passes the engine a fixed host name and the real client address from the connection, never values that a client sends. It also enforces the limits described above:

grep -E 'proxy_set_header (Host|X-Forwarded-For|X-Real-IP)|client_max_body_size|proxy_read_timeout|limit_conn whisper' /etc/nginx/sites-available/sac-whisper-cpp-server

client_max_body_size 25m; proxy_set_header Host 127.0.0.1:8080; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_read_timeout 900s; limit_conn whisper_perip 2; limit_conn whisper_total 4;

nginx also refuses an upload whose Origin is another web site (HTTP 403), because browsers resend Basic Auth credentials on cross-site requests. It also removes the engine's permissive cross-origin header from every response.

Point a DNS record at the instance, then set the name the service is served as. This generates a new self-signed certificate for the name, updates nginx and the URLs in /root/whisper-cpp-server-credentials.txt, and keeps the name across reboots:

sudo whisper-set-hostname <your-domain>

To use a certificate from your own certificate authority instead, copy it over the generated pair and reload nginx. Running whisper-set-hostname again replaces it with a self-signed one:

sudo install -m 0644 <your-domain>.crt /etc/ssl/whisper-cpp-server/whisper.crt
sudo install -m 0600 <your-domain>.key /etc/ssl/whisper-cpp-server/whisper.key
sudo nginx -t && sudo systemctl reload nginx

whisper.cpp runs on the CPU, so speed scales with vCPUs. These timings were measured on this image on a 2 vCPU instance (the same vCPU count as m5.large) with the bundled ggml-base.en model:

• 4 second synthesised sentence: about 4.3 seconds, mostly fixed per-request cost.
• 11 second recording (jfk.wav): about 5 seconds.
• 88 seconds of speech: about 23 seconds, roughly 3.9 times faster than real time.

A 10 minute recording therefore takes about 2.5 to 3 minutes on m5.large. The engine uses about 470 MB of memory with this model.

For sustained or batch work, choose a compute-optimised type with more vCPUs (c5.xlarge, c5.2xlarge). The engine uses every vCPU (WHISPER_THREADS=auto, up to 16). Requests are processed one at a time and the rest wait in turn. For more throughput, run several instances behind a load balancer.

The engine reads /etc/whisper-cpp-server/server.env. The variables are:

• WHISPER_MODEL: the GGML model file. The image default is /var/lib/whisper-cpp-server/models/ggml-base.en.bin.
• WHISPER_THREADS: CPU threads, or auto (the vCPU count, at most 16). The image default is auto.
• WHISPER_EXTRA_ARGS: extra whisper-server options, for example --language auto with a multilingual model. The image default is empty.

The bind address is fixed to 127.0.0.1:8080 by /usr/local/sbin/whisper-server-launch. Keep it on loopback, because the engine has no authentication of its own. After editing the file, run sudo systemctl restart whisper-server.

ggml-base.en balances accuracy and speed and is English only. Larger models are more accurate and slower. Multilingual models drop the .en suffix. To use another model from the same repository (here ggml-small.en.bin):

sudo curl -fL -o /var/lib/whisper-cpp-server/models/ggml-small.en.bin \
  https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small.en.bin
sudo sed -i 's#^WHISPER_MODEL=.*#WHISPER_MODEL=/var/lib/whisper-cpp-server/models/ggml-small.en.bin#' /etc/whisper-cpp-server/server.env
sudo systemctl restart whisper-server

Check the licence of any model you add. The models in that repository are conversions of OpenAI's Whisper checkpoints (MIT), but community fine-tunes elsewhere can carry terms that forbid commercial use.

The upload limit, the per-request time limit and the concurrency limits are in /etc/nginx/sites-available/sac-whisper-cpp-server (client_max_body_size, proxy_read_timeout, limit_conn). After a change, run sudo nginx -t && sudo systemctl reload nginx.

To set a new password, give it to htpasswd on standard input. Start the line with a space so that most shells keep it out of your history. nginx uses the new password at once:

printf '%s' '<new-password>' | sudo htpasswd -i -B -C 10 /etc/nginx/whisper-cpp-server.htpasswd whisper

Then update WHISPER_PASSWORD in /root/whisper-cpp-server-credentials.txt so the file does not hold an old password.

Check that the model matches the checksum recorded at build time:

sha256sum /var/lib/whisper-cpp-server/models/ggml-base.en.bin

a03779c86df3323075f5e796cb2ce5029f00ec8869eee3fdfb897afe36c6d002 /var/lib/whisper-cpp-server/models/ggml-base.en.bin

To read the licence notices and the provenance record, run:

ls /opt/whisper/ head -12 /opt/whisper/MODEL-PROVENANCE.txt

LICENSE.openai-whisper LICENSE.whisper-cpp MODEL-PROVENANCE.txt whisper.cpp Server - bundled speech recognition model provenance ================================================================ Model file : ggml-base.en.bin Location : /var/lib/whisper-cpp-server/models/ggml-base.en.bin Size : 147964211 bytes SHA-256 : a03779c86df3323075f5e796cb2ce5029f00ec8869eee3fdfb897afe36c6d002 SHA-1 : 137c40403d78fd54d454da0f9bd998f78703390c (as published in the source repository's model card) Source : https://huggingface.co/ggerganov/whisper.cpp/resolve/5359861c739e955e79d9a303bcbc70fb988958b1/ggml-base.en.bin Provenance chain:

These are the main components and their paths:

• whisper.cpp server 1.9.5: /usr/local/bin/whisper-server
• Launcher (loopback bind): /usr/local/sbin/whisper-server-launch
• Engine settings: /etc/whisper-cpp-server/server.env
• Model: /var/lib/whisper-cpp-server/models/ggml-base.en.bin
• Test clips: /usr/local/share/whisper-cpp-server/samples/
• Licences and provenance: /opt/whisper/
• Service unit: /etc/systemd/system/whisper-server.service
• nginx site: /etc/nginx/sites-available/sac-whisper-cpp-server
• Password store (bcrypt): /etc/nginx/whisper-cpp-server.htpasswd
• Certificate and key: /etc/ssl/whisper-cpp-server/
• First boot script: /usr/local/sbin/whisper-cpp-server-firstboot.sh
• Host name helper: /usr/local/sbin/whisper-set-hostname
• Credentials: /root/whisper-cpp-server-credentials.txt (mode 0600)

To check the server binary and the service status, run:

/usr/local/bin/whisper-server --help 2>&1 | head -3 systemctl status whisper-server --no-pager | head -3

usage: /usr/local/bin/whisper-server [options] ● whisper-server.service - whisper.cpp HTTP server (speech to text, loopback behind nginx; sac) Loaded: loaded (/etc/systemd/system/whisper-server.service; enabled; preset: enabled) Drop-In: /etc/systemd/system/whisper-server.service.d

• Allow ports 22, 80 and 443 in the security group from your own networks only.
• Keep the per-instance password secret, and rotate it if it may have been shared (Step 11).
• Use your own DNS name and a certificate your clients trust (Step 9).
• Keep the engine on loopback, and check the licence of any model you add.
• Keep the operating system patched. Unattended security upgrades are on, and sudo apt-get update && sudo apt-get upgrade applies everything else.