Elasticsearch

Sign in to the AWS Management Console, open the EC2 service, and select Launch instance. Under Application and OS Images, choose AWS Marketplace AMIs and search for Elasticsearch. Select the SecureAnyCloud listing, choose Select, then choose Continue on the subscription summary.

Pick an instance type of m5.large or larger as a balanced default. Elasticsearch is a JVM workload that benefits from a heap of roughly 50 percent of host memory, but never above approximately 30 GiB. Beyond that, the JVM stops using compressed object pointers and per-object memory overhead increases. For light search workloads, m5.large (8 GiB) is sufficient. For heavier ingest or analytics workloads, use m5.xlarge, m5.2xlarge or larger.

Choose your EC2 key pair under Key pair (login). Under Network settings, select your VPC and subnet, then create or select a security group. It must allow inbound TCP 22 from your management network and inbound TCP 9200 only from the application servers that need it. Do not expose port 9200 to the public internet. Once a client is authenticated, the REST API exposes administrative operations, including index deletion and cluster reconfiguration.

Leave the root volume at the default size. The image automatically attaches a dedicated 30 GiB data volume, mounted at /var/lib/elasticsearch, for index storage.

Select Launch instance. First-boot initialization takes up to a minute after the instance state becomes Running and the status checks pass. During that time Elasticsearch starts the JVM and initializes the security index, and the first-boot service generates the elastic superuser password.

The command below launches an instance from the SecureAnyCloud Elasticsearch Marketplace AMI into an existing subnet and security group. Replace the values in angle brackets as follows:

• <ami-id>: the AMI ID shown on the Marketplace listing
• <key-name>: your EC2 key pair name
• <subnet-id>: your subnet ID
• <security-group-id>: a security group that allows inbound TCP 22 from your management network and inbound TCP 9200 from your application servers

aws ec2 run-instances \
  --image-id <ami-id> \
  --instance-type m5.large \
  --key-name <key-name> \
  --subnet-id <subnet-id> \
  --security-group-ids <security-group-id> \
  --block-device-mappings '{"DeviceName":"/dev/sda1","Ebs":{"VolumeSize":30,"VolumeType":"gp3"}}' \
  --metadata-options 'HttpTokens=required,HttpEndpoint=enabled' \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=elasticsearch-node-1}]'

The image attaches the dedicated 30 GiB index storage volume automatically. To start with more index storage, enlarge that volume on the Storage step in the console, or add a second block device mapping in the CLI.

Connect over SSH as the default login user for the operating system variant you launched. The login user differs by variant:

Operating system variant SSH login user

Ubuntu 24.04 ubuntu

Replace <key-file> with the path to your private key file and <instance-public-ip> with the public IP address or DNS name of the instance.

ssh -i <key-file> ubuntu@<instance-public-ip>

The instance generates the elastic superuser password on first boot and writes it to /root/elasticsearch-credentials.txt, which only root can read. Print it from inside the SSH session:

sudo cat /root/elasticsearch-credentials.txt

The file records the URL the API is reachable on, the superuser name, and the plain-text password. Save the password somewhere safe. It is stored in plain text only in this file, and rotating it through the supported tool overwrites the value.

Check that the Elasticsearch service is active and listening on its REST API port. The first command reports active for both the daemon and the SecureAnyCloud first-boot service. The second confirms that a process is listening on TCP 9200.

systemctl is-active elasticsearch.service elasticsearch-firstboot.service

ss -tlnp | grep ':9200 '

The Elasticsearch process runs as the elasticsearch system user. The bundled OpenJDK runtime is at /usr/share/elasticsearch/jdk and is the only Java on the image.

Read the password into a shell variable, then query the root endpoint. The elastic superuser authenticates over HTTP basic auth. The root endpoint reports the node name, the cluster name, the version and the Lucene library version.

PASS=$(sudo grep '^elasticsearch.elastic.pass=' /root/elasticsearch-credentials.txt | cut -d= -f2-)
curl -s -u "elastic:${PASS}" http://127.0.0.1:9200/

Inspect cluster health. On this single-node image the status is green or yellow. green means every primary shard and its replica is active. yellow means one or more replicas are unassigned, because a replica cannot be assigned to the same node as its primary and the cluster has only one node.

PASS=$(sudo grep '^elasticsearch.elastic.pass=' /root/elasticsearch-credentials.txt | cut -d= -f2-)
curl -s -u "elastic:${PASS}" http://127.0.0.1:9200/_cluster/health | python3 -m json.tool

Create a new index, add a document to it, search for the document, then delete the index. This end-to-end check confirms that the cluster accepts writes, refreshes the search index and answers queries.

PASS=$(sudo grep '^elasticsearch.elastic.pass=' /root/elasticsearch-credentials.txt | cut -d= -f2-)
curl -s -u "elastic:${PASS}" -X PUT http://127.0.0.1:9200/books

PASS=$(sudo grep '^elasticsearch.elastic.pass=' /root/elasticsearch-credentials.txt | cut -d= -f2-)
curl -s -u "elastic:${PASS}" -X POST "http://127.0.0.1:9200/books/_doc?refresh=true" \
  -H 'Content-Type: application/json' \
  -d '{"title":"The Pragmatic Programmer","author":"Hunt and Thomas","year":1999}'

PASS=$(sudo grep '^elasticsearch.elastic.pass=' /root/elasticsearch-credentials.txt | cut -d= -f2-)
curl -s -u "elastic:${PASS}" "http://127.0.0.1:9200/books/_search?q=author:Hunt"

PASS=$(sudo grep '^elasticsearch.elastic.pass=' /root/elasticsearch-credentials.txt | cut -d= -f2-)
curl -s -u "elastic:${PASS}" -X DELETE http://127.0.0.1:9200/books

List every index on the cluster, with storage and document counts, using the _cat indices endpoint. The trailing ?v adds a header row.

PASS=$(sudo grep '^elasticsearch.elastic.pass=' /root/elasticsearch-credentials.txt | cut -d= -f2-)
curl -s -u "elastic:${PASS}" "http://127.0.0.1:9200/_cat/indices?v"

Rotate the elastic superuser password with the supported reset tool. The flag -i prompts for the new password interactively. The flag -b generates a strong random password and prints it on stdout. After rotation, update /root/elasticsearch-credentials.txt so future shells pick up the new value.

sudo /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic -i

Use the prompt variant in interactive sessions only. The non-interactive variant suits automation:

sudo /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic -b

The image ships with a conservative heap of -Xms512m -Xmx1g, so the daemon starts cleanly on the smallest instance type. On a production instance, raise the heap to roughly 50 percent of host memory, but never above approximately 30 GiB. The heap is set in /etc/elasticsearch/jvm.options.d/sac-heap.options.

cat /etc/elasticsearch/jvm.options.d/sac-heap.options

To change it on a running instance, edit the file and restart Elasticsearch:

sudo sed -i 's/^-Xms.*/-Xms4g/' /etc/elasticsearch/jvm.options.d/sac-heap.options
sudo sed -i 's/^-Xmx.*/-Xmx4g/' /etc/elasticsearch/jvm.options.d/sac-heap.options
sudo systemctl restart elasticsearch.service

The Elasticsearch data directory is its own filesystem on its own EBS volume, mounted at /var/lib/elasticsearch. Index data lives there. Confirm the mount and check free space with:

findmnt /var/lib/elasticsearch

df -h /var/lib/elasticsearch

To grow the index store, modify the EBS volume in the AWS console or with the CLI. Then extend the filesystem on the instance with resize2fs against the volume's device. The data volume is independent of the operating system disk, so you can grow it without disturbing the rest of the instance.

Elasticsearch snapshots write a logical backup to a registered repository. A common repository type is s3, which requires the AWS S3 repository plugin. The plugin is available from the official Elastic packages. Install it with the bundled plugin tool, restart Elasticsearch, then register the repository over the REST API.

sudo /usr/share/elasticsearch/bin/elasticsearch-plugin install --batch repository-s3
sudo systemctl restart elasticsearch.service

Register a repository and take a snapshot. Replace <your-bucket> and <your-base-path> with the S3 bucket and key prefix for your snapshots. Make sure the instance's IAM role can read and write that bucket.

PASS=$(sudo grep '^elasticsearch.elastic.pass=' /root/elasticsearch-credentials.txt | cut -d= -f2-)
curl -s -u "elastic:${PASS}" -X PUT "http://127.0.0.1:9200/_snapshot/s3-backups" \
  -H 'Content-Type: application/json' \
  -d '{"type":"s3","settings":{"bucket":"<your-bucket>","base_path":"<your-base-path>"}}'
curl -s -u "elastic:${PASS}" -X PUT "http://127.0.0.1:9200/_snapshot/s3-backups/snap-1?wait_for_completion=true"

The image disables HTTP TLS by default, so the daemon is reachable over plain HTTP with basic authentication out of the box. For production traffic, enable HTTPS with a certificate from your own certificate authority.

Place a PEM certificate and private key on the instance, then point Elasticsearch at them in elasticsearch.yml. The reference settings below use example paths for your own files. Restart Elasticsearch after editing.

sudo tee -a /etc/elasticsearch/elasticsearch.yml > /dev/null <<EOF
# sac: HTTPS on the REST layer with your own certificate authority.
# Replace the paths below with your PEM certificate and private key. xpack.security.http.ssl.enabled: true xpack.security.http.ssl.certificate: /etc/elasticsearch/certs/node.crt xpack.security.http.ssl.key: /etc/elasticsearch/certs/node.key xpack.security.http.ssl.certificate_authorities: ["/etc/elasticsearch/certs/ca.crt"] EOF
sudo systemctl restart elasticsearch.service

After you enable HTTPS, every client must use https:// and a trusted certificate. The user guide for the SecureAnyCloud ELK Stack covers the full TLS bootstrap in more depth.

Keep the operating system patched with the standard package manager. The official Elastic 8.x APT repository is already configured on the image, so Elasticsearch upgrades arrive through the normal package update process.

Inspect the JVM and node-level statistics to spot saturated heaps, full disks or pending tasks:

PASS=$(sudo grep '^elasticsearch.elastic.pass=' /root/elasticsearch-credentials.txt | cut -d= -f2-)
curl -s -u "elastic:${PASS}" "http://127.0.0.1:9200/_cat/nodes?v&h=name,heap.percent,ram.percent,cpu,load_1m,node.role"

To free disk space on a busy cluster, delete unused indices with DELETE /<index-name>, or use index lifecycle management policies to expire indices automatically. The cluster log is at /var/log/elasticsearch/elasticsearch.log and is rotated by the system logrotate.