Deployment guide

Apache Hop

Overview

Apache Hop (the Hop Orchestration Platform) is a visual data orchestration and integration platform, and the modern successor to Pentaho Data Integration / Kettle. You design pipelines (data transformations) and workflows (orchestration) in a browser, then run them locally or on a dedicated execution server.

The SecureAnyCloud image installs Apache Hop 2.18.1 with two runtimes ready to use: Hop Web, the browser-based visual editor served by the official Apache Hop Web container, and Hop Server, the headless pipeline and workflow execution server. Hop Web is published behind an nginx reverse proxy on port 80 with HTTP Basic authentication. Hop Server listens on port 8081 with its own Basic authentication.

A unique password is generated on the first boot of every VM. All projects, pipelines, workflows and Hop configuration live on a dedicated Azure data disk mounted at /var/lib/hop, and the image ships the official Apache Hop samples project so the editor has real content to explore.

What you get:

Apache Hop 2.18.1 (Hop Orchestration Platform) on OpenJDK 21
Hop Web, the browser-based visual pipeline and workflow editor, run as the official apache/hop-web container on 127.0.0.1:8080
Hop Server, the headless execution server, on 127.0.0.1:8081 with Basic authentication (user cluster)
nginx on :80 as a reverse proxy to Hop Web with HTTP Basic authentication (user admin)
A per-VM password generated on first boot and recorded in a root-only file
A dedicated Azure data disk at /var/lib/hop for projects, pipelines, workflows and Hop configuration
The official Apache Hop samples project, preloaded so the editor opens with example pipelines and workflows
docker.service, hop-web.service, hop-server.service and nginx.service as systemd units, enabled and active
An unauthenticated /health endpoint for Azure Load Balancer health probes
24/7 SecureAnyCloud support

Prerequisites

You need an active Azure subscription, an SSH key pair, and a VNet with a subnet in the target region.

Standard_B2ms (2 vCPU / 8 GiB RAM) is a reasonable starting point. Size up for heavier pipelines.

Configure the network security group (NSG) to allow inbound 22/tcp from your management network and 80/tcp for Hop Web. Hop Web serves plain HTTP on port 80, so terminate TLS in front of it with your own domain for production.

Step 1 - Deploy from the Azure Marketplace

Sign in to the Azure Portal, choose Create a resource, search the Marketplace for Apache Hop by SecureAnyCloud, and select Create.

On the Basics tab, pick your subscription, resource group, region and size. Under Administrator account, choose SSH public key and paste your key. Under Inbound port rules, allow SSH (22) and HTTP (80).

Review the dedicated data disk on the Disks tab, then select Review + create and Create.

Step 2 - Deploy from the Azure CLI

Alternatively, deploy from the command line:

# Accept the Marketplace terms once per subscription (required before the first CLI deploy)
az vm image terms accept --urn sac1647283583153:apache-hop-ubuntu-24-04:default:latest
az vm create \
  --resource-group <your-rg> \
  --name apache-hop \
  --image sac1647283583153:apache-hop-ubuntu-24-04:default:latest \
  --size Standard_B2ms \
  --admin-username azureuser \
  --ssh-key-values ~/.ssh/id_ed25519.pub \
  --vnet-name <your-vnet> --subnet <your-subnet> \
  --public-ip-sku Standard
az vm open-port --resource-group <your-rg> --name apache-hop --port 80 --priority 1010

Step 3 - Connect to your VM

Connect to the VM over SSH:

ssh azureuser@<vm-public-ip>

Step 4 - Confirm the services are running

Check that all four services are active:

systemctl is-active docker.service hop-web.service hop-server.service nginx.service

All four report active. Hop Web runs as the official apache/hop-web container on the loopback address 127.0.0.1:8080, and nginx fronts it on port 80. Hop Server runs as the dedicated hop user on 127.0.0.1:8081.

All projects, pipelines, workflows and Hop configuration live on the dedicated Azure data disk mounted at /var/lib/hop.

Step 5 - Retrieve your password

Your VM generates a single unique password on first boot and uses it for both Hop Web (admin) and Hop Server (cluster). Read it from the root-only file:

sudo cat /root/hop-credentials.txt

This file contains HOP_ADMIN_USER (admin), HOP_CLUSTER_USER (cluster) and HOP_ADMIN_PASSWORD, along with the Hop Web and Hop Server URLs. Store the password somewhere safe.

Step 6 - Confirm the health endpoint

Request the unauthenticated health endpoint that nginx serves for load balancers and probes:

curl -s http://localhost/health

It returns ok. This endpoint never requires authentication, so you can use it for an Azure Load Balancer health probe.

Step 7 - Confirm the Hop Web container

Hop Web runs as a container managed by hop-web.service. Check that it is up:

sudo docker ps --filter name=hop-web --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}'

You will see the hop-web container Up and publishing 127.0.0.1:8080->8080/tcp.

Step 8 - Sign in to Hop Web

Browse to http://<vm-public-ip>/. When your browser prompts for HTTP Basic credentials, sign in with username admin and the password from Step 5. Hop Web opens on the preloaded samples project.

The file explorer on the left lists the project's pipelines, workflows, transforms and datasets. Double-click any pipeline (.hpl) to open it on the canvas. Use the project and file explorer to browse the full set of example pipelines and workflows shipped with the image.

Step 9 - Confirm authentication from the command line

nginx protects Hop Web with HTTP Basic authentication. A wrong password returns 401, and the per-VM password returns 200.

The command embeds your unique password, so run it interactively rather than from a script. Substitute the value of HOP_ADMIN_PASSWORD from Step 5 for <HOP_ADMIN_PASSWORD>:

curl -s -o /dev/null -w '%{http_code}\n' -u admin:<HOP_ADMIN_PASSWORD> http://127.0.0.1/

The response is 200. Repeating the command with a wrong password returns 401.

Step 10 - Use the Hop Server execution engine

Hop Server runs your pipelines and workflows headlessly and exposes a status page on port 8081, protected by Basic authentication for the cluster user. To view the status page from the command line, substitute your password for <HOP_ADMIN_PASSWORD>:

curl -s -u cluster:<HOP_ADMIN_PASSWORD> http://127.0.0.1:8081/hop/status/

The status page lists running pipelines and workflows and the server configuration. To run your pipelines and workflows on the server, point the Hop Server metadata element in Hop Web at 127.0.0.1:8081 with the cluster credentials.

Step 11 - Confirm project data lives on the dedicated disk

Your Hop projects, pipelines, workflows and Hop configuration are stored on the dedicated Azure data disk, so they survive OS changes and can be resized independently. Check the mount:

findmnt /var/lib/hop

The mount is backed by a separate Azure data disk captured into the image and re-provisioned on every VM.

Maintenance

Projects and pipelines: Create and edit projects, pipelines and workflows in Hop Web. Everything is stored under /var/lib/hop/project on the data disk.
Hop Server: The execution server configuration (including the cluster credential) is at /var/lib/hop/server/hop-server-config.xml. Restart with sudo systemctl restart hop-server after edits.
Hop Web: The editor runs as the hop-web container. Restart it with sudo systemctl restart hop-web.
Storage: Projects, pipelines, workflows and configuration live under /var/lib/hop on the data disk. Back up that volume to protect your work.
TLS: Hop Web serves plain HTTP on port 80. Front it with TLS (for example, certbot) and your own domain for production use.
Security patches: unattended-upgrades remains enabled, so the OS continues to receive security updates automatically.

Support

SecureAnyCloud provides 24/7 expert support for this image. Contact sales@secureanycloud.com.

Deploy on Azure

Launch Apache Hop on Ubuntu 24.04 LTS on Azure with 24/7 support from SecureAnyCloud.

View this image

Need Help?

Our support team is available 24/7.

sales@secureanycloud.com

Deploy on Azure

Overview

This image runs Apache Hop, the Hop Orchestration Platform, an open source data engineering and data orchestration platform for visually designing and running data pipelines and workflows. Hop is the successor to Pentaho Data Integration (Kettle) and provides a browser-based, drag-and-drop editor for building, running and monitoring data integration, transformation and orchestration logic.

What you get:

Hop Web, the browser-based visual pipeline and workflow editor, supervised by systemd. It runs as the official apache/hop-web container on the loopback interface on port 8080. An nginx reverse proxy on port 80 fronts it with HTTP Basic authentication, so the editor is only reachable after you sign in.
Hop Server, the headless pipeline and workflow execution server, supervised by systemd. It runs as a dedicated unprivileged hop service user on the loopback interface on port 8081 and has its own HTTP Basic authentication. Use it to run pipelines and workflows on a schedule or remotely.
Apache Hop 2.18.1 on OpenJDK 21, installed from the Ubuntu 24.04 archive. The Hop client distribution (which provides the execution server and the hop-run command line runner) is installed under /opt/hop
A dedicated EBS data volume mounted at /var/lib/hop for Hop projects, pipelines, workflows and the execution server configuration. It is separate from the operating system disk, so you can resize the data tier independently
Loopback-only Hop runtimes: nothing is exposed to the network except SSH on port 22 and the authenticated Hop Web editor on port 80
24/7 SecureAnyCloud support

Per-instance credentials

The image generates a fresh password on the first boot of every deployed instance and applies it to both the Hop Web editor (user admin) and the Hop Server execution server (user cluster). Because Hop Server can run pipelines remotely, replacing its default credentials on every instance is essential, and this image does that automatically. Two instances launched from the same Amazon Machine Image never share a password, and the well-known Hop stock defaults (cluster / cluster) are never present on a deployed instance.

The generated password is written to /root/hop-credentials.txt with mode 0600, so only the root user can read it.

Prerequisites

Before you deploy this image, make sure you have:

An Amazon Web Services account where you can launch EC2 instances
IAM permissions to launch instances, create security groups, and subscribe to AWS Marketplace products
An EC2 key pair in the target Region for SSH access to the instance
A VPC and subnet in the target Region, with a security group that allows inbound port 22 from your management network and inbound port 80 from the trusted networks that reach the Hop Web editor
The AWS CLI (version 2), installed locally, if you plan to deploy from the command line

Launch the instance from AWS Marketplace

Open the product page in the AWS Marketplace and choose Continue to Subscribe, then Continue to Configuration.
Select the region and the delivery option (the Ubuntu 24.04 AMI), then Continue to Launch.
Choose an instance type. The recommended type is m5.large. Hop runs a JVM and a container, so give it at least 8 GiB of memory for production workloads.
Select your key pair, VPC, subnet and a security group that allows inbound port 22 (SSH) and port 80 (the Hop Web editor) from your trusted networks, then launch.

Launch the instance from the AWS CLI

Alternatively, launch from the command line. Replace the AMI ID, key pair name, security group and subnet with your own values:

aws ec2 run-instances \
  --image-id <ami-id> \
  --instance-type m5.large \
  --key-name <your-key-pair> \
  --security-group-ids <your-security-group> \
  --subnet-id <your-subnet> \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=apache-hop}]'

Connect to your instance

Connect over SSH as the default login user for your operating system variant. This listing ships one variant:

OS variant SSH login user

Ubuntu 24.04 ubuntu

ssh <login-user>@<public-ip>

Retrieve your password

The per-instance password is written to a root-only file. Read it with sudo:

sudo cat /root/hop-credentials.txt

The file records the Hop Web URL, the admin user for the editor, the generated password, and the Hop Server URL with the cluster user. The same password works for both the editor and the execution server.

Verify the server is healthy

Confirm that all four product services are active:

sudo systemctl is-active docker hop-web hop-server nginx

Confirm the unauthenticated health endpoint returns ok:

curl -fsS http://127.0.0.1/health

Confirm that Hop Web (8080) and Hop Server (8081) are listening on the loopback interface:

LISTEN="$(sudo ss -tln)" printf '%s\n' "$LISTEN" | grep -E '127.0.0.1:8080|127.0.0.1:8081'

Open the Hop Web editor

Browse to the instance over HTTP on port 80:

http://<public-ip>/

When prompted, sign in as the user admin with the password from /root/hop-credentials.txt. The Hop Web visual editor opens with the bundled samples project, which contains ready-to-run example pipelines and workflows, plus the sac-hello pipeline this image ships. Drag transforms onto the canvas, connect them and build your first pipeline.

To confirm that the reverse proxy enforces authentication, test it directly on the instance. An unauthenticated request is refused with 401, and a request with the correct credentials returns 200:

curl -s -o /dev/null -w 'unauthenticated: HTTP %{http_code}\n' http://127.0.0.1/
curl -s -o /dev/null -w 'authenticated: HTTP %{http_code}\n' -u "admin:<HOP_ADMIN_PASSWORD>" http://127.0.0.1/

Run a Pipeline

The image ships a small deterministic pipeline, sac-hello, that generates exactly ten rows and passes them to an output transform. Run it with the hop-run command line runner to confirm that the execution engine works end to end. The output transform reads and writes exactly ten rows:

OUT="$(sudo -u hop env JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64 HOP_CONFIG_FOLDER=/var/lib/hop/server HOME=/var/lib/hop/server /bin/bash /opt/hop/hop-run.sh --project=samples --file='${PROJECT_HOME}/sac-hello.hpl' --runconfig=local 2>&1)" printf '%s\n' "$OUT" | grep -E 'generate 10 rows\.0 - Finished processing|output\.0 - Finished processing' printf '%s\n' "$OUT" | grep -q 'output.0 - Finished processing (I=0, O=0, R=10, W=10, U=0, E=0)' \
  && echo 'PIPELINE ROUND-TRIP OK: output transform read and wrote exactly 10 rows' \
  || { echo 'PIPELINE ROUND-TRIP FAILED'; exit 1; }

You can also run the same pipeline from the Hop Web editor. Open sac-hello.hpl from the file browser, click the run button, choose the local run configuration and launch. The execution log panel shows each transform finishing with ten rows.

Use the Hop Server execution server

The Hop Server execution server runs on the loopback interface on port 8081 with HTTP Basic authentication as the cluster user (the same per-instance password). It is not exposed to the network. To reach its status page from your workstation, forward the port over SSH:

ssh -L 8081:127.0.0.1:8081 <login-user>@<public-ip>

Then browse to http://127.0.0.1:8081/hop/status/ and sign in as cluster. From here you can submit pipelines and workflows for remote execution and monitor their status. To confirm that the server enforces authentication, test it directly on the instance:

curl -s -o /dev/null -w 'unauthenticated: HTTP %{http_code}\n' http://127.0.0.1:8081/hop/status/
curl -s -o /dev/null -w 'authenticated: HTTP %{http_code}\n' -u "cluster:<HOP_ADMIN_PASSWORD>" http://127.0.0.1:8081/hop/status/

Filesystem and service layout

Path Purpose

/opt/hop Apache Hop client distribution (hop-server.sh, hop-run.sh, hop-conf.sh)

/var/lib/hop/project The Hop project (pipelines, workflows, metadata), mounted into the Hop Web container

/var/lib/hop/config The Hop Web container configuration mount

/var/lib/hop/server Hop Server configuration folder and hop-server-config.xml

/root/hop-credentials.txt Per instance credentials (mode 0600, root only)

/etc/nginx/.hop_htpasswd nginx Basic auth password file for the admin user

The /var/lib/hop tree is a dedicated EBS data volume mounted by filesystem UUID, so you can resize it independently of the operating system disk.

Product services:

Service Role

docker.service Container runtime for the Hop Web editor

hop-web.service The apache/hop-web container on 127.0.0.1:8080

hop-server.service The Hop Server execution server on 127.0.0.1:8081

nginx.service Reverse proxy and Basic auth on port 80

Start, stop and check status

Check the status of the product services:

sudo systemctl status hop-web hop-server nginx

To restart the editor or the execution server:

sudo systemctl restart hop-web hop-server

Rotate the password

To set a new password after first boot, update the nginx Basic auth file and the Hop Server configuration, then reload nginx. Replace the value with your own strong secret:

sudo htpasswd -b /etc/nginx/.hop_htpasswd admin '<new-password>'
sudo systemctl reload nginx

Update the <password> element in /var/lib/hop/server/hop-server-config.xml to the same value. Then restart the execution server with sudo systemctl restart hop-server so the cluster credential matches.

Security recommendations

The Hop Web editor is served over plain HTTP with Basic authentication. Restrict inbound port 80 in your security group to trusted networks. For production, place your own TLS termination (for example, an Application Load Balancer or an nginx TLS server block with a CA-signed certificate) in front of the editor so credentials are not sent in clear text.
The Hop Server execution server can run pipelines remotely, which is powerful. This image binds it to the loopback interface only and never exposes it to the network. Keep it that way and reach it over an SSH tunnel, or restrict it tightly if you choose to expose it.
Keep the per-instance password secret and rotate it as described above. Never restore the Hop stock cluster / cluster default.
Apply operating system security updates regularly.

Backups and maintenance

Your pipelines, workflows and project metadata live under /var/lib/hop/project. Back up that directory, or snapshot the EBS data volume, to preserve your work. Because the data tier is a separate volume, you can snapshot it independently of the operating system disk.

Screenshots

The Hop Web visual editor, showing the welcome screen and the bundled samples project:

The sac-hello pipeline open on the visual canvas, with the generate rows transform feeding an output transform:

The run options dialog, where you choose the run configuration and launch a pipeline:

A successful pipeline run: both transforms complete (green ticks) and the execution log confirms the rows processed:

Support

SecureAnyCloud provides 24/7 technical support for this product by email at sales@secureanycloud.com and via live chat. Our engineers assist with Hop deployment, upgrades, pipeline and workflow design, execution server configuration and performance tuning. Critical issues receive a one-hour average response time.

Apache Hop, Hop and Apache are trademarks of the Apache Software Foundation. All other product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.

Deploy on AWS

Launch Apache Hop on AWS with 24/7 support from SecureAnyCloud.

View on Marketplace

Need Help?

Our support team is available 24/7.

sales@secureanycloud.com

Deploy on AWS