Running TestZeus Hercules with docker#


Author	Juha-Matti Hietala
Last Updated	30.07.2025
Status	In Progress

Requirements#

* Ubuntu 24.04 * Docker installed

Introduction#

This guide is taken pretty much straight from the official TestZeus Hercules documentation with a few additions from myself, as I couldn't get it to running properly. I have mentioned in the text to what point I managed to get and the official guide will continue further.

QEMU#

TestZeus Hercules docker image supports arm64 processor architecture and the Ubuntu 24.04 architecture uses amd64. When trying to run the image there will be and error message that the image's architecture is not supported. A way to fix this I found was to emulate the amr64 processor architecture with QEMU. From wikipedia: "QEMU is a free and open-source hypervisor. It emulates the machine’s processor through dynamic binary translation and provides a set of different hardware and device models for the machine, enabling it to run a variety of guest operating systems."

There are different ways to try to fix this problem, like building the docker image again with the right architecture, but due to time constraints I chose emulation as it seemed the fastest solution, not necessary the best.

Installing QEMU#

First look what platforms your docked driver supports:

sudo docker buildx ls

You should see something like this:

NAME/NODE DRIVER/ENDPOINT STATUS  PLATFORMS
default * docker                  
 default default         running linux/amd64, linux/386

Assuming there isn't linux/arm64 listed, next install QEMU:

sudo apt-get install -y qemu qemu-user-static

Check supperted platforms again:

sudo docker buildx ls

Now there should be several more showing:

NAME/NODE    DRIVER/ENDPOINT             STATUS  PLATFORMS
default      docker                              
  default    default                     running linux/amd64, linux/386, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/arm/v7, linux/arm/v6

Using Docker#

Pull the image:

sudo docker pull testzeus/hercules:latest

Non-Bulk Mode (Default)#

Prepare your local directory structure:

mkdir -p opt/input opt/output opt/test_data
cp your-test.feature opt/input/
cp your-test-data.json opt/test_data/

Run the container with environment file and specify platform to linux/arm64, otherwise it tries to run it with amd64. Example .env file a step further below. Create the .env file to where you run the command:

sudo docker run --platform linux/arm64 --env-file=.env \
  -v ./agents_llm_config.json:/testzeus-hercules/agents_llm_config.json \
  -v ./opt:/testzeus-hercules/opt \
  --rm -it testzeus/hercules:latest

Example .env file, change accordingly#

LLM_MODEL_NAME=PUT YOUR MODEL NAME HERE, for example: gpt-4o, gpt-4o
LLM_MODEL_API_KEY=PUT YOUR MODEL API KEY HERE
# LLM_MODEL_BASE_URL=IF YOU ARE USING OpenAI, remove this line, otherwise add the base URL, for example: https://api.groq.com/openai/v1

# If you want to use json file for config (see sample agents_llm_config.json and the Readme)
# AGENTS_LLM_CONFIG_FILE=agents_llm_config.json
# AGENTS_LLM_CONFIG_FILE_REF_KEY=The key for the config stanza to use from the file
# PROJECT_SOURCE_ROOT=folder where to save data.

# optional
# CDP_ENDPOINT_URL=ws://localhost:9222/devtools/browser/b12de40a-fc1c-4ebb-81d9-046e901f376d

TOKENIZERS_PARALLELISM=false
MODE=prod
HEADLESS=false
RECORD_VIDEO=false
TAKE_SCREENSHOTS=true
BROWSER_TYPE=chromium
CAPTURE_NETWORK=true
HF_HOME=./.cache
# select the device from this file https://github.com/microsoft/playwright/blob/main/packages/playwright-core/src/server/deviceDescriptorsSource.json
# RUN_DEVICE="iPhone 15 Pro Max"
# GEO_API_KEY=
# GEO_PROVIDER=maps_co

# LOAD_EXTRA_TOOLS=true
# EXECUTE_BULK=true

Important Note: Docker runs are always in headless mode (no visible browser). To connect to a visible browser, you'll need to use the CDP_ENDPOINT_URL environment variable to connect to an external browser instance, more on that further down.

This is how far I got successfully. Now the image runs but for some reason seems to try to open a browser and gives an error message, even though the image should run in headless mode by default. The .env file has HEADLESS=true also but doesn't help. Tried external browser, which is explained more further down, but ran out of time to get it working, had timeout on trying to connect. Might be the solution if worked on further.

Bulk Mode in Docker#

Set up environment file (.env):

EXECUTE_BULK=true
# ... other environment variables ...

Prepare multiple test directories locally:

mkdir -p opt/tests/test1/input opt/tests/test1/test_data
mkdir -p opt/tests/test2/input opt/tests/test2/test_data
# Copy your feature files and test data to respective directories

Run the container:

sudo docker run --platform linux/arm64 --env-file=.env \
  -v ./agents_llm_config.json:/testzeus-hercules/agents_llm_config.json \
  -v ./opt:/testzeus-hercules/opt \
  --rm -it testzeus/hercules:latest

Expected outcome: - Tests execute sequentially in headless mode - Each test directory processed independently - Results stored in respective test directories under opt/tests/

Connecting to External Browser (Docker Mode)#

To run tests with a visible browser while using Docker:

Set up a Chrome instance with remote debugging enabled

Add to your .env file:

CDP_ENDPOINT_URL=ws://your-chrome-host:9222

This allows Hercules running in Docker to connect to an external browser instance, enabling visible test execution even when running in a container.

To set cookies for the browser context, you can use the BROWSER_COOKIES environment variable:

export BROWSER_COOKIES='[{"name": "session", "value": "123456", "domain": "example.com", "path": "/", "httpOnly": true, "secure": true}]'

The cookies must be provided as a JSON array of cookie objects with the following properties: - name: Cookie name (required) - value: Cookie value (required) - domain: Cookie domain (required) - path: Cookie path (defaults to "/") - httpOnly: Whether the cookie is HTTP-only (defaults to false) - secure: Whether the cookie is secure (defaults to false) - expires: Cookie expiration time in seconds since epoch (defaults to -1, session cookie)

Cookies are added to the browser context after it's created, ensuring compatibility with all browser types (Chromium, Firefox, WebKit).

Remote Browser Testing Platforms#

Hercules supports running tests on various remote browser platforms using the CDP (Chrome DevTools Protocol) endpoint URL. Here's how to configure different providers:

BrowserStack#

Set up your credentials:

export BROWSERSTACK_USERNAME=your_username
export BROWSERSTACK_ACCESS_KEY=your_access_key

Generate CDP URL:

export CDP_ENDPOINT_URL=$(python helper_scripts/browser_stack_generate_cdp_url.py)

Note: Video recording is not supported with BrowserStack as it uses the connect API.

LambdaTest#

Set up your credentials:

export LAMBDATEST_USERNAME=your_username
export LAMBDATEST_ACCESS_KEY=your_access_key

Generate CDP URL:

export CDP_ENDPOINT_URL=$(python helper_scripts/lambda_test_generate_cdp_url.py)

Note: Video recording is not supported with LambdaTest as it uses the connect API.

BrowserBase#

Set the CDP URL directly:

export CDP_ENDPOINT_URL=wss://connect.browserbase.com?apiKey=your_api_key

BrowserBase supports connect_over_cdp, so video recording will work on the test execution host.

AnchorBrowser#