Hub Configuration

This document describes the configuration options available for the Apheris Hub.

Request Timeout Configuration

The Apheris Hub includes timeout mechanisms to automatically handle requests that remain in pending or accepted states for extended periods. These timeouts help maintain system health by preventing stale requests from accumulating.

Environment Variables

APH_HUB_PENDING_REQUEST_TIMEOUT

Description: Maximum duration a request can remain in the pending state before being automatically marked as failed.

Default: 12h (12 hours)

Format: Duration string (e.g., 30m, 2h, 12h, 24h)

Purpose: When a request is first submitted to the Hub, it enters the pending state while waiting to be accepted by a model worker. If the request remains in this state beyond the configured timeout, the Hub's dispatcher automatically marks it as failed. This prevents requests from remaining indefinitely in the pending state when workers are unavailable or system issues prevent request acceptance.

Example:

--env=APH_HUB_PENDING_REQUEST_TIMEOUT=1h

APH_HUB_ACCEPTED_REQUEST_TIMEOUT

Description: Maximum duration a request can remain in the accepted state before being automatically marked as failed.

Default: 12h (12 hours)

Format: Duration string (e.g., 30m, 2h, 12h, 24h)

Purpose: After a request is accepted by a model worker, it transitions to the accepted state before moving to running. If a request remains in the accepted state beyond the configured timeout (for example, due to worker failures or resource constraints), the Hub's dispatcher automatically marks it as failed. This ensures that accepted but stalled requests do not block system resources indefinitely.

Example:

--env=APH_HUB_ACCEPTED_REQUEST_TIMEOUT=2h

Configuration in Deployment

These environment variables can be set when deploying the Apheris Hub container:

docker run \
  --name=apheris-hub \
  --detach \
  --rm \
  --network=apheris-hub \
  --platform=linux/amd64 \
  --user=$(id -u):$(stat -c '%g' /var/run/docker.sock) \
  --publish=127.0.0.1:8080:8080 \
  --mount=type=bind,src=$HOME/apheris-hub,dst=$HOME/apheris-hub \
  --mount=type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \
  --env=APH_HUB_API_KEY=KEY_PROVIDED_BY_APHERIS \
  --env=APH_HUB_INPUT_ROOT_DIRECTORY=$HOME/apheris-hub/input \
  --env=APH_HUB_OUTPUT_ROOT_DIRECTORY=$HOME/apheris-hub/output \
  --env=APH_HUB_DATABASE_FILE_NAME=$HOME/apheris-hub/db/hub.db \
  --env=APH_HUB_PENDING_REQUEST_TIMEOUT=12h \
  --env=APH_HUB_ACCEPTED_REQUEST_TIMEOUT=12h \
  quay.io/apheris/hub:1.0.0

Request Lifecycle

The timeout configurations correspond to specific phases in the request lifecycle:

1. Pending → The request is waiting to be accepted by a model worker
2. Accepted → The request has been accepted and is waiting to start running
3. Running → The request is actively being processed by the model
4. Done or Failed → The request has completed successfully or encountered an error

The timeout mechanisms monitor requests in the pending and accepted states, automatically transitioning them to failed if they exceed their respective timeout durations.

Note

These timeouts only apply to requests in the pending and accepted states. Once a request enters the running state, it is not subject to these timeout mechanisms and will run until completion or until the underlying task fails.