Hub Configuration

This document describes advanced configuration options for the Apheris Hub. For basic deployment configuration (storage paths, database connection, authentication, model selection), see the Docker Deployment Guide or Kubernetes Deployment Guide.

Request Timeout Configuration

The Apheris Hub includes timeout mechanisms to automatically handle requests that remain in pending or accepted states for extended periods. These timeouts help maintain system health by preventing stale requests from accumulating.

Environment Variables

APH_HUB_PENDING_REQUEST_TIMEOUT

Description: Maximum duration a request can remain in the pending state before being automatically marked as failed.

Default: 12h (12 hours)

Format: Duration string (e.g., 30m, 2h, 12h, 24h)

Purpose: When a request is first submitted to the Hub, it enters the pending state while waiting to be accepted by a model worker. If the request remains in this state beyond the configured timeout, the Hub's dispatcher automatically marks it as failed. This prevents requests from remaining indefinitely in the pending state when workers are unavailable or system issues prevent request acceptance.

Example:

APH_HUB_PENDING_REQUEST_TIMEOUT=1h

APH_HUB_ACCEPTED_REQUEST_TIMEOUT

Description: Maximum duration a request can remain in the accepted state before being automatically marked as failed.

Default: 12h (12 hours)

Format: Duration string (e.g., 30m, 2h, 12h, 24h)

Purpose: After a request is accepted by a model worker, it transitions to the accepted state before moving to running. If a request remains in the accepted state beyond the configured timeout (for example, due to worker failures or resource constraints), the Hub's dispatcher automatically marks it as failed. This ensures that accepted but stalled requests do not block system resources indefinitely.

Example:

APH_HUB_ACCEPTED_REQUEST_TIMEOUT=2h

Configuration in Deployment

These environment variables can be set when deploying the Apheris Hub container. You can configure them via the Docker deployment script (using APH_HUB_* environment variables) or through the Helm chart (via hub.env).

Docker deployment

To override the default timeouts when using the Docker deployment script, set the environment variables before running the script:

APH_HUB_PENDING_REQUEST_TIMEOUT=1h \
APH_HUB_ACCEPTED_REQUEST_TIMEOUT=2h \
./deploy_apherisfold

The script forwards these values directly into the Hub container.

Kubernetes deployment

To configure the same timeouts in a Kubernetes deployment, add entries under hub.env in your values file:

hub:
  env:
    - name: APH_HUB_PENDING_REQUEST_TIMEOUT
      value: "1h"
    - name: APH_HUB_ACCEPTED_REQUEST_TIMEOUT
      value: "2h"

Request Lifecycle

The timeout configurations correspond to specific phases in the request lifecycle:

1. Pending → The request is waiting to be accepted by a model worker
2. Accepted → The request has been accepted and is waiting to start running
3. Running → The request is actively being processed by the model
4. Done or Failed → The request has completed successfully or encountered an error

The timeout mechanisms monitor requests in the pending and accepted states, automatically transitioning them to failed if they exceed their respective timeout durations.

Note

These timeouts only apply to requests in the pending and accepted states. Once a request enters the running state, it is not subject to these timeout mechanisms and will run until completion or until the underlying task fails.