Docker Configuration Options

Inference servers have a number of configurable parameters that you set with environment variables. To set an environment variable with docker run, use the -e flag:

docker run -it --rm -e ENV_VAR_NAME=env_var_value -p 9001:9001 --gpus all roboflow/roboflow-inference-server-gpu:latest

This page covers the options you are most likely to change. For the complete list, see Environment Variables.

Networking

HOST: string (default 0.0.0.0). Sets the host address used by HTTP interfaces.

PORT: integer (default 9001). Sets the port used by HTTP interfaces.

ALLOW_ORIGINS: string (default *). Sets the allow_origins property on the CORS middleware used with FastAPI for HTTP interfaces. Multiple values can be provided separated by a comma, for example ALLOW_ORIGINS=orig1.com,orig2.com.

Inference behavior

CLASS_AGNOSTIC_NMS: boolean (default False). Sets the default non-maximum suppression (NMS) behavior for detection models (object detection, instance segmentation, and similar). When True, NMS is class agnostic, so overlapping detections from different classes may be removed based on the IoU threshold. When False, only overlapping detections from the same class are considered for removal.

MAX_CANDIDATES: integer (default 3000). The maximum number of candidates for detection.

MAX_DETECTIONS: integer (default 300). The maximum number of detections returned by a model.

FIX_BATCH_SIZE: boolean (default False). When True, the batch size is fixed to the maximum batch size configured for this server.

MAX_ACTIVE_MODELS: integer (default 8). The maximum number of models the internal model manager keeps in memory at one time. By default the model queue removes the least recently accessed model when making space for a new one.

NUM_WORKERS: integer (default 1). The number of workers used by HTTP interfaces.

CLIP model options

CLIP_VERSION_ID: string (default ViT-B-16). Sets the OpenAI CLIP version used by all /clip routes. Available versions are RN101, RN50, RN50x16, RN50x4, RN50x64, ViT-B-16, ViT-B-32, ViT-L-14-336px, and ViT-L-14.

CLIP_MAX_BATCH_SIZE: integer (default 8). Sets the max batch size accepted by the CLIP model inference functions.

Model cache

MODEL_CACHE_DIR: string (default /tmp/cache). Sets the container path for the root model cache directory.

TENSORRT_CACHE_PATH: string (default: the value of MODEL_CACHE_DIR). Sets the container path to the TensorRT cache directory. Setting this path together with a mounted host volume reduces the cold start time of TensorRT-based servers.

Persistent model cache

By default model weights are stored inside the container at /tmp/cache and are lost on container restart or system reboot. For production deployments, mount a persistent host volume to preserve downloaded weights:

# Create a persistent cache directory on the host
mkdir -p /var/lib/roboflow/cache

# Run the container with a persistent cache
docker run -d \
  -p 9001:9001 \
  -v /var/lib/roboflow/cache:/tmp/cache \
  -e MODEL_CACHE_DIR=/tmp/cache \
  roboflow/roboflow-inference-server-cpu:latest

Things to keep in mind:

• The host path should be on persistent storage, not in /tmp.
• The mounted directory needs appropriate permissions for the container user (typically UID 1000 or root, depending on the image).
• A persistent cache lets you pre-populate weights before deployment and keeps them across container updates.

See Offline weights download for more on pre-downloading and caching weights.

HTTPS / TLS

ENABLE_HTTPS: boolean (default False). When set, the Inference Server serves traffic over HTTPS instead of HTTP, reading the certificate and private key from SSL_CERTFILE and SSL_KEYFILE.

SSL_CERTFILE: string (default /etc/inference/certs/server.crt) and SSL_KEYFILE: string (default /etc/inference/certs/server.key). Paths to the PEM-encoded certificate and private key inside the container. The defaults are convenient mount points, so usually you only need to bind your cert and key into /etc/inference/certs/ and set ENABLE_HTTPS=true.

SSL_KEYFILE_PASSWORD: string (optional). Set this if your private key is encrypted.

SSL_CA_CERTS: string (optional). Set this to a CA bundle when you need client certificate verification (mTLS).

Full walkthrough with self-signed certificates: Serving Inference over HTTPS.

Secure Gateway

SECURE_GATEWAY: string (default unset). Sets the address of a Roboflow Secure Gateway for air-gapped deployments. Roboflow API and model download traffic is routed through this proxy. Traffic that cannot be proxied is disabled or rerouted as follows:

• The Inference version check (which calls api.github.com) is force-disabled: DISABLE_VERSION_CHECK is set to True even if explicitly configured otherwise.
• If WORKFLOWS_STEP_EXECUTION_MODE=remote is combined with WORKFLOWS_REMOTE_API_TARGET=hosted, step execution falls back to local with a warning, because the hosted Roboflow inference endpoints cannot be reached through the gateway proxy. To keep remote execution, use WORKFLOWS_REMOTE_API_TARGET=self-hosted and point LOCAL_INFERENCE_API_URL (default http://127.0.0.1:9001) at an Inference server reachable inside the gateway perimeter.
• Third-party integrations (Google Vision and Gemini direct-key paths, Stability AI, Twilio media upload, webhooks to external hosts) are not proxied and will fail unless the gateway network permits them.

The legacy LICENSE_SERVER environment variable is still accepted but deprecated.