Roboflow Docs

Install on Linux

Install and run the Roboflow Inference Server on Linux with the Inference CLI, Docker, or Docker Compose, on CPU, GPU, or TensorRT.

The easiest way to start the correct container for your machine, with good default settings (a cache volume and a secure, non-privileged execution mode), is to let the CLI choose and start it with inference server start. Install Docker first:

pip install inference-cli
inference server start

Manually starting the container

If you want more control over the container settings, start it yourself.

The core CPU Docker image includes support for OpenVINO acceleration on x64 CPUs via onnxruntime. Heavy models like SAM2 may run too slowly (dozens of seconds per image) to be practical; if you need them, use a CUDA-capable GPU.

The primary use cases for CPU inference are processing still images (for example NSFW classification of uploads or document verification) or infrequent sampling of frames from a video (for example occupancy tracking of a parking lot).

To get started with CPU inference, use the roboflow/roboflow-inference-server-cpu:latest container.

sudo docker run -d \
    --name inference-server \
    --read-only \
    -p 9001:9001 \
    --volume ~/.inference/cache:/tmp:rw \
    --security-opt="no-new-privileges" \
    --cap-drop="ALL" \
    --cap-add="NET_BIND_SERVICE" \
    roboflow/roboflow-inference-server-cpu:latest

Docker Compose

If you use Docker Compose for your application, the equivalent YAML is:

version: "3.9"

services:
  inference-server:
    container_name: inference-server
    image: roboflow/roboflow-inference-server-cpu:latest

    read_only: true
    ports:
      - "9001:9001"

    volumes:
      - "${HOME}/.inference/cache:/tmp:rw"

    security_opt:
      - no-new-privileges
    cap_drop:
      - ALL
    cap_add:
      - NET_BIND_SERVICE

Roboflow Enterprise plans add a Helm chart for Kubernetes deployments, networking solutions for OT networks, and customized support and installation packages. Contact the sales team to learn more.

Next steps