Pick the installation method that matches your platform. All paths start the server on port 9001.
Docker is the preferred way to run Inference (see why Docker). It works on Linux, macOS, Windows, Jetson, and other Docker-capable devices.
Install Docker first (plus the NVIDIA Container Toolkit if you have a CUDA-enabled GPU), then install and run the Inference CLI:
pip install inference-cli && inference server startThis automatically chooses and configures the optimal container for your machine.
Run an Inference Server on Windows with the native desktop app, with no Docker required.
- Download the latest installer and run it.
- When the install finishes, it offers to launch the Inference Server.
- To stop the server, close the terminal window it opens.
- To start it again later, find Roboflow Inference in your Start Menu.
See Install on Windows for details and the Docker alternative.
Run an Inference Server on an Apple Silicon Mac with the native desktop app, with no Docker required.
- Download the DMG and open it.
- Drag the Roboflow Inference app to your Applications folder.
- Double-click the app in Applications to start the server.
See Install on Mac for details, the Docker alternative, and MPS acceleration.
Requirements
Inference adapts to your machine and runs faster on more powerful hardware. The floor is a 64-bit processor, 4 GB of RAM, and 20 GB of free disk space. Docker is required for the container paths above.
| Target | Hardware | OS | Docker image |
|---|---|---|---|
| CPU | 64-bit CPU, 4 GB RAM, 20 GB free disk. Heavy models (e.g. SAM2) may be too slow to be practical. | Linux, macOS, or Windows 10/11 with WSL 2 | roboflow/roboflow-inference-server-cpu |
| GPU | CUDA-capable NVIDIA GPU with the NVIDIA Container Toolkit installed. Recommended for larger models and live video. | Linux (or Windows 10/11 with WSL 2) | roboflow/roboflow-inference-server-gpu |
| NVIDIA Jetson | Jetson Orin device (Orin NX 16 GB or above recommended), running JetPack 4.5, 4.6, 5.x, or 6.x. Allow ~10 GB free disk for the image. | JetPack / L4T | roboflow/roboflow-inference-server-jetson-* (JetPack-specific, auto-selected) |
See Minimum Requirements for the full list of supported and suggested devices.
Device-specific guides
Special installation notes and performance tips by device:
- Linux
- Windows
- Mac
- NVIDIA Jetson
- Raspberry Pi
- Other devices
- Deploy in your own cloud - AWS, Azure, or GCP
If you cannot run Docker at all, the Inference Library runs models in your own Python process instead of a server.
Running the container yourself
You do not usually pick the image by hand: inference server start detects your hardware and runs docker run for you with recommended security settings, caching, and platform-specific options. If you would rather manage the container yourself, use the CPU image on a CPU-only host, or the GPU image with --gpus all on a CUDA host.
sudo docker run -d \
--name inference-server \
--read-only \
-p 9001:9001 \
--volume ~/.inference/cache:/tmp:rw \
--security-opt="no-new-privileges" \
--cap-drop="ALL" \
--cap-add="NET_BIND_SERVICE" \
roboflow/roboflow-inference-server-cpu:latestInstall the NVIDIA Container Toolkit first, then add --gpus all:
sudo docker run -d \
--name inference-server \
--gpus all \
--read-only \
-p 9001:9001 \
--volume ~/.inference/cache:/tmp:rw \
--security-opt="no-new-privileges" \
--cap-drop="ALL" \
--cap-add="NET_BIND_SERVICE" \
roboflow/roboflow-inference-server-gpu:latestYour platform's guide has a "Manually starting the container" section with the exact flags for that device.
Updating
Docker images default to the :latest tag. To move to the newest server, pull the latest image, or re-run inference server start, which pulls it for you:
docker pull roboflow/roboflow-inference-server-gpu:latestFor reproducible deployments, pin a specific version tag instead of :latest so an update never changes behavior unexpectedly, for example roboflow/roboflow-inference-server-gpu:<version>. Browse available tags on Docker Hub, and update deliberately by bumping the pinned tag.
Securing your server
A self-hosted server does not enforce authentication, encryption, or network restrictions by default, so securing it is your responsibility. Before exposing it beyond local development traffic, review Securing a Self-Hosted Server.
Using your new server
Once the server is running, call it over its HTTP API or with the Inference SDK. See Run a model for the first request, and Docker configuration options for tuning the container.
Enterprise considerations
A Helm chart is available for enterprise cloud deployments, and enterprise networking solutions that support deployment in OT networks are available on request.
Roboflow also offers customized support and installation packages and a pre-configured Jetson-based edge device suitable for rapid prototyping. Contact the sales team if you are part of a large organization and want to learn more. See Enterprise Deployment for the full feature set.