Inference is tested and supported on x64 and ARM processors, optionally with an NVIDIA/CUDA GPU. Running on other devices may be possible but is not officially tested or supported.
Other GPUs
Hardware acceleration on non-NVIDIA, non-Apple GPUs is not currently supported, but ONNX Runtime has additional execution providers for AMD/ROCm, Arm NN, Rockchip, and others.
If you install one of these runtimes, you can enable it with the ONNXRUNTIME_EXECUTION_PROVIDERS environment variable. For example:
export ONNXRUNTIME_EXECUTION_PROVIDERS="[ROCMExecutionProvider,OpenVINOExecutionProvider,CPUExecutionProvider]"This is untested and performance improvements are not guaranteed. Acceleration of non-CUDA GPUs is unlikely to work inside Docker. See the Mac install guide for an example of how to run the server outside of a container.
Other edge devices
Roboflow has SDKs for running object detection natively on other deployment targets, including TensorFlow.js in a web browser, native Swift on iOS via CoreML, and Snap Lens Studio. See the SDKs overview for the full list.
For additional functionality, such as running Workflows and other types of models on another device, connect to an Inference Server over HTTP with the Inference SDK.