RF-DETR

RF-DETR Object Detection

RF-DETR is Roboflow's transformer-based real-time detection model. Run inference against COCO-pretrained object detection checkpoints through the Serverless Cloud API, or self-host using Roboflow Inference.

RF-DETR Object Detection API

The steps below run RF-DETR through the Serverless Cloud API and visualize results with supervision.

1

Get your API Key

Create a Roboflow account, find your key on the Roboflow API settings page and make it available to your shell:

Copy

export ROBOFLOW_API_KEY="your-key-here"

2

Install the dependencies

These two packages call the model and draw its results:

Copy

pip install -U inference-sdk supervision

3

Run the model

Run rfdetr-small on a sample image and annotate boxes and labels:

Copy

import os
import cv2
import supervision as sv
from inference_sdk import InferenceHTTPClient

image_url = "https://media.roboflow.com/quickstart/traffic.jpg"
image = sv.load_image_from_url(image_url)

client = InferenceHTTPClient(
    api_url="https://serverless.roboflow.com",
    api_key=os.environ["ROBOFLOW_API_KEY"],
)
result = client.infer(image, model_id="rfdetr-small")

detections = sv.Detections.from_inference(result)

annotated = sv.BoxAnnotator().annotate(image.copy(), detections)
annotated = sv.LabelAnnotator().annotate(annotated, detections)

cv2.imwrite("traffic-annotated.png", annotated)

RF-DETR Object Detection models and benchmarks

Pass any of these aliases as the model_id when running inference. The SDK resolves each alias to the underlying Roboflow project version.

RF-DETR Instance Segmentation

RF-DETR also provides instance segmentation checkpoints that predict masks alongside boxes. Run them through the Serverless Cloud API, or self-host using Roboflow Inference.

RF-DETR Instance Segmentation API

Set your API key and install the dependencies as shown above, then run a segmentation checkpoint and draw its masks:

import os
import cv2
import supervision as sv
from inference_sdk import InferenceHTTPClient

image_url = "https://media.roboflow.com/quickstart/traffic.jpg"
image = sv.load_image_from_url(image_url)

client = InferenceHTTPClient(
    api_url="https://serverless.roboflow.com",
    api_key=os.environ["ROBOFLOW_API_KEY"],
)
result = client.infer(image, model_id="rfdetr-seg-preview")

detections = sv.Detections.from_inference(result)

annotated = sv.MaskAnnotator().annotate(image.copy(), detections)
annotated = sv.LabelAnnotator().annotate(annotated, detections)

cv2.imwrite("traffic-annotated.png", annotated)

RF-DETR Instance Segmentation models and benchmarks

Pass any of these aliases as the model_id when running inference. The SDK resolves each alias to the underlying Roboflow project version. Figures are mask mAP on COCO val.

RF-DETR Keypoint Detection

RF-DETR keypoint detection is a preview checkpoint, pretrained on COCO person keypoints. Run it through the Serverless Cloud API, or self-host using Roboflow Inference.

RF-DETR Keypoint Detection API

Set your API key and install the dependencies as shown above, then run rfdetr-keypoint-preview and annotate keypoints:

import os
import cv2
import numpy as np
import supervision as sv
from inference_sdk import InferenceHTTPClient

COCO_KEYPOINTS = [
    "nose", "left_eye", "right_eye", "left_ear", "right_ear",
    "left_shoulder", "right_shoulder", "left_elbow", "right_elbow",
    "left_wrist", "right_wrist", "left_hip", "right_hip",
    "left_knee", "right_knee", "left_ankle", "right_ankle",
]

image_url = "https://media.roboflow.com/notebooks/examples/person-walking.png"
image = sv.load_image_from_url(image_url)

client = InferenceHTTPClient(
    api_url="https://serverless.roboflow.com",
    api_key=os.environ["ROBOFLOW_API_KEY"],
)
result = client.infer(image, model_id="rfdetr-keypoint-preview")

# Each prediction lists only the keypoints the model can see, so place them in
# the 17 COCO slots the skeleton expects and mark the missing ones not visible.
xy, visible = [], []
for prediction in result["predictions"]:
    found = {point["class"]: (point["x"], point["y"]) for point in prediction["keypoints"]}
    xy.append([found.get(name, (0, 0)) for name in COCO_KEYPOINTS])
    visible.append([name in found for name in COCO_KEYPOINTS])

key_points = sv.KeyPoints(xy=np.array(xy, dtype=np.float32), visible=np.array(visible))

annotated = sv.EdgeAnnotator(color=sv.Color.GREEN, thickness=3).annotate(image.copy(), key_points)
annotated = sv.VertexAnnotator(color=sv.Color.RED, radius=5).annotate(annotated, key_points)
cv2.imwrite("person-walking-annotated.png", annotated)

RF-DETR Keypoint Detection models and benchmarks

Pass the alias as the model_id when running inference. Accuracy is COCO val AP50-95 scored with object keypoint similarity (OKS), the standard COCO keypoint metric, so it does not compare to the box and mask mAP above.

Roboflow does not publish a prebuilt TensorRT engine for this checkpoint yet, so Inference runs it on ONNX Runtime even when you install the inference-models[trt10] extra. Reaching the latency in the table means building the engine yourself with the rfdetr package.

† Keypoint accuracy and latency are the figures published in the RF-DETR benchmarks: latency is TensorRT FP16 on 1x NVIDIA T4 at batch size 1, timing the model only. That is a different GPU and a different pipeline from the tables above, so the two sets of latencies do not compare directly.

* Latency is measured with Roboflow Inference on 1x NVIDIA L4, batch size 1, mean of 1,000 inferences (100 warmup). The default inference-gpu install runs ONNX on the CUDA execution provider; adding the inference-models[trt10] extra selects a prebuilt TensorRT FP16 engine automatically. FP16 matches FP32 accuracy within 0.2 mAP on COCO val2017. Accuracy is the published COCO val spec (see the RF-DETR announcement).