Luxonis
    Our new model ZOO works with DepthAI V3. Find out more in our documentation.
    Model Details
    Model Description
    RF-DETR Nano is a real-time transformer-based object detection model from Roboflow's RF-DETR family. It is the smallest detection variant in the lineup, designed for low-latency inference while retaining strong COCO accuracy. The model is built on a DINOv2-based vision transformer backbone and available at different size variants.
    • Developed by: Roboflow
    • Shared by:
    • Model type: Computer Vision
    • License:
    • Resources for more information:
    To deploy RF-DETR to Luxonis RVC4 devices some modifications need to be made to the model architecture. This are mainly there to make the operations compatible with the device accelerator. Please refer to to learn more about this. Note that currently only FP16_STANDARD precision is supported for this model family.
    Training Details
    Training Data
    The base RF-DETR Nano detection checkpoint is trained on the dataset. Official RF-DETR detection models are evaluated on COCO and RF100-VL, and this export uses the COCO label space.
    The exported parser metadata contains 91 COCO category slots (including background and unused category IDs), which matches the standard COCO indexing used by the packaged parser, while the underlying detection task covers the standard 80 foreground COCO classes.
    Testing Details
    Metrics
    The following benchmark numbers are taken from the . COCO metrics are reported for the validation split
    ModelCOCO AP50COCO AP50:95RF100-VL AP50RF100-VL AP50:95Params
    (M)
    RF-DETR Nano67.648.485.057.730.5
    Technical Specifications
    Input/Output Details
    • Input:
      • Name: input
        • Info: NCHW BGR un-normalized image
    • Outputs:
      • Name: dets
        • Info: Bounding box tensor of shape [1, 300, 4] for up to 300 detections
      • Name: labels
        • Info: Per-detection COCO class score tensor of shape [1, 300, 91]
    Model Architecture
    • Backbone: DINOv2 windowed-small vision transformer encoder (dinov2_windowed_small)
    • Neck: RF-DETR feature projection over the P4 scale
    • Head: RF-DETR detection transformer head with 2 decoder layers and 300 object queries
    For Nano specifically, the upstream configuration uses feature taps at transformer blocks [3, 6, 9, 12], a patch size of 16, and a default resolution of 384x384.
    Throughput
    Model variant: rfdetr-nano:coco-384x384
    • Input shape: [1, 3, 384, 384] • Output shapes: [[1, 300, 4], [1, 300, 91]]
    • Params (M): 26.863 • GFLOPs: 16.556
    PlatformPrecisionThroughput (infs/sec)Power Consumption (W)
    RVC4FP1629.083.69
    * Benchmarked with , using 2 threads (and the DSP runtime in balanced mode for RVC4).
    * Parameters and FLOPs are obtained from the exported ONNX files with the package.
    Utilization
    Models converted for RVC platforms can be used for inference on OAK devices. DepthAI pipelines are used to define the information flow linking the device, inference model, and the output parser. Below, we present the most crucial utilization steps for this model. Please consult the docs for more information.
    Install DAIv3 library:
    pip install depthai
pip install depthai-nodes
    Define model:
    model_description = dai.NNModelDescription(
    "luxonis/rfdetr-nano:coco-384x384"
)

nn = pipeline.create(ParsingNeuralNetwork).build(
    <CameraNode>, model_description
)
    The model is automatically parsed by DAI and it outputs the message (bounding boxes, labels, and confidence scores for the detected objects).
    Get model output(s):
    while pipeline.isRunning():
    nn_output: dai.ImgDetections = parser_output_queue.get()
    Example
    You can quickly run the model using the script. It automatically downloads the model, creates a DepthAI pipeline, runs the inference, and displays the results using the DepthAI visualizer tool. To try it out, run:
    python3 main.py \
    --model luxonis/rfdetr-nano:coco-384x384
    RF-DETR Nano
    RF-DETR is a real-time transformer architecture for object detection and instance segmentation developed by Roboflow.
    License
    Apache 2.0
    Commercial use
    Downloads
    2
    Tasks
    Object Detection
    Model Types
    ONNX
    Model Variants
    NameVersionAvailable ForCreated AtDeploy
    		RVC41 day ago
    Luxonis - Robotic vision made simple.
    XYouTubeLinkedInGitHub