RT Super Resolution

Our new model ZOO works with DepthAI V3. Find out more in our documentation.

0 Likes

Model Details

Model Description

Real Time (RT) Super Resolution is a robust light-weight model for recovering image resolution. It requires no guide image to perform the enhancement, thus falling under the category of single-image super-resolution (SISR) rather than guided super-resolution (GSR). The model is based on framework with a few modifications in the objective function.

Developed by: Rishik Mourya
Model type: Computer Vision
License:
Resources for more information:

Photo by Kampus Production from

Training Details

Training Data

Testing Details

Metrics

No data available.

Technical Specifications

Input/Output Details

Input:
- Name: image
  - Info: NCHW BGR un-normalized image
Output:
- Name: enhanced_image
  - Info: NCHW RGB 4x upscaled image

Model Architecture

architecture consisting of Residual-in-Residual Dense Block(s).

Throughput

Model variant: rt-super-resolution:50x50

• Input shape: [1, 3, 50, 50] • Output shape: [1, 3, 200, 200]

• Params (M): 1.339 • GFLOPs: 5.929

Platform	Precision	Throughput (infs/sec)	Power Consumption (W)
RVC2	FP16	15.93	N/A
RVC4	INT8	582.07	4.54

* Benchmarked with , using 2 threads (and the DSP runtime in balanced mode for RVC4).

* Parameters and FLOPs are obtained from the package.

Quantization

RVC4 version of the model was quantized using a HubAI General dataset.

Utilization

Models converted for RVC Platforms can be used for inference on OAK devices. DepthAI pipelines are used to define the information flow linking the device, inference model, and the output parser (as defined in model head(s)). Below, we present the most crucial utilization steps for the particular model. Please consult the docs for more information.

Install DAIv3 and depthai-nodes libraries:

pip install depthai
pip install depthai-nodes

Define model:

model_description = dai.NNModelDescription(
    "luxonis/rt-super-resolution:50x50"
)

nn = pipeline.create(ParsingNeuralNetwork).build(
    <CameraNode>, model_description
)

Inspect model head(s):

ImageOutputParser that outputs message (upscaled image).

Get parsed output(s):

while pipeline.isRuning():
    parser_output: dai.ImgFrame = parser_output_queue.get()

Example

You can quickly run the model using our script. It automatically downloads the model, creates a DepthAI pipeline, runs the inference, and displays the results using our DepthAI visualizer tool. To try it out, run:

python3 main.py \
    --model luxonis/rt-super-resolution:50x50

Single-image super-resolution model.
License	MIT Commercial use
Downloads	150
Tasks	Super Resolution
Model Types	ONNX

Model Variants

Name	Version	Available For	Created At	Deploy
			7 months ago
		RVC2, RVC4	11 months ago