Your model, fast enough on your own device.
A typical case: an object detector at FP32 runs at 9 FPS on a fanless Jetson Orin Nano, and the product needs 30. I measure on the real board, find where the time goes, and report what it takes to close the gap.
NVIDIA Jetson · TensorRT · GStreamer · embedded Linux
The problem I fix
You have a trained model, or a working GPU prototype, and a target device. Something on the device is not good enough.
- Frame rate
- The model works on a desktop GPU but runs at a fraction of the needed frame rate on the device.
- Latency
- End-to-end latency, from camera to output, is too high for the product.
- Power and heat
- Power draw or heat is over budget on a fanless or battery device.
- INT8 accuracy
- INT8 quantisation lost too much accuracy.
The same applies when you move to a cheaper or newer chip, such as a newer Jetson or a Hailo or Qualcomm NPU, and must not lose accuracy, or when several camera streams have to run on one board.
How I work on it
I measure first, on the real board. I profile the whole pipeline, not just the model: capture, decode, pre-processing, inference, post-processing and display. That shows where the time and the power go.
Then I apply the fix the measurements point to: TensorRT engines, FP16 or INT8 with calibration, changes to the model or the input size, GStreamer pipeline changes, or CUDA for custom steps.
Every number I report carries its conditions: device, precision, model and dataset.
Edge performance review
A fixed-scope first step. I profile your model on your board, find where the time goes, and report what is achievable and what it would take.
-
Profile on your board
I run your model and its pipeline on your hardware and time every stage, from capture to display.
-
Find where the time goes
I trace the frame time and the power draw to the stages that cost the most.
-
Report what is achievable and what it takes
You get the numbers with their conditions, and the changes that would move them.
- You get
-
- Profiling report
- Bottleneck analysis
- Recommended steps with expected effect
- A one-hour readout call
- Scope
- About 20 hours over three weeks · remote · NDA first.
- For
-
- Engineering leads at small and mid-size product teams in the Nordics and the EU that ship a camera or vision feature on an embedded board: industrial monitoring, robotics, agritech, medtech devices, sports and video products.
- Integrators and engineering consultancies that need a deployment specialist for a fixed work package.
- Hardware and module vendors whose customers need models running on their boards.
About
Umair Mukati, PhD
I hold a PhD in multi-view image processing from the Technical University of Denmark (DTU), an MS in computational imaging and a BE in industrial electronics.
My industry work is deploying computer vision and ML on embedded and edge hardware: GStreamer pipelines, CUDA GPU kernels, embedded ARM deployment, camera colour calibration and on-device super-resolution. I have published IEEE research on image and video compression, deep decompression and view synthesis.
Infernordics is a one-person consultancy, so you work with me directly. I am based in the Copenhagen area, Denmark, and work remotely, in English, with teams in the Nordics and the EU.
Contact
Book a 20-minute call or email me. I reply within two working days.
- hello@infernordics.dk