I want to create an app that draws bounding boxes around objects in real-time. Should I use a Cloud API or stick to on-device processing for this? I’m concerned about the latency of sending video frames to a server. Are there any ready-made Flutter widgets that handle the bounding box overlays automatically?
3 answers
For real-time bounding boxes, on-device is the only way to go. Cloud latency makes the boxes "lag" behind the physical object. You should use google_ml_kit_object_detection. It provides the coordinates of the detected objects. To draw the boxes, you can use a CustomPainter widget over your CameraPreview. You’ll need to calculate a "scale factor" to map the coordinates from the ML model's coordinate system (often smaller, like 300x300) to the actual screen size of the phone. This ensures the boxes align perfectly with the objects in the viewfinder.
How do you handle different aspect ratios? Most cameras are 4:3 or 16:9, but the model input is usually a square. Doesn't that distort the detection accuracy?
I used a Stack widget with the CameraPreview at the bottom and a transparent Canvas on top. It’s a very lightweight way to handle the UI for detection.
That's the standard pattern, Monica. It keeps the UI layer very clean and allows for easy updates to the box colors or labels based on the detection confidence.
Kevin, it definitely can! The trick is to "center crop" the image rather than stretching it to fit the model's square input. In Flutter, you can do this by taking the center portion of the camera frame. When you get the results back, you just have to add the crop offset back to the coordinates before drawing them on the CustomPainter. This keeps the aspect ratio consistent and ensures the AI "sees" the objects clearly without any weird horizontal or vertical stretching.