I want to help visually impaired users by building an app that describes what the camera sees. I need to combine a vision model with a Text-to-Speech (TTS) engine. What are the best Flutter plugins for TTS, and how do I chain the AI's text output into a voice response without a noticeable delay?
3 answers
To build this, you should use flutter_tts for the voice part and the Gemini gemini-pro-vision model for the description. The workflow is: capture the image, send it to Gemini with a prompt like "Describe this scene for a blind person in one sentence," and then pass the resulting string to flutterTts.speak(). To reduce delay, you should set the flutter_tts engine to "await" the AI response but initialize the TTS engine during app startup so it's "warm." Also, use a lower-resolution image for the AI (like 720p) to speed up the upload and processing time.
Is there a way to do this entirely offline? Sending photos to the cloud for every scan might be too slow for someone trying to navigate a room in real-time.
Don't forget to handle the volume and audio focus! If the user is listening to music, your app should "duck" the music volume while the AI is speaking its description.
Excellent point, Brenda. Small UX details like audio ducking make a huge difference in how "native" and helpful an accessibility app feels to the user.
Michael, for a truly real-time offline experience, you should use the google_ml_kit_image_labeling package. It won't give you a poetic "description" like Gemini, but it can identify objects (e.g., 'Chair', 'Door', 'Person') in milliseconds. You can then use a simple Dart logic to concatenate these labels: "I see a chair and a door." Since it runs on-device, the feedback to the user is almost instantaneous, which is much safer and more practical for navigation-based accessibility tools.