We are testing our <LiveKit (AI voice)> agent in a mobile environment where background noise is a major factor. The agent keeps stopping its response because it thinks the background noise is an interruption. How can we tune the sensitivity of the 'interruption_threshold' to ensure the agent only stops when the user is actually speaking directly to it?
3 answers
Dealing with noise in <LiveKit (AI voice)> requires a two-pronged approach: software noise suppression and VAD tuning. First, make sure you are using the 'silero' plugin for VAD, as it is much better at filtering out non-human sounds than basic energy-based detectors. You can set the 'threshold' to about 0.6 or 0.7 for noisy environments. Also, look into the 'prefix_padding_ms' setting; increasing this allows the agent to ignore short, sharp noises that might otherwise be interpreted as a user starting a sentence.
Can we use a "Push-to-Talk" mode with the SDK to bypass the VAD logic entirely for mobile?
I found that using the 'NoiseCancellation' processor on the track before it hits the agent works wonders.
Great point, Laura. Integrating Krisp or a similar WebRTC-compatible noise suppressor at the edge can save the
Yes, that is a solid fallback. You can manually control the 'allow_interruptions' flag in the agent session. By tying this to a UI button, you ensure the agent only listens when the user intends it, which is a lifesaver for apps used in public transport or busy offices.