Is it possible to access the current audio stream in a Twilio call?

Question

Twilio can provide call recording , but it is not in real time. Is it possible to write an application that processes the caller’s sound in real time and answers after processing the sound? I would like some software to “listen” to the speaker and respond programmatically.

Answer 1

Two years later, Twilio released a use case that I tried to do on my own. They now have a real-time speech recognition service built into Programmable Voice. This is in the public beta: https://www.twilio.com/blog/2017/05/introducing-speech-recognition.html

Answer 2

Twilio does not offer a way to handle audio as an IVR input, as far as I know. They suggest using number entry, but it’s not as smart as what you are going to do: https://www.twilio.com/docs/api/twiml/gather .

You can, however, listen to a call that is currently in process, with a trick. It should be configured as a conference. A conference can do anything a regular disk can do. You can disable some additional features, and then you can use the twilio js library to seamlessly join the conference and listen to the call. I suppose if you were very ambitious, you could use some kind of speech in text-based software to do all kinds of things through the Twilio client.

Watch annyang! for some interactive speech with text: https://www.talater.com/annyang/

Answer 3

For people who are still searching, Twilio now has Voice Streams that cover this use case! This is a twiml verb that will stream audio over a web socket to your server.