Inbound With Speech

UAS API - Inbound With Speech Sample Application

Filename:

samples/inbound_with_speech.py

Description:

An inbound application that prompts the caller to say a number between sixteen and ninety nine. After ringing and answering an inbound call, the application primes the speech recogniser to start recognising speech after the next play has ended. It selects a predefined grammar, SixteenToNinetyNine, for the recognition job. The application then uses TTS to ask for the caller's age, which should be between sixteen and ninety nine.

The caller can respond by pressing digits instead of speaking. If this happens, the application will collect the DTMF digits that were pressed. Finally, the application will speak back to the caller what was said, or pressed, using TTS.

Code:

# -*- coding: utf-8 -*-
"""
A simple application that answers an inbound call, speaks some TTS, 
detects some speech and plays back the speech that was detected.

    Actions:
        - check the channel state
        - ring and answer
        - play some tts
        - detect a spoken response
        - play what was detected
        - hang up

The speech detection grammar used is pre-defined.

"""

from prosody.uas import Hangup, Error
import time

__uas_version__  = "0.0.1"
__uas_identify__ = "application"

def main(channel, application_instance_id, file_man, my_log, application_parameters):
    return_code = 0
    try:
        # check the incoming channel state
        state = channel.state()

        if state == channel.State.CALL_INCOMING:
            state = channel.ring()   # this can raise a Hangup exception
            if state == channel.State.RING_INCOMING:
                state = channel.answer() # this can raise a Hangup exception
        else:
            raise Hangup('No inbound call, state is {0}'.format(state))
        if state != channel.State.ANSWERED:
            raise Hangup('Failed to answer inbound call, state is {0}'.format(state))
        
        my_log.info("Answered an inbound call") # log at info level

        channel.FileRecorder.start("speech_test_{0}.wav".format(application_instance_id))
        # prepare the speech detector, speech detection will begin as soon
        # as the prompt has finished playing
        my_grammar = channel.SpeechDetector.Grammar()
        my_grammar.create_from_predefined('SixteenToNinetyNine')
        if channel.SpeechDetector.prime(my_grammar, channel.SpeechDetector.SpeechDetectorTrigger.ONPLAYEND) is False:
            raise Error("Speech detector failed to start")

        # Say the TTS prompt, the text will ask the caller their age, a value from sixteen to ninety nine.
        cause = channel.FilePlayer.say("<acu-engine name='Polly'><voice name='Amy'>How old are you?</voice></acu-engine>")
        if cause != channel.FilePlayer.Cause.NORMAL:
            raise Error("Say prompt failed: cause is {0}".format(cause))

        # Now get the recognised speech, can be words or DTMF digits
        response = channel.SpeechDetector.get_recognised_speech()
        if channel.SpeechDetector.cause() == channel.SpeechDetector.Cause.BARGEIN:
            # age is 16 to 99, so we want two digits
            age = channel.DTMFDetector.get_digits(count=2)
        else:
            age = response.get_recognised_words_as_string()
        if not age:
            to_say = "Sorry, I did not hear that."
        else:
            to_say = "Your age is {0}.".format(age)
        # Say the recognised speech.
        cause = channel.FilePlayer.say("<acu-engine name='Polly'><voice name='Amy'>{0}</voice></acu-engine>".format(to_say))

        if cause != channel.FilePlayer.Cause.NORMAL:
            raise Error("TTS player returned {0}: expected {1}".format(cause, channel.FilePlayer.Cause.NORMAL))
        
        # Bye bye.
        cause = channel.FilePlayer.say("Bye bye.")
        if cause != channel.FilePlayer.Cause.NORMAL:
            raise Error("Say bye bye failed: cause is {0}".format(cause))
        
    except Hangup as exc:
        my_log.info("Hangup exception reports: {0}".format(exc))
         # in this app a hangup is not an error, return a positive value
        return_code = -100
        
    except Error as exc:
        # for error conditions return a negative value
        my_log.error("Error exception reports: {0}".format(exc))
        return_code = -101
        
    except Exception as exc:
        # an unexpected exception, return a negative value
        my_log.exception("Unexpected exception reports: {0}".format(exc))
        return_code = -102
        
    finally:
        channel.FileRecorder.stop()
        if channel.state() != channel.State.IDLE:
            channel.hang_up()

    return return_code

Filename:

Samples\C#\InboundWithSpeech\InboundWithSpeech.cs

Description:

First we ring and answer the call. Then we create a Grammar object that will determine the words that the SpeechDetector on the call will try to recognise. This is passed to the Prime method which readies the SpeechDetector to start recognising speech input after the next play has ended. We give the SpeechDetector 10 seconds to recognise some speech input.

Having primed the SpeechDetector we play a prompt to the caller and then wait for any speech input to be recognised. If the Cause is Normal we have a valid recognised result and play back the recognised word to the call.

Note: if the SpeechDetector didn't recognise one of the expected words or was interrupted by a press on the caller's telephone keypad we could then also use the DTMFDetector to identify the keys pressed. This would provide an alternative means of number input if the caller's environment is too loud or disturbed for accurate recognition.

Code:

using System;
using AMSClassLibrary;
using UASAppAPI;

// An inbound application that prompts the caller to say some text 
// then reads the recognised text back to the caller and hangs up.
//
// Requires:
// -
namespace InboundWithSpeech
{
    // The application class.
    // This must have the same name as the assembly and must inherit from either 
    // UASInboundApplication or UASOutboundApplication.
    // It must override the Run method.
    public class InboundWithSpeech : UASInboundApplication
    {
        enum ReturnCode
        {
            // Success Codes:
            Success = 0,
            // ... any positive integer

            // Fail Codes:
            // -1 to -99 reserved
            ExceptionThrown = -100,
            PlayInterrupted = -101,
            NoResponse = -102
        }

        // This is the entry point for the application
        public override int Run(UASCallChannel channel, string applicationParameters)
        {
            this.Trace.TraceInfo("Start - appParms [{0}]", applicationParameters);
            ReturnCode reply = ReturnCode.Success;

            try
            {
                // Ring for 2 seconds
                channel.Ring(2);

                // Answer the call
                CallState state = channel.Answer();
                if (state == CallState.Answered)
                {
                    this.Trace.TraceInfo("Answered");

                    // Prime the speech recognition to listen when the prompt has finished playing
                    Grammar listenFor = Grammar.CreateFromAlternatives(new String[] { "apricots", "apples", "pears", "pomegranates" });
                    if (!channel.SpeechDetector.Prime(SpeechDetectorTrigger.OnPlayEnd, listenFor, 10))
                    {
                        this.Trace.TraceError("Failed to prime speech recognition");
                        reply = ReturnCode.PlayInterrupted;
                    }

                    // Prompt the caller to make a selection by saying one of the following
                    FilePlayerCause playCause = channel.FilePlayer.Say(
                        "Hello. You've reached the Inbound With Speech sample. Please say one of the following: apricots, apples, pears, pomegranates");
                    if (FilePlayerCause.Normal != playCause)
                    {
                       this.Trace.TraceError("Say failed or was interrupted");
                        reply = ReturnCode.PlayInterrupted;
                    }
                    else
                    {
                        // Now wait for speech recognition
                        SpeechDetectorResult result;
                        SpeechDetectorCause speechCause = channel.SpeechDetector.GetRecognisedSpeech(out result);
                        if (speechCause == SpeechDetectorCause.Normal)
                        {
                            this.Trace.TraceInfo("Got speech result: {0}", result.ToString());

                            // Now say the recognised text using TTS.
                            playCause = channel.FilePlayer.Say(String.Join(" ", result.RecognisedWords.ToArray()));
                            if (FilePlayerCause.Normal != playCause)
                            {
                                this.Trace.TraceError("Say failed or was interrupted");
                                reply = ReturnCode.PlayInterrupted;
                            }
                        }
                        else
                        {
                            this.Trace.TraceInfo("GetRecognisedSpeech returned {0}", speechCause.ToString());
                            this.Trace.TraceError("No response was recognised");
                            reply = ReturnCode.NoResponse;
                        }
                    }

                    // Say Goodbye using the default voice
                    channel.FilePlayer.Say("Goodbye.");

                    // Ensure the call is hung up.
                    channel.HangUp();
                }
            }
            catch (Exception e)
            {
                this.Trace.TraceError("Exception caught: {0}", e.Message);
                reply = ReturnCode.ExceptionThrown;
            }

            this.Trace.TraceInfo("Completed with return code {0}", reply);
            return (int)reply;
        }
    }
}

Filename:

Samples\VB\InboundWithSpeech\InboundWithSpeech.vb

Description:

First we ring and answer the call. Then we create a Grammar object that will determine the words that the SpeechDetector on the call will try to recognise. This is passed to the Prime method which readies the SpeechDetector to start recognising speech input after the next play has ended. We give the SpeechDetector 10 seconds to recognise some speech input.

Having primed the SpeechDetector we play a prompt to the caller and then wait for any speech input to be recognised. If the Cause is Normal we have a valid recognised result and play back the recognised word to the call.

Note: if the SpeechDetector didn't recognise one of the expected words or was interrupted by a press on the caller's telephone keypad we could then also use the DTMFDetector to identify the keys pressed. This would provide an alternative means of number input if the caller's environment is too loud or disturbed for accurate recognition.

Code:

Imports AMSClassLibrary
Imports UASAppAPI

' An inbound application that prompts the caller to say some text 
' then reads the recognised text back to the caller And hangs up.
'
' Requires:
' -
Namespace InboundWithSpeech

    ' The application class.
    ' This must have the same name as the assembly and must inherit from either 
    ' UASInboundApplication or UASOutboundApplication.
    ' It must override the Run method.
    Public Class InboundWithSpeech
        Inherits UASInboundApplication

        ' Possible return codes
        Enum ReturnCode
            ' Success Codes:
            Success = 0
            ' ... any positive integer

            ' Fail Codes:
            ' -1 to -99 reserved
            ExceptionThrown = -100
            PlayInterrupted = -101
            NoResponse = -102
        End Enum

        ' This is the entry point for the application
        Overrides Function Run(ByVal channel As UASCallChannel,
                               ByVal applicationParameters As String) _
                               As Integer

            Me.Trace.TraceInfo("Start - appParms [{0}]", applicationParameters)
            Dim reply As ReturnCode = ReturnCode.Success

            Try
                ' Ring for 2 seconds
                channel.Ring(2)

                ' Answer the call
                Dim state As CallState
                state = channel.Answer()

                If state = CallState.Answered Then

                    Me.Trace.TraceInfo("Call answered")

                    ' Prime the speech recognition to listen when the prompt has finished playing
                    Dim options As String() = {"apricots", "apples", "pears", "pomegranates"}
                    Dim listenFor As Grammar = Grammar.CreateFromAlternatives(options)
                    If Not channel.SpeechDetector.Prime(SpeechDetectorTrigger.OnPlayEnd, listenFor, 10) Then
                        Me.Trace.TraceError("Failed to prime speech recognition")
                        reply = ReturnCode.PlayInterrupted
                    End If

                    ' Prompt the caller to make a selection by saying one of the following
                    Dim playCause As FilePlayerCause = channel.FilePlayer.Say(
                        "Hello. You've reached the Inbound With Speech sample. Please say one of the following: " _
                        + String.Join(", ", options))
                    If FilePlayerCause.Normal <> playCause Then
                        Me.Trace.TraceError("Say failed or was interrupted")
                        reply = ReturnCode.PlayInterrupted
                    Else
                        ' Now wait for speech recognition
                        Dim result As SpeechDetectorResult = Nothing
                        Dim speechCause As SpeechDetectorCause = channel.SpeechDetector.GetRecognisedSpeech(result)
                        If speechCause = SpeechDetectorCause.Normal Then
                            Me.Trace.TraceInfo("Got speech result: {0}", result.ToString())

                            ' Now say the recognised text using TTS.
                            playCause = channel.FilePlayer.Say(String.Join(" ", result.RecognisedWords.ToArray()))
                            If FilePlayerCause.Normal <> playCause Then
                                Me.Trace.TraceError("Say failed or was interrupted")
                                reply = ReturnCode.PlayInterrupted
                            End If
                        Else
                            Me.Trace.TraceInfo("GetRecognisedSpeech returned {0}", speechCause.ToString())
                            Me.Trace.TraceError("No response was recognised")
                            reply = ReturnCode.NoResponse
                        End If
                    End If

                    ' Say Goodbye using the default voice
                    channel.FilePlayer.Say("Goodbye.")

                    ' Ensure the call is hung up.
                    channel.HangUp()
                End If

            Catch ex As Exception
                Me.Trace.TraceError("Exception thrown {0}", ex.Message)
                reply = ReturnCode.ExceptionThrown
            End Try

            Me.Trace.TraceInfo("Completed with return code {0}", reply)
            Return reply

        End Function

    End Class

End Namespace

Cloud documentation

Filename:

Description:

Code:

Filename:

Description:

Code:

Filename:

Description:

Code:

Contact Aculab