Skip to content

[speech-to-text] Connection immediately closed in Android build, works fine in editor #691

Open
Open
[speech-to-text] Connection immediately closed in Android build, works fine in editor#691
@jpingen

Description

@jpingen
jpingen
opened on Jul 29, 2022

Hi, I'm having issues with getting speech-to-text to work in an Android build. Everything works fine in the Unity Editor.

Unity version: 2021.3.6f1
Watson unity-sdk: v6.0.0
IBM unity-sdk-core: v1.2.4
Scripting backend: IL2CPP
Api compatibility level: .NET Framework (I tried switching to .NET Standard 2.1, but it results in a compilation error IBMSdkCore\Connection\RESTConnector.cs(668,24): error CS0656: Missing compiler required member 'Microsoft.CSharp.RuntimeBinder.CSharpArgumentInfo.Create')

So everything works fine in the Editor, but when I run the Android build, the websocket connection is immediately closed. Similar to #542, but I'm using the Frankfurt endpoint (eu-de).
Edit: also tested on London (eu-gb) and Dallas (us-south), problem still the same.

Logs:

: RESTConnector: URL: https://iam.cloud.ibm.com/identity/token
: IBM.Cloud.SDK.Connection.<ProcessRequestQueue>d__33:MoveNext()
: IBM.Cloud.SDK.Utilities.Routine:MoveNext()
: UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
:
: RESTConnector: URL: https://iam.cloud.ibm.com/identity/token
: IBM.Cloud.SDK.Connection.<ProcessRequestQueue>d__33:MoveNext()
: IBM.Cloud.SDK.Utilities.Routine:MoveNext()
: UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
:
: RESTConnector: URL: https://api.eu-de.assistant.watson.cloud.ibm.com/v2/assistants/[redacted-assistant-id]/sessions?version=2021-11-27
: IBM.Cloud.SDK.Connection.<ProcessRequestQueue>d__33:MoveNext()
: IBM.Cloud.SDK.Utilities.Routine:MoveNext()
: UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
:
: [07-29-2022 08:55:14][WSConnector][WARNING] No case for URL for wss://. Replacing https:// with wss://.
: IBM.Cloud.SDK.Debug.DebugReactor:ProcessLog(LogRecord)
: IBM.Cloud.SDK.LogSystem:ProcessLog(LogRecord)
: IBM.Cloud.SDK.Log:Warning(String, String, Object[])
: IBM.Cloud.SDK.Connection.WSConnector:FixupURL(String)
: IBM.Cloud.SDK.Connection.WSConnector:CreateConnector(Authenticator, String, String, String)
: IBM.Watson.SpeechToText.V1.SpeechToTextService:CreateListenConnector()
: IBM.Watson.SpeechToText.V1.SpeechToTextService:StartListening(OnRecognize, OnRecognizeSpeaker)
: 
: SpeechToText.CreateListenConnector(): Created listen socket. Model: nl-NL_Telephony, parsedParams:
: IBM.Watson.SpeechToText.V1.SpeechToTextService:CreateListenConnector()
: IBM.Watson.SpeechToText.V1.SpeechToTextService:StartListening(OnRecognize, OnRecognizeSpeaker)
:
: WSConnector.Send(): Sending TextMessage message: {"action":"start","content-type":"audio/l16;rate=16000;channels=1;","inactivity_timeout":-1,"interim_results":true,"max_alternatives":1,"profanity_filter":false,"smart_formatting":true,"speaker_labels":false,"timestamps":true,"word_confidence":true,"speech_detector_sensitivity":0,5,"background_audio_suppression":0,"processing_metrics":false}
: IBM.Cloud.SDK.Connection.WSConnector:Send(Message, Boolean)
: IBM.Watson.SpeechToText.V1.SpeechToTextService:SendStart()
: IBM.Watson.SpeechToText.V1.SpeechToTextService:OnListen(AudioData)
:
: SpeechToText.SendStart(): SendStart() with the following params: {"action":"start","content-type":"audio/l16;rate=16000;channels=1;","inactivity_timeout":-1,"interim_results":true,"max_alternatives":1,"profanity_filter":false,"smart_formatting":true,"speaker_labels":false,"timestamps":true,"word_confidence":true,"speech_detector_sensitivity":0,5,"background_audio_suppression":0,"processing_metrics":false}
: IBM.Watson.SpeechToText.V1.SpeechToTextService:SendStart()
: IBM.Watson.SpeechToText.V1.SpeechToTextService:OnListen(AudioData)
:
: WSConnector.OnWSClose(): Reason: An exception has occurred while receiving. - Code: 1006 - WasClean: False
: IBM.Cloud.SDK.Connection.WSConnector:OnWSClose(Object, CloseEventArgs)
: IBM.Cloud.SDK.Plugins.WebSocketSharp.WebSocket:close(PayloadData, Boolean, Boolean, Boolean)
: System.Threading.Tasks.TaskToApm:Begin(Task, AsyncCallback, Object)
: IBM.Cloud.SDK.Plugins.WebSocketSharp.<>c__DisplayClass57_0:<ReadBytesAsync>b__0(IAsyncResult)
: System.Threading.Tasks.TaskToApm:Begin(Task, AsyncCallback, Object)
: IBM.Cloud.SDK.Plugins.WebSocketSharp.<>c__DisplayClass57_0:<ReadBytesAsync>b__0(IAsyncResult)
: System.Threading.Tasks.TaskToApm:Begin(Task, AsyncCallback, Object)
: IBM.Cloud.SDK.Plugins.WebSocketSharp.<>c__DisplayClass57_0:<ReadBytesAsync>b__0(IAsyncResult)
: System.Threading.Tasks.TaskToApm:Begin(Task, AsyncCallback, Object)
: IBM.Cloud.SDK.Plugins.WebSocketSharp.<>c__DisplayClass57_0:<ReadBytesAsync>b__0(IAsyncResult)
: System.Threading.Tasks.TaskToApm:Begin(Task, AsyncCallback, Object)
: IBM.Cloud.SDK.Plugins.WebSocketSharp
: SpeechToText.OnListenClosed(): OnListenClosed(), State = DISCONNECTED
: IBM.Watson.SpeechToText.V1.SpeechToTextService:OnListenClosed(WSConnector)
: IBM.Cloud.SDK.Connection.<ProcessReceiveQueue>d__49:MoveNext()
: IBM.Cloud.SDK.Utilities.Routine:MoveNext()
: UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
:
: SpeechToText.KeepAlive(): KeepAlive exited.
: IBM.Watson.SpeechToText.V1.<KeepAlive>d__242:MoveNext()
: IBM.Cloud.SDK.Utilities.Routine:MoveNext()
: UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)

I have also enabled logging on the Websocket in WSConnector:

29-7-2022 10:55:15|Warn |WSConnector.SendMessages|The current logging level has been changed to Trace.
29-7-2022 10:55:15|Debug|WebSocket.sendHandshakeRequest|A request to the server:
                         GET /v1/recognize?model=nl-NL_Telephony HTTP/1.1
                         User-Agent: websocket-sharp/1.0
                         Host: api.eu-de.speech-to-text.watson.cloud.ibm.com
                         Upgrade: websocket
                         Connection: Upgrade
                         Sec-WebSocket-Key: [redacted]
                         Sec-WebSocket-Version: 13
                         Authorization: Bearer [redacted]
29-7-2022 10:55:16|Debug|WebSocket.sendHandshakeRequest|A response to this request:
                         HTTP/1.1 101 Switching Protocols
                         sec-websocket-accept: [redacted]
                         Content-Type: application/octet-stream
                         Server: watson-gateway
                         strict-transport-security: max-age=31536000; includeSubDomains;
                         x-dp-watson-tran-id: 36a4772e-ae7f-4e2a-a996-1e37e6931fd0
                         x-request-id: 36a4772e-ae7f-4e2a-a996-1e37e6931fd0
                         x-global-transaction-id: 36a4772e-ae7f-4e2a-a996-1e37e6931fd0
                         X-EdgeConnect-MidMile-RTT: 0
                         X-EdgeConnect-Origin-MEX-Latency: 534
                         Date: Fri, 29 Jul 2022 08:55:15 GMT
                         Upgrade: websocket
                         Connection: Upgrade
29-7-2022 10:55:17|Fatal|TaskToApm.Begin|IBM.Cloud.SDK.Plugins.WebSocketSharp.WebSocketException: The header part of a frame could not be read.
                           at IBM.Cloud.SDK.Plugins.WebSocketSharp.WebSocketFrame.processHeader (System.Byte[] header) [0x00000] in <00000000000000000000000000000000>:0 
                           at IBM.Cloud.SDK.Plugins.WebSocketSharp.WebSocketFrame+<>c__DisplayClass73_0.<readHeaderAsync>b__0 (System.Byte[] bytes) [0x00000] in <00000000000000000000000000000000>:0 
                           at IBM.Cloud.SDK.Plugins.WebSocketSharp.Ext+<>c__DisplayClass57_0.<ReadBytesAsync>b__0 (System.IAsyncResult ar) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.TaskToApm.Begin (System.Threading.Tasks.Task task, System.AsyncCallback callback, System.Object state) [0x00000] in <00000000000000000000000000000000>:0 
                           at IBM.Cloud.SDK.Plugins.WebSocketSharp.Ext+<>c__DisplayClass57_0.<ReadBytesAsync>b__0 (System.IAsyncResult ar) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.TaskToApm.Begin (System.Threading.Tasks.Task task, System.AsyncCallback callback, System.Object state) [0x00000] in <00000000000000000000000000000000>:0 
                           at IBM.Cloud.SDK.Plugins.WebSocketSharp.Ext+<>c__DisplayClass57_0.<ReadBytesAsync>b__0 (System.IAsyncResult ar) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.TaskToApm.Begin (System.Threading.Tasks.Task task, System.AsyncCallback callback, System.Object state) [0x00000] in <00000000000000000000000000000000>:0 
                           at IBM.Cloud.SDK.Plugins.WebSocketSharp.Ext+<>c__DisplayClass57_0.<ReadBytesAsync>b__0 (System.IAsyncResult ar) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.TaskToApm.Begin (System.Threading.Tasks.Task task, System.AsyncCallback callback, System.Object state) [0x00000] in <00000000000000000000000000000000>:0 
                           at IBM.Cloud.SDK.Plugins.WebSocketSharp.Ext+<>c__DisplayClass57_0.<ReadBytesAsync>b__0 (System.IAsyncResult ar) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.TaskToApm.Begin (System.Threading.Tasks.Task task, System.AsyncCallback callback, System.Object state) [0x00000] in <00000000000000000000000000000000>:0 
                           at IBM.Cloud.SDK.Plugins.WebSocketSharp.Ext+<>c__DisplayClass57_0.<ReadBytesAsync>b__0 (System.IAsyncResult ar) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action action, System.Boolean allowInlining, System.Threading.Tasks.Task& currentTask) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.Task.FinishContinuations () [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.Task`1[TResult].TrySetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].SetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0 
                           at Mono.Net.Security.MobileAuthenticatedStream.StartOperation (Mono.Net.Security.MobileAuthenticatedStream+OperationType type, Mono.Net.Security.AsyncProtocolRequest asyncRequest, System.Threading.CancellationToken cancellationToken) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run () [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action action, System.Boolean allowInlining, System.Threading.Tasks.Task& currentTask) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.Task.FinishContinuations () [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.Task`1[TResult].TrySetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].SetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0 
                           at Mono.Net.Security.AsyncProtocolRequest.StartOperation (System.Threading.CancellationToken cancellationToken) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run () [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action action, System.Boolean allowInlining, System.Threading.Tasks.Task& currentTask) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.Task.FinishContinuations () [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.Task`1[TResult].TrySetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].SetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0 
                           at Mono.Net.Security.AsyncProtocolRequest.ProcessOperation (System.Threading.CancellationToken cancellationToken) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run () [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action action, System.Boolean allowInlining, System.Threading.Tasks.Task& currentTask) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.Task.FinishContinuations () [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.Task`1[TResult].TrySetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].SetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0 
                           at Mono.Net.Security.AsyncProtocolRequest.InnerRead (System.Threading.CancellationToken cancellationToken) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run () [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action action, System.Boolean allowInlining, System.Threading.Tasks.Task& currentTask) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.Task.FinishContinuations () [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.Task`1[TResult].TrySetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].SetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0 
                           at Mono.Net.Security.MobileAuthenticatedStream.InnerRead (System.Boolean sync, System.Int32 requestedSize, System.Threading.CancellationToken cancellationToken) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run () [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action action, System.Boolean allowInlining, System.Threading.Tasks.Task& currentTask) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.Task.FinishContinuations () [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.Task`1[TResult].TrySetResult (TResult result) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.Tasks.ValueTask`1+ValueTaskSourceAsTask+<>c[TResult].<.cctor>b__4_0 (System.Object state) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Net.Sockets.Socket+<>c.<.cctor>b__367_10 (System.IAsyncResult ares) [0x00000] in <00000000000000000000000000000000>:0 
                           at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00000] in <00000000000000000000000000000000>:0 
29-7-2022 10:55:17|Trace|TaskToApm.Begin|Begin closing the connection.
29-7-2022 10:55:17|Debug|WebSocket.close|Was clean?: False
                           sent: False
                           received: False
29-7-2022 10:55:17|Trace|TaskToApm.Begin|End closing the connection.
29-7-2022 10:55:17|Info |ExecutionContext.RunInternal|The connection has already been closed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions