[Home]

Summary:ASTERISK-30481: chan_console: Hangs on placing calls
Reporter:Sebastian Gutierrez (sum)Labels:
Date Opened:2023-03-30 10:38:03Date Closed:
Priority:MinorRegression?
Status:Open/NewComponents:Channels/General
Versions:20.2.0 Frequency of
Occurrence		Constant
Related
Issues:
Environment:Ubuntu 22 - 5.15.0-69-genericAttachments:
Description:Asterisk from scratch no modifications after the 2 call to this basic dialplan asterisk gets unresponsive

{code:none}
[test]

exten=> 1,1,Answer()
exten=> 1,2,Set(result=${CURL(https://dummyjson.com/products)})
exten=> 1,3,NoOp(${result})
{code}
Comments:By: Asterisk Team (asteriskteam) 2023-03-30 10:38:07.955-0500

The severity of this issue has been automatically downgraded from "Blocker" to "Major". The "Blocker" severity is reserved for issues which have been determined to block the next release of Asterisk. This severity can only be set by privileged users. If this issue is deemed to block the next release it will be updated accordingly during the triage process.
By: Asterisk Team (asteriskteam) 2023-03-30 10:38:08.539-0500

Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. Please note that log messages and other files should not be sent to the Sangoma Asterisk Team unless explicitly asked for. All files should be placed on this issue in a sanitized fashion as needed.

A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report.

Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].

Please note that once your issue enters an open state it has been accepted. As Asterisk is an open source project there is no guarantee or timeframe on when your issue will be looked into. If you need expedient resolution you will need to find and pay a suitable developer. Asking for an update on your issue will not yield any progress on it and will not result in a response. All updates are posted to the issue when they occur.

Please note that by submitting data, code, or documentation to Sangoma through JIRA, you accept the Terms of Use present at [https://www.asterisk.org/terms-of-use/|https://www.asterisk.org/terms-of-use/].
By: Joshua C. Colp (jcolp) 2023-03-30 10:48:38.078-0500

We suspect that you have a deadlock occurring within Asterisk. Please follow the instructions on the wiki [1] for obtaining debug relevant to a deadlock. Once you have that information, attach it to the issue. Be sure the instructions are followed exactly as the debug may otherwise not be useful.

Thanks!

[1] https://wiki.asterisk.org/wiki/display/AST/Getting+a+Backtrace#GettingaBacktrace-GettingInformationForADeadlock


By: Sebastian Gutierrez (sum) 2023-03-30 13:15:27.995-0500

the core dumper gets stuck but I can show you the held locks, seems an issue with console (Im testing using console dial from CLI)

{code:none}
ucontactx*CLI> core show locks

=======================================================================
=== 20.2.0
=== Currently Held Locks
=======================================================================
===
=== <pending> <lock#> (<file>): <lock type> <line num> <function> <lock name> <lock addr> (times locked)
===
=== Thread ID: 0xffff5d9750a0 LWP:173335 (netconsole           started at [ 1569] asterisk.c listener())
=== ---> Waiting for Lock #0 (chan_console.c): MUTEX 865 cli_console_dial pvt 0xaaaacef700c0 (1)
       asterisk __ast_pthread_mutex_lock()
       asterisk __ao2_lock()
       [0xffffa0624afc] chan_console.so chan_console.c:866 cli_console_dial()
       asterisk ast_cli_command_full()
       asterisk ast_cli_command_multiple_full()
       asterisk <unknown>()
       asterisk <unknown>()
       [0xffffa40dd5c8] libc.so.6 pthread_create.c:442 start_thread()
       [0xffffa4145d1c] libc.so.6 clone.S:82 thread_start()
=== --- ---> Locked Here: chan_console.c line 281 (stream_monitor)
=== -------------------------------------------------------------------
===
=== Thread ID: 0xffff5d8e90a0 LWP:173336 (pbx_thread           started at [ 4727] pbx.c ast_pbx_start())
=== ---> Lock #0 (channel.c): MUTEX 2576 ast_hangup chan 0xffff64008b00 (1)
       asterisk __ast_pthread_mutex_lock()
       asterisk __ao2_lock()
       asterisk ast_hangup()
       asterisk <unknown>()
       asterisk <unknown>()
       asterisk <unknown>()
       [0xffffa40dd5c8] libc.so.6 pthread_create.c:442 start_thread()
       [0xffffa4145d1c] libc.so.6 clone.S:82 thread_start()
=== ---> Waiting for Lock #1 (chan_console.c): MUTEX 410 stop_stream pvt 0xaaaacef700c0 (1)
       asterisk __ast_pthread_mutex_lock()
       asterisk __ao2_lock()
       [0xffffa0624afc] chan_console.so chan_console.c:866 cli_console_dial()
       asterisk ast_cli_command_full()
       asterisk ast_cli_command_multiple_full()
       asterisk <unknown>()
       asterisk <unknown>()
       [0xffffa40dd5c8] libc.so.6 pthread_create.c:442 start_thread()
       [0xffffa4145d1c] libc.so.6 clone.S:82 thread_start()
=== --- ---> Locked Here: chan_console.c line 281 (stream_monitor)
=== -------------------------------------------------------------------
===
=======================================================================
{code}
By: Joshua C. Colp (jcolp) 2023-03-30 13:18:08.954-0500

Are you using chan_console to do dialing? If that is what is blocked (as it seems to be), then the problem is in chan_console which is extended support.
By: Sebastian Gutierrez (sum) 2023-03-30 13:23:02.408-0500

yes using console to make tests on the dialplan

I can see this error

{code}
] ERROR[175513]: utils.c:1027 lock_info_destroy: Thread 'stream_monitor       started at [  390] chan_console.c start_stream()' still has a lock! - 'pvt' (0xaaaae2b4eee0) from 'stream_monitor' in chan_console.c:281!
{code}
By: Joshua C. Colp (jcolp) 2023-03-30 13:26:18.576-0500

Does this happen when not using chan_console?
By: Sebastian Gutierrez (sum) 2023-03-30 13:32:59.412-0500

using originate works ok
By: Joshua C. Colp (jcolp) 2023-03-30 13:36:35.358-0500

Okay, I will mark this as accepted since there does appear to be a deadlock issue within chan_console but there is no time frame on when anyone would look at it, or even if they will.
By: Sebastian Gutierrez (sum) 2023-03-30 13:38:43.750-0500

ok, no problem as I was using console for testing
By: N A (InterLinked) 2023-04-03 13:37:36.418-0500

Pa_ReadStream is a blocking call, so keeping console pvt locked there seems likely to cause all sorts of issues.

If you move the console_pvt_lock/unlock at line 281 to wrap the pvt->owner check a couple lines below, does this resolve your deadlock issue (without causing further issues)? A quick test of this for me effected a big improvement. However, I haven't studied how this will affect other things.