Summary:	ASTERISK-16674: Random deadlock in res_config_mysql
Reporter:	Ricardo Landim (ricardolandim)	Labels:
Date Opened:	2010-09-14 08:36:22	Date Closed:
Priority:	Minor	Regression?	No
Status:	Open/New	Components:	Addons/res_config_mysql
Versions:		Frequency of Occurrence
Related Issues:
Environment:		Attachments:	( 0) asterisk-bt-full-20100916-153646.txt ( 1) asterisk-bt-full-20100916-153833.txt ( 2) asterisk-bt-full-20100916-154026.txt ( 3) asterisk-bt-full-20100917-111315.txt
Description:	Asterisk randomly stops processing SIP calls. I cannot find a pattern of external events triggering it. I have around 700 registers in realtime.... Asterisk version 1.4.32 I have compiled asterisk with DEBUG_THREADS. When this happers 'core show locks' presents this... I am reporting 4 events (core show locks)... I believe this bug is the same issue ASTERISK-14113 (fixed for 1.6.x but not fixed for 1.4.x) ****** ADDITIONAL INFORMATION ****** Event 1) ======================================================================= === Currently Held Locks ============================================== ======================================================================= === === <file> <line num> <function> <lock name> <lock addr> (times locked) === === Thread ID: -1213891696 (do_monitor           started at [17123] chan_sip.c restart_monitor()) === ---> Lock #0 (chan_sip.c): MUTEX 16790 sipsock_read &netlock 0xb7b06140 (1) === ---> Lock #1 (chan_sip.c): MUTEX 4873 find_call &p->lock 0x90f1f58 (1) === ---> Lock #2 (res_config_mysql.c): MUTEX 330 update_mysql &mysql_lock 0xb7601680 (1) === ------------------------------------------------------------------- === ======================================================================= Event 2) ======================================================================= === Currently Held Locks ============================================== ======================================================================= === === <file> <line num> <function> <lock name> <lock addr> (times locked) === === Thread ID: -1213334640 (do_monitor           started at [17123] chan_sip.c restart_monitor()) === ---> Lock #0 (chan_sip.c): MUTEX 17069 do_monitor &monlock 0xb7b8de80 (1) === ---> Lock #1 (res_config_mysql.c): MUTEX 330 update_mysql &mysql_lock 0xb7689680 (1) === ------------------------------------------------------------------- === === Thread ID: -1229177968 (pbx_thread           started at [ 2645] pbx.c ast_pbx_start()) === ---> Waiting for Lock #0 (chan_sip.c): MUTEX 17112 restart_monitor &monlock 0xb7b8de80 (1) === --- ---> Locked Here: chan_sip.c line 17069 (do_monitor) === ------------------------------------------------------------------- === ======================================================================= Event 3) ======================================================================= === Currently Held Locks ============================================== ======================================================================= === === <file> <line num> <function> <lock name> <lock addr> (times locked) === === Thread ID: -1213777008 (do_monitor           started at [17123] chan_sip.c restart_monitor()) === ---> Lock #0 (chan_sip.c): MUTEX 16790 sipsock_read &netlock 0xb7b22140 (1) === ---> Lock #1 (chan_sip.c): MUTEX 4873 find_call &p->lock 0x8458da8 (1) === ---> Lock #2 (res_config_mysql.c): MUTEX 330 update_mysql &mysql_lock 0xb761d680 (1) === ------------------------------------------------------------------- === ======================================================================= Event 4) ======================================================================= === Currently Held Locks ============================================== ======================================================================= === === <file> <line num> <function> <lock name> <lock addr> (times locked) === === Thread ID: -1211081840 (do_devstate_changes  started at [  363] devicestate.c ast_device_state_engine_init()) === ---> Waiting for Lock #0 (res_config_mysql.c): MUTEX 115 realtime_mysql &mysql_lock 0xb760f680 (1) === --- ---> Locked Here: res_config_mysql.c line 330 (update_mysql) === ------------------------------------------------------------------- === === Thread ID: -1213834352 (do_monitor           started at [17123] chan_sip.c restart_monitor()) === ---> Lock #0 (chan_sip.c): MUTEX 16790 sipsock_read &netlock 0xb7b14140 (1) === ---> Lock #1 (chan_sip.c): MUTEX 4873 find_call &p->lock 0x8bc45f8 (1) === ---> Lock #2 (res_config_mysql.c): MUTEX 330 update_mysql &mysql_lock 0xb760f680 (1) === ------------------------------------------------------------------- === =======================================================================
Comments:	By: Ricardo Landim (ricardolandim) 2010-09-14 08:42:22 When this occurs, all peers are UNREACHABLE.... By: Leif Madsen (lmadsen) 2010-09-14 14:37:08 It might be useful for you to provide an unoptimized backtrace when this happens as well. By: Ricardo Landim (ricardolandim) 2010-09-14 14:57:47 ok lmadsen... I will recompile to provide an unoptimized backtrace! I saw that the status issue has been changed to acknowledged. This bug is already known? By: Leif Madsen (lmadsen) 2010-09-14 15:04:09 No, acknowledged just means debugging information has been provided. The lock information may be enough, but I'm being proactive about asking for additional information that may be required. By: Leif Madsen (lmadsen) 2010-09-14 15:41:28 Developers have confirmed that we do indeed need a backtrace to move this issue forward. By: Ricardo Landim (ricardolandim) 2010-09-15 06:41:30 Asterisk randomly stops processing SIP calls but will not crash. I can stop it with a "stop now" in the CLI. What would be the best method to generate the backtrace?   1. get PID with ps aux   2. enter "gdb /usr/sbin/asterisk PID"   3. enter "bt" while in gdb (or of the "bt full")   4. enter "thread apply all bt" ???????????? By: Ricardo Landim (ricardolandim) 2010-09-16 13:51:45 I am sending 3 bt full... anexxed! By: Ricardo Landim (ricardolandim) 2010-09-17 09:15:21 more one bt_full! annexed! By: Leif Madsen (lmadsen) 2010-10-04 10:43:12 No need to send any more backtraces. This issue will be triaged and assigned to a developer as time and resources permit. By: Matt Jordan (mjordan) 2013-01-15 11:18:06.378-0600 Please note that {{res_config_mysql}} is an extended support module. Developer resources for it typically come from the open source community. Response times may reflect that.