Summary:	ASTERISK-25640: pbx: Deadlock on features reload and state change hint.
Reporter:	Krzysztof Trempala (kristophert)	Labels:
Date Opened:	2015-12-21 01:46:58.000-0600	Date Closed:	2016-01-11 17:39:22.000-0600
Priority:	Blocker	Regression?	Yes
Status:	Closed/Complete	Components:	Addons/General
Versions:	11.20.0 13.6.0	Frequency of Occurrence	One Time
Related Issues:	is related to ASTERISK-25394 pbx: Incorrect device and presence state when changing hint details
Environment:	Ubuntu 12.04	Attachments:
Description:	Deadlock on features reload and state change hint. Lock on &context_merge_lock in function ast_change_hint() add in ASTERISK-25394 Logs from core: {code} Thread 58 (Thread 0xb185fb40 (LWP 23499)): #0 0xffffe424 in ?? () #1 0x00005bcb in ?? () #2 0x00000002 in ?? () #3 0x00000080 in ?? () #4 0xb733c5a2 in __lll_lock_wait () from /home/ncp/proj/ncpbx.1.03.1040.6f2ae8b/asterisk/main/../../man/e_pc/target/lib/libpthread.so.0 #5 0xb7337ebb in _L_lock_764 () from /home/ncp/proj/ncpbx.1.03.1040.6f2ae8b/asterisk/main/../../man/e_pc/target/lib/libpthread.so.0 #6 0xb7337d75 in pthread_mutex_lock () from /home/ncp/proj/ncpbx.1.03.1040.6f2ae8b/asterisk/main/../../man/e_pc/target/lib/libpthread.so.0 #7 0xb76cc6d4 in pthread_mutex_lock () from /home/ncp/proj/ncpbx.1.03.1040.6f2ae8b/asterisk/main/../../man/e_pc/target/lib/libc.so.6 #8 0x0813c183 in __ast_pthread_mutex_lock (filename=0x8211434 "pbx.c", lineno=6026, func=0x8217a86 "ast_change_hint", mutex_name=0x82132cb "&context_merge_lock", t=0x825bb20) at lock.c:313 ==> Wait for lock &context_merge_lock (Lock added in https://issues.asterisk.org/jira/browse/ASTERISK-25394 [^]) #9 0x0816eafe in ast_change_hint (oe=0xb3649830, ne=0xb287ccc0) at pbx.c:6026 #10 0x0817bf65 in add_priority (con=0xb32924c0, tmp=0xb287ccc0, el=0xb706e928, e=0xb3649830, replace=1) at pbx.c:10192 #11 0x0817c86e in ast_add_extension2_lockopt (con=0xb32924c0, replace=1, extension=0xb185e2c2 "708", priority=-1, label=0x0, callerid=0x0, application=0xb185e272 "park:708@parkedcalls_700", data=0x0, datad=0,    registrar=0x820048a "features", lock_context=1) at pbx.c:10390 #12 0x0817c116 in ast_add_extension2 (con=0xb32924c0, replace=1, extension=0xb185e2c2 "708", priority=-1, label=0x0, callerid=0x0, application=0xb185e272 "park:708@parkedcalls_700", data=0x0, datad=0, registrar=0x820048a "features") at pbx.c:10261 ==> Lock &conlock in function find_context_locked() at pbx.c:9907 #13 0x0817b076 in ast_add_extension (context=0x82e30fc "parkedcalls_700", replace=1, extension=0xb185e2c2 "708", priority=-1, label=0x0, callerid=0x0, application=0xb185e272 "park:708@parkedcalls_700", data=0x0, datad=0,    registrar=0x820048a "features") at pbx.c:9909 #14 0x08114db6 in park_add_hints (context=0x82e30fc "parkedcalls_700", start=701, stop=709) at features.c:5850 #15 0x08115800 in parkinglot_activate (parkinglot=0x82e300c) at features.c:6042 #16 0x08117dcb in parkinglot_activate_cb (obj=0x82e300c, arg=0x0, flags=2) at features.c:7064 #17 0x0808e44b in internal_ao2_callback (c=0x82e279c, flags=OBJ_NODATA, cb_fn=0x8117d99, arg=0x0, data=0x0, type=DEFAULT, tag=0x0, file=0x0, line=0, func=0x0) at astobj2.c:1109 #18 0x0808e844 in __ao2_callback (c=0x82e279c, flags=OBJ_NODATA, cb_fn=0x8117d99 <parkinglot_activate_cb>, arg=0x0) at astobj2.c:1214 #19 0x08118195 in load_config (reload=1) at features.c:7147 #20 0x0811886e in ast_features_reload () at features.c:7265 #21 0x081188dc in handle_features_reload (e=0x8259088, cmd=-4, a=0xb185e674) at features.c:7283 #22 0x080dd78a in ast_cli_command_full (uid=-1, gid=-1, fd=189, s=0xb2ab4a21 "features reload") at cli.c:2693 #23 0x0814bcd4 in action_command (s=0xb185f130, m=0xb185ea20) at manager.c:4047 #24 0x08151416 in process_message (s=0xb185f130, m=0xb185ea20) at manager.c:5283 #25 0x08151e04 in do_message (s=0xb185f130) at manager.c:5496 #26 0x08152398 in session_do (data=0xb26fc684) at manager.c:5617 #27 0x081bf4a6 in handle_tcptls_connection (data=0xb26fc684) at tcptls.c:694 #28 0x081cef50 in dummy_start (data=0xb36081c8) at utils.c:1320 #29 0xb7335d4c in start_thread () from /home/ncp/proj/ncpbx.1.03.1040.6f2ae8b/asterisk/main/../../man/e_pc/target/lib/libpthread.so.0 #30 0xb76bebae in clone () from /home/ncp/proj/ncpbx.1.03.1040.6f2ae8b/asterisk/main/../../man/e_pc/target/lib/libc.so.6 {code} {code} Thread 11 (Thread 0xb6c97b40 (LWP 2864)): #0 0xffffe424 in ?? () #1 0x00000b30 in ?? () #2 0x00000002 in ?? () #3 0x00000080 in ?? () #4 0xb733c5a2 in __lll_lock_wait () from /home/ncp/proj/ncpbx.1.03.1040.6f2ae8b/asterisk/main/../../man/e_pc/target/lib/libpthread.so.0 #5 0xb7337ebb in _L_lock_764 () from /home/ncp/proj/ncpbx.1.03.1040.6f2ae8b/asterisk/main/../../man/e_pc/target/lib/libpthread.so.0 #6 0xb7337d75 in pthread_mutex_lock () from /home/ncp/proj/ncpbx.1.03.1040.6f2ae8b/asterisk/main/../../man/e_pc/target/lib/libpthread.so.0 #7 0xb76cc6d4 in pthread_mutex_lock () from /home/ncp/proj/ncpbx.1.03.1040.6f2ae8b/asterisk/main/../../man/e_pc/target/lib/libc.so.6 #8 0x0813c183 in __ast_pthread_mutex_lock (filename=0x8211434 "pbx.c", lineno=12343, func=0x82172de "ast_rdlock_contexts", mutex_name=0x821718b "&conlock", t=0x825bb00) at lock.c:313 ==> Wait for lock &conlock at pbx.c:12343 #9 0x08183393 in ast_rdlock_contexts () at pbx.c:12343 #10 0x0816bf0a in ast_hint_extension (c=0x0, context=0xb6c9720c "vsx-hints", exten=0xb6c9725c "3007") at pbx.c:5033 #11 0x0816f28c in ast_get_hint (hint=0xb6c96eac "@\360\064\266\334nÉśgpÉś\334nÉś", hintsize=512, name=0x0, namesize=0, c=0x0, context=0xb6c9720c "vsx-hints", exten=0xb6c9725c "3007") at pbx.c:6176 #12 0x081530fa in manager_state_cb (context=0xb6c9720c "vsx-hints", exten=0xb6c9725c "3007", info=0xb6c970f4, data=0x0) at manager.c:5861 #13 0x0816c9a0 in execute_state_callback (cb=0x815307e <manager_state_cb>, context=0xb6c9720c "vsx-hints", exten=0xb6c9725c "3007", data=0x0, reason=AST_HINT_UPDATE_DEVICE, hint=0xb247215c, device_state_info=0x0) at pbx.c:5308 ===> Lock &context_merge_lock at pbx.c:5540 #14 0x0816d98d in handle_statechange (datap=0xb6f0c598) at pbx.c:5608 #15 0x081bd062 in tps_processing_function (data=0x82dc22c) at taskprocessor.c:330 #16 0x081cef50 in dummy_start (data=0x82dc2c0) at utils.c:1320 #17 0xb7335d4c in start_thread () from /home/ncp/proj/ncpbx.1.03.1040.6f2ae8b/asterisk/main/../../man/e_pc/target/lib/libpthread.so.0 #18 0xb76bebae in clone () from /home/ncp/proj/ncpbx.1.03.1040.6f2ae8b/asterisk/main/../../man/e_pc/target/lib/libc.so.6 {code}
Comments:	By: Asterisk Team (asteriskteam) 2015-12-21 01:47:00.788-0600 Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report. Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].