====================================================================== Deadlock between dialplan execution and module unload command: early detection, only two threads locked. Investigation with gdb on core generated with SIGQUIT deadlock of threads 59/LWP34269 and 31/LWP34270 : (gdb) thread 59 [Switching to thread 59 (Thread 0x7f2540aa4700 (LWP 34269))] #0 0x00007f2542ff69b8 in futex_wait (private=0, expected=1, futex_word=0x8a2098 ) at ../sysdeps/unix/sysv/linux/futex-internal.h:61 61 in ../sysdeps/unix/sysv/linux/futex-internal.h (gdb) bt #0 0x00007f2542ff69b8 in futex_wait (private=0, expected=1, futex_word=0x8a2098 ) at ../sysdeps/unix/sysv/linux/futex-internal.h:61 #1 futex_wait_simple (private=0, expected=1, futex_word=0x8a2098 ) at ../sysdeps/nptl/futex-internal.h:135 #2 __pthread_rwlock_rdlock_slow (rwlock=rwlock@entry=0x8a2090 ) at pthread_rwlock_rdlock.c:68 #3 0x00007f2542ff6bf7 in __GI___pthread_rwlock_rdlock (rwlock=rwlock@entry=0x8a2090 ) at pthread_rwlock_rdlock.c:177 #4 0x0000000000529742 in __ast_rwlock_rdlock (filename=filename@entry=0x62f1f0 "pbx_app.c", line=line@entry=97, func=func@entry=0x62f79a <__PRETTY_FUNCTION__.15388> "pbx_findapp", t=t@entry=0x8a2090 , name=name@entry=0x62f1e1 "&(&apps)->lock") at lock.c:911 #5 0x000000000056ec0e in pbx_findapp (app=0x7f251c75be8b "Set") at pbx_app.c:97 #6 0x0000000000566115 in pbx_extension_helper (c=c@entry=0x7f251c656b20, context=0x7f251c6574d8 "sub_init_globals", exten=exten@entry=0x7f251c657528 "s", priority=priority@entry=43, label=label@entry=0x0, callerid=callerid@entry=0x0, action=action@entry=E_SPAWN, found=found@entry=0x7f2540aa3cec, combined_find_spawn=combined_find_spawn@entry=1, con=0x0) at pbx.c:2848 #7 0x0000000000567e34 in ast_spawn_extension (combined_find_spawn=1, found=0x7f2540aa3cec, callerid=0x0, priority=43, exten=0x7f251c657528 "s", context=, c=0x7f251c656b20) at pbx.c:4110 #8 __ast_pbx_run (c=c@entry=0x7f251c656b20, args=args@entry=0x0) at pbx.c:4285 #9 0x00000000005693fd in pbx_thread (data=data@entry=0x7f251c656b20) at pbx.c:4605 #10 0x00000000005d3d39 in dummy_start (data=) at utils.c:1235 #11 0x00007f2542ff234c in start_thread (arg=0x7f2540aa4700) at pthread_create.c:334 #12 0x00007f25428ce50d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 (gdb) frame 4 #4 0x0000000000529742 in __ast_rwlock_rdlock (filename=filename@entry=0x62f1f0 "pbx_app.c", line=line@entry=97, func=func@entry=0x62f79a <__PRETTY_FUNCTION__.15388> "pbx_findapp", t=t@entry=0x8a2090 , name=name@entry=0x62 911 res = pthread_rwlock_rdlock(&t->lock); (gdb) print t->lock $4 = {__data = {__lock = 0, __nr_readers = 0, __readers_wakeup = 1, __writer_wakeup = 0, __nr_readers_queued = 1, __nr_writers_queued = 0, __writer = 34270, __shared = 0, __rwelision = 0 '\000', __pad1 = "\000\000\000\000\000\000", __pad2 = 0, __flags = 0}, __size = "\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000ޅ", '\000' , __align = 0} => pbx_findapp() is blocked on AST_RWLIST_RDLOCK(&apps). apps lock belongs to thread 31/34270 (see __writer). => (59) thread 59/34269 is waiting for thread 31/34270. (gdb) thread 31 [Switching to thread 31 (Thread 0x7f2540930700 (LWP 34270))] #0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135 135 ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S: Aucun fichier ou dossier de ce type. (gdb) bt #0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135 #1 0x00007f2542ff4929 in __GI___pthread_mutex_lock (mutex=mutex@entry=0x8a1d60 ) at ../nptl/pthread_mutex_lock.c:116 #2 0x000000000052969e in __ast_pthread_mutex_lock (filename=filename@entry=0x62b5eb "pbx.c", lineno=lineno@entry=8334, func=func@entry=0x62e890 <__PRETTY_FUNCTION__.20010> "ast_rdlock_contexts", mutex_name=mutex_name@entry=0x62b7ea "&conlock", t=t@entry=0x8a1d60 ) at lock.c:315 #3 0x000000000056b3cc in ast_rdlock_contexts () at pbx.c:8334 #4 unreference_cached_app (app=app@entry=0x7f251c001970) at pbx.c:6064 #5 0x000000000056f601 in ast_unregister_application ( app=0x7f24cffe1ee0 ) at pbx_app.c:408 #6 0x00007f24cff92e6f in ?? () #7 0x00007f251c001b30 in ?? () #8 0x00007f251c001ab0 in ?? () #9 0x00007f251c001b30 in ?? () #10 0x00007f251c006c48 in ?? () #11 0x0000000000000000 in ?? () (gdb) frame 3 #3 0x000000000056b3cc in ast_rdlock_contexts () at pbx.c:8334 8334 return ast_mutex_lock(&conlock); (gdb) print conlock $5 = {mutex = {__data = {__lock = 2, __count = 1, __owner = 34269, __nusers = 1, __kind = 1, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = "\002\000\000\000\001\000\000\000݅\000\000\001\000\000\000\001", '\000' , __align = 4294967298}, track = 0x0, tracking = 1} => unreference_cached_app() is blocked, through ast_rdlock_contexts(), on ast_mutex_lock(&conlock). conlock mutex belongs to thread 59/34269 (see __owner). => (31) thread 31/34270 is waiting for thread 59/34269. Summary : => (59) thread 59/34269 is waiting for thread 31/34270. => (31) thread 31/34270 is waiting for thread 59/34269. Thread 59, executing the dialplan, took conlock mutex at the beginning of pbx_extension_helper(), and now is asking for the lock on apps list in pbx_findapp(). Thread 31, handling ast_unregister_application() (due to an AMI command to unload a module), took the lock on apps list, now is asking conlock mutex in unreference_cached_app() function. ====================================================================== Deadlock between dialplan execution and module unload command: late detection, more threads locked due to CLI commands Investigation with gdb on core generated with SIGQUIT deadlock of threads 55/LWP50303 and 9/LWP50305, more threads taken into the deadlock: (gdb) thread 58 [Switching to thread 58 (Thread 0x7f85033a6700 (LWP 12781))] #0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135 135 ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S: Aucun fichier ou dossier de ce type. (gdb) bt #0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135 #1 0x00007f85d9d90929 in __GI___pthread_mutex_lock (mutex=mutex@entry=0x8a1d60 ) at ../nptl/pthread_mutex_lock.c:116 #2 0x000000000052969e in __ast_pthread_mutex_lock (filename=filename@entry=0x62b5eb "pbx.c", lineno=lineno@entry=8334, func=func@entry=0x62e890 <__PRETTY_FUNCTION__.20010> "ast_rdlock_contexts", mutex_name=mutex_name@entry=0x62b7ea "&conlock", t=t@entry=0x8a1d60 ) at lock.c:315 #3 0x0000000000563ae6 in ast_rdlock_contexts () at pbx.c:8334 #4 show_dialplan_helper (fd=90, context=context@entry=0x0, exten=exten@entry=0x0, dpc=dpc@entry=0x7f85033a4f90, rinclude=rinclude@entry=0x0, includecount=includecount@entry=0, includes=includes@entry=0x7f85033a4fb0) at pbx.c:5408 #5 0x000000000056437f in handle_show_dialplan (e=, cmd=, a=0x7f85033a5430) at pbx.c:5651 #6 0x00000000004cbbc6 in ast_cli_command_full (uid=uid@entry=0, gid=gid@entry=0, fd=fd@entry=90, s=s@entry=0x7f85033a5730 "dialplan show") at cli.c:2775 #7 0x00000000004cbcac in ast_cli_command_multiple_full (uid=0, gid=0, fd=90, size=size@entry=14, s=s@entry=0x7f85033a5a00 "dialplan show") at cli.c:2801 #8 0x0000000000455e99 in netconsole (vconsole=vconsole@entry=0x8c2d10 ) at asterisk.c:1596 #9 0x00000000005d3d39 in dummy_start (data=) at utils.c:1235 #10 0x00007f85d9d8e34c in start_thread (arg=0x7f85033a6700) at pthread_create.c:334 #11 0x00007f85d966a50d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 (gdb) frame 3 #3 0x0000000000563ae6 in ast_rdlock_contexts () at pbx.c:8334 8334 return ast_mutex_lock(&conlock); (gdb) print conlock $1 = {mutex = {__data = {__lock = 2, __count = 1, __owner = 50303, __nusers = 1, __kind = 1, __spins = 0, __elision = 0, __list = { __prev = 0x0, __next = 0x0}}, __size = "\002\000\000\000\001\000\000\000\177\304\000\000\001\000\000\000\001", '\000' , __align = 4294967298}, track = 0x0, tracking = 1} => show_dialplan_helper() is blocked, through ast_rdlock_contexts(), on ast_mutex_lock(&conlock). conlock mutex belongs to thread 55/50303 (see __owner). => (58) thread 58/12781 is waiting for thread 55/50303. (gdb) thread 57 [Switching to thread 57 (Thread 0x7f85d7748700 (LWP 12660))] #0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135 135 ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S: Aucun fichier ou dossier de ce type. (gdb) bt #0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135 #1 0x00007f85d9d90929 in __GI___pthread_mutex_lock (mutex=mutex@entry=0x8a1d60 ) at ../nptl/pthread_mutex_lock.c:116 #2 0x000000000052969e in __ast_pthread_mutex_lock (filename=filename@entry=0x62b5eb "pbx.c", lineno=lineno@entry=8334, func=func@entry=0x62e890 <__PRETTY_FUNCTION__.20010> "ast_rdlock_contexts", mutex_name=mutex_name@entry=0x62b7ea "&conlock", t=t@entry=0x8a1d60 ) at lock.c:315 #3 0x000000000056046f in ast_rdlock_contexts () at pbx.c:8334 #4 complete_show_dialplan_context (word=0x7f85a8002c63 "", state=0, line=, pos=2) at pbx.c:5361 #5 0x00000000005643e9 in complete_show_dialplan_context (line=, state=, pos=, word=) at pbx.c:5676 #6 handle_show_dialplan (e=, cmd=, a=0x7f85d77470d0) at pbx.c:5628 #7 0x00000000004c59e8 in __ast_cli_generator (text=text@entry=0x7f85a8002c54 "dialplan show ", word=word@entry=0x7f85a8002c63 "", state=state@entry=0, lock=1) at cli.c:2696 #8 0x00000000004c8c7b in ast_cli_generator (state=0, word=0x7f85a8002c63 "", text=0x7f85a8002c54 "dialplan show ") at cli.c:2710 #9 ast_cli_generatornummatches (word=0x7f85a8002c63 "", text=0x7f85a8002c54 "dialplan show ") at cli.c:2527 #10 handle_commandnummatches (e=, cmd=, a=0x7f85d7747430) at cli.c:1388 #11 0x00000000004cbbc6 in ast_cli_command_full (uid=uid@entry=0, gid=gid@entry=0, fd=fd@entry=87, s=s@entry=0x7f85d7747730 "_COMMAND NUMMATCHES \"dialplan show \" \"\"") at cli.c:2775 #12 0x00000000004cbcac in ast_cli_command_multiple_full (uid=0, gid=0, fd=87, size=size@entry=40, s=s@entry=0x7f85d7747a00 "_COMMAND NUMMATCHES \"dialplan show \" \"\"") at cli.c:2801 #13 0x0000000000455e99 in netconsole (vconsole=vconsole@entry=0x8c2c68 ) at asterisk.c:1596 #14 0x00000000005d3d39 in dummy_start (data=) at utils.c:1235 #15 0x00007f85d9d8e34c in start_thread (arg=0x7f85d7748700) at pthread_create.c:334 #16 0x00007f85d966a50d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 (gdb) frame 3 #3 0x000000000056046f in ast_rdlock_contexts () at pbx.c:8334 8334 return ast_mutex_lock(&conlock); (gdb) print conlock $2 = {mutex = {__data = {__lock = 2, __count = 1, __owner = 50303, __nusers = 1, __kind = 1, __spins = 0, __elision = 0, __list = { __prev = 0x0, __next = 0x0}}, __size = "\002\000\000\000\001\000\000\000\177\304\000\000\001\000\000\000\001", '\000' , __align = 4294967298}, track = 0x0, tracking = 1} => complete_show_dialplan_context() is blocked, through ast_rdlock_contexts(), on ast_mutex_lock(&conlock). conlock mutex belongs to thread 55/50303 (see __owner). => (57) thread 57/12660 is waiting for thread 55/50303. (gdb) thread 56 [Switching to thread 56 (Thread 0x7f85d7938700 (LWP 63267))] #0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135 135 ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S: Aucun fichier ou dossier de ce type. (gdb) bt #0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135 #1 0x00007f85d9d90929 in __GI___pthread_mutex_lock (mutex=mutex@entry=0x89f1d0 ) at ../nptl/pthread_mutex_lock.c:116 #2 0x000000000052969e in __ast_pthread_mutex_lock (filename=filename@entry=0x621d60 "loader.c", lineno=lineno@entry=757, func=func@entry=0x622720 <__PRETTY_FUNCTION__.28528> "ast_module_helper", mutex_name=mutex_name@entry=0x621db2 "&(&module_list)->lock", t=t@entry=0x89f1d0 ) at lock.c:315 #3 0x0000000000527d9d in ast_module_helper (line=, word=0x7f85c00008e6 "", pos=3, state=0, rpos=3, needsreload=0) at loader.c:757 #4 0x00000000004c59e8 in __ast_cli_generator (text=text@entry=0x7f85c00008d4 "module show like ", word=word@entry=0x7f85c00008e6 "", state=state@entry=0, lock=1) at cli.c:2696 #5 0x00000000004c8c7b in ast_cli_generator (state=0, word=0x7f85c00008e6 "", text=0x7f85c00008d4 "module show like ") at cli.c:2710 #6 ast_cli_generatornummatches (word=0x7f85c00008e6 "", text=0x7f85c00008d4 "module show like ") at cli.c:2527 #7 handle_commandnummatches (e=, cmd=, a=0x7f85d7937430) at cli.c:1388 #8 0x00000000004cbbc6 in ast_cli_command_full (uid=uid@entry=0, gid=gid@entry=0, fd=fd@entry=80, s=s@entry=0x7f85d7937730 "_COMMAND NUMMATCHES \"module show like \" \"\"") at cli.c:2775 #9 0x00000000004cbcac in ast_cli_command_multiple_full (uid=0, gid=0, fd=80, size=size@entry=43, s=s@entry=0x7f85d7937a00 "_COMMAND NUMMATCHES \"module show like \" \"\"") at cli.c:2801 #10 0x0000000000455e99 in netconsole (vconsole=vconsole@entry=0x8c2bc0 ) at asterisk.c:1596 #11 0x00000000005d3d39 in dummy_start (data=) at utils.c:1235 #12 0x00007f85d9d8e34c in start_thread (arg=0x7f85d7938700) at pthread_create.c:334 #13 0x00007f85d966a50d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 (gdb) frame 3 #3 0x0000000000527d9d in ast_module_helper (line=, word=0x7f85c00008e6 "", pos=3, state=0, rpos=3, needsreload=0) at loader.c:757 757 AST_DLLIST_LOCK(&module_list); (gdb) frame 2 #2 0x000000000052969e in __ast_pthread_mutex_lock (filename=filename@entry=0x621d60 "loader.c", lineno=lineno@entry=757, func=func@entry=0x622720 <__PRETTY_FUNCTION__.28528> "ast_module_helper", mutex_name=mutex_name@entry=0x621db2 "&(&module_list)->lock", t=t@entry=0x89f1d0 ) at lock.c:315 315 res = pthread_mutex_lock(&t->mutex); (gdb) print t->mutex $3 = {__data = {__lock = 2, __count = 1, __owner = 50305, __nusers = 1, __kind = 1, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = "\002\000\000\000\001\000\000\000\201\304\000\000\001\000\000\000\001", '\000' , __align = 4294967298} => ast_module_helper() is blocked, through AST_DLLIST_LOCK(&module_list), on module_list mutex. module_list mutex beslongs to thread 9/50305 (see __owner). => (56) thread 56/63267 is waiting for thread 9/50305. (gdb) thread 55 [Switching to thread 55 (Thread 0x7f85d7650700 (LWP 50303))] #0 0x00007f85d9d929b8 in futex_wait (private=0, expected=1, futex_word=0x8a2098 ) at ../sysdeps/unix/sysv/linux/futex-internal.h:61 61 ../sysdeps/unix/sysv/linux/futex-internal.h: Aucun fichier ou dossier de ce type. (gdb) bt #0 0x00007f85d9d929b8 in futex_wait (private=0, expected=1, futex_word=0x8a2098 ) at ../sysdeps/unix/sysv/linux/futex-internal.h:61 #1 futex_wait_simple (private=0, expected=1, futex_word=0x8a2098 ) at ../sysdeps/nptl/futex-internal.h:135 #2 __pthread_rwlock_rdlock_slow (rwlock=rwlock@entry=0x8a2090 ) at pthread_rwlock_rdlock.c:68 #3 0x00007f85d9d92bf7 in __GI___pthread_rwlock_rdlock (rwlock=rwlock@entry=0x8a2090 ) at pthread_rwlock_rdlock.c:177 #4 0x0000000000529742 in __ast_rwlock_rdlock (filename=filename@entry=0x62f1f0 "pbx_app.c", line=line@entry=97, func=func@entry=0x62f79a <__PRETTY_FUNCTION__.15388> "pbx_findapp", t=t@entry=0x8a2090 , name=name@entry=0x62f1e1 "&(&apps)->lock") at lock.c:911 #5 0x000000000056ec0e in pbx_findapp (app=0x7f85c46d150b "Set") at pbx_app.c:97 #6 0x0000000000566115 in pbx_extension_helper (c=c@entry=0x7f85c4005470, context=0x7f85c4005e28 "sub_init_globals", exten=exten@entry=0x7f85c4005e78 "s", priority=priority@entry=32, label=label@entry=0x0, callerid=callerid@entry=0x0, action=action@entry=E_SPAWN, found=found@entry=0x7f85d764fcec, combined_find_spawn=combined_find_spawn@entry=1, con=0x0) at pbx.c:2848 #7 0x0000000000567e34 in ast_spawn_extension (combined_find_spawn=1, found=0x7f85d764fcec, callerid=0x0, priority=32, exten=0x7f85c4005e78 "s", context=, c=0x7f85c4005470) at pbx.c:4110 #8 __ast_pbx_run (c=c@entry=0x7f85c4005470, args=args@entry=0x0) at pbx.c:4285 #9 0x00000000005693fd in pbx_thread (data=data@entry=0x7f85c4005470) at pbx.c:4605 #10 0x00000000005d3d39 in dummy_start (data=) at utils.c:1235 #11 0x00007f85d9d8e34c in start_thread (arg=0x7f85d7650700) at pthread_create.c:334 #12 0x00007f85d966a50d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 (gdb) frame 5 #5 0x000000000056ec0e in pbx_findapp (app=0x7f85c46d150b "Set") at pbx_app.c:97 97 AST_RWLIST_RDLOCK(&apps); (gdb) frame 4 #4 0x0000000000529742 in __ast_rwlock_rdlock (filename=filename@entry=0x62f1f0 "pbx_app.c", line=line@entry=97, func=func@entry=0x62f79a <__PRETTY_FUNCTION__.15388> "pbx_findapp", t=t@entry=0x8a2090 , name=name@entry=0x62f1e1 "&(&apps)->lock") at lock.c:911 911 res = pthread_rwlock_rdlock(&t->lock); (gdb) print t->lock $5 = {__data = {__lock = 0, __nr_readers = 0, __readers_wakeup = 1, __writer_wakeup = 0, __nr_readers_queued = 1, __nr_writers_queued = 0, __writer = 50305, __shared = 0, __rwelision = 0 '\000', __pad1 = "\000\000\000\000\000\000", __pad2 = 0, __flags = 0}, __size = "\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\201\304", '\000' , __align = 0} => pbx_findapp() is blocked, through AST_RWLIST_RDLOCK(&apps), on apps list lock. apps list lock belongs to thread 9/50305 (see __owner). => (55) thread 55/50303 is waiting for thread 9/50305. (gdb) thread 29 [Switching to thread 29 (Thread 0x7f8558631700 (LWP 4586))] #0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135 135 ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S: Aucun fichier ou dossier de ce type. (gdb) bt #0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135 #1 0x00007f85d9d90929 in __GI___pthread_mutex_lock (mutex=mutex@entry=0x8a1d60 ) at ../nptl/pthread_mutex_lock.c:116 #2 0x000000000052969e in __ast_pthread_mutex_lock (filename=filename@entry=0x62b5eb "pbx.c", lineno=lineno@entry=8334, func=func@entry=0x62e890 <__PRETTY_FUNCTION__.20010> "ast_rdlock_contexts", mutex_name=mutex_name@entry=0x62b7ea "&conlock", t=t@entry=0x8a1d60 ) at lock.c:315 #3 0x0000000000565b6a in ast_rdlock_contexts () at pbx.c:8334 #4 pbx_extension_helper (c=0x0, context=0x7f852c021218 "gw_atssip_in", exten=0x7f8565de8b9a , priority=1, label=label@entry=0x0, callerid=0x7f8565deac70 , action=action@entry=E_MATCH, found=found@entry=0x0, combined_find_spawn=combined_find_spawn@entry=0, con=0x0) at pbx.c:2831 #5 0x00000000005678db in ast_exists_extension (c=, context=, exten=, priority=, callerid=) at pbx.c:4085 #6 0x00007f8565d679ec in ?? () #7 0x00007f8558630300 in ?? () #8 0x00007f852c02e0c5 in ?? () #9 0x00007f8558620073 in ?? () #10 0x00007f8565d67972 in ?? () #11 0x00007f855862f690 in ?? () #12 0x00007f855862f560 in ?? () #13 0x312e30313a706973 in ?? () #14 0x3138312e352e3030 in ?? () #15 0x00007f852c034700 in ?? () #16 0x0000000000000000 in ?? () (gdb) frame 3 #3 0x0000000000565b6a in ast_rdlock_contexts () at pbx.c:8334 8334 return ast_mutex_lock(&conlock); (gdb) print conlock $6 = {mutex = {__data = {__lock = 2, __count = 1, __owner = 50303, __nusers = 1, __kind = 1, __spins = 0, __elision = 0, __list = { __prev = 0x0, __next = 0x0}}, __size = "\002\000\000\000\001\000\000\000\177\304\000\000\001\000\000\000\001", '\000' , __align = 4294967298}, track = 0x0, tracking = 1} => pbx_extension_helper() is blocked, through ast_rdlock_contexts(), on ast_mutex_lock(&conlock). conlock mutex besongs to thread 55/50303 (see __owner). => (29) thread 29/4586 is waiiting for thread 55/50303. (gdb) thread 9 [Switching to thread 9 (Thread 0x7f85d7840700 (LWP 50305))] #0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135 135 ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S: Aucun fichier ou dossier de ce type. (gdb) bt #0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135 #1 0x00007f85d9d90929 in __GI___pthread_mutex_lock (mutex=mutex@entry=0x8a1d60 ) at ../nptl/pthread_mutex_lock.c:116 #2 0x000000000052969e in __ast_pthread_mutex_lock (filename=filename@entry=0x62b5eb "pbx.c", lineno=lineno@entry=8334, func=func@entry=0x62e890 <__PRETTY_FUNCTION__.20010> "ast_rdlock_contexts", mutex_name=mutex_name@entry=0x62b7ea "&conlock", t=t@entry=0x8a1d60 ) at lock.c:315 #3 0x000000000056b3cc in ast_rdlock_contexts () at pbx.c:8334 #4 unreference_cached_app (app=app@entry=0x7f85c40101b0) at pbx.c:6064 #5 0x000000000056f601 in ast_unregister_application (app=0x7f8566e3fee0 ) at pbx_app.c:408 #6 0x00007f8566df0e6f in ?? () #7 0x00007f85c4699620 in ?? () #8 0x00007f85c46995a0 in ?? () #9 0x00007f85c4699620 in ?? () #10 0x00007f85c4bde818 in ?? () #11 0x0000000000000000 in ?? () (gdb) frame 3 #3 0x000000000056b3cc in ast_rdlock_contexts () at pbx.c:8334 8334 return ast_mutex_lock(&conlock); (gdb) print conlock $7 = {mutex = {__data = {__lock = 2, __count = 1, __owner = 50303, __nusers = 1, __kind = 1, __spins = 0, __elision = 0, __list = { __prev = 0x0, __next = 0x0}}, __size = "\002\000\000\000\001\000\000\000\177\304\000\000\001\000\000\000\001", '\000' , __align = 4294967298}, track = 0x0, tracking = 1} => unreference_cached_app() is blocked, through ast_rdlock_contexts(), on ast_mutex_lock(&conlock). conlock mutex belongs to thread 55/50303 (see __owner). => (9) thread 9/50305 is waiting for thread 55/50303. Summary : Deadlock of threads 55/50303 (executing the dialplan) and thread 9/50305 (unregistering an application). Four more threads are blocked by one of these two threads. 1) Deadlock: => (55) thread 55/50303 is waiting for thread 9/50305. => (9) thread 9/50305 is waiting thread 55/50303. 2) Collateral effect: => (58) thread 58/12781 is waiting thread 55/50303. => (57) thread 57/12660 is waiting thread 55/50303. => (29) thread 29/4586 is waiting thread 55/50303. => (56) thread 56/63267 is waiting thread 9/50305.