[Home]

Summary:ASTERISK-24752: Crash in bridge_manager_service_req when bridge is destroyed by ARI during shutdown
Reporter:Richard Mudgett (rmudgett)Labels:
Date Opened:2015-02-02 15:59:06.000-0600Date Closed:2015-02-11 11:29:40.000-0600
Priority:MajorRegression?
Status:Closed/CompleteComponents:Core/Bridging Core/General
Versions:13.1.1 Frequency of
Occurrence		One Time
Related
Issues:
Environment:Attachments:( 0) threads.txt
Description:The full trace is attached.

The relevant thread is here:
{noformat}
#0  0x080a2592 in bridge_manager_service_req ()
#0  0x080a2592 in bridge_manager_service_req ()
No symbol table info available.
#1  0x080a2b65 in bridge_queue_action_nodup ()
No symbol table info available.
#2  0x080a2b9a in ast_bridge_queue_action ()
No symbol table info available.
#3  0x080a2cc7 in bridge_dissolve ()
No symbol table info available.
#4  0x080a48e6 in ast_bridge_destroy ()
No symbol table info available.
#5  0x00cec473 in stasis_app_bridge_destroy (bridge_id=0x96ae581 "0d77e47d-dccd-4c8c-96f1-64becabdee8f") at res_stasis.c:794
       bridge = 0xb2f9421c
#6  0x08e200e4 in ast_ari_bridges_destroy (headers=0x96ab258, args=0xb71a5e38, response=0xb71a5f64) at ari/resource_bridges.c:862
       bridge = 0xb2f9421c
#7  0x08e1c5d3 in ast_ari_bridges_destroy_cb (ser=0x967316c, get_params=0x0, path_vars=0x96ae528, headers=0x96ab258, response=0xb71a5f64) at res_ari_bridges.c:397
       args = {bridge_id = 0x96ae581 "0d77e47d-dccd-4c8c-96f1-64becabdee8f"}
       i = 0x0
       body = 0x0
#8  0x00b1019e in ast_ari_invoke (ser=0x967316c, uri=0xb71a605c "bridges/0d77e47d-dccd-4c8c-96f1-64becabdee8f", method=AST_HTTP_DELETE, get_params=0x0, headers=0x96ab258, response=0xb71a5f64) at res_ari.c:560
       root = 0x990e244
       handler = 0x8e22840
       path_vars = 0x96ae528
       path = 0x0
       path_segment = 0x0
       callback = 0x8e1c55f <ast_ari_bridges_destroy_cb>
       __PRETTY_FUNCTION__ = "ast_ari_invoke"
#9  0x00b111ad in ast_ari_callback (ser=0x967316c, urih=0xb17ba4, uri=0xb71a605c "bridges/0d77e47d-dccd-4c8c-96f1-64becabdee8f", method=AST_HTTP_DELETE, get_params=0x0, headers=0x96ab258) at res_ari.c:965
       conf = 0x97d6a14
       response_body = 0x996fa70
       user = 0x97dcdf4
       response = {message = 0x0, headers = 0x997d2b8, response_code = 0, response_text = 0x0, no_response = 0}
       post_vars = 0x0
       __PRETTY_FUNCTION__ = "ast_ari_callback"
#10 0x081501c2 in handle_uri ()
No symbol table info available.
#11 0x08150ff3 in httpd_process_request ()
No symbol table info available.
#12 0x0815141e in httpd_helper_thread ()
No symbol table info available.
#13 0x081f6515 in handle_tcptls_connection ()
No symbol table info available.
#14 0x08206eb8 in dummy_start ()
No symbol table info available.
#15 0x00a24a49 in start_thread () from /lib/libpthread.so.0
No symbol table info available.
#16 0x004ba63e in clone () from /lib/libc.so.6
No symbol table info available.
{noformat}

Looking at {{bridge_manager_service_req}}, there's only so many pointers that could have gone wrong:
{code}
static void bridge_manager_service_req(struct ast_bridge *bridge)
{
struct bridge_manager_request *request;

ao2_lock(bridge_manager);
if (bridge_manager->stop) {
ao2_unlock(bridge_manager);
return;
}

/* Create the service request. */
request = ast_calloc(1, sizeof(*request));
if (!request) {
/* Well. This isn't good. */
ao2_unlock(bridge_manager);
return;
}
ao2_ref(bridge, +1);
request->bridge = bridge;

/* Put request into the queue and wake the bridge manager. */
AST_LIST_INSERT_TAIL(&bridge_manager->service_requests, request, node);
ast_cond_signal(&bridge_manager->cond);
ao2_unlock(bridge_manager);
}
{code}

Generally, I'd suspect it would be the {{bridge_manager}} itself... but that'd be odd, since the {{bridge_manager}} is only disposed of on Asterisk shutdown.

Except, that is what is happening:

{noformat}
Thread 99 (Thread 0xb775c780 (LWP 28363)):
#0  0x0809af9a in hash_ao2_find_next ()
#1  0x080995b3 in internal_ao2_traverse ()
#2  0x08099750 in __ao2_callback ()
#3  0x08099dbe in container_destruct ()
#4  0x08098394 in internal_ao2_ref ()
#5  0x08098665 in __ao2_ref ()
#6  0x080986c3 in __ao2_cleanup ()
#7  0x08116214 in aco_deinit ()
#8  0x0808e12a in ast_run_atexits ()
#9  0x08090567 in really_quit ()
#10 0x0808ff18 in quit_handler ()
#11 0x08097a00 in main ()
{noformat}

Since we're in the middle of our {{ast_run_atexits}}, it's pretty safe to assume that the {{bridge_manager}} is no more. Or at least, is in the process of becoming no more.
Comments: