Summary:	ASTERISK-18724: crash in __ao2_ref_debug
Reporter:	Matthew Nicholson (mnicholson)	Labels:
Date Opened:	2011-10-14 13:59:37	Date Closed:	2011-11-01 17:40:27
Priority:	Minor	Regression?	No
Status:	Closed/Complete	Components:	Core/General
Versions:	1.8.7.0 10.0.0-beta2	Frequency of Occurrence	Occasional
Related Issues:
Environment:		Attachments:	( 0) btfull.txt ( 1) fopen-error-check1.diff ( 2) refs
Description:	In __ao2_ref_debug(), we don't check the return value of fopen(). If fopen() fails for some reason, asterisk crashes. {noformat} (gdb) bt #0  0x0053fbf9 in vfprintf () from /lib/libc.so.6 #1  0x00549e42 in fprintf () from /lib/libc.so.6 #2  0x0808aced in __ao2_ref_debug (user_data=0xadbe3558, delta=-1, tag=0x81cc5b4 "deref object via container destroy", file=0x81cc4c1 "astobj2.c",    line=959, funcname=0x81cc6d3 "cd_cb_debug") at astobj2.c:214 #3  0x0808ad69 in cd_cb_debug (obj=0xadbe3558, arg=0x0, flag=1) at astobj2.c:959 #4  0x0808bae4 in internal_ao2_callback (c=0xabab00c0, flags=OBJ_UNLINK, cb_fn=0x808ad30, arg=0x0, data=0x0, type=DEFAULT,    tag=0x81cc5d8 "container_destruct_debug called", file=0x81cc4c1 "astobj2.c", line=988, funcname=0x81cc6ba "container_destruct_debug") at astobj2.c:681 #5  0x0808c606 in __ao2_callback_debug (_c=0xabab00c0) at astobj2.c:778 #6  container_destruct_debug (_c=0xabab00c0) at astobj2.c:988 #7  0x0808aa12 in internal_ao2_ref (user_data=0xabab00c0, delta=0) at astobj2.c:262 #8  0x081134d0 in ast_format_cap_destroy (cap=0xaba48f90) at format_cap.c:97 #9  0x080b7355 in ast_request (type=0x997f7d48 "agent", request_cap=0xb71c890, requestor=0xb74eb48, data=0x997f7d4e, cause=0x997f7edc) at channel.c:5738 #10 0x002d3cd5 in ring_entry (qe=0x997f9cb8, tmp=0x99b99588, busies=0x997f9bb4) at app_queue.c:3115 #11 0x002d59ff in ring_one (qe=0x997f9cb8, outgoing=0x99bb9220, busies=0x997f9bb4) at app_queue.c:3291 #12 0x002dc30a in try_calling (qe=0x997f9cb8, options=<value optimized out>, announceoverride=0x997f9c3e "", url=0x997f9c3d "", tries=0x997fae3c,    noption=0x997fae38, agi=0x0, macro=0x0, gosub=0x0, ringing=0) at app_queue.c:4636 #13 0x002e06bc in queue_exec (chan=0xb74eb48, data=0x997fcf04 "uut_queue,cn,,,20") at app_queue.c:6070 #14 0x0815183b in pbx_exec (c=0xb74eb48, app=0x9386528, data=0x997fcf04 "uut_queue,cn,,,20") at pbx.c:1542 #15 0x081624ed in pbx_extension_helper (c=0xb74eb48, con=0x0, context=0xb74f240 "queue", exten=0xb74f290 "100", priority=17, label=0x0,    callerid=0x9ba7b8a0 "1122", action=E_SPAWN, found=0x997ff328, combined_find_spawn=1) at pbx.c:4260 #16 0x081655f4 in ast_spawn_extension (c=0xb74eb48, args=0x0) at pbx.c:4911 #17 __ast_pbx_run (c=0xb74eb48, args=0x0) at pbx.c:5002 #18 0x08168220 in pbx_thread (data=0xb74eb48) at pbx.c:5337 #19 0x081aaba3 in dummy_start (data=0xb86cec0) at utils.c:1010 #20 0x00680832 in start_thread () from /lib/libpthread.so.0 #21 0x005d5e0e in clone () from /lib/libc.so.6 (gdb) frame 2 #2  0x0808aced in __ao2_ref_debug (user_data=0xadbe3558, delta=-1, tag=0x81cc5b4 "deref object via container destroy", file=0x81cc4c1 "astobj2.c",    line=959, funcname=0x81cc6d3 "cd_cb_debug") at astobj2.c:214 214 fprintf(refo, "%p %s%d   %s:%d:%s (%s) [@%d]\n", user_data, (delta<0? "":"+"), delta, file, line, funcname, tag, obj ? obj->priv_data.ref_counter : -1); (gdb) info locals refo = 0x0 obj = 0xadbe3528 {noformat} Notice refo is NULL.
Comments:	By: Matthew Nicholson (mnicholson) 2011-10-14 14:00:38.678-0500 I have attached a full bt and the last 10 lines from /tmp/refs (the file fopen() tries to open). By: Matthew Nicholson (mnicholson) 2011-10-14 14:18:58.049-0500 In my situation it appears that fopen is failing because asterisk is out of available file descriptors. By: Matthew Nicholson (mnicholson) 2011-10-19 14:26:47.535-0500 Uploaded fopen-error-check1.diff. This patch simply checks for errors with fopen and does not attempt to write the file if fopen fails. This would render the ref debugging file useless as some reference operations would be missing so perhaps a message should be logged if fopen fails. By: Richard Mudgett (rmudgett) 2011-11-01 17:40:27.658-0500 Fixed by v1.8 SVN commit -r342487.  Wdokees also reported.