Summary:ASTERISK-12366: [patch] crash can occur when ast_channel_free() tries to free a chan_iax2 tech_pvt
Reporter:Peter Grayson (jpgrayson)Labels:
Date Opened:2008-07-11 20:09:10Date Closed:2008-07-16 15:40:14
Versions:Frequency of
Environment:Attachments:( 0) chan_iax2_tech_pvt_crash.patch
Description:There apparently exists some sort of race condition in chan_iax2 that leads to a channel's tech_pvt member remaining non-null even after iaxs[PTR_TO_CALLNO(c->tech_pvt)] has been set to null.

Note the following backtrace:

Program terminated with signal 11, Segmentation fault.
Thread 1 (process 30298):
#0  0x002665f9 in free () from /lib/tls/libc.so.6
#1  0x0807f3ce in ast_channel_free (chan=0x25b9) at channel.c:1235
#2  0x08085d5a in ast_hangup (chan=0x81ff1b0) at channel.c:1524
#3  0x080cde91 in __ast_pbx_run (c=0x81ff1b0) at pbx.c:2576
#4  0x080cf8ee in pbx_thread (data=0x81ff1b0) at pbx.c:2636
ASTERISK-1  0x080fe5d5 in dummy_start (data=0x0) at utils.c:867
ASTERISK-2  0x003f33cc in start_thread () from /lib/tls/libpthread.so.0
ASTERISK-3  0x002ccc3e in clone () from /lib/tls/libc.so.6

The crash occurs in ast_channel_free() when it tries to free() memory at address 0x25b9. This is chan->tech_pvt. I am not sure why gdb prints the wrong argument in the backtrack for frame #1. Obviously tech_pvt is not used as a pointed, but as an index into iaxs and iaxsl, so it's no surprise that attempting to free() tech_pvt caused a crash.

In channel.c, immediately preceding the ast_channel_free() call is a call to chan->tech->hangup(chan). Also note that in this case, chan->flags is 0.

This problem has been observed at least three times in the past three weeks running asterisk- I have not confirmed this bug against or against head of svn, but I also do not see any changes in those versions that would likely affect this outcome.

The dynamics at play in chan_iax2 concerning iaxs[callno], channel objects, iax2_destroy(), iax2_predestroy(), iax2_hangup(), etc. are incredibly complicated. Thus it is not at all clear to me what set of circumstances precipitate this problem.
Comments:By: Peter Grayson (jpgrayson) 2008-07-11 20:14:33

I have submitted a patch that I believe will mitigate this problem. By pre

This change is safe because the expected post-condition for iax2_hangup() is
that c->tech_pvt == NULL. In the preceding 'if' block, iax2_predestroy() does
just that. It's also safe because that previous 'if' test just determined that
iaxs[callno] no longer exists -- this is very strong evidence that the only
possible outcomes from the channel object retaining a non-null tech_pvt are undesirable.

It will be several weeks until I have enough runtime on my systems to validate that this does alleviate the symptom.

Even without validating that it fixes the observed problem, I think this patch may stand on its own legs.

By: Digium Subversion (svnbot) 2008-07-16 15:40:12

Repository: asterisk
Revision: 131421

U   branches/1.4/channels/chan_iax2.c

r131421 | russell | 2008-07-16 15:40:11 -0500 (Wed, 16 Jul 2008) | 7 lines

Always ensure that the channel's tech_pvt reference is NULL after calling the
destroy callback.
(closes issue ASTERISK-12366)
Reported by: jpgrayson
     chan_iax2_tech_pvt_crash.patch uploaded by jpgrayson (license 492)