Summary:	ASTERISK-05900: Crash in meetme: *** glibc detected *** double free or corruption (!prev): (0x......)
Reporter:	kuj (kuj)	Labels:
Date Opened:	2005-12-24 14:23:15.000-0600	Date Closed:	2008-01-15 16:08:54.000-0600
Priority:	Critical	Regression?	No
Status:	Closed/Complete	Components:	Applications/app_meetme
Versions:		Frequency of Occurrence
Related Issues:
Environment:		Attachments:	( 0) bt.txt ( 1) bt3.txt ( 2) bt4.txt ( 3) bt5.txt ( 4) bt8.txt ( 5) console3.txt ( 6) debug.txt ( 7) debug3.txt ( 8) debug4.txt ( 9) debugfull5.txt (10) debugfull8.txt (11) extensions.conf (12) sipcrash2.txt
Description:	This may be another datapoint in the series of recently reported crashes. Scenario: UA1 (Polycom) dials into conference line, is placed into conference as a user, waiting for admin, listening to MOH. UA2 (eyebeam) dialing into same conf line (Ext. 2600 in attached extensions.conf), entering conf room number as prompted by Background command, but intention is to enter as admin (11-digit room number per attached extension.conf). While entering conf room number, and before entering full 11-digit room no. or being placed into Meetme, * crashes according to attached bt.txt. Note that UA2 has not been placed into conference yet. Attached files (bt, debug, sipcrash2 console log) were captured from * built with dont-optimize, so should yield valid backtrace. While it crashes in app_meetme, I doubt the root cause to this is in app_meetme, so feel free to reclassify. This crash is reproducible here, although the exact number of DTMF digits required to crash * varies. ****** ADDITIONAL INFORMATION ****** This crash does not happen on SVN trunk version 7520M or earlier.
Comments:	By: Tilghman Lesher (tilghman) 2005-12-24 16:57:22.000-0600 Additional backtrace info needed: (gdb) frame 6 (gdb) p *fr By: Tilghman Lesher (tilghman) 2005-12-24 17:01:03.000-0600 Come to think of it, please apply the debugging patch is 6032.  Being able to see two different systems with the same bogus frame crash may help to diagnose the issue faster. By: Mark Spencer (markster) 2005-12-24 18:28:33.000-0600 Should be fixed in SVN trunk ASTERISK-7424, sorry about that.  Feel free to reopen if the problem still occurs. By: kuj (kuj) 2005-12-24 21:01:57.000-0600 Gotta reopen, as the latest meetme changes don't fix it. I'll attach more files, one set just with the meetme change (7620), the other built on 7620 with Corydon's debug patch from ASTERISK-5976032. By: kuj (kuj) 2005-12-24 21:07:54.000-0600 Files uploaded. bt3, console3 and debug3 go together (SVN trunk 7620). bt4 and debug4 are the other set (SVN trunk 7620 + Corydon's debug patch). debug4 contains a "full" logger config, thus no separate console log. By: Tilghman Lesher (tilghman) 2005-12-24 23:17:29.000-0600 Could you rerun the bt4 with error,warning,notice turned on, as well as debug,verbose (in logger.conf)? By: kuj (kuj) 2005-12-25 13:31:09.000-0600 Here you go: debugfull5 has a "full" log as requested. I believe debug4 had that as well. However, I couldn't find your debug prints (from the patch in ASTERISK-5976032) in either of them. I did verify the patch was applied, though. Rebuilt from scratch also (make clean dont-optimize). By: Tilghman Lesher (tilghman) 2005-12-25 14:18:30.000-0600 Uh, there's something wrong with your "full" log.  That only has verbose and debug enabled.  I'd expect to see at least a few NOTICE, WARNING, and ERROR messages. Perhaps you forgot to do a 'logger reload' after changing logger.conf ? By: kuj (kuj) 2005-12-25 14:36:08.000-0600 Nothing wrong with the log. I just trimmed it to a "relevant" time window: from when UA2 (eyebeam softphone) registers, to when the crash occurs. No ERRORs, WARNINGs or NOTICEs are logged during that time. I do see a few NOTICEs and WARNINGs prior to the softphone registering, no ERRORs though. None of those logs are from your debug code. Right now I'm single-stepping back through the svn timeline, trying to determine which updates are causing this issue. By: kuj (kuj) 2005-12-25 15:14:45.000-0600 svn trunk 7547M introduces the issue. Anything before that does not crash. I can take the latest svn (7626) and just replace app_meetme.c with a pre-7547 version and it works fine. However, I suspect that the changes to channel.[ch] in 7547 may also play a role, as the crash is triggered *before* UA2 is placed into the meetme. (Recall that UA1 is waiting in the meetme room, waiting with MoH for the leader to arrive. I then start dialing on UA2, which causes the crash to occur. On UA2, I have not yet been prompted for the conf. room no., though, so UA2 cannot be in the meetme yet) By: Mark Spencer (markster) 2005-12-25 17:13:51.000-0600 I'm having some trouble with your backtrace.  Versions 7620 to 7626 have a "break" at line 2276, where your backtrace claims to have an ast_frfree()...  Are you using any features with meetme?  If so, if you use "plain meetme" does the problem still occur? By: Mark Spencer (markster) 2005-12-25 17:40:02.000-0600 I am theorizing that the crash has to do with the fact that we're using just one pseudo channel for both making announcements and for recording the channel / doing the conversions.  I think we may have to break down and open *two* pseudos, one for announcements, one for recording, alas. By: Mark Spencer (markster) 2005-12-25 17:47:50.000-0600 Okay, it should *really* be fixed no in SVN trunk 7627.  Again feel free to reopen if i still didnt' get it :)  Merry Christmas / Happy Holidays! By: Russell Bryant (russell) 2005-12-25 18:06:48.000-0600 these crashes are related to optimizations only present in the trunk, so no changes are necessary for the 1.2 branch By: kuj (kuj) 2005-12-25 18:27:39.000-0600 Don't know what to say, but the mystery goes on. Same thing happens with 7627 applied. To be on the safe side, I got rid of existing sources, downloaded complete source (7627), deleted all existing modules, built source with dont-optimize, installed, had to add format_mp3.so as single "foreign" module (after compiling it against the 7627 source tree) and could still replicate the problem. See bt8 and debugfull8. This time, the line number from the backtrace matches the real source. Sorry for that, don't know how it happened. Merry Christmas to you guys, too! I'm going to give in for the day now! By: Mark Spencer (markster) 2005-12-26 16:21:59.000-0600 You've got me stumped.  Sounds like we're going to have to find a way to get together on IRC and go through this together.  Find me as "kram" in irc.freenode.net.  I will need root access on your machine and will need you to create the problem.  Thanks! By: Mark Spencer (markster) 2005-12-26 16:56:56.000-0600 Okay also trying another fix attempt.  Update to latest SVN trunk ASTERISK-7442 and see if it goes away! By: kuj (kuj) 2005-12-26 17:20:48.000-0600 Mark, 7640 was it! Thanks much! 7627 (local channel variation), when run together with 7640, yielded some really bad (i.e. scratchy, noisy) sound when users were placed into the conference after the conf admin joined (conf join announcement was bad, afterwards ok). I rolled back app_meetme.c to 7620 and ran it together with 7640 without those problems. No crash so far. I recommend to roll back app_meetme.c to 7620 and not use the local channel variation of 7627. Thanks again! By: Matt O'Gorman (mogorman) 2006-01-10 09:57:28.000-0600 Marko... close your bugs... fixed in commmit 7640 By: Digium Subversion (svnbot) 2008-01-15 16:08:48.000-0600 Repository: asterisk Revision: 7620 U   trunk/apps/app_meetme.c ------------------------------------------------------------------------ r7620 | markster | 2008-01-15 16:08:48 -0600 (Tue, 15 Jan 2008) | 2 lines Fix multiple free of a frame (bug ASTERISK-5900) ------------------------------------------------------------------------ http://svn.digium.com/view/asterisk?view=rev&revision=7620 By: Digium Subversion (svnbot) 2008-01-15 16:08:54.000-0600 Repository: asterisk Revision: 7627 U   trunk/apps/app_meetme.c ------------------------------------------------------------------------ r7627 | markster | 2008-01-15 16:08:54 -0600 (Tue, 15 Jan 2008) | 3 lines Add "local channel" variation so that we don't read/write to the same channel...  (bug ASTERISK-5900) ------------------------------------------------------------------------ http://svn.digium.com/view/asterisk?view=rev&revision=7627