Summary:	ASTERISK-04418: [fixed-need testers] Crash with Audiocode UA set to DTMF notify
Reporter:	Jon Brüel (jb)	Labels:
Date Opened:	2005-06-15 10:32:27	Date Closed:	2008-01-15 15:43:08.000-0600
Priority:	Critical	Regression?	No
Status:	Closed/Complete	Components:	Channels/chan_sip/Interoperability
Versions:		Frequency of Occurrence
Related Issues:
Environment:		Attachments:	( 0) 4534.SegfaultFixRev0.patch.txt ( 1) asterisk.cap ( 2) asterisk2.cap ( 3) Audiocode_Crash.txt ( 4) Audiocode_crash_gdb.txt ( 5) Audiocode_crash_gdb_HEAD.txt ( 6) Audiocode_crash_gdb_HEAD_as_per_morning_20-6-05.txt ( 7) Audiocode_crash_gdb_NEWHEAD.txt ( 8) Audiocode_no_crash_CLI.txt ( 9) chan_sip_subscribe_partial.diff
Description:	The system crashes when used together with an Audiocode UA when a DTMF parameters i the Audiocode is set to "Notify". The parameters is found under the following menu structure: Protocol Management, Protocol Definition, General, Out of band DTMF format. The crash happens when answering a call. Asterisk stays up for about 1,5 seconds before the conversation is broken. ****** ADDITIONAL INFORMATION ****** Audiocode version: 4.30.19.7. The SIP debug has been attached.
Comments:	By: Russell Bryant (russell) 2005-06-15 11:22:34 Please provide a backtrace from the crash. start asterisk with -g make it crash gdb ./asterisk core.<pid> (where <pid> is the process id of the instance of Asterisk that just crashed) (gdb) bt (gdb) bt full By: Michael Jerris (mikej) 2005-06-15 11:24:06 And also please test with at least current stable.. but also current cvs head.  You are using almost 3 month old code.  Lets see if it is already fixed before trying to fix it. By: Jon Brüel (jb) 2005-06-16 05:33:38 I would very much like to test with the latest CVS STABLE. I tried to get it a week ago, and the version from MArch, version 1.0.7, was the latest. If there is a later STABLE version, I'll ask you to instruct me how to get it. By: Michael Jerris (mikej) 2005-06-16 07:26:59 www.voip-info.org under, asterisk installation, asterisk download.  You need to download from cvs. By: Jon Brüel (jb) 2005-06-16 10:18:02 I have downloaded the latest version using MikeJ's directions, and I still end up with the same version from March 25th. Is this tha latest stable version, or do I do somthing wrong? By: Michael Jerris (mikej) 2005-06-16 10:57:10 Please find somone on IRC in #asterisk or #asterisk-bugs to assist you with this. By: Jon Brüel (jb) 2005-06-17 08:54:56 Well, I don't find any reason to "find someone". According to http://www.voip-info.org/tiki-index.php?page=Asterisk+Download, the latest stable version is 1.0.7, which is the version I have tested on. I will now attach the gdb information. By: Michael Jerris (mikej) 2005-06-17 09:00:37 per bug guidelines, this needs to be tested with current head. By: Mark Spencer (markster) 2005-06-17 09:03:00 This problem doesn't look like it can occur in CVS head.  I'd like to have some confirmation of that.  The problem is that ast_rtcp_read() is being called when there is no tcp structure. By: Jon Brüel (jb) 2005-06-17 12:22:50 OK, then, I'll do it some day in the not so far future. It is somewhat cumbersome because of the number of changes needs to make the HEAD work in our setup (due to the changes database structures). But I guess this a way an VoIP operator as us could contribute to the Asterisk community.. By: Jon Brüel (jb) 2005-06-17 15:01:24 Well, it wasn't as bad as anticipated. I have now tried it with the HEAD as of today (17-6-05). This time I hear a beep before it crashes, but, unfortunately, it still crashes. It also crash when a call is initiated from the Audiocode side. I have made a new dump. By: Jon Brüel (jb) 2005-06-17 15:42:59 A hint: We have tried the same setup with the STABLE 1.0.7 version on another machine, where we use 10 digit SIP addresses instead og 12 digit adresses. On that setup, we hear a beep, but the system does not crash. By: Mark Spencer (markster) 2005-06-18 12:52:30 The trace labeled "head" is clearly not from CVS head (the line numbers do not match).  Can you please try again with latest CVS head and produce a core from CVS head so that we can analyze it better, thanks! By: Jon Brüel (jb) 2005-06-19 04:31:08 Ok, this is a new trace, hope it is right this time. By: Mark Spencer (markster) 2005-06-19 17:48:02 Okay this should be fixed in CVS head now.  If it's still a problem, please include a SIP debug in addition to the backtrace.  Thanks! By: Jon Brüel (jb) 2005-06-20 02:10:26 Well, it still crashes, I have attached one file with sip debug in the biginning followed by the gdb stuff. By: Kevin P. Fleming (kpfleming) 2005-06-20 18:58:17 The SIP trace you supplied unfortunately does not include the packet from the Audiocodes box that is causing Asterisk to crash (because Asterisk crashes before printing it out). If you can, please use Ethereal to get a packet trace of the communication between the Asterisk server and the Audiocodes device, so we can see what the Audiocodes unit is sending. By: Jon Brüel (jb) 2005-06-21 04:36:16 A tcpdump capture file is now attached, which I hope you can use. By: Kevin P. Fleming (kpfleming) 2005-06-21 12:03:08 Yes, that trace is very helpful, thanks for uploading it. I believe the problem may be that the Audiocodes box is using the same Call-ID for the telephone-event SUBSCRIBE that it's using for the actual call, and by default Asterisk does not compare tags to match calls. Try this again with 'pedantic=yes' in the general section of your sip.conf file and see if you still have the same problem. By: Jon Brüel (jb) 2005-06-22 02:20:19 Setting pedandic to yes does not change the problem. Please bear in mind that the crash does not happen in our "private customer dialplan" where we use 2 less digits in the SIP accounts. By: Kevin P. Fleming (kpfleming) 2005-06-22 15:32:03 Just double-checking that your 'pedandic' was a typo in the bugnote, and not in your sip.conf file :-) Can you provide a trace of a _working_ call with the Audiocodes unit in the same configuration? By: Jon Brüel (jb) 2005-06-22 23:14:45 Yes, I'm aware of the pedantic typo, where I wrote pedandic. The spelling was correct in the configuration, though, and it lead to some other problems as well: Incoming rings were received, but could not be answered. In the mean time, I have downgraded the system because we do run it as a production system, I I have been asked not to dabble with the stability. Soon, I'll set it up on another blade and provide you with the requested information. By: Jon Brüel (jb) 2005-07-05 09:45:50 As requested, I have now attached a CLI screen dump, where the Audiocode/asterisk combination does not crash. The firmware version of the Audiocode is a step higher than the previous dump, where it did crash, but we have tested that it does also crash with this setup when set to out of band DTMF. The trace starts with the incoming call to the Audiocode. By: Kevin P. Fleming (kpfleming) 2005-07-05 21:50:20 OK, that trace shows the Audiocodes unit configured for RFC2833 DTMF, not SIP INFO. That's an entirely different configuration, so let's go back to the beginning: were you having this problem with the unit in SIP INFO mode, or RFC2833 mode? If it's the latter, then we are going to need to see the RTP stream as well between the units to figure out what is going on. By: Michael Jerris (mikej) 2005-07-12 17:39:43 Unfortunately we are unable to troubleshoot this issue without additional information.  If we do not get response we will have to close this issue out. By: Jon Brüel (jb) 2005-07-13 00:34:57 The system crashes 1,5 seconds after the audio connection, and a beep is heard just before. That was the rtp in words. If you need the rtp as a file please, advice me how I capture it. By: Michael Jerris (mikej) 2005-07-13 07:53:53 the same way you did before should be fine.  If you can set up logger to capture debug to file, both sip and rtp debug, along with verbose set 255, and a packet capture of the same traffic (as the offending traffic is likely not going to be in the debug as before) as well as answering kpflemings questions, that should do it. By: Olle Johansson (oej) 2005-07-20 13:50:29 Any updates? We're waiting for information in order to be able to find this bug. By: Jon Brüel (jb) 2005-07-21 10:37:32 I have now attached a tcpdump, thus the rtp should be included. After answer, I say 1-2-3 a couple of times. After approx. 2 seconds a beep is heard and the Asterisk crashes. I'll be away on holiday from 23th of July to 8th of August, but may be able to supply you with more data tomorrow - if required. By: Michael Jerris (mikej) 2005-07-21 10:45:10 can you supply the asterisk sip debug with verbose as well today? By: Jon Brüel (jb) 2005-07-22 09:49:54 I did not have time to supply a sip debug and trace, but you should be able to use the debugs and traces supplied before, see (0029200,06-20-05 02:10. If it does not give you the information required, we must wait till after the 7th of August. By: dbruce (dbruce) 2005-07-22 12:12:56 The problem is that the audiocodes is configured for DTMF NOTIFY. Asterisk only supports in-band, rfc2833 and SIP INFO. The notify method has the device subscribe for "event: telephone-event". This subscribe is interpreted by the current CVS implimentation as a presence subscription (since the only support for subscriptions is for MWI or presence). As kpfleming pointed out, the subscribe uses the same call-id for the subscribe as for the invite, so the subscribe is matched to the private structure for the INVITE. As part of the subscription processing, the RTP subsystem for the call is torn down. Once asterisk tries to process the next RTP frame for the call with an RTP read or write, the call no longer has an RTP structure, and asterisk crashes. This is definately a configuration error! you have the device configured to use a method that is not supported. Your bug report specifically says 'the Audiocode is set to "Notify"'. The Subscribe/Notify method is NOT supported. You need to configure the device to use SIP INFO or rfc2833 for DTMF to work. Alternately, you can use the patch in bugnote ASTERISK-3564 so that the subscribe if handled properly (when the subscribe is received, asterisk will respond with a "489 Bad Event" and ignore the subscribe). By: Michael Jerris (mikej) 2005-07-22 15:22:19 Ok, I will try to look at the patch on 3644 to see if I can get a smaller patch to review that will at least keep this from crashing. By: Michael Jerris (mikej) 2005-07-22 15:28:13 Nevermind, that's a huge patch. By: Michael Jerris (mikej) 2005-07-22 15:29:04 xylome, by any chance could you create a quick small patch for this that will address the segfault? By: Jon Brüel (jb) 2005-07-22 21:41:22 Looks good. As a telecommunications operator, we can't live with a central device which breaks down if the subscriber configures the UA improperly, so I would appreciate the patch to be included in the code. Thanks. By: dbruce (dbruce) 2005-07-22 22:53:57 Here is a patch against CVS-HEAD-2005-07-22 to resolve problems with unrecognized subscribe requests. Most of the credit goes to xylome. THIS IS NOT A COMPLETE FIX!!! Get the patch for ASTERISK-3564 into CVS for a complete fix and proper presence support!! Disclaimer on file. By: Michael Jerris (mikej) 2005-07-22 23:07:50 also uploaded a patch (from file) that may fix just the segfault itself.  If this avoids the segfault, it should go it as well for safety.  Can somone test this out as well... (both patches seperately to verify that they both fix the segfault at least. By: dbruce (dbruce) 2005-07-23 04:35:02 MikeJ: Although your patch will prevent the segfault after the RTP is torn down, by itself it will cause a call with signalling only. With only signalling present, the RTP bridge will collapse and the call will be disconnected. It is good insurance against segfaults for other situations that cause the tear down of RTP, but does not fix the cause of the problem. In the same vien, the patch I provided does not "fix" the problem either. It will reject the subscription for telephone events, allowing the call to proceed with signalling and audio... but DTMF will not be passed. Both patches are Methods to prevent asterisk from crashing. The only way to solve the problem completely is to improve subscription handling and add support for telphone event subscriptions. This is not a trivial matter, as is demonstrated by the patch for ASTERISK-3564. The advantage of ASTERISK-3564 is that it provides a basis for subscription event handling for telephone events and shared call appearances and bridged line appearances. By: Olle Johansson (oej) 2005-07-23 05:04:48 This bug is reported against stable. 3644 will not be added to stable, so let's concentrate on fixing the bug, not adding functionality. Stable needs to deny subscriptions for events it doesn't support and handle RTP better. By: Michael Jerris (mikej) 2005-07-23 10:34:12 ok, so we all agree... now we just need somone to test... By: Kevin P. Fleming (kpfleming) 2005-07-25 13:45:00 OK, it seems clear that the fix for this in CVS HEAD will come from bug ASTERISK-3613644 and related patches... so what is the status of this problem for CVS v1-0? Have either/both of these patches been tested there? By: Mark Spencer (markster) 2005-07-28 14:22:46 Please confirm this does not crash in current CVS head.  I've made a change which should take care of it, and we'll move the device state discussion back to 3644 where it belongs.  Thanks! By: Michael Jerris (mikej) 2005-08-03 13:25:04 This should be fixed, please re-open if this is still an issue.  Thanks By: Jon Brüel (jb) 2005-08-24 02:51:59 I have tested it again with CVS HEAD, and it still is an issue. The crash pattern is the same as before: after around 2 seconds, a beep is head and it crashes. By: Michael Jerris (mikej) 2005-08-24 03:10:11 can you test on cvs head with the patch from ASTERISK-3564 applied? By: Olle Johansson (oej) 2005-08-24 03:27:52 I'll check into this. 3644 denies a subscription that subscribes to unknown events. But I am not sure on how we handle subscriptions within a current dialogue. There's a risk we are tearing the channel down. Any references to this event? Any documents out there? By: Olle Johansson (oej) 2005-08-24 03:33:57 Looking at the code of handle_request_subscribe we assume this is a totally new dialog. As of now, we should reject subscriptions within a dialog. I'll update 3644 to do that. By: Jon Brüel (jb) 2005-08-24 14:39:57 Please inform me when retesting on an Audiocode is required. I would prefer the code incorporated in the head, as otherwise, I'll need to spend time on learning the patchwork technicalities. By: Kevin P. Fleming (kpfleming) 2005-08-30 23:00:16 ASTERISK-3613644 has been merged, so let's keep this one moving and get some re-testing done ASAP. By: Jon Brüel (jb) 2005-09-01 02:48:08 Still the same crash with CVS-v1-0-30/24/05. By: Michael Jerris (mikej) 2005-09-01 06:36:01 Can we have this tested with CVS head please. By: Jon Brüel (jb) 2005-09-01 08:50:25 Well, it should be the HEAD. I used the "asterisk-update.sh update dev" command and during the download, information was shown indicating that it was the HEAD. I said no to download addons. Please advice me, if I should upgrade it in a different way. By: Michael Jerris (mikej) 2005-09-01 08:59:49 what does show version say? By: Jon Brüel (jb) 2005-09-01 10:09:01 It says: Asterisk CVS-v1-0-09/01/05-15:27:23 built by root@blade1 on a i686 running Linux blade1*CLI> By: Michael Jerris (mikej) 2005-09-01 10:40:23 That is still the 1-0 branch.  Please do a fresh checkout into a different directory using cvs co asterisk, remove all the installed asterisk modules, and make and install the new checkout of asterisk. By: Jon Brüel (jb) 2005-09-01 12:12:29 I'm leaving for a two week training course, and I'll redo the test after. By: Michael Jerris (mikej) 2005-09-01 14:07:34 Ken-  Can you test this please. By: Jon Brüel (jb) 2005-09-01 15:10:47 I have tried to get the HEAD version to work. The compiled version did load OK (with the HEAD shown in the version description), but I do use app_dbodbc heavely, so I needed to have that included. After the HEAD version of the app_dbodbc was included, I got - during startup of Asterisk - the error message undefined symbol: ast_direct_realtime. I have tried to change the config.c and recomplile, as my knowledge about using the patches is nonexistant. So we are struck for now. If we can find a way, I may be able to retest before the break mentioned in the previous note. By: Michael Jerris (mikej) 2005-09-02 21:15:24 This should be resolved in HEAD, and outside of anything conclusive otherwise I am going to assume it is.  A 1.0 variation of the chan_sip_subscribe_partial.diff patch should resolve this issue in stable.  Marking this issue pending stable for inclusion in 1.0.  Can somone please create a chan_sip_subscribe_partial.diff variant that will apply to the 1.0 branch so that it can be tested.  Thanks By: Jon Brüel (jb) 2005-09-03 05:41:53 I get compile errors, so I can't proceed. I'll be leaving on a workshop for two weeks tomorrow, but I'll try to do the test before, if I can get through. The last lines including the error were: chan_zap.c: In function `handle_pri_show_debug': chan_zap.c:9130: warning: implicit declaration of function `pri_get_debug' chan_zap.c: In function `setup_zap': chan_zap.c:10418: error: `PRI_SWITCH_QSIG' undeclared (first use in this function) chan_zap.c: In function `load_module': chan_zap.c:10686: warning: passing arg 1 of `pri_set_error' from incompatible pointer type chan_zap.c:10687: warning: passing arg 1 of `pri_set_message' from incompatible pointer type make[1]: *** [chan_zap.o] Error 1 make[1]: Leaving directory `/usr/src/asterisk/channels' make: *** [subdirs] Error 1 [root@blade1 asterisk]# By: silik0n (silik0n) 2005-09-27 15:21:40 I've done some more checking on this with head. there appears yo still be a problem with DTMF set to NOTIFY on the MP10Xs. Instead of Segfaulting asterisk, the calls get dropped from the MP10X's end. Workaround is to use Info(Cisco) on the MP10X.  Trying to get a debug of this, but the device is in production at a remote site so its sort of hard to gather the data By: silik0n (silik0n) 2005-09-27 15:22:28 Ack, Submitted too soon. If anyone has one of these available for testing please contact me on IRC (SwK) By: Olle Johansson (oej) 2005-10-04 12:41:27 Logged in to system to check. Found out that Asterisk parses the Digest header incorrectly (separate patch in another issue report) but the audiocode return the nonce like nonce="laskdfj" " and propably does a bad calculation because the nonce is wrong. This will be reported to Audiocodes by issuer. By: Russell Bryant (russell) 2005-10-04 13:48:49 Does this issue only relate to 1.0?  Otherwise, it should be changed from the 'pending stable' status ... By: Morten Isaksen (misaksen) 2005-10-05 08:04:15 At the request of Olle I have testet this. Asterisk CVS-HEAD (checked out today) and Audiocodes MP-124 (firmware 4.60A.008.006). Asterisk does not crash when DTMF Notify is set. You can make calls, but DTMF is ignored. I tested against the Asterisk voicemail and Asterisk ignored any digits a pressed. By: Michael Jerris (mikej) 2005-10-05 08:40:05 Ok, this is confirmed fixed in cvs head, can anyone test this against cvs v-1-0 please. By: Morten Isaksen (misaksen) 2005-10-05 09:07:22 I just checked it with Asterisk 1.0.9 and it crashes.    -- Executing VoiceMailMain("SIP/070001-71e6", "s070001") in new stack Oct  5 17:05:19 DEBUG[8028]: rtp.c:1195 ast_rtp_write: Ooh, format changed from unknown to ulaw Oct  5 17:05:19 DEBUG[8028]: channel.c:1128 ast_settimeout: Scheduling timer at 160 sample intervals    -- Playing 'vm-login' (language 'en') Oct  5 17:05:20 DEBUG[8011]: chan_sip.c:841 __sip_ack: Stopping retransmission on '2577618449hMKz@10.131.2.1' of Response 2: Found By: Olle Johansson (oej) 2005-10-05 09:16:00 misaksen: Thanks a lot for testing! I am happy for every bug I can remove from the 1.2 horizon... !!! By: Michael Jerris (mikej) 2005-10-05 09:17:18 can you also check from the 1-0 branch checked out from cvs, I beleive a fix went in there but did not make 1.0.9. By: Morten Isaksen (misaksen) 2005-10-05 10:10:48 I'll check tomorrow when I am back at the office... By: silik0n (silik0n) 2005-10-05 12:33:09 Ok Guys I have noticed that with the 4.40 Firmware this problem may still exist. Also, I had a problem of not being able to get the 4.60 firmware on the MP104 to register correctly and had to down-grading the box to 4.40 makes the MP-104. Can we get someone verify or discount this? By: Morten Isaksen (misaksen) 2005-10-22 14:26:44 One of our MP-124 is beeing repaired at the moment, so I do not have access to the spare MP-124 at the moment. I will try to test this when we get it back from repair. By: Russell Bryant (russell) 2005-11-15 14:09:56.000-0600 If anyone is still seeing any issues in cvs head or 1.2, please open a new bug.  Thanks! By: Digium Subversion (svnbot) 2008-01-15 15:43:08.000-0600 Repository: asterisk Revision: 6243 U   trunk/channels/chan_sip.c ------------------------------------------------------------------------ r6243 | markster | 2008-01-15 15:43:08 -0600 (Tue, 15 Jan 2008) | 2 lines Don't delete RTP if there is a channel, even if we get REGISTER/SUBSCRIBE (bug ASTERISK-4418) ------------------------------------------------------------------------ http://svn.digium.com/view/asterisk?view=rev&revision=6243