Summary:	ASTERISK-11548: CCM sends multiple INVITEs which causes MusicOnHold to execute on a channel
Reporter:	Leif Madsen (lmadsen)	Labels:
Date Opened:	2008-02-29 16:27:16.000-0600	Date Closed:	2011-06-07 14:07:28
Priority:	Minor	Regression?	No
Status:	Closed/Complete	Components:	Channels/chan_sip/Interoperability
Versions:		Frequency of Occurrence
Related Issues:
Environment:		Attachments:	( 0) Cisco_7940_remoteagent_inbound.log ( 1) debug.ciscotoasterisk-goodpedantic ( 2) debug.ciscotoasterisk-goodpedantic-1.2.26.2 ( 3) debug.remoteagent-badcallpedantic ( 4) debug.remoteagent-badcallpedantic-1.2.26.2 ( 5) debug.remoteagent-bad-with12110 ( 6) issue_12110.patch ( 7) tcpdump.remoteagent-badcall-1.2.26.2.cap ( 8) tcpdump.remoteagent-badcall-20080312.cap ( 9) tcpdump.remoteagent-badcall-unpatched.cap
Description:	Found this for a client who is using a CCM gateway and terminating calls to SIP phones attached to Asterisk. I've added some traces from the console with history dumped. You'll notice in the good calls that you get 3 or 4 INVITE,100,200,ACK loops, then the CCM stops sending the invites, and the call remains setup. In 1.2.26.2, you'll see that happen, with the Starting Music on Hold/Stopping Music on Hold each time that loop happens. On 1.2 though, the music on hold never actually is heard. On 1.4, when the same thing happens, the music on hold actually is executed and does not stop. This is preventing him from moving to 1.4. ****** ADDITIONAL INFORMATION ****** This too me looks like a bug with the Cisco, but if we can handle it gracefully or something, that'd be the ideal. Let me know how we should proceed. You'll notice a good call is a call from: Cisco Phone --> CCM --> Asterisk --> SIP phone A bad call (music on hold and oddities) is: DID --> CCM --> (database lookup to perform routing... ICM) --> Asterisk -- SIP phone The bad calls that did a lookup in the ICM seem to be the ones that don't work and have the odd INVITE,100,200,ACK loop.
Comments:	By: Ryan Ritchie (kreator) 2008-03-06 21:49:03.000-0600 blitzrage sent the diff file to me and we tried applying it to a 1.4.18 build tonight.  Thanks for the patch, by the way.  We did see some change in behaviour but not a full resolution.  The Looping Started/Stopped MOH message is gone. New behaviour: - Call rings on agent's phone. - Agent answers the call but hears MOH - After 3-5 seconds the call is torn down and a routing rule message is displayed Please see "debug.remoteagent-bad-with12110" in the attachments to see the newly observed behaviour. By: Kevin P. Fleming (kpfleming) 2008-03-11 17:25:20 First, I'd like to see if we can get a tcpdump/wireshark capture of the traffic between the CCM and Asterisk and between Asterisk and the SIP phone for one of these failing calls. A packet capture is much easier to analyze than a console log :-) The packet capture should be made from an unmodified Asterisk installation, not one with any patches applied. Second, a number of us have analyzed the information that is provided here and concluded that it appears that Asterisk is doing *exactly* what the CCM is asking;  it asks us to put the call on hold, and we do that. For some reasons it keeps on asking over and over again (and we keep the call on hold), and it never puts the call back into normal ('sendrecv', 2-way audio) mode. You didn't experience the same symptoms with Asterisk 1.2 because music-on-hold handling in Asterisk 1.2 was written very differently and in many cases did not work correctly; you happened to find a case where a combination of bugs in our MOH handling masked the problems being caused by CCM. This code was rewritten in Asterisk 1.4, and now you are seeing the bad effects this behavior causes. Once we have a packet capture of a complete call (start to finish) we can be more sure of what we are seeing, but it certainly appears that the CCM is misbehaving and that Asterisk can't be expected to do anything other than what it is doing. By: Ryan Ritchie (kreator) 2008-03-11 23:35:37 Just uploaded the two requested captures grabbed with tcpdump.     tcpdump.remoteagent-badcall-unpatched.cap <-- 1.4.18     tcpdump.remoteagent-badcall-1.2.26.2.cap  <-- 1.2.26.2 Observed behaviour: - Agent (called party) picks up phone and hears MOH (10.132.252.55)   Called party does not hear the calling party - Calling party hears the called party just fine and does not hear any MOH - Call stays connected until either party disconnects. Elements: 10.132.253.245 <-- Asterisk 10.132.252.55  <-- Cisco 7940 Phone registered to Asterisk 10.132.20.10   <-- CCM 5.1 Publisher By: Kevin P. Fleming (kpfleming) 2008-03-12 13:21:51 Please try the attached issue_12110.patch against Asterisk 1.4; Asterisk has some code to explicitly take calls *off* hold when receiving an INVITE with no SDP, which as added to interoperate with at least one known system that uses them that way. The problem here is that CCM expects the opposite behavior; when it sends an INVITE with no SDP, it's expecting use not to change the state of the call, but we are. If the patch works for you, then we'll need to make a config option to make this behavior in Asterisk something that can be controlled. By: Ryan Ritchie (kreator) 2008-03-12 16:44:40 Should I assume that this patch is written against the 1.4.18 release tarball?  Seems to have applied OK. root@rsi-nod-asterisk1:/usr/src/1.4.18/asterisk-1.4.18.12110# patch -p0 < 12110.patch patching file channels/chan_sip.c Hunk #1 succeeded at 13786 (offset -33 lines). I'll be trying this tonight at 22:00 PDT. By: Ryan Ritchie (kreator) 2008-03-13 00:38:44 Thanks  for the patch, kpfleming.  Sadly, this did not solve the issue, just caused a different ungraceful behaviour. Please see "tcpdump.remoteagent-badcall-20080312.cap" to see what changed. Patched behaviour: - Call rings on the agent's (called party) phone - Agent answers and hears MOH from the Asterisk server - Cisco client sees a "requested operation failed" popup <- *NEW* - Calling Party gets their call dropped instantly when the called party answers. <- *NEW* - Still say the Starting/Stopped MOH loop condition within the asterisk console. By: Kevin P. Fleming (kpfleming) 2008-03-13 08:16:29 Well, I'm not sure what to tell you. In this trace Asterisk is doing *exactly* what the CCM is asking it to do. The initial call setup proceeds normally, then the CCM asks Asterisk to put the call on hold (the media session is set to 'inactive'). After that, the CCM sends a series of INVITE requests with no SDP included. The RFCs do not say what should happen when this occurs. In older releases of Asterisk, and 1.4 with the patch I supplied, Asterisk does not change the state of the session, it just replies to the INVITE as normal. In unpatched Asterisk 1.4, Asterisk takes the session off hold (the media session returns to 'sendrecv'), which your CCM does not handle well, it asks Asterisk to put the call back on hold. This code in Asterisk was put there explicitly to support Cisco Call Manager by oej when he was at a SIP interoperability event early last year, so it's really bizarre why your CCM is acting so strangely. In any case, there are only two things we can do when we receive an INVITE with no SDP: we can either respond with the session's current state without changing it, or we can bring the session back from hold. We have now tried both alternatives with your CCM, and neither of them results in usable behavior for your system. I really don't see that there is anything else that Asterisk can do to help this situation. By: Olle Johansson (oej) 2008-03-17 05:07:51 The idea with a re-invite without SDP is to ask for the other side to reset whatever state they want. Since we never put SIP phones on remote hold, we simply say "take us off hold, please". By: Kevin P. Fleming (kpfleming) 2008-03-17 12:33:17 Well, that may be true, but with this user's CCM, whether we change state or not when responding to the INVITE-without-SDP he has a broken system. By: Ryan Ritchie (kreator) 2008-03-17 12:59:22 I am going to be capturing a debug between a Cisco 7940 running SIP load (non-CCM) 8.8 with the same call scenario to see what a Cisco phone does in this same scenario.  I have configured a SIP Trunk definition identical to the one that points to Asterisk but with a different dial-pattern pointed at the phone directly.  From Cisco's perspective this is just another Non-CCM SIP peer.  I'll upload the wireshark traces once I have them. By: Ryan Ritchie (kreator) 2008-03-20 13:24:41 Two updates: We now have a full 1.4 testbed in place with a transcoder card installed setup to mirror the configuration of our production servers.  We can test various patches now without the need to schedule after-hours activities or interrupt normal business. Also, after talking to oej, I've defined another SIP Trunk on the CCMs that points directly at an extension on a Cisco 7940 running the non-CCM 8.8 SIP load.  The results of that have been posted as "Cisco_7940_remoteagent_inbound.log". Exhibited behaviour was that the call rang through, we had 3 seconds of initial two-way silence, then both parties were able to simeltaneously hear each other. By: Leif Madsen (lmadsen) 2008-04-17 13:38:32 /housekeeping Any updates on this, or can we close this issue? By: Ryan Ritchie (kreator) 2008-04-17 13:43:39 Please keep the case open.  We have this escalated within Cisco and it looks so far like it may be a misconfiguration related to a setting called "Require MTP" that is defined on the SIP Trunk.  Right now we are awaiting instructions for configuring a software MTP within one of our voice gateways that will suffice for the requirement.  I would like to ensure that the resolution, if this works, reaches this case for the sake of others that may also hit this issue. Thanks, Ryan By: Leif Madsen (lmadsen) 2008-04-17 14:01:39 Setting this status to feedback since it doesn't appear to be an issue with Asterisk, but leaving it open per the reporter (kreator) so he can leave feedback once it is resolved on the Cisco side of things. By: Leif Madsen (lmadsen) 2008-06-09 13:42:20 Since this appears to be an issue with the way the CCM sends messages to Asterisk, I'm closing this. Feel free to have a bug marshall re-open the issue if need be.