Summary:	ASTERISK-10781: Asterisk doesn't reply well to in-dialog OPTIONS in pedantic mode
Reporter:	Iñaki Baz Castillo (ibc)	Labels:
Date Opened:	2007-11-15 14:04:13.000-0600	Date Closed:	2008-07-01 06:44:40
Priority:	Minor	Regression?	No
Status:	Closed/Complete	Components:	Channels/chan_sip/Interoperability
Versions:		Frequency of Occurrence
Related Issues:
Environment:		Attachments:	( 0) debug-Asterisk-in-dialog-OPTIONS-404.txt ( 1) debug-OPTIONS-indialog-404-pedantic-no.txt ( 2) debug-OPTIONS-indialog-404-pedantic-yes.txt ( 3) debug-OPTIONS-wrong-tags-indialog-481-pedantic-yes.txt
Description:	A way to monitorize a SIP dialog is by sending in-dialog OPTIONS so the UAC/UAS replies with 200 OK if that dialog exists or 404 if not. Asterisk with pedantic=no replies a "404 Not Found" correctly but in pedantic=yes it replies with a "481 Call/Transaction Does Not Exist". I'm not 100% sure (I'll try to confirm it) but I think it should reply with a 404 instead of 481 in this case (OPTIONS in-dialog).
Comments:	By: Iñaki Baz Castillo (ibc) 2007-11-15 15:41:19.000-0600 Ok, I'm sorry, Asterisk does it well. I've tested other SIP UAC/UAS (as Twinkle, very RFC compliant) and it replies with a 481 as well if the dialog doesn't exist. So I'm sorry and I ask someone to close this report as INVALID. By: Mark Michelson (mmichelson) 2007-11-15 18:29:59.000-0600 Closing at reporter's request. By: Iñaki Baz Castillo (ibc) 2007-11-17 13:47:44.000-0600 Sorry, I reopen this bug because now I've confirmed it exists: As I said, RFC 3261 says that a way to monitorize a SIP dialog is by sending in-dialog OPTIONS so the UAC/UAS replies with 200 OK if that dialog exists or 404/481 if not. With Twinkle I can send an in-dialog OPTIONS (menu "Call" - "Terminal capabilities" during a call), so it lets me testing it. Two cases: 1) The phone sends the INVITE to Asterisk: If Twinkle calls to 500@asterisk (and that extension exists) the Asterisk replies a 200 OK with "Contact: sip:500@IP_Asterisk". So if Twinkle sends now and in-dialog OPTIONS it's sent to "sip:500@IP_Asterisk" and Asterisk replies with a "200 OK": If during same dialog I send the same in-dialog OPTIONS but changing the Call-ID (or From/To tag in pedantic mode) the Asterisk replies a 404 (481 in pedantic mode), so it's OK, it works :) 2) Asterisk sends the INVITE to Twinkle: In this case Asterisk sends an INVITE with "Contact: sip:asterisk@IP_Asterisk". Of course, "asterisk" is not a valid extension when calling to Asterisk. So Twinkle accepts the call and sends an in-dialog OPTIONS:  OPTIONS sip:asterisk@IP_Asterisk SIP/2.0  Call-ID: "the correct value"  From: ...  To: ...  ... so because "asterisk" is not a valid extension it replies with a 404 :( But in fact it should reply with a 200 OK since the session exists, and since this is an in-dialog request it should match the Call-ID (and From/To tags if pedantic mode) against the active sessions and reply with 200 OK if the dialog exists. Nortel and Cisco gateways use those in-dialog OPTIONS to monitorize the status of a dialog, so if Asterisk generates the INVITE but replies with 404 when receiving an in-dialog OPTIONS then the gateway would end the call. By: Iñaki Baz Castillo (ibc) 2007-11-17 14:04:13.000-0600 So, Asterisk doesn't reply well to in-dialog OPTIONS in pedanticmode = yes or no. in-dialog OPTIONS is just replied with a 200 OK from Asterisk if Asterisk is the called. By: Olle Johansson (oej) 2007-11-18 11:47:36.000-0600 As always, you need to include a full SIP debug output (see the bug guidelines). Thanks. By: Iñaki Baz Castillo (ibc) 2007-11-18 18:49:02.000-0600 Ok oej, you are right. I've attached an Asterisk SIP debug now. It is a call from Asterisk to Twinkle (200) and during the call Twinkle sends an in-dialog OPTIONS, but Asterisk replies 404. I've added some comments in the trace to explain what occurs. Hope it helps. If not please ask me whatever you need. Regards. By: Olle Johansson (oej) 2007-11-19 01:23:19.000-0600 Thanks. However, you only have the debug output here, I need it all - verbose and the actual SIP messages too. Set verbose and debug to 5, turn sip debug on and capture again. Sorry for the extra work. I think the issue here is not the 404, because you propably don't have an extension called "asterisk", it's that it causes the call to hang up on a non-fatal in-dialog transaction. We are always requested to answer to OPTIONs as if it was an INVITE, regardless if it's in-dialog or not. I need to see more output though, before I start working on a patch and try to fix this. Thanks for your patience. By: Olle Johansson (oej) 2007-11-19 01:39:37.000-0600 When testing again, please make sure you have the latest 1.4 code from subversion as well as testing with both pedantic modes. thank you. By: Iñaki Baz Castillo (ibc) 2007-11-19 04:16:35.000-0600 > I think the issue here is not the 404, because you propably don't have > an extension called "asterisk" No, I haven't it. > it's that it causes the call to hang up on a non-fatal in-dialog > transaction. Not sure if I understand what you mean, but the call is not hanged up when I send an in-dialog OPTIONS to "asterisk@IP" (a non existing extension). About the debug, now I'll  capture it in the "full" log message:  full => notice,warning,error,debug,verbose Anyway let me some time to install the SVN version and try it since I was using 1.4.13. Really thanks for all your patience ;) By: Iñaki Baz Castillo (ibc) 2007-11-19 09:08:14.000-0600 Hope those new 2 attached files can help. They are the same as before: - Asterisk (fromuser=asterisk) calls to 200 (Twinkle). - Twinkle sends in-dialog OPTIONS to sip:asterisk@IP - Asterisk replies 404 because that extension doesn't exist, but it should in fact match the dialog, not the extension. By: Olle Johansson (oej) 2007-11-19 09:16:50.000-0600 If the extension doesn't exist, we should always answer 404... By: Olle Johansson (oej) 2007-11-19 09:18:07.000-0600 This is a bug: After anwering the OPTIONS request: [Nov 19 16:00:53] DEBUG[10056] chan_sip.c: SIP message could not be handled, bad request: 63bbf9cb58098fcb796d4e2e74048e17@80.95.0.111 Then Asterisk sends a BYE. Bad. By: Olle Johansson (oej) 2007-11-19 09:18:51.000-0600 Sorry, we do not send a bye, still that's something reported as an error. By: Olle Johansson (oej) 2007-11-19 09:20:07.000-0600 I don't see any difference in the way OPTIONs are handled in these two files. By: Iñaki Baz Castillo (ibc) 2007-11-19 09:27:23.000-0600 Why? it's perfectly possible that an Asterisk user A can call to an Asterisk user B, but B can't call to A: sip.conf:  [user_a]  context = context_a  [user_b]  context = context_b extensions.conf:  [context_a]  exten => user_b,1,Dial(SIP/user_b)  [context_b]  ;can't call to user_a So in this case, imagine user_a calls to user_b. The "Contact" header in the INVITE says "<sip:user_a@IP_Asterisk>". The call is established and user_b (that could be a gateway) sends periodically in-dialog OPTIONS to user_a@IP_Asterisk in order to test if the dialog remains open for a correct CDR. But since user_b can't call to user_a then it will get a 404. This is a wrong reply since the dialog does exist. IMHO: Asterisk shouldn't match the RURI username for in-dialog OPTIONS, just the Call-ID (and the From/To tags if pedantic=yes), and reply 200 OK if the dialog exists, and 404/481 if not. This behaviour of in-dialog OPTION is not very cleared explaind in RFC3261 (but it's in fact), and many vendors implement it (Nortel, Cisco). By: Olle Johansson (oej) 2007-11-19 09:29:50.000-0600 You report an issue with the pedantic setting here. I don't see that in the log files you provide. I don't agree with reporting anything else than what we would respond to an INVITE (or in-dialog re-INVITE). That would be to go way outside the specs. By: Iñaki Baz Castillo (ibc) 2007-11-19 09:34:15.000-0600 Ops, how many commnets XD > Then Asterisk sends a BYE. Bad. The BYE is sent by Twinkle (just because I press hang-up in Twinkle XD). > I don't see any difference in the way OPTIONs are handled in these two > files. Maybe, I assume that the difference could just occur if Asterisk accept the message (now it rejects it because not valid extension), so it would check the From/To tags (if pedantic=yes). Anyway my report is that Asterisk has a bad behaviour for in-dialog OPTIONS even if pedantic = yes or no. By: Iñaki Baz Castillo (ibc) 2007-11-19 09:37:36.000-0600 > I don't agree with reporting anything else than what we would respond > to an INVITE (or in-dialog re-INVITE). That would be to go way outside > the specs. Which specs do you mean? SIP RFC's specs or Asterisk SIP specs? in-dialog OPTION does exist in SIP world with the aim I describe here. Regards. By: Olle Johansson (oej) 2007-11-19 09:47:38.000-0600 Show me an IETF spec that says that we should answer on dialog status and nothing else if the OPTION is sent in-dialog. From RFC3261: 11.2 Processing of OPTIONS Request The response to an OPTIONS is constructed using the standard rules for a SIP response as discussed in Section 8.2.6. The response code chosen MUST be the same that would have been chosen had the request been an INVITE. That is, a 200 (OK) would be returned if the UAS is ready to accept a call, a 486 (Busy Here) would be returned if the UAS is busy, etc. This allows an OPTIONS request to be used to determine the basic state of a UAS, which can be an indication of whether the UAS will accept an INVITE request. An OPTIONS request received within a dialog generates a 200 (OK) response that is identical to one constructed outside a dialog and does not have any impact on the dialog. -------------- This says very clearly that OPTIONS outside and inside of a dialog should be treated exactly the same way. By: Iñaki Baz Castillo (ibc) 2007-11-19 09:57:16.000-0600 ok, let me some time to find it. Regards. By: Iñaki Baz Castillo (ibc) 2007-11-19 14:43:09.000-0600 oej, Asterisk behaviour for OPTIONS is not at described in section 11.2 of RFC 3261 as you suggest. In that section it said that an in-dialog OPTIONS should be considered as an initial request OPTIONS, so just reply with 200 OK if the extension/username called exists and accepts calls. Ok, this explaines why if I send an in-dialog OPTIONS to asterisk@IP I receive a 404 (in case "asterisk" is not a valid extension for me). But now note the new attached file "debug-OPTIONS-wrong-tags-indialog-481-pedantic-yes.txt" in which Twinkle (200) calls to Asterisk (51): - During the call I send an in-dialog OPTION with a Ruby code I've done (respecting RURI, Call-ID and From/To tags of the extablished dialog) and Asterisk replies me 200 OK. - But later during the same call I change the Call-ID (or From/To tags) in my Ruby program and send an in-dialog OPTIONS. In this case Asterisk replies me with 481 (in pedantic=yes) or 404 (in pedantic=no). Why?? it makes no sense. If Asterisk's behaviour is as in 11.2 section it should reply 200 OK because the extension of the RURI in the in-dialog OPTIONS exists, regardless of the values of Call-ID and From/To tags. So we can expect a totally 11.2 compliant behaviour (alway reply 200 OK) or a "maybe not so RFC specs compliant" behavior and allowing in-dialog OPTIONS to monitorize dialog status by repling 200 OK (if that dialog exists) or 481 (if not). The last behaviour is used by Cisco gateways at least (and I think by Nortel too). By: Olle Johansson (oej) 2007-11-19 14:46:52.000-0600 "- But later during the same call I change the Call-ID (or From/To tags) in my Ruby program and send an in-dialog OPTIONS. In this case Asterisk replies me with 481 (in pedantic=yes) or 404 (in pedantic=no)." This is confusing. If you change caller ID or from/to tags, this is a totally different dialog and we should answer 481 in pedantic yes and 404 if pedantic no. That's absolutely correct behaviour. By: Iñaki Baz Castillo (ibc) 2007-11-19 16:11:05.000-0600 So in fact if the OPTIONS is in-dialog it's not treated as an initial request since the dialog parameters are matched. But *just* if the "Contact" header is a valid extension for the UAC sending the OPTIONS. IMHO this is not a solid behaviour, it seems a mix between what 11.2 section says and what devices monitorizing dialogs using OPTIONS do. I tell again the case where A calls B but B is not able to call A. In this case only A can send in-dialog OPTIONS to B (if B sends it it will get a 404). So, in order to get a 200 OK for an in-dialog OPTIONS it must occurs: - The UAC sending the OPTIONS can call to the username in received "Contact" header. - The dialog parameters match an existing dialog. Anyway, if definitively you consider this as the expected behaviour I just can **wish** Asterisk to support in-dialog OPTIONS (matching parameters) even if the caller can't call to he other. By: Jesus Rodriguez (jesusr) 2007-11-19 17:20:05.000-0600 Dear Olle, I agree with Iñaki that Asterisk behavior regarding OPTIONS in-dialog is wrong. Following your arguments, point 11.2 of the rfc3261: "The response code chosen MUST be the same that would have been chosen had the request been an INVITE". Keeping in mind that you are sending an OPTIONS with a To tag, you must reply with a 200OK if the dialog exists or with a "481 Call/Transaction Does Not Exist" as you would do for a (re-)INVITE with a To tag that does (not) exist in the UAS. Also, 12.2.2 UAS Behavior (requests within a dialog) states: If the UAS wishes to reject the request because it does not wish to recreate the dialog, it MUST respond to the request with a 481 (Call/Transaction Does Not Exist) status code and pass that to the server transaction. OPTIONS in-dialog is becoming a very common keepalive mechanism used by VoIP providers and vendors to check that calls (dialogs) are alive both in users and gateways/softswitches and avoid, specially, billing problems and would be great if Asterisk has the right behavior. Regards. By: Iñaki Baz Castillo (ibc) 2007-11-19 17:45:31.000-0600 Another argument: Again the case where A can call B but B can't call A: - A calls B. - During the dialog B **can** put A on-hold with a re-INVITE (so the extension is not matched for re-INVITE). - But during same dialog B **can't** send an in-dialog OPTIONS to A (the extension is matched so gets 404). This is not consistent, B can send an in-dialog INVITE (and extension is not matched) but not an in-dialog OPTIONS (because the extension is invalid for B). IMHO not sense. Asterisk in-dialog OPTIONS has not similar behaviour as in-dialog INVITE. By: Olle Johansson (oej) 2007-11-21 02:37:35.000-0600 The big issue here, and cause of this issue, is that you are sending out a call without a proper caller ID. If you set the caller ID number to something that exists in the dialplan and can be reached, asterisk would answer 200 OK on the options. There is another big issue in Asterisk with OPTIONs handling though, which is that since we don't authenticate, we might check the wrong context and end up with the wrong answer. Adding authentication to OPTIONs seems like a lot of extra processing in many cases where it's only used as a keep-alive and the answer is ignored... So we have three cases: 1. OPTIONs out of dialog for keepalives ("ping") 2. OPTIONs out of dialog for callee capability checking/extension checking 3. OPTIONs inside a dialog Seems to me like we handle 2 and 3 the wrong way. I'm still not very convinced on the in-dialog always answering 200 OK, so that's something we need to check with other implementations. In this case the OPTION is using a request uri that doesn't exist in the dialplan, so an INVITE formed like this would fail too. Maybe not a re-invite where you mostly ignore the request URI, which might be the point here. By: Jason Parker (jparker) 2008-04-14 16:35:43 oej, what do we want to do here then? By: Iñaki Baz Castillo (ibc) 2008-05-15 09:08:02 More info: There is a RFC handling those issues:  RFC 5057 - Multiple Dialog Usages in the Session Initiation Protocol Anyway it's not clear after reading it, but I've got a nice explanation in sip-implementators maillist (by Paul Kyzivat from Cisco): ---------------------- Now that I look at it, section 5.3 of 5057 may be less than crystal clear on this point. If the OPTIONS is sent within a dialog, AFAIK it will indeed monitor dialog state to the extent that it will fail with a 481 if there is no dialog matching its from- and to-tags and Call-ID. However it will not provide any information on the status of any particular usage within that dialog. *If* it is known that the dialog only had a single usage (the normal case) then it provides *some* evidence of the continued existence of that usage. But it is in general impossible to know there was a single usage. For instance, a REFER may have been sent within the dialog, resulting in a refer subscription usage sharing the dialog. Then the INVITE-usage could have gone away leaving only the refer subscription within the dialog. An OPTIONS at that time would still conclude that the dialog existed even though the INVITE did not. This would typically not be a problem since the INVITE, REFER, and OPTIONS are typically all sent by the same entity, but that depends upon application design. ----------------- By: Olle Johansson (oej) 2008-05-18 03:37:19 This is a feature request as well as a bug report, so we need to keep this open By: Olle Johansson (oej) 2008-05-18 03:38:55 A side note: Over wine and cheese in Barcelona, I got convinced that we need to support in-dialog OPTIONs properly... By: Iñaki Baz Castillo (ibc) 2008-05-19 02:09:40 Yeah, the wine in-dialog is always nice :) By: Digium Subversion (svnbot) 2008-07-01 06:44:25 Repository: asterisk Revision: 126789 U   branches/1.4/channels/chan_sip.c ------------------------------------------------------------------------ r126789 | oej | 2008-07-01 06:44:21 -0500 (Tue, 01 Jul 2008) | 6 lines Report 200 OK to all in-dialog OPTIONs requests (to confirm that the dialog exist). Don't bother checking the request URI. (closes issue ASTERISK-10781) Reported by: ibc ------------------------------------------------------------------------ http://svn.digium.com/view/asterisk?view=rev&revision=126789