[Home]

Summary:ASTERISK-16848: [patch] Malformed XML response
Reporter:Nivaldo Montenegro Jr (nivaldomjunior)Labels:
Date Opened:2010-10-22 07:53:02Date Closed:2011-03-10 10:09:12.000-0600
Priority:TrivialRegression?No
Status:Closed/CompleteComponents:Core/ManagerInterface
Versions:Frequency of
Occurrence
Related
Issues:
Environment:Attachments:( 0) 18186-httpheadernewline.diff
( 1) Asterisk_16.pcap
( 2) Asterisk_18.pcap
( 3) issue18186.patch
( 4) manager.diff.txt
( 5) res_phoneprov.c.diff
Description:Using the AJAM and XML to access the manager interface we receive a malformed XML response.
We tested using a java and a php application, and both had received the ajax response with error. The final tag </ajax-response> is not there.

We tested using the browser and the results are:

Chrome - Didn't worked. No </ajax-response> tag:
<ajax-response>
<response type='object' id='unknown'><generic response='Success' message='Authentication accepted' /></response>

Safari - Didn't worked. The </ajax-response> tag is there, but is not complete. The > is not there:
<ajax-response>
<response type='object' id='unknown'><generic response='Success' message='Authentication accepted' /></response>
</ajax-response

Firefox - Worked. The XML response is complete:

?
<ajax-response>
?
<response type="object" id="unknown">
<generic response="Success" message="Queue summary will follow"/>
</response>
?
<response type="object" id="unknown">
<generic event="QueueSummary" queue="Fila-Unimed" loggedin="0" available="0" callers="0" holdtime="0" talktime="0" longestholdtime="0"/>
</response>
?
<response type="object" id="unknown">
<generic event="QueueSummary" queue="fila-unimed-nne" loggedin="0" available="0" callers="0" holdtime="0" talktime="0" longestholdtime="0"/>
</response>
?
<response type="object" id="unknown">
<generic event="QueueSummaryComplete"/>
</response>
</ajax-response>



****** ADDITIONAL INFORMATION ******

The manager.conf:

[general]
enabled = yes
webenabled = yes
port=5038
bindaddr = 0.0.0.0
[admin]
secret = test
permit=0.0.0.0/0.0.0.0
read=system,call,log,verbose,command,agent,user,originate
write=system,call,log,verbose,command,agent,user,originate

The http.conf:
[general]
enabled=yes
bindaddr=0.0.0.0
bindport=8088
prefix=asterisk
enablestatic=yes

Comments:By: David Woolley (davidw) 2010-10-22 09:18:42

My guess would be that there is no final newline.  In that case, the bug will be in the clients.  You need to provide a wireshark, etc., trace showing the exact TCP payload, or look at the source code.
By: Nivaldo Montenegro Jr (nivaldomjunior) 2010-10-22 11:45:27

Ok. I will use the tcpdump to get the packages.
But it's very strange, because on asterisk 1.6 everything works fine. The java application, the php application, chrome, safari and firefox.
By: Nivaldo Montenegro Jr (nivaldomjunior) 2010-10-22 13:14:34

I colected the packages on 1.6 and on 1.8.

That's the result:

Asterisk 1.6: It sends the content type on the same package of the XML:

17:21:36.878366 IP (tos 0x0, ttl 64, id 15773, offset 0, flags [DF], proto TCP (6), length 187) voxeasy.omniorb > bd.62742: P, cksum 0x41b0 (correct), 1:136(135) ack 481 win 215 <nop,nop,timestamp 114468092 965905716>
0x0000:  0050 56c0 0008 000c 296e 7f8f 0800 4500  .PV.....)n....E.
0x0010:  00bb 3d9d 4000 4006 67f0 ac10 1e8e ac10  ..=.@.@.g.......
0x0020:  1e01 1f98 f516 71b4 9967 f171 493e 8018  ......q..g.qI>..
0x0030:  00d7 41b0 0000 0101 080a 06d2 a4fc 3992  ..A...........9.
0x0040:  8d34 4854 5450 2f31 2e31 2032 3030 204f  .4HTTP/1.1.200.O
0x0050:  4b0d 0a53 6572 7665 723a 2041 7374 6572  K..Server:.Aster
0x0060:  6973 6b2f 312e 362e 312e 3130 0d0a 4461  isk/1.6.1.10..Da
0x0070:  7465 3a20 5375 6e2c 2031 3020 4f63 7420  te:.Sun,.10.Oct.
0x0080:  3230 3130 2032 303a 3231 3a33 3620 4252  2010.20:21:36.BR
0x0090:  540d 0a43 6f6e 6e65 6374 696f 6e3a 2063  T..Connection:.c
0x00a0:  6c6f 7365 0d0a 4361 6368 652d 436f 6e74  lose..Cache-Cont
0x00b0:  726f 6c3a 206e 6f2d 6361 6368 652c 206e  rol:.no-cache,.n
0x00c0:  6f2d 7374 6f72 650d 0a                   o-store..
17:21:36.878597 IP (tos 0x0, ttl 64, id 49932, offset 0, flags [DF], proto TCP (6), length 52) bd.62742 > voxeasy.omniorb: ., cksum 0x1476 (correct), 481:481(0) ack 136 win 65535 <nop,nop,timestamp 965905716 114468092>
0x0000:  000c 296e 7f8f 0050 56c0 0008 0800 4500  ..)n...PV.....E.
0x0010:  0034 c30c 4000 4006 e307 ac10 1e01 ac10  .4..@.@.........
0x0020:  1e8e f516 1f98 f171 493e 71b4 99ee 8010  .......qI>q.....
0x0030:  ffff 1476 0000 0101 080a 3992 8d34 06d2  ...v......9..4..
0x0040:  a4fc                                     ..
17:21:36.878655 IP (tos 0x0, ttl 64, id 15774, offset 0, flags [DF], proto TCP (6), length 313) voxeasy.omniorb > bd.62742: P, cksum 0xef61 (correct), 136:397(261) ack 481 win 215 <nop,nop,timestamp 114468092 965905716>
0x0000:  0050 56c0 0008 000c 296e 7f8f 0800 4500  .PV.....)n....E.
0x0010:  0139 3d9e 4000 4006 6771 ac10 1e8e ac10  .9=.@.@.gq......
0x0020:  1e01 1f98 f516 71b4 99ee f171 493e 8018  ......q....qI>..
0x0030:  00d7 ef61 0000 0101 080a 06d2 a4fc 3992  ...a..........9.
0x0040:  8d34 436f 6e74 656e 742d 7479 7065 3a20  .4Content-type:.
0x0050:  7465 7874 2f78 6d6c 0d0a 4361 6368 652d  text/xml..Cache-
0x0060:  436f 6e74 726f 6c3a 206e 6f2d 6361 6368  Control:.no-cach
0x0070:  653b 0d0a 5365 742d 436f 6f6b 6965 3a20  e;..Set-Cookie:.
0x0080:  6d61 6e73 6573 7369 6f6e 5f69 643d 2236  mansession_id="6
0x0090:  3838 3137 6162 3422 3b20 5665 7273 696f  8817ab4";.Versio
0x00a0:  6e3d 2231 223b 204d 6178 2d41 6765 3d36  n="1";.Max-Age=6
0x00b0:  300d 0a0d 0a3c 616a 6178 2d72 6573 706f  0....<ajax-respo
0x00c0:  6e73 653e 0a3c 7265 7370 6f6e 7365 2074  nse>.<response.t
0x00d0:  7970 653d 276f 626a 6563 7427 2069 643d  ype='object'.id=
0x00e0:  2775 6e6b 6e6f 776e 273e 3c67 656e 6572  'unknown'><gener
0x00f0:  6963 2072 6573 706f 6e73 653d 2753 7563  ic.response='Suc
0x0100:  6365 7373 2720 6d65 7373 6167 653d 2741  cess'.message='A
0x0110:  7574 6865 6e74 6963 6174 696f 6e20 6163  uthentication.ac
0x0120:  6365 7074 6564 2720 2f3e 3c2f 7265 7370  cepted'./></resp
0x0130:  6f6e 7365 3e0a 3c2f 616a 6178 2d72 6573  onse>.</ajax-res
0x0140:  706f 6e73 653e 0a                        ponse>.


Asterisk 1.8: It sends the content type on a package before the XML package:

14:15:14.311599 IP (tos 0x0, ttl 64, id 3389, offset 0, flags [DF], proto TCP (6), length 344) maquinavirtual.local.omniorb > MacBook-Pro-de-Nivaldo-Junior.local.62717: P, cksum 0xf8db (correct), 1:293(292) ack 508 win 215 <nop,nop,timestamp 6130134 965903770>
0x0000:  0023 32bf 2f58 000c 2935 cf23 0800 4500  .#2./X..)5.#..E.
0x0010:  0158 0d3d 4000 4006 dee2 c0a8 e61e c0a8  .X.=@.@.........
0x0020:  e610 1f98 f4fd f94f 84ea c9eb 7ee3 8018  .......O....~...
0x0030:  00d7 f8db 0000 0101 080a 005d 89d6 3992  ...........]..9.
0x0040:  859a 4854 5450 2f31 2e31 2032 3030 204f  ..HTTP/1.1.200.O
0x0050:  4b0d 0a53 6572 7665 723a 2041 7374 6572  K..Server:.Aster
0x0060:  6973 6b2f 312e 382e 300d 0a44 6174 653a  isk/1.8.0..Date:
0x0070:  2046 7269 2c20 3232 204f 6374 2032 3031  .Fri,.22.Oct.201
0x0080:  3020 3137 3a31 353a 3134 2047 4d54 0d0a  0.17:15:14.GMT..
0x0090:  436f 6e6e 6563 7469 6f6e 3a20 636c 6f73  Connection:.clos
0x00a0:  650d 0a43 6163 6865 2d43 6f6e 7472 6f6c  e..Cache-Control
0x00b0:  3a20 6e6f 2d63 6163 6865 2c20 6e6f 2d73  :.no-cache,.no-s
0x00c0:  746f 7265 0d0a 436f 6e74 656e 742d 4c65  tore..Content-Le
0x00d0:  6e67 7468 3a20 3134 360d 0a43 6f6e 7465  ngth:.146..Conte
0x00e0:  6e74 2d74 7970 653a 2074 6578 742f 786d  nt-type:.text/xm
0x00f0:  6c0d 0a43 6163 6865 2d43 6f6e 7472 6f6c  l..Cache-Control
0x0100:  3a20 6e6f 2d63 6163 6865 3b0d 0a53 6574  :.no-cache;..Set
0x0110:  2d43 6f6f 6b69 653a 206d 616e 7365 7373  -Cookie:.mansess
0x0120:  696f 6e5f 6964 3d22 3531 3238 3034 3939  ion_id="51280499
0x0130:  223b 2056 6572 7369 6f6e 3d31 3b20 4d61  ";.Version=1;.Ma
0x0140:  782d 4167 653d 3630 0d0a 5072 6167 6d61  x-Age=60..Pragma
0x0150:  3a20 5375 7070 7265 7373 4576 656e 7473  :.SuppressEvents
0x0160:  0d0a 0d0a 0d0a                           ......
14:15:14.311744 IP (tos 0x0, ttl 64, id 3390, offset 0, flags [DF], proto TCP (6), length 198) maquinavirtual.local.omniorb > MacBook-Pro-de-Nivaldo-Junior.local.62717: FP, cksum 0xfb08 (correct), 293:439(146) ack 508 win 215 <nop,nop,timestamp 6130134 965903770>
0x0000:  0023 32bf 2f58 000c 2935 cf23 0800 4500  .#2./X..)5.#..E.
0x0010:  00c6 0d3e 4000 4006 df73 c0a8 e61e c0a8  ...>@.@..s......
0x0020:  e610 1f98 f4fd f94f 860e c9eb 7ee3 8019  .......O....~...
0x0030:  00d7 fb08 0000 0101 080a 005d 89d6 3992  ...........]..9.
0x0040:  859a 3c61 6a61 782d 7265 7370 6f6e 7365  ..<ajax-response
0x0050:  3e0a 3c72 6573 706f 6e73 6520 7479 7065  >.<response.type
0x0060:  3d27 6f62 6a65 6374 2720 6964 3d27 756e  ='object'.id='un
0x0070:  6b6e 6f77 6e27 3e3c 6765 6e65 7269 6320  known'><generic.
0x0080:  7265 7370 6f6e 7365 3d27 5375 6363 6573  response='Succes
0x0090:  7327 206d 6573 7361 6765 3d27 4175 7468  s'.message='Auth
0x00a0:  656e 7469 6361 7469 6f6e 2061 6363 6570  entication.accep
0x00b0:  7465 6427 202f 3e3c 2f72 6573 706f 6e73  ted'./></respons
0x00c0:  653e 0a3c 2f61 6a61 782d 7265 7370 6f6e  e>.</ajax-respon
0x00d0:  7365 3e0a                                se>.
By: Paul Belanger (pabelanger) 2010-10-22 15:14:23

Please attach the trace to the issue, not in your note.
By: Brandon Kruse (bkruse) 2010-10-27 11:33:23

Even in the wireshark you just pasted, the response is clear at the bottom: </ajax-response>

Please provide a wireshark dump (wireshark, then export), otherwise, we cannot reproduce the issue and will close this bug.

Thanks,

-bk
By: Nivaldo Montenegro Jr (nivaldomjunior) 2010-10-27 11:47:24

Hi! I was travelling... sorry for that...

I just sent two files, on both the response of the </ajax-response> come from asterisk. But i don't know why some clients does not understand the XML.
Clients with problem: Chrome, Safari, Java and PHP using SimpleXML.

On asterisk 16.pcap you can use the filter ip.addr == 192.168.230.2 (this is the IP address of the asterisk 1.6 server)

On asterisk 18.pcap you can use the filter ip.addr == 192.168.230.30 (this is the IP address of the asterisk 1.8 server)

Thanks,
By: David Woolley (davidw) 2010-10-27 12:12:06

The only obvious difference is that there is a bogus empty line at the start of the XML payload in 1.8.  I'm not sure if XML allows that or not.  I don't think the fact that the real start of payload is in a different fragment from the end of the header is relevant.

As an incidental observation, this is showing severe IP fragmentation because of a very short MTU (about 500 bytes) and no use of Path MTU discovery.  Efficient operation with this requires that you explicity set MTU in the routing tables, or on the interface.


By: David Woolley (davidw) 2010-10-27 12:55:46

The productions in the XML specification don't seem to allow white space before the prolog.
By: Nivaldo Montenegro Jr (nivaldomjunior) 2010-10-28 08:42:21

Yes. You're right, i just tested on the browser and the XML has an empty line before the XML. Probably this empty line is causing the problem.
By: Paul Belanger (pabelanger) 2010-11-04 18:09:49

A shot in the dark, based on comment here, mind testing the patch?  I don't guarantee a fix :)
By: Nivaldo Montenegro Jr (nivaldomjunior) 2010-11-05 11:08:34

I just tested the patch and it worked using this clients: Chrome, PHP and Safari.
The java client we are still testing.
I will add a new note after the test.
By: Walter Doekes (wdoekes) 2010-11-05 14:57:35

(That patch will break when http_header is non-empty.)
By: Jason Parker (jparker) 2011-01-04 16:29:49.000-0600

This patch should solve the issue a little better.  It removes the "extra" CRLF in ast_http_send, and forces all instances of http_header to end with a CRLF.  Then, two CRLFs will be sent at the end whether http_header is NULL or not.
By: Jason Parker (jparker) 2011-02-01 11:06:48.000-0600

Anybody able to provide testing results here, so this can be committed?
By: Nivaldo Montenegro Jr (nivaldomjunior) 2011-02-01 11:31:42.000-0600

I will test the new patch and report the result.
By: Digium Subversion (svnbot) 2011-03-01 16:25:46.000-0600

Repository: asterisk
Revision: 309204

U   branches/1.8/main/http.c

------------------------------------------------------------------------
r309204 | qwell | 2011-03-01 16:25:46 -0600 (Tue, 01 Mar 2011) | 7 lines

Fix consistency of CRLFs on HTTP headers that get sent out.

(closes issue ASTERISK-16848)
Reported by: nivaldomjunior
Patches:
     18186-httpheadernewline.diff uploaded by qwell (license 4)

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=309204
By: Digium Subversion (svnbot) 2011-03-01 16:26:39.000-0600

Repository: asterisk
Revision: 309209

_U  trunk/
U   trunk/main/http.c

------------------------------------------------------------------------
r309209 | qwell | 2011-03-01 16:26:39 -0600 (Tue, 01 Mar 2011) | 14 lines

Merged revisions 309204 via svnmerge from
https://origsvn.digium.com/svn/asterisk/branches/1.8

........
 r309204 | qwell | 2011-03-01 16:25:44 -0600 (Tue, 01 Mar 2011) | 7 lines
 
 Fix consistency of CRLFs on HTTP headers that get sent out.
 
 (closes issue ASTERISK-16848)
 Reported by: nivaldomjunior
 Patches:
       18186-httpheadernewline.diff uploaded by qwell (license 4)
........

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=309209
By: Andrew Latham (lathama) 2011-03-09 16:44:08.000-0600

This change breaks res_phoneprov.  I assume that res_phoneprov is doing some compensation for this.
By: Andrew Latham (lathama) 2011-03-10 06:48:54.000-0600

Patch uploaded, tested, Happy :)
By: Terry Wilson (twilson) 2011-03-10 10:00:23.000-0600

There is also a missing \r\n in manager.c. patch added. I'll go ahead and commit both. Thanks!
By: Andrew Latham (lathama) 2011-03-10 10:02:40.000-0600

Thanks, I was beginning to worry that others would start a pile of tickets...
By: Digium Subversion (svnbot) 2011-03-10 10:05:46.000-0600

Repository: asterisk
Revision: 310240

U   branches/1.8/main/manager.c
U   branches/1.8/res/res_phoneprov.c

------------------------------------------------------------------------
r310240 | twilson | 2011-03-10 10:05:46 -0600 (Thu, 10 Mar 2011) | 13 lines

Add \r\n to remaining http headers passed to ast_http_send

r309204 changed the behavior of ast_http_send. It now requires headers
to be passed with trailing \r\n. This change updates the remaining
instances in the code that did not pass the \r\n.

(closes issue ASTERISK-16848)
Reported by: nivaldomjunior
Patches:
     res_phoneprov.c.diff uploaded by lathama (license 1028)
     manager.diff.txt uploaded by twilson (license 396)
Tested by: lathama

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=310240
By: Digium Subversion (svnbot) 2011-03-10 10:09:11.000-0600

Repository: asterisk
Revision: 310241

_U  trunk/
U   trunk/main/manager.c
U   trunk/res/res_phoneprov.c

------------------------------------------------------------------------
r310241 | twilson | 2011-03-10 10:09:10 -0600 (Thu, 10 Mar 2011) | 20 lines

Merged revisions 310240 via svnmerge from
https://origsvn.digium.com/svn/asterisk/branches/1.8

........
 r310240 | twilson | 2011-03-10 10:05:45 -0600 (Thu, 10 Mar 2011) | 13 lines
 
 Add \r\n to remaining http headers passed to ast_http_send
 
 r309204 changed the behavior of ast_http_send. It now requires headers
 to be passed with trailing \r\n. This change updates the remaining
 instances in the code that did not pass the \r\n.
 
 (closes issue ASTERISK-16848)
 Reported by: nivaldomjunior
 Patches:
       res_phoneprov.c.diff uploaded by lathama (license 1028)
       manager.diff.txt uploaded by twilson (license 396)
 Tested by: lathama
........

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=310241