Summary:	ASTERISK-16772: Many "rtp.c: RTP Read too short" errors followed by memory exhaustions and crash
Reporter:	Mark Hampton (mhampton)	Labels:
Date Opened:	2010-10-05 18:48:58	Date Closed:	2011-06-07 14:01:02
Priority:	Critical	Regression?	No
Status:	Closed/Complete	Components:	Channels/chan_sip/General
Versions:		Frequency of Occurrence
Related Issues:
Environment:		Attachments:
Description:	Occasionally a channel will get numerous rtp.c: RTP Read too short errors with seemingly random numbers.  When this happens, I often get a memory exhaustion error followed by a crash.  This has been happening in both 1.6.2.13 and 1.6.2.14-rc1.  It has happened several times over the past week. ****** ADDITIONAL INFORMATION ****** [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (-213, expecting 196344) [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (42, expecting 72) [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (-213, expecting 224116) [Oct  5 18:17:09] NOTICE[26095] rtp.c: Unknown RTP codec 77 received from 'xx.xx.xx.xx' [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (-213, expecting 186928) [Oct  5 18:17:09] NOTICE[26095] rtp.c: Unknown RTP codec 61 received from 'xx.xx.xx.xx' [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (-213, expecting 14376) [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (-213, expecting 190024) [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (42, expecting 56) [Oct  5 18:17:09] NOTICE[26095] rtp.c: Unknown RTP codec 52 received from 'xx.xx.xx.xx' [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (42, expecting 64) [Oct  5 18:17:09] NOTICE[26095] rtp.c: Unknown RTP codec 2 received from 'xx.xx.xx.xx' [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (42, expecting 204592) [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (42, expecting 197408) [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (42, expecting 68) [Oct  5 18:17:09] NOTICE[26095] channel.c: Dropping incompatible voice frame on SIP/XXX-00003215 of format g723 since our native format has changed to 0x4 (ulaw) [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (42, expecting 64) [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (42, expecting 86368) [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (42, expecting 18556) [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (42, expecting 123408) [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (42, expecting 154240) [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (42, expecting 152528) [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (-213, expecting 134696) [Oct  5 18:17:09] NOTICE[26095] channel.c: Dropping incompatible voice frame on SIP/XXX-00003215 of format g723 since our native format has changed to 0x4 (ulaw) [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (42, expecting 113248) [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (42, expecting 70140) [Oct  5 18:17:09] WARNING[26095] rtp.c: RTP Read too short (42, expecting 148100) [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (-213, expecting 12308) [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (-213, expecting 32744) [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (42, expecting 60) [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (-213, expecting 224116) [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (42, expecting 86368) [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (-213, expecting 240552) [Oct  5 18:17:10] NOTICE[26095] rtp.c: Unknown RTP codec 54 received from 'xx.xx.xx.xx' [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (42, expecting 91704) [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (-213, expecting 204592) [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (42, expecting 68932) [Oct  5 18:17:10] NOTICE[26095] rtp.c: Unknown RTP codec 15 received from 'xx.xx.xx.xx' [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (42, expecting 204592) [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (-213, expecting 149452) [Oct  5 18:17:10] NOTICE[26095] rtp.c: Unknown RTP codec 117 received from 'xx.xx.xx.xx' [Oct  5 18:17:10] NOTICE[26095] rtp.c: Unknown RTP codec 25 received from 'xx.xx.xx.xx' [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (-213, expecting 30776) [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (42, expecting 60) [Oct  5 18:17:10] NOTICE[26095] rtp.c: Unknown RTP codec 29 received from 'xx.xx.xx.xx' [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (42, expecting 240552) [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (42, expecting 118052) [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (42, expecting 72) [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (42, expecting 64) [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (42, expecting 140868) [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (42, expecting 240552) [Oct  5 18:17:10] NOTICE[26095] rtp.c: Comfort noise support incomplete in Asterisk (RFC 3389). Please turn off on client if possible. Client IP: xx.xx.xx.xx [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (42, expecting 48) [Oct  5 18:17:10] NOTICE[26095] rtp.c: Unknown RTP codec 85 received from 'xx.xx.xx.xx' [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (42, expecting 22612) [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (42, expecting 16484) [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (42, expecting 260048) [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (-213, expecting 146012) [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (42, expecting 86368) [Oct  5 18:17:10] WARNING[26095] rtp.c: RTP Read too short (42, expecting 52) [Oct  5 18:17:10] ERROR[26095] /root/asterisk-1.6.2.14-rc1/include/asterisk/utils.h: Memory Allocation Failure in function ast_frdup at line 515 of frame.c
Comments:	By: Stefan Schmidt (schmidts) 2010-10-06 13:21:05 Could you please add the backtrace if you have a core dump and also debug logs if this is easy to reproduce for you. have a look at the doc/backtrace.txt file to see how to generate a backtrace we need and dont forget to set the DONT OPTIMIZE Flag and recompile asterisk before creating the backtrace. thank you By: Mark Hampton (mhampton) 2010-10-12 11:35:51 I'm sorry; my system is in production and this behavior only manifests under heavy call volume (ie > 250 calls in progress).  I didn't have it setup to dump a core file, and I'm reluctant to experiment with a production system any more than I already have. Yes I read the bug report guidelines, however I thought it best to report the bug with whatever limited information I had available so that at least you would be aware there's an issue.  I don't expect that you will be able to provide a fix for me based on the limited information I have provided. As a workaround, I modified rtp.c to drop calls when "RTP Read too short" is encountered.  This has resolved the crashing, but at the price of occasional dropped calls.  Better 1 - 3 dropped calls a day than 250 dropped calls when Asterisk runs out of memory and crashes. By: Mark Hampton (mhampton) 2010-10-18 15:41:39 Just an update:  I reverted back to 1.6.2.11 (without my hacks), and the crashes are gone. By: Leif Madsen (lmadsen) 2011-01-06 14:43:41.000-0600 As there is nothing we can do with this issue as it stands, I'm going to suspend it. Please reopen when you have the information required to move it forward.