[Home]

Summary:ASTERISK-01048: WARNING[3065886]: chan_iax2.c:2512 iax2_send: Out of trunk data space on call number 16389, dropping
Reporter:zoa (zoa)Labels:
Date Opened:2004-02-18 09:11:47.000-0600Date Closed:2004-09-25 02:49:39
Priority:MajorRegression?No
Status:Closed/CompleteComponents:Core/General
Versions:Frequency of
Occurrence
Related
Issues:
Environment:Attachments:
Description:There seems to be a little bug in the iax2 code, every now and then i get this error, and when this happens, all the calls on the trunk get interrupted for a little time, since this happens a lot, this is a big problem for me.

This happens quite a lot when i have a lot of trunking going on (> 15 calls)

Both servers are running latest cvs, (also happened with an old cvs).

I searched the mailinglists and only found something not applicable about an iax.conf setting.



****** ADDITIONAL INFORMATION ******

Feb 18 14:51:11 WARNING[3065886]: chan_iax2.c:2512 iax2_send: Out of trunk data space on call number 16389, dropping
Feb 18 14:51:11 WARNING[3032099]: chan_iax2.c:2512 iax2_send: Out of trunk data space on call number 16399, dropping
Feb 18 14:51:11 WARNING[3056662]: chan_iax2.c:2512 iax2_send: Out of trunk data space on call number 16416, dropping
Feb 18 14:51:11 WARNING[3018776]: chan_iax2.c:2512 iax2_send: Out of trunk data space on call number 16394, dropping
Feb 18 14:51:11 WARNING[3058726]: chan_iax2.c:2512 iax2_send: Out of trunk data space on call number 16420, dropping
Feb 18 14:51:11 WARNING[3060777]: chan_iax2.c:2512 iax2_send: Out of trunk data space on call number 16427, dropping
Feb 18 14:51:11 WARNING[3016732]: chan_iax2.c:2512 iax2_send: Out of trunk data space on call number 16422, dropping
Feb 18 14:51:11 WARNING[3048484]: chan_iax2.c:2512 iax2_send: Out of trunk data space on call number 16390, dropping
Feb 18 14:51:11 WARNING[2998309]: chan_iax2.c:2512 iax2_send: Out of trunk data space on call number 16407, dropping
Feb 18 14:51:11 WARNING[3057695]: chan_iax2.c:2512 iax2_send: Out of trunk data space on call number 16417, dropping
Feb 18 14:51:11 WARNING[3050515]: chan_iax2.c:2512 iax2_send: Out of trunk data space on call number 16396, dropping
Feb 18 14:51:11 WARNING[3047449]: chan_iax2.c:2512 iax2_send: Out of trunk data space on call number 16384, dropping
Feb 18 14:51:11 WARNING[3064875]: chan_iax2.c:2512 iax2_send: Out of trunk data space on call number 16439, dropping
Feb 18 14:51:11 WARNING[3061793]: chan_iax2.c:2512 iax2_send: Out of trunk data space on call number 16428, dropping
Feb 18 14:51:11 WARNING[3062826]: chan_iax2.c:2512 iax2_send: Out of trunk data space on call number 16430, dropping
Feb 18 14:51:11 WARNING[3059752]: chan_iax2.c:2512 iax2_send: Out of trunk data space on call number 16424, dropping
Comments:By: zoa (zoa) 2004-02-18 09:13:16.000-0600

oh i have a TE410p in both machines (in one of them i also have an x100p), they are 100ms away and have 5ms jitter on average.

both servers have an intel MB and are dual xeons.
By: zoa (zoa) 2004-02-18 09:17:07.000-0600

related posts to the mailinglist: (that were not applicable for me)

http://lists.digium.com/pipermail/asterisk-dev/2004-January/002952.html
http://lists.digium.com/pipermail/asterisk-dev/2004-January/002951.html
By: Brian West (bkw918) 2004-02-18 12:05:41.000-0600

This happens when you have one end configured for trunking and the other isn't.  Can you post both ends of the iax.conf and the extensions.conf that does the dial for both ends?  We need to make sure its two way.
By: zoa (zoa) 2004-02-18 12:09:14.000-0600

[xxx]
type=friend
secret=xxx
host=xxx
disallow=all
allow=ilbc
context=xxx
;auth=plaintext,rsa,md5
qualify=yes
trunk=yes
;port=5036
;sendani=no
accountcode=xxx

on the other machine
[xxx]
type=friend
secret=xxx
host=xxx
disallow=all
allow=ilbc
context=xxx
trunk=yes
qualify=yes


iax2 trunk debug on both servers reveals that i have trunking in both directions.

The trunking works sweet but every now and then it drops the audio for a little while.
By: zoa (zoa) 2004-02-18 12:32:02.000-0600

as bkw suggested, i now removed all possible dial lines that did not use the trunking layout. lets see what it gets me.

I don't know if this should be considered a bug, it isnt very easy behaviour :)

Btw: if i want to use 2 different extensions on the destination server, would i need to make two trunks then ?
By: zoa (zoa) 2004-02-18 12:41:56.000-0600

Feb 18 19:24:56 WARNING[10251]: chan_iax2.c:4065 socket_read: midget packet received (0 of 4 min)

any idea what could cause this ? maybe a registration packet (its in todays cvs and was not in 10 januari's cvs)

comes up every few minutes.
By: zoa (zoa) 2004-02-18 15:11:32.000-0600

I tried with 45 people, and got that bug at least once a minute, when it happens, both trunked calls and non-trunked calls have interuptions in their conversations. On that amount of calls, it seems its quite impossible to have a good conversation.
By: Mark Spencer (markster) 2004-02-18 17:14:21.000-0600

What codec are you running?
By: zoa (zoa) 2004-02-19 02:19:32.000-0600

i'm on 100% ilbc.

When doing around 45 simultaneous calls yesterday, there was no other traffic on any of the 2 machines.

xlite (ilbc) -> B -> IAX2 (ilbc) ->  A -> CARRIER.

(with the both trunked as nontrunked calls being interrupted, i meant: when the test was going bad a collegue called me by iax2 (not trunked) from a server C to server A to the carrier to my mobile phone. And it was impossible to have a normal conversation.)

the destination A server is the one showing all the trunking errors.
By: zoa (zoa) 2004-02-19 04:38:46.000-0600

My collegue has the impression that most iax2 warnings, cause audio drops on all the lines. (eg this error: received mini frame before first full voice frame... chan_iax2.c:4930)
By: zoa (zoa) 2004-02-19 04:41:33.000-0600

(last error is something i also get a lot, current setup: trunk=no on both ends)
By: zoa (zoa) 2004-02-21 04:10:00.000-0600

anthm also seems to have this problem, tclark doesnt, both do a large amount of trunking.

I'm using ilbc, anthm is using gsm only.
By: Brian West (bkw918) 2004-02-21 20:46:37.000-0600

I don't recall the guys name but he was in #asterisk a few weeks back and said it was related to one end not trunking.  I wonder if one end has just gone dumb and decided not to trunk thus causing this?!?!?!
By: zoa (zoa) 2004-02-25 04:17:39.000-0600

I disabled hyperthreading on the receiving machine as recommended by someone in #asterisk, problem continues.

Any suggestions on how to debug this ?
It only happens frequently on heavy load (+20 calls) which makes it a lot more difficult to debug.

Meanwhile i also tried simulating a lot of calls coming from the other server and sending audio to that other iax2 server. (but not sending audio as i don't know how to simulate that).

I get no trunking errors then.

I tried using GSM codec instead of iLBC, but my server doesnt seem to like GSM.

Any suggestions from someone with good C knowledge on what might cause this problem ?

I'm waiting to get another public ip, once i get it (hopefully today) i'll try with another termination server.
By: zoa (zoa) 2004-02-25 09:07:02.000-0600

k, i got the other server online.

an running with +/- 15 calls in the trunk for 20 minutes now.
no out of frame data seen yet.
By: zoa (zoa) 2004-02-27 04:34:06.000-0600

still no out of frame data noticed. (after several days).

<kram> maybe we're running out of CPU
<kram> probably just needs to be updated

-> new server is a slower dual opteron 1,4ghz.
previous server was a dual 3.06ghz xeon.
By: zoa (zoa) 2004-03-14 05:37:10.000-0600

citats & bonbon, since you both were able to reproduce this, could you add more information on your findings ?
By: tan (tan) 2004-03-18 06:09:16.000-0600

We have noticed that CPU is constantly at 99.7% although we only have around 5 calls connected.

I can't reproduce exactly when we get the out of trunk space errors.
By: revk (revk) 2004-03-18 07:49:38.000-0600

I have this as well. I have a trunked link to another system. I also make calls using a guest@ non trucked connection using an ENUM type look up. If I call one of our numbers it resolves in ENUM to the other system and makes a guest non trunked call to it. The other system knows the number are here and calls me via iax2 trunked, and when I answer it does a native brigded link from this system to itself. I had assumed that this was the problem, but from the comments above it suggests the problem is that the outgoing call is not trunked and the incoming one is, and the native bridge means we are talking to ourselves with one end trunked and the other end non trunked. I get about a second of audio and the call drops with that message. Maybe it should not attempt iax-iax native bridging if one link is trunked and the other is not - perhaps this needs fixing. Obviously I can fix my dialling plan to remove the loop, but that is not the point.
By: Brian West (bkw918) 2004-04-20 18:48:03

I wanna get a closer look at this one anyone have a pair of asterisk server we can have attach each other?  and watch...?
By: Mark Spencer (markster) 2004-04-20 18:52:28

bonbon only reproduces this on reload, are you having the same symptom, zoa?
By: zoa (zoa) 2004-04-21 03:28:24

no i had this all the time, i'm not seeing out of trunk data at the moment, but i'm not using that much trunking atm and also am using different codecs & different servers and different kernels for these connections.

I'm extremely busy atm, thats why i'm not a lot on irc and don't do a lot on mantis atm.

When i crawl out of this bulgarian mess i'll be happy to test some more.
(Unless other major disasters happen here, an earthquake or so, i think i'll be ready for more tests on saturday.)
By: tan (tan) 2004-04-21 04:34:30

Bear in mind that we have 5130 iax.conf entries. When this number is reduced to 130 then we don't see any trunk usage errors. If any useful information will come out of it, I can reduce the number of iax.conf entries in decrements of 1000 until we stop seeing trunk error messages.
By: tan (tan) 2004-04-21 12:57:54

We're running mainly ilbc and gsm codecs. Our iax.conf is as follows:

disallow=all
allow=ilbc
allow=gsm
allow=ulaw
allow=alaw
allow=g729

We can generate the error with just a single call up i.e. 2 channels.
By: Mark Spencer (markster) 2004-04-27 11:15:29

I've put some serious enhancments to IAX2 trunking in CVS (head only, of course).  These should totally eliminate this message and cause no loss of audio even on a reload.  Please feel free to test.
By: Mark Spencer (markster) 2004-04-27 17:00:29

Any feedback here?
By: Mark Spencer (markster) 2004-04-28 00:37:12

Fixed in CVS