[Home]

Summary:ASTERISK-08045: [branch] chan_iax2.c:3782 iax2_trunk_queue: Maximum trunk data space exceeded
Reporter:Marco D'Ambrosio (marcodmb)Labels:
Date Opened:2006-11-02 04:16:49.000-0600Date Closed:2007-01-12 21:26:36.000-0600
Priority:MajorRegression?No
Status:Closed/CompleteComponents:Channels/chan_iax2
Versions:Frequency of
Occurrence
Related
Issues:
Environment:Attachments:( 0) 12iax.diff
Description:On mid-high load servers (>80 channels, 40 cuncurrent calls), when the CLI starts flooding messages, the other end will get timeout and the calls will get dropped. I've to stop and restart Asterisk manually.
(issue already reported but not fixed, http://bugs.digium.com/view.php?id=2442)

****** ADDITIONAL INFORMATION ******

I've tried to modify the MAX_TRUNKDATA variable in chan_iax2.c from 200 to 800 channels:
#define MAX_TRUNKDATA           640 * 800       /*!< 40ms, uncompressed linear * 800 channels */
/*#define MAX_TRUNKDATA         640 * 200*/     /*!< 40ms, uncompressed linear * 200 channels */
but the issue is still present.

Asterisk A (that crashes) is connected with Asterisk B and C via IAX2 trunk and "switch" statements to share the dialplan. Any suggestion? Dundi? or must I disable "trunking" on IAX2?

This is what I see in console:
(...)
Nov  2 10:28:40 WARNING[20655]: chan_iax2.c:3782 iax2_trunk_queue: Maximum trunk data space exceeded to 10.251.0.41:4569
Nov  2 10:28:40 WARNING[20525]: chan_iax2.c:3782 iax2_trunk_queue: Maximum trunk data space exceeded to 10.251.24.5:4569
Nov  2 10:28:40 WARNING[20052]: chan_iax2.c:3782 iax2_trunk_queue: Maximum trunk data space exceeded to 10.251.0.41:4569
Nov  2 10:28:40 WARNING[20418]: chan_iax2.c:3782 iax2_trunk_queue: Maximum trunk data space exceeded to 10.251.24.5:4569
Nov  2 10:28:40 WARNING[20655]: chan_iax2.c:3782 iax2_trunk_queue: Maximum trunk data space exceeded to 10.251.0.41:4569
Nov  2 10:28:40 WARNING[20525]: chan_iax2.c:3782 iax2_trunk_queue: Maximum trunk data space exceeded to 10.251.24.5:4569
Nov  2 10:28:40 WARNING[20052]: chan_iax2.c:3782 iax2_trunk_queue: Maximum trunk data space exceeded to 10.251.0.41:4569
Nov  2 10:28:40 WARNING[20418]: chan_iax2.c:3782 iax2_trunk_queue: Maximum trunk data space exceeded to 10.251.24.5:4569
Nov  2 10:28:40 WARNING[20655]: chan_iax2.c:3782 iax2_trunk_queue: Maximum trunk data space exceeded to 10.251.0.41:4569
Nov  2 10:28:40 WARNING[20525]: chan_iax2.c:3782 iax2_trunk_queue: Maximum trunk data space exceeded to 10.251.24.5:4569
Nov  2 10:28:40 WARNING[20052]: chan_iax2.c:3782 iax2_trunk_queue: Maximum trunk data space exceeded to 10.251.0.41:4569
(...)
Comments:By: Marco D'Ambrosio (marcodmb) 2006-11-02 10:43:11.000-0600

I've now set iaxcompat=yes in iax.conf... A multi-threading survey of dial-plan extensions could solve the issue.
Please note that we have more than 15000 calls/day.
By: Anthony LaMantia (alamantia) 2006-11-06 16:20:24.000-0600

not having to much experiance with the iax code base i can't say to much about but but.  after looking at this for a little bit

it looks like in send_trunk() we may me able to free up some space.. inside of

iax2_trunk_peer->trunkdataalloc

and avoid running into this error.. or maybe allow a configurable option to allow the asterisk user to have trunkdatalloc keep realloc()'ing ( in iax2_trunk_queue)rather then setting a hard-coded limit in size of chan_iax2.c
By: Marco D'Ambrosio (marcodmb) 2006-11-07 02:40:37.000-0600

Yes, Alamantia.
I think this is a good idea...

Meanwhile I'm testing our production server with iaxcompat=yes, and seems to work ok (about 30000 calls).

asterisk*CLI> show uptime
System uptime: 4 days, 23 hours, 4 minutes, 12 seconds
By: Anthony LaMantia (alamantia) 2006-11-10 15:55:54.000-0600

i have placed the modifications to chan_iax and the sample configuartion in a testing branch.

asterisk/team/anthonyl/8267/

all you would have to add to your iax.conf is
trunkrealloc=yes

which would allow you to bypass the hard limit and keep allocating memory in the queue.
By: Marco D'Ambrosio (marcodmb) 2006-11-10 18:37:39.000-0600

Ok, I'll try patching it.
However with iaxcompat=yes I've no more flooding messages (in /var/log/asterisk/messages) like this:
chan_iax2.c: Maximum trunk data space exceeded to (...)
---------------
MAX_TRUNKDATA is still set at 640x800
#define MAX_TRUNKDATA 640 * 800 /*!< 40ms, uncompressed linear * 800 channels
I would change soon this to default, to test if iaxcompat=yes really solved the issue:
#define MAX_TRUNKDATA 640 * 200 /*!< 40ms, uncompressed linear * 200 channels
Without iaxcompat and MAX_TRUNKDATA at 640 * 800 our server "flooded" again.

If the flooding messages appear, I'll try setting trunkrealloc=yes.
Is better to set iaxcompat or trunkrealloc to "yes"?

Thanks a lot for you precious support!
Marco


By: Anthony LaMantia (alamantia) 2006-11-20 13:18:04.000-0600

marcodmb, any updates on testing out the changes made in my branch?
By: Marco D'Ambrosio (marcodmb) 2006-11-20 14:14:35.000-0600

Our server are now up&running without any error :)
We could probably close this issue.

Thank you for your support!
Marco
By: Anthony LaMantia (alamantia) 2006-11-20 14:23:50.000-0600

let's keep this open until we can get this commited into asterisk.
By: Anthony LaMantia (alamantia) 2006-11-28 16:20:41.000-0600

at the moment i am making some changes to this branch, cli additions that report current trunk memory usage/stats as well as shrinking the amount of memory used by these trunks to prevent it from expanding and just eating up more memory then needed.
By: Jared Smith (jsmith) 2006-11-29 10:52:58.000-0600

You may want to check out oej's iaxtrunkfix branch at http://svn.digium.com/view/asterisk/team/oej/iaxtrunkfix-1.4/, which makes it so that when adding another call to the trunk would exceed the MTU, it simply starts another trunk...

... not sure if that's exactly what you're looking for, but it may help.
By: Marco D'Ambrosio (marcodmb) 2006-11-30 03:00:12.000-0600

In our production server (Asterisk 1.2.13), after two week, this morning is happened once again... :(
(...)
Nov 30 09:46:58 WARNING[20046] chan_iax2.c: Maximum trunk data space exceeded to 10.251.0.41:4569
Nov 30 09:46:58 WARNING[22906] chan_iax2.c: Maximum trunk data space exceeded to 10.251.0.41:4569
(...)

I'm testing your fixes in a backup server...
Marco
By: Marco D'Ambrosio (marcodmb) 2006-11-30 05:09:06.000-0600

For alamantia:
what is the patched revision, in your testing branch, of chan_iax2.c for my 1.2.13 production version?
http://svn.digium.com/view/asterisk/team/anthonyl/8267/channels/chan_iax2.c?rev=48091&view=log

I would like to rebuild only chan_iax2.c to test "trunkrealloc=yes".
Thank you
By: Anthony LaMantia (alamantia) 2006-11-30 15:54:39.000-0600

marcodmb, i will design a patch for 1.2 and upload it shortly for you.
the testing branch for this issue is off of the current trunk and contains the new configuration option as well as some cli commands that are in development for reporting current iax2 trunking stats.


By: Anthony LaMantia (alamantia) 2006-12-04 12:52:25.000-0600

i have uploaded the diff off 1.2svn for you. it's a simplified version of the current 8267 branch in my team directory.
By: Marco D'Ambrosio (marcodmb) 2006-12-04 13:34:09.000-0600

Thank you very much! I'll try this patch as soon as possible...

Marco
By: Anthony LaMantia (alamantia) 2006-12-06 16:03:08.000-0600

any updates on the performance of the patch?
By: Marco D'Ambrosio (marcodmb) 2006-12-07 06:48:16.000-0600

Seems that everything is ok, at the moment...
Bye Marco
By: Anthony LaMantia (alamantia) 2006-12-15 18:16:11.000-0600

is this still working good for you, or have you ran into any problems?
By: Marco D'Ambrosio (marcodmb) 2006-12-18 05:51:26.000-0600

I don't have any problem for now... thanks!
By: Anthony LaMantia (alamantia) 2007-01-12 17:43:47.000-0600

hey file :), sorry for assigning yet another bug to you.
but what do you think about putting a change such as this into trunk?
By: Joshua C. Colp (jcolp) 2007-01-12 21:26:35.000-0600

Put into trunk as of revision 50676. /me runs away