Summary:ASTERISK-10291: At high load audio drops following - [Sep 13 09:55:35] DEBUG[10689] channel.c: Didn't get a frame from channel: Zap/1-1
Reporter:Paul W (kwakwaversal)Labels:
Date Opened:2007-09-13 07:43:24Date Closed:2007-09-18 15:56:55
Versions:Frequency of
Environment:Attachments:( 0) asterisk.conf
( 1) err-mike-gaz-2007-09-12a-drop-zap.txt
( 2) err-paul-gaz-2007-09-12a-drop-iax.txt
( 3) iax.conf
( 4) zapata.conf
( 5) zap-iax-2007-09-12-drop-zap.txt
Description:Our setup is ZAP -> {IAX2} -> {IAX2} -> SIP (agents), where the IAX2 -> IAX2 connection is over a 10mb EES.

We recently moved over to asterisk in our call centre and under low calls everything runs smoothly, we only noticed problems with calls when it was an extremely busy part of the day (30-40 channels).  After moving through different asterisk versions from 1.4.6 to 1.4.11 and still having the same problems I took it off production and started delving a little deeper.

To get the strain of a production server I use sipp and send 45 calls over the IAX2 link `sipp -sn uac -d 300000 -s 2005 172.16.0.XX -l 45` (45 calls lasting 5 mins).  At the same time I send 7 calls through ZAP and over the IAX2 link. Each call is Answered() and listens to music on hold on the other end.

At this point, half the new calls which go through ZAP (both directions) result in immediate loss of audio or loss of audio part way through a call (I have attached debug dumps of the two kinds).

To make things more complicated, the audio seems to only be dropped one way, and that way is not consistent each time.  Most times it's the IAX2 link but I have logs of the ZAP channel being dropped.  I recorded some calls to check the audio is being dropped as the log suggests and it is.

N.B. I have a tendency to waffle and not describe things articulately so I apologise now if this information I've provided isn't accurate enough :)


Card: DIGTE205P

Asterisk is compiled from source with the following

$> Debian lenny/sid
$> Linux asterisk-isher 2.6.22-2-486 #1 Thu Aug 30 23:45:36 UTC 2007 i686 GNU/Linux

$> dpkg -l '*zap*'
ii  libzap-dev                 1:1.0.1-2                  Zapata telephony interface library (development)
ii  libzap1                    1:1.0.1-2                  Zapata telephony interface library (runtime)
ii  zaptel                     1:           zapata telephony utilities
un  zaptel-modules             <none>                     (no description available)
ii  zaptel-modules-2.6.18-4-48 1:1.2.11.dfsg-1+2.6.18.dfs zaptel modules for Linux (kernel 2.6.18-4-486).
ii  zaptel-modules-2.6.18-5-48 1:1.4.1~dfsg-3+2.6.18.dfsg zaptel modules for Linux (kernel 2.6.18-5-486).
ii  zaptel-modules-2.6.22-2-48 1:  zaptel modules for Linux (kernel 2.6.22-2-486).
ii  zaptel-source              1:           Zapata telephony interface (source code for kernel driver)
Comments:By: Paul W (kwakwaversal) 2007-09-16 17:13:33

My apologies, after looking into this problem more I believe I was mistaken, it turns out the problem is with IAX -> IAX.  I came to that conclusion as the ZAP -> {IAX} server has an IAX connection to two other asterisk boxes.  When I'm recreating a high load and make a call on the same network going through the 2nd * IAX link, the audio is fine.

I'm not sure the best way to debug this, if I turn debugging on the the destination * IAX box it crashes it.  With debugging diabled its fine (both boxes run 1.4.11)

After searching around I've found the issue is similar to bug 8325.  Should I reopen that issue with my problems or start a new one?

By: Paul W (kwakwaversal) 2007-09-18 13:18:07

Ok I think you can close this issue, seems like the problem is with IAX.  Over a certain amount of concurrent calls (between 30 and 40) the one way audio problem hits.  It's intermittent but it's easily reproducible when the IAX connection is saturated with concurrent calls.

There are no errors in * to signify the problem even at high debug levels so I'm unable to give a lot of information.  When IAX debug is on, as there are so many IAX frames being whizzed between the two * boxes I can't pinpoint the information which might be handy.

I'm quite sure there's nothing wrong with the 10mb EES as we've changed IAX to SIP and can send 75 calls over no problem now.  It's a shame because I really wanted to use IAX because of much less bandwidth overhead but IAX isn't stable enough for us.

By: Jason Parker (jparker) 2007-09-18 15:56:54

Closed per reporter.

I'd say to go ahead and open a new bug report detailing the information you have at this point, unless you can say for sure that the problem occurs under the same circumstances as ASTERISK-8098.