Summary:ASTERISK-00598: IAX phones and CPU usage problem
Reporter:klictel (klictel)Labels:
Date Opened:2003-11-30 11:45:52.000-0600Date Closed:2011-06-07 14:05:20
Versions:Frequency of
Description:Hi all,

We have an asterisk server running on redhat 9, with a TE410P card connected to 4 PRIs, on a dual 2.6GHz xeon server with 2GB of RAM. We also have 10-12 agents using IAX based softphones on winxp. Almost all of the usage consists of outbound calls done automatically through the PRIs and connected  to an agent once answered. The agents perform the 'ready' operation via IAX by calling iaxc_call() to call an extension that in turn calls AgentLogin. The 'Not ready' operation is done automatically once the phone call ends by invoking iaxc_dump_call(). This ready/not-ready sequence is made repeatedly, for every call connected to an agent.

We noticed that the CPU usage of Asterisk if increasing with time. When we start Asterisk, everything seems ok, but with time the CPU usage goes higher and higher. After a day of work or so, the 'top' command shows that Asterisk is consuming around 90-95% of the CPU, some times 100%, and the voice on the softphones suffers dramatically.

When all the agents become Not Ready by doing an iaxc_dump_call(), the CPU goes down to 1-2%. As soone as one softphone does a iaxc_call() the CPU jumps up again. This situation is very serious to us especially since we are planning to deploy over 50 agents on this system.

Any ideas, hints, recommendations are greatly appreciated.


Comments:By: Brian West (bkw918) 2003-11-30 12:33:36.000-0600

dont use RH9... it has issues.

By: klictel (klictel) 2003-12-01 08:59:53.000-0600

Thanks for the recommendation. Are those issues with Asterisk documented somewhere? Are they regarding the POSIX threading?

By: Brian West (bkw918) 2003-12-02 10:09:10.000-0600

Not really sure.  I do know that alot of people have issues with the broken redhat 9 distro.

By: Brian West (bkw918) 2003-12-06 11:11:26.000-0600

export LD_ASSUME_KERNEL=2.4.1

By: Brian West (bkw918) 2003-12-10 17:50:58.000-0600

Any input on this

By: klictel (klictel) 2003-12-11 11:57:53.000-0600

I have already tried setting LD_ASSUME_KERNEL and didn't fix anything. What it did is change the usage of the CPU. Instead of having the CPU usage go up and up without much fluctuation, it started showing peaks then droping, but always going higher and higher until it suffocates.
Now the problem is solved.
We did a few changes. We updated the SMP kernel, updated zaptel, made sure the program irqbalance is running, migrated the softphones from IAX to IAX2 and fixed a bug in the threads creation in one of our custom applications (the treads were not marked as detach). When we were having the problem, I noticed a certain thread consuming more and more CPU cycles. Its CPU usage was growing with the CPU usage that the asterisk process was showing. Several times I used 'gdb' to break into that thread and it was always in chan_iax.c. This motivated me to move to IAX2.

The guys from Digium support, especially Mike, were of a great help.
I would consider this problem closed.