[Home]

Summary:ASTERISK-09545: Out of idle IAX2 threads for I/O, pausing
Reporter:Arseny Chernov (ars888)Labels:
Date Opened:2007-05-30 10:38:10Date Closed:2007-06-04 18:39:36
Priority:CriticalRegression?No
Status:Closed/CompleteComponents:Channels/chan_iax2
Versions:Frequency of
Occurrence
Related
Issues:
Environment:Attachments:
Description:Dear colleagues,

In continuation of bug 0009822 submission ( http://bugs.digium.com/view.php?id=9822 ) let me report another issue that hangs out our softswitch . These problems might be connected, because in the issue noted threads are a part of core dumps, yet I considered to use two different bug submissions to track them down.

We get a plenty of "Out of idle IAX2 threads for I/O, pausing" messages when the amount of IAX peers exceeds 20...25, after several seconds all calls are disconnected. If the amount of IAX peers exceed 35...40 - core dump (as per bug 0009822 )

Same server.

Versions:

zaptel-1.4.2.1
asterisk-1.4.4
libpri-1.4.0


Hardware:

MB: Tomcat K8E (S2865)
RAM: 1Gb
HD: 1 x ST3808110AS
&ASTERISK-1035;&ASTERISK-1049;:

CPU: AMD Dual Core AMD Opteron(tm) Processor 170 stepping 02 2Ghz
cpu MHz         : 2009.300
cache size      : 1024 KB
cpu cores       : 2
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
                  mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall
                  nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy ts fid vid ttp
bogomips        : 8039.83 (2 cores taken into consideration)

Modules:

lsmod|grep zt
ztdummy                 5000  0
zaptel                181956  41 ztdummy
rtc                    12372  1 ztdummy

Interrupts:

cat /proc/interrupts
          CPU0       CPU1
 0:  391943392        267    IO-APIC-edge  timer
 1:       2168          1    IO-APIC-edge  i8042
 8:     558216          1    IO-APIC-edge  rtc
 9:          0          0   IO-APIC-level  acpi
50:  580954350  291073889   IO-APIC-level  eth1
217: 1442708362 2594162104   IO-APIC-level  eth0
225:     647585     901282   IO-APIC-level  libata
233:          0          0   IO-APIC-level  libata
NMI:          0          0
LOC:  391961455  391961454
ERR:          0
MIS:          0


Comments:By: timrobbins (timrobbins) 2007-06-01 05:09:11

Are you using realtime IAX users/peers? I've seen this kind of problem happen when the realtime backend (e.g. MySQL) was taking too long to respond, thus stalling the IAX worker threads.

Try increasing iaxthreadcount/iaxmaxthreadcount in iax.conf. For example:
iaxthreadcount=64
iaxmaxthreadcount=64

However, in my experience, this doesn't help very much. As soon as one of the worker threads gets blocked, the rest seem to follow.
By: Steve Davies . (stevedavies) 2007-06-01 08:33:17

Hi,

Please apply the patches from bugs 9663 and 9666 and see if they help.  I suspect the deadlock in 9663 could be blocking up your iax2 and that results in running out of iax2 threads.

IE: running out of threads is a symptom and not the problem.  (Which is what timrobbins said)

WIth Mihai I'm trying to get the multithreaded iax2 stabilised.  SO your feedback will be helpful (please post on the 9663 and 9666 bugs to help me get momentum to get them merged).

There are more places where segfaults can occur, but my changes + Mihai's has made my IAX2 switch box 10x more stable.

Regards,
Steve Davies
By: Russell Bryant (russell) 2007-06-04 18:39:35

This is probably fixed in 1.4 and trunk by revisions 67020 and 67022.  If you have any further problems, feel free to reopen this bug or file a new one.

Thanks!