[Home]

Summary:ASTERISK-07896: After ~20 - 30 minutes of load asterisk IAX deadlocked, and do not accept any more calls and not stop on "stop now". Needs kill
Reporter:Anton Vazir (vazir)Labels:
Date Opened:2006-10-08 07:13:33Date Closed:2006-10-09 10:53:04
Priority:BlockerRegression?No
Status:Closed/CompleteComponents:Core/General
Versions:Frequency of
Occurrence
Related
Issues:
Environment:Attachments:
Description:Putting a live load on IAX2

after ${SUBJ} I do get the following.
Btw: Before that (during normal operation lots of the following messages appers:
[Oct  8 17:00:06] NOTICE[23291]: sched.c:284 ast_sched_del: Attempted to delete nonexistent schedule entry 63408!

and when deadloked, it start just producing the following messages continously:

[Oct  8 16:58:31] NOTICE[8240]: chan_iax2.c:1720 iax2_destroy: Avoiding IAX destroy deadlock
[Oct  8 16:58:31] NOTICE[8240]: chan_iax2.c:1720 iax2_destroy: Avoiding IAX destroy deadlock
[Oct  8 16:58:31] NOTICE[8240]: chan_iax2.c:1720 iax2_destroy: Avoiding IAX destroy deadlock
[Oct  8 16:58:31] NOTICE[8240]: chan_iax2.c:1720 iax2_destroy: Avoiding IAX destroy deadlock


****** ADDITIONAL INFORMATION ******

switch:/usr/src/ASTERISK_SVN/asterisk# svn info
Path: .
URL: http://svn.digium.com/svn/asterisk/trunk
Repository UUID: 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Revision: 44649
Node Kind: directory
Schedule: normal
Last Changed Author: rizzo
Last Changed Rev: 44641
Last Changed Date: 2006-10-07 04:56:59 -0400 (Sat, 07 Oct 2006)
Properties Last Updated: 2006-10-07 05:40:07 -0400 (Sat, 07 Oct 2006)
Comments:By: Anton Vazir (vazir) 2006-10-08 07:21:10

Number of simultaneous call attempts arround ~ 30

I've got an additional message what might help

[Oct  8 17:27:36] NOTICE[22556]: chan_iax2.c:6196 socket_read: Out of idle IAX2 threads for I/O, pausing!

and the call scheme is

Asterisk 1.2.12.1 (PSTN) --> IAX2 --> Asterisk (trunk) --> voip

So, the one which is deadlocking = Asterisk (trunk)

but once I've got the same on the trunk -> trunk communication, so it seems to be unrelated that the calls are originated by 1.2.12.1


By: Anton Vazir (vazir) 2006-10-08 07:28:47

[Oct  8 17:27:36] NOTICE[22549]: chan_iax2.c:1720 iax2_destroy: Avoiding IAX destroy deadlock
[Oct  8 17:27:36] NOTICE[22556]: chan_iax2.c:6196 socket_read: Out of idle IAX2 threads for I/O, pausing!
[Oct  8 17:27:36] NOTICE[22549]: chan_iax2.c:1720 iax2_destroy: Avoiding IAX destroy deadlock


By: Paul Cadach (pcadach) 2006-10-08 07:40:32

I've seen "Out of idle IAX2 threads for I/O" and like for scheduling too.
By: Anton Vazir (vazir) 2006-10-08 08:12:25

While compiled with DO_CRASH it crashes in 5 seconds after start with the following backtrace.

#0  0x080eb983 in ast_sched_del (con=0x819fc98, id=11773) at sched.c:286
286                     CRASH;
(gdb) bt
#0  0x080eb983 in ast_sched_del (con=0x819fc98, id=11773) at sched.c:286
#1  0xb650b0a4 in iax2_destroy_helper (pvt=0x82a11a8) at chan_iax2.c:1670
#2  0xb650b441 in iax2_destroy (callno=14) at chan_iax2.c:1731
#3  0xb652371f in socket_process (thread=0x81ae610) at chan_iax2.c:6913
#4  0xb652834c in iax2_process_thread (data=0x81ae610) at chan_iax2.c:7608
ASTERISK-1  0x080fa878 in dummy_start (data=0x81a2770) at utils.c:544
ASTERISK-2  0xb7fc4b63 in start_thread () from /lib/tls/libpthread.so.0
ASTERISK-3  0xb6fa918a in clone () from /lib/tls/libc.so.6
(gdb)


By: Anton Vazir (vazir) 2006-10-08 09:09:28

UPDATE - while increasing number of simultaneous calls (40+) live of asterisk shortens to not more than 5 minutes. So it linearry depends to the number of the calls passed. Increasing threads variables in iax2.conf not helping at all.


By: Joshua C. Colp (jcolp) 2006-10-09 10:53:04

Fixed in 1.4 as of revision 44759 and trunk as of revision 44762.