[Home]

Summary:ASTERISK-15861: [patch] Asterisk crashes while core restart (#0 0x000000000050683c in term_beep (el=0x16cdd9b0) at term.c:865)
Reporter:sybasesql (sybasesql)Labels:
Date Opened:2010-03-23 06:22:02Date Closed:2010-10-03 23:03:10
Priority:CriticalRegression?No
Status:Closed/CompleteComponents:Applications/General
Versions:Frequency of
Occurrence
Related
Issues:
Environment:Attachments:( 0) 20100703__issue17080.diff.txt
( 1) 20100716__issue17080.diff.txt
( 2) 20100718__issue17080.diff.txt
( 3) 20100721__issue17080.diff.txt
( 4) Asterisk-1.6.2.6_x86_64.txt
( 5) Asterisk-1.6.2.6_x86_64.txt-DONT_OPTIMIZE.txt
( 6) Asterisk-1.6.2.6_x86_64.txt-DONT_OPTIMIZE+20100716__issue17080.diff.txt
( 7) Asterisk-1.6.2.9_x86_64.txt-DONT_OPTIMIZE+20100718__issue17080_2.diff.txt
( 8) Asterisk-1.6.2.9_x86_64.txt-DONT_OPTIMIZE+20100718__issue17080.diff.txt
( 9) asterisk-1.6.2.9+20100703__issue17080.diff.txt
Description:Asterisk always crashes while execute by cron "core restart when convenient".
I attached gdb output to that issue.

****** STEPS TO REPRODUCE ******

/usr/sbin/asterisk -r -x "core restart when convenient"


****** ADDITIONAL INFORMATION ******

#0  0x000000000050683c in term_beep (el=0x16cdd9b0) at term.c:865
865             if (GoodStr(T_bl))
(gdb) bt
#0  0x000000000050683c in term_beep (el=0x16cdd9b0) at term.c:865
#1  0x0000000000508739 in el_gets (el=0x16cdd9b0, nread=0x7fff29de316c) at read.c:544
#2  0x0000000000435940 in main (argc=<value optimized out>, argv=0x7fff29de3288) at asterisk.c:3706
#3  0x000000366321d994 in __libc_start_main () from /lib64/libc.so.6
#4  0x000000000041b019 in SSL_accept ()
ASTERISK-1  0x00007fff29de3278 in ?? ()
ASTERISK-2  0x0000000000000000 in ?? ()
Comments:By: Leif Madsen (lmadsen) 2010-03-23 09:32:46

I'm seeing a lot of <value optimized out> in your backtrace. Have you enabled DONT_OPTIMIZE in the Compiler Flags section of menuselect? (per doc/backtrace.txt of your Asterisk source)
By: sybasesql (sybasesql) 2010-03-23 09:43:24

No I don't have DONT_OPTIMIZE flag. I'll try set DONT_OPTIMIZE flag, rebuild rpm and post gdb output again.
By: sybasesql (sybasesql) 2010-03-23 10:18:49

I attached gdb output from asterisk after execute "core restart when convenient".
Asterisk was build with DONT_OPTIMIZE flag.
By: Paul Belanger (pabelanger) 2010-06-29 12:49:29

Please retest with the latest 1.6.2 branch.  I'm curious if this is still an issue;.
By: sybasesql (sybasesql) 2010-06-29 23:43:22

The problem still exists on Asterisk 1.6.2.7 x86_64
By: Paul Belanger (pabelanger) 2010-06-30 08:06:53

@sybasesql: Unfortunately that is not the latest branch.

svn co http://svn.asterisk.org/svn/asterisk/branches/1.6.2
By: sybasesql (sybasesql) 2010-06-30 09:00:37

Unfortunately I can't check on latest branch, only on release, because this is production system.
By: Paul Belanger (pabelanger) 2010-06-30 10:40:05

1.6.2.9 should include the changes I'm looking for you to test.
By: sybasesql (sybasesql) 2010-07-03 04:00:15

Bug still exist on Asterisk 1.6.2.9 x86_64.
By: sybasesql (sybasesql) 2010-07-04 23:28:35

Bug still exist on Asterisk 1.6.2.9 x86_64 with your patch 20100703__issue17080.diff.txt.
I attached gdb output.
By: Tilghman Lesher (tilghman) 2010-07-06 10:56:00

Are you running on a multi-processor box?
By: sybasesql (sybasesql) 2010-07-06 11:10:02

Yes. The server have Dual Intel Xeon Processors E5420 (12M Cache, 2.50 GHz, 1333 MHz FSB).

dmesg output:
Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz stepping 06
Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz stepping 06
Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz stepping 06
Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz stepping 06
Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz stepping 06
Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz stepping 06
Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz stepping 06

/proc/cpu output:
processor       : 0-7
vendor_id       : GenuineIntel
cpu family      : 6
model           : 23
model name      : Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz
stepping        : 6
cpu MHz         : 2500.143
cache size      : 6144 KB
physical id     : 1
siblings        : 4
core id         : 3
cpu cores       : 4
apicid          : 7
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 lahf_lm
bogomips        : 5037.48
clflush size    : 64
cache_alignment : 64
address sizes   : 38 bits physical, 48 bits virtual
power management:
By: Tilghman Lesher (tilghman) 2010-07-16 16:10:55

Okay, new patch uploaded.
By: sybasesql (sybasesql) 2010-07-18 04:30:47

Bug still exist on Asterisk 1.6.2.9 x86_64 with your patch  20100716__issue17080.diff.txt.
I attached gdb output in file Asterisk-1.6.2.6_x86_64.txt-DONT_OPTIMIZE+20100716__issue17080.diff.txt.
By: Tilghman Lesher (tilghman) 2010-07-18 10:39:18

Okay, new patch uploaded.
By: sybasesql (sybasesql) 2010-07-19 00:27:37

Bug still exist on Asterisk 1.6.2.9 x86_64 with your patch 20100716__issue17080.diff.txt.
I attached gdb output in file Asterisk-1.6.2.9_x86_64.txt-DONT_OPTIMIZE+20100718__issue17080.diff.txt.
By: Tilghman Lesher (tilghman) 2010-07-19 00:59:18

That crash should not be possible with the latest patch.  Are you sure you were running the very latest patch?
By: Tilghman Lesher (tilghman) 2010-07-19 01:02:33

You'll likely need to provide remote access to the machine in question, since we're not able to reproduce this issue here.  Please come onto irc.freenode.net, channel #asterisk-bugs, mention this issue, and be prepared to provide login details.  This is best done between the hours of 10:00 and 18:00, Central Time (American).
By: sybasesql (sybasesql) 2010-07-20 12:23:45

I recompile all from scratch, but unfortunately the asterisk core dump.
See attached file.
P.s. For security reasons I can not give remote access to our production servers.
By: Tilghman Lesher (tilghman) 2010-07-21 15:20:05

Well, instead of trying to unwind it all and add checks, the simpler solution is probably not to close it at all, unless we're executing it from the console thread.  New patch uploaded.
By: sybasesql (sybasesql) 2010-07-23 02:16:54

After applying the latest patch crash has disappeared.
By: Digium Subversion (svnbot) 2010-07-23 11:42:24

Repository: asterisk
Revision: 278981

U   branches/1.4/main/asterisk.c

------------------------------------------------------------------------
r278981 | tilghman | 2010-07-23 11:42:24 -0500 (Fri, 23 Jul 2010) | 8 lines

Avoid race with consolethread on shutdown (on parallel processors).

(closes issue ASTERISK-15861)
Reported by: sybasesql
Patches:
      20100721__issue17080.diff.txt uploaded by tilghman (license 14)
Tested by: sybasesql

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=278981
By: Digium Subversion (svnbot) 2010-07-23 11:43:33

Repository: asterisk
Revision: 278982

_U  trunk/
U   trunk/main/asterisk.c

------------------------------------------------------------------------
r278982 | tilghman | 2010-07-23 11:43:33 -0500 (Fri, 23 Jul 2010) | 15 lines

Merged revisions 278981 via svnmerge from
https://origsvn.digium.com/svn/asterisk/branches/1.4

........
 r278981 | tilghman | 2010-07-23 11:42:25 -0500 (Fri, 23 Jul 2010) | 8 lines
 
 Avoid race with consolethread on shutdown (on parallel processors).
 
 (closes issue ASTERISK-15861)
  Reported by: sybasesql
  Patches:
        20100721__issue17080.diff.txt uploaded by tilghman (license 14)
  Tested by: sybasesql
........

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=278982
By: Digium Subversion (svnbot) 2010-07-23 11:44:52

Repository: asterisk
Revision: 278983

_U  branches/1.6.2/
U   branches/1.6.2/main/asterisk.c

------------------------------------------------------------------------
r278983 | tilghman | 2010-07-23 11:44:52 -0500 (Fri, 23 Jul 2010) | 22 lines

Merged revisions 278982 via svnmerge from
https://origsvn.digium.com/svn/asterisk/trunk

................
 r278982 | tilghman | 2010-07-23 11:43:34 -0500 (Fri, 23 Jul 2010) | 15 lines
 
 Merged revisions 278981 via svnmerge from
 https://origsvn.digium.com/svn/asterisk/branches/1.4
 
 ........
   r278981 | tilghman | 2010-07-23 11:42:25 -0500 (Fri, 23 Jul 2010) | 8 lines
   
   Avoid race with consolethread on shutdown (on parallel processors).
   
   (closes issue ASTERISK-15861)
    Reported by: sybasesql
    Patches:
          20100721__issue17080.diff.txt uploaded by tilghman (license 14)
    Tested by: sybasesql
 ........
................

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=278983