[Home]

Summary:ASTERISK-05219: Asterisk 1.2b1 crash on Local channel hangup
Reporter:Matt Florell (mflorell)Labels:
Date Opened:2005-10-03 06:41:15Date Closed:2011-06-07 14:10:38
Priority:MinorRegression?No
Status:Closed/CompleteComponents:Core/General
Versions:Frequency of
Occurrence
Related
Issues:
Environment:Attachments:
Description:I've been using 1.2beta1 in production for several weeks now and this is the first crash I've had with it. Here's the gdb backtrace of the core file:

(gdb) bt
#0  0x4002aa12 in pthread_mutex_lock () from /lib/libpthread.so.0
#1  0x403c5105 in local_hangup (ast=0x82b5580) at chan_local.c:370
#2  0x0805fabd in ast_hangup (chan=0x82b5580) at channel.c:1025
#3  0x4066ea53 in dial_exec_full (chan=0x829d380, data=0x829d380,
peerflags=0xb49f955c) at app_dial.c:1643
#4  0x4066cb15 in dial_exec (chan=0xb49f8754, data=0xb49f8754) at
app_dial.c:1670
ASTERISK-1  0x0808c52f in pbx_extension_helper (c=0x829d380, con=0xb49f8754,
context=0x829d4d0 "demo", exten=0x829d5c4 "919205287002", priority=2,
label=0x0, callerid=0x82004f8 "Dial", action=0)
  at pbx.c:553
ASTERISK-2  0x0808d0ff in __ast_pbx_run (c=0x829d380) at pbx.c:2151
ASTERISK-3  0x0808dd08 in pbx_thread (data=0x829d380) at pbx.c:2438
ASTERISK-4  0x4002a54e in pthread_start_thread () from /lib/libpthread.so.0
ASTERISK-5  0x401d6b8a in clone () from /lib/libc.so.6
(gdb) info thread
* 1 process 15607  0x4002aa12 in pthread_mutex_lock () from /lib/libpthread.so.0
Comments:By: Kevin P. Fleming (kpfleming) 2005-10-04 19:52:03

Please read the bug posting guidelines; this is not a useful backtrace, since it only shows one thread, and it was apaprently generated from an optimized build of Asterisk.
By: Matt Florell (mflorell) 2005-10-04 20:57:49

This is a system that ran fine for over a month in a high-volume workload. Should I always install my production systems with "make clean; make valgrind; make install" so that I can send a good backtrace?

What kind of performance hit does a system take by not having an optimized installation?

Here is the full bt, even if it doesn't help I wanted to make sure it was in the bug tracker in case someone else runs into this issue:

(gdb) bt
#0  0x4002aa12 in pthread_mutex_lock () from /lib/libpthread.so.0
#1  0x403c5105 in local_hangup (ast=0x82b5580) at chan_local.c:370
#2  0x0805fabd in ast_hangup (chan=0x82b5580) at channel.c:1025
#3  0x4066ea53 in dial_exec_full (chan=0x829d380, data=0x829d380, peerflags=0xb49f955c) at app_dial.c:1643
#4  0x4066cb15 in dial_exec (chan=0xb49f8754, data=0xb49f8754) at app_dial.c:1670
ASTERISK-1  0x0808c52f in pbx_extension_helper (c=0x829d380, con=0xb49f8754, context=0x829d4d0 "demo",
   exten=0x829d5c4 "919205287002", priority=2, label=0x0, callerid=0x82004f8 "Dial", action=0) at pbx.c:553
ASTERISK-2  0x0808d0ff in __ast_pbx_run (c=0x829d380) at pbx.c:2151
ASTERISK-3  0x0808dd08 in pbx_thread (data=0x829d380) at pbx.c:2438
ASTERISK-4  0x4002a54e in pthread_start_thread () from /lib/libpthread.so.0
ASTERISK-5  0x401d6b8a in clone () from /lib/libc.so.6
(gdb) bt full
#0  0x4002aa12 in pthread_mutex_lock () from /lib/libpthread.so.0
No symbol table info available.
#1  0x403c5105 in local_hangup (ast=0x82b5580) at chan_local.c:370
       p = (struct local_pvt *) 0x0
       isoutbound = -1264613500
       f = {frametype = 4, subclass = 1, datalen = 0, samples = 0, mallocd = 0, offset = 0, src = 0x0, data = 0x0,
 delivery = {tv_sec = 0, tv_usec = 0}, prev = 0x0, next = 0x0}
       cur = (struct local_pvt *) 0xb49f8754
       prev = (struct local_pvt *) 0x0
       ochan = (struct ast_channel *) 0x0
       glaredetect = 0
#2  0x0805fabd in ast_hangup (chan=0x82b5580) at channel.c:1025
       res = 0
#3  0x4066ea53 in dial_exec_full (chan=0x829d380, data=0x829d380, peerflags=0xb49f955c) at app_dial.c:1643
       res2 = -1264613548
       loopcount = 136958848
       res = -1
       u = (struct localuser *) 0x817efd0
       info = 0xb49f8814 "Zap"
       timeout = 0xb49f8827 "30"
       number = 0xb49f8818 "g2/19205287002"
       rest = 0x0
       cur = 0x0
       privdb = '\0' <repeats 255 times>
       s = 0x82b5580 "Local/919205287002@demo-6187,1<ZOMBIE>"
       privcid = '\0' <repeats 255 times>
       privintro = '\0' <repeats 1023 times>
       announcemsg = '\0' <repeats 255 times>
       ann = 0x82b5580 "Local/919205287002@demo-6187,1<ZOMBIE>"
       outgoing = (struct localuser *) 0x0
       tmp = (struct localuser *) 0x824d278
       peer = (struct ast_channel *) 0x82b5580
       to = 6145
       hasmacro = 0
       privacy = 0
       screen = 0
       no_save_intros = 0
       no_screen_callerid = 0
       announce = 0
       resetcdr = 0
       numbusy = 0
       numcongestion = 0
       numnochan = 0
       cause = 0
       numsubst = "g2/19205287002", '\0' <repeats 65 times>
       restofit = '\0' <repeats 79 times>
       cidname = '\0' <repeats 79 times>
       transfer = 0xb49f882a "o"
       l = 0x829d380 "Local/919205287002@demo-6187,2"
       url = 0x0
       privdb_val = 0
       calldurationlimit = 0
       now = 0
       config = {features_caller = {flags = 0}, features_callee = {flags = 0}, start_time = {tv_sec = 1128103416,
   tv_usec = 19026}, feature_timer = 0, timelimit = 0, play_warning = 0, warning_freq = 0, warning_sound = 0x0,
 end_sound = 0x0, start_sound = 0x0, firstpass = 0, flags = 0}
       timelimit = 0
       play_warning = 0
       warning_freq = 0
       warning_sound = 0x0
       end_sound = 0x0
       start_sound = 0x0
       limitptr = 0x82b5580 "Local/919205287002@demo-6187,1<ZOMBIE>"
       limitdata = '\0' <repeats 255 times>
       sdtmfptr = 0x82b5580 "Local/919205287002@demo-6187,1<ZOMBIE>"
       dtmfcalled = 0x0
       dtmfcalling = 0x0
       stack = 0x0
       var = 0x829d380 "Local/919205287002@demo-6187,2"
       mac = 0x82b5580 "Local/919205287002@demo-6187,1<ZOMBIE>"
       macroname = 0x0
       status = "ANSWER\000R", '\0' <repeats 247 times>
       toast = "0", '\0' <repeats 78 times>
       play_to_caller = 0
       play_to_callee = 0
       sentringing = 0
       moh = 0
       mohclass = 0x0
       outbound_group = 0x0
       macro_result = 0x829d380 "Local/919205287002@demo-6187,2"
       macro_transfer_dest = 0x829d380 "Local/919205287002@demo-6187,2"
       digit = 0
       result = 0
       start_time = 1128103391
       answer_time = 1128103416
       end_time = 1128103416
       app = (struct ast_app *) 0x829d380
       dblgoto = 0x0
       priority_jump = 0
       oo = (struct localuser *) 0xb49f8754
#4  0x4066cb15 in dial_exec (chan=0xb49f8754, data=0xb49f8754) at app_dial.c:1670
       peerflags = {flags = 4096}
ASTERISK-1  0x0808c52f in pbx_extension_helper (c=0x829d380, con=0xb49f8754, context=0x829d4d0 "demo",
   exten=0x829d5c4 "919205287002", priority=2, label=0x0, callerid=0x82004f8 "Dial", action=0) at pbx.c:553
       e = (struct ast_exten *) 0x0
       sw = (struct ast_switch *) 0x0
       data = 0x0
       foundcontext = 0x829d4d0 "demo"
       newstack = 1
       res = 0
       status = 5
       incstack = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x4017bc14 "\201\v", 0x0, 0x0,
 0x40235ff4 "\034\215\021", 0xb49ff7b4 "\001\200\237\237\237\237\a\2373\237\2373\237",
 0xb49ff790 "\237`x\027@\237%h\017\b\237\237",
 0x4017ceb0 "\211~\020\213E\024\211~\f\211~\004\205t$\213U\024\213}\211V\024\211~\030\211V\b1\211\216\230",
 0xb49ff7b4 "\001\200\237\237\237\237\a\2373\237\2373\237",
 0xb49ff8e4 "\"V0930140310005218409\" <0000000000>", 0xb49ff933 "", 0x0, 0xb49ff933 "", 0x40235ff4 "\034\215\021",
 0x80f6825 "\"%s\" <%s>", 0xb49ff8d0 "`\030\b\027\by#@\221\002@", 0xb49ff8a0 "\2374\001\026@\237P",
 0x40177860 "\213\215\f\2119\2150t\t\213(\006",
 0xb49ff7b4 "\001\200\237\237\237\237\a\2373\237\2373\237", 0x80f6825 "\"%s\" <%s>",
 0xb49ff8d8 "y#@\221\002@", 0xb49ff8e4 "\"V0930140310005218409\" <0000000000>", 0x0, 0xb49ff854 "",
 0xb49ff7b4 "\001\200\237\237\237\237\a\2373\237\2373\237",
 0xfbad8001 <Address 0xfbad8001 out of bounds>, 0xb49ff8e4 "\"V0930140310005218409\" <0000000000>",
 0xb49ff8e4 "\"V0930140310005218409\" <0000000000>", 0xb49ff8e4 "\"V0930140310005218409\" <0000000000>",
 0xb49ff8e4 "\"V0930140310005218409\" <0000000000>", 0xb49ff907 "", 0xb49ff933 "",
 0xb49ff8e4 "\"V0930140310005218409\" <0000000000>", 0xb49ff933 "", 0x0 <repeats 14 times>,
 0xffffffff <Address 0xffffffff out of bounds>, 0x0 <repeats 13 times>, 0x40235a00 "", 0x0 <repeats 18 times>,
 0x40235ff4 "\034\215\021", 0x8205f30 "\"V0930140310005218409\" <0000000000>", 0x817b4d8 "0000000000",
 0xb49ff8bc "\200)\b\t\b\237P",
 0x40160134 "\213]\211]\220\220\220\220\220U\211\203\020\211]\213U\f\215M\020y!\201^\r",
 0xb49ff8e4 "\"V0930140310005218409\" <0000000000>", 0x50 <Address 0x50 out of bounds>, 0x80f6825 "\"%s\" <%s>",
 0xb49ff8d0 "`\030\b\027\by#@\221\002@", 0xb49ff8e4 "\"V0930140310005218409\" <0000000000>",
 0x829d380 "Local/919205287002@demo-6187,2", 0x809d8c2 "\017D$ \211\215L$ P",
 0xb49ff8e4 "\"V0930140310005218409\" <0000000000>", 0x50 <Address 0x50 out of bounds>, 0x80f6825 "\"%s\" <%s>",
 0x818d560 "V0930140310005218409", 0x817b4d8 "0000000000", 0x402379c0 "", 0x4002b091 "\201c\177", 0x0,
 0x39305622 <Address 0x39305622 out of bounds>, 0x34313033 <Address 0x34313033 out of bounds>,
 0x30313330 <Address 0x30313330 out of bounds>, 0x32353030 <Address 0x32353030 out of bounds>,
 0x30343831 <Address 0x30343831 out of bounds>, 0x3c202239 <Address 0x3c202239 out of bounds>,
 0x30303030 <Address 0x30303030 out of bounds>, 0x30303030 <Address 0x30303030 out of bounds>,
 0x3e3030 <Address 0x3e3030 out of bounds>, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
 0x809d3ec "\213T$\030\213t$\034\211\223\200\002", 0xb49ff93c "}=C\200)\b\237", 0x0, 0x0, 0x0}
       passdata = "Zap/g2/19205287002|30|o", '\0' <repeats 8168 times>
       stacklen = 0
       tmp = "\033[1;36;40mDial\033[0;37;40m", '\0' <repeats 55 times>
       tmp2 = "\033[1;35;40mLocal/919205287002@demo-6187,2\033[0;37;40m", '\0' <repeats 29 times>
       tmp3 = "\033[1;35;40mZap/g2/19205287002|30|o\033[0;37;40m\000m", '\0' <repeats 8146 times>
       atmp = '\0' <repeats 79 times>
       atmp2 = '\0' <repeats 8291 times>
ASTERISK-2  0x0808d0ff in __ast_pbx_run (c=0x829d380) at pbx.c:2151
       digit = 0
       exten = '\0' <repeats 255 times>
       pos = 0
       waittime = 0
       res = 0
ASTERISK-3  0x0808dd08 in pbx_thread (data=0x829d380) at pbx.c:2438
No locals.
ASTERISK-4  0x4002a54e in pthread_start_thread () from /lib/libpthread.so.0
No symbol table info available.
ASTERISK-5  0x401d6b8a in clone () from /lib/libc.so.6
No symbol table info available.
By: Olle Johansson (oej) 2005-10-25 04:21:19

Matt, please attach output like that as an attachment - it makes it easier to read the files.

* Does the problem still exist in CVS head?
* Any way this can be repeated?
* If so, can you turn on thread debugging in the Makefile and see if this is a locking problem in chan_local ? (Extremely wild guess)
/O
By: Matt Florell (mflorell) 2005-10-25 09:36:05

The system ran for a month before this crash happened and it hasn't happened in the last 3 weeks. I was not able to duplicate it despite my best efforts.

We do a tremendous amount of Local/ channel calls on this system every day(>50,000) so if there is a bug it must take just the right circumstances for it to show itself.

You can close the bug and I will reopen if this happens again.
By: Olle Johansson (oej) 2005-10-25 09:39:32

Suspended on reporter's request until we find the bug again.

Cheers, Matt!

/Olle