| Summary: | ASTERISK-10583: [crash] FreeBSD: Crash if compile with option DEBUG_THREADS DONT_OPTIMIZE MALLOC_DEBUG | ||
| Reporter: | Yuri (ys) | Labels: | |
| Date Opened: | 2007-10-21 10:55:15 | Date Closed: | 2007-10-22 16:17:04 |
| Priority: | Critical | Regression? | No |
| Status: | Closed/Complete | Components: | Core/BuildSystem |
| Versions: | Frequency of Occurrence | ||
| Related Issues: | |||
| Environment: | Attachments: | ( 0) lock.h.diff | |
| Description: | FreBSD 5_RELEASE and 6_RELEASE are crashed at startup , if asterisk compiled with options DEBUG_THREADS and DONT_OPTIMIZE. This can be always reproduced at any version of FreeBSD. #./sbin/asterisk Bus error (core dumped) #gdb ./sbin/asterisk ./asterisk.core GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd"... Core was generated by `asterisk'. Program terminated with signal 10, Bus error. Reading symbols from /lib/libncurses.so.5...done. Loaded symbols for /lib/libncurses.so.5 Reading symbols from /lib/libm.so.3...done. Loaded symbols for /lib/libm.so.3 Reading symbols from /lib/libcrypto.so.3...done. Loaded symbols for /lib/libcrypto.so.3 Reading symbols from /usr/lib/libpthread.so.1...done. Loaded symbols for /usr/lib/libpthread.so.1 Reading symbols from /lib/libc.so.5...done. Loaded symbols for /lib/libc.so.5 Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x080bc834 in ast_log (level=4, file=0x813b7e0 "/home/asterisk2/src/asterisk-1.4/include/asterisk/lock.h", line=313, function=0x813bb94 "__ast_pthread_mutex_lock", fmt=0x813b888 "%s line %d (%s): Error: mutex '%s' is uninitialized.\n") at logger.c:689 689 if (!(buf = ast_dynamic_str_thread_get(&log_buf, LOG_BUF_INIT_SIZE))) (gdb)bt 2 full #0 0x080bc834 in ast_log (level=4, file=0x813b7e0 "/home/asterisk2/src/asterisk-1.4/include/asterisk/lock.h", line=313, function=0x813bb94 "__ast_pthread_mutex_lock", fmt=0x813b888 "%s line %d (%s): Error: mutex '%s' is uninitialized.\n") at logger.c:689 chan = (struct logchannel *) 0x283656bf buf = (struct ast_dynamic_str *) 0xbfa00308 t = -1077941088 tm = {tm_sec = 240, tm_min = 0, tm_hour = -1080032584, tm_mday = 674647487, tm_mon = 240, tm_year = 0, tm_wday = 0, tm_yday = 674647410, tm_isdst = 0, tm_gmtoff = 0, tm_zone = 0x0} date = '\0' <repeats 220 times>, "?J6(", '\0' <repeats 21 times>, "\f\000\000\b\000\000\000?+=(" ap = 0x0 __PRETTY_FUNCTION__ = "ast_log" #1 0x0807294c in __ast_pthread_mutex_lock (filename=0x813b75f "astmm.c", lineno=120, func=0x813bc0c "__ast_alloc_region", mutex_name=0x813b774 "®lock", t=0x816f780) at lock.h:312 res = 674650024 canlog = -11 __PRETTY_FUNCTION__ = "__ast_pthread_mutex_lock" (More stack frames follow...) (gdb) frame 1 #1 0x0807294c in __ast_pthread_mutex_lock (filename=0x813b75f "astmm.c", lineno=120, func=0x813bc0c "__ast_alloc_region", mutex_name=0x813b774 "®lock", t=0x816f780) at lock.h:312 warning: Source file is more recent than executable. 312 __ast_mutex_logger("%s line %d (%s): Error: mutex '%s' is uninitialized.\n", (gdb) t *p No symbol "p" in current context. (gdb) p *t $1 = {mutex = 0x0, track = 0, file = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, lineno = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, reentrancy = 0, func = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, thread = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, reentr_mutex = 0x0} ****** ADDITIONAL INFORMATION ****** Found in 1.4.13 and present in 1.4 SVN branch. | ||
| Comments: | By: Yuri (ys) 2007-10-21 10:56:54 Correction: Crash options: DEBUG_THREADS MALLOC_DEBUG By: Dmitry Andrianov (dimas) 2007-10-21 11:31:57 Take a look at http://bugs.digium.com/view.php?id=10571 There is confirmed race condition in thread (actually mutex) debugging code which was fixed in revision 85994. Since this commit happened after 1.4.13 was released, you may want getting latest 1.4 from SVN for testing. By: Yuri (ys) 2007-10-21 11:44:04 Now, I try to compile revision 86572 from SVN 1.4 branch. The problemm is still exist. Ivan, say me in IRC, that problem my be in AST_MUTEX_DEFINE_STATIC_NOTRACKING(..), defined at ./include/asterisk/lock.h And. Make sure, that pthread_mutex_t are pointer to structure in FreeBSD and Solaris.. By: Volnikov Ivan (ivan) 2007-10-22 03:25:01 I think, it is a problem only for BSD based systems. "ys" say, that just with MALLOC_DEBUG that problem happends. By: Yuri (ys) 2007-10-22 10:05:27 I upload patch, that fix the problemm in deadloop. If reglock is is uninitialized, we have the loop: ast_mutex_lock(®lock) -> __ast_mutex_logger() -> ast_log() -> __ast_calloc() -> __ast_alloc_region() -> ast_mutex_lock(®lock) And, I don't understand why ast_mutex_lock(®lock); can be called before AST_MUTEX_DEFINE_STATIC_NOTRACKING(reglock); ? By: Digium Subversion (svnbot) 2007-10-22 10:23:16 Repository: asterisk Revision: 86726 U branches/1.4/include/asterisk/lock.h ------------------------------------------------------------------------ r86726 | russell | 2007-10-22 10:23:16 -0500 (Mon, 22 Oct 2007) | 4 lines Update the static mutex initializer to include the initialization of the internal mutex used to protect the lock debugging data. (closes issue ASTERISK-10583, patch suggested by Ivan) ------------------------------------------------------------------------ By: Digium Subversion (svnbot) 2007-10-22 10:24:17 Repository: asterisk Revision: 86734 _U trunk/ U trunk/include/asterisk/lock.h ------------------------------------------------------------------------ r86734 | russell | 2007-10-22 10:24:17 -0500 (Mon, 22 Oct 2007) | 12 lines Merged revisions 86726 via svnmerge from https://origsvn.digium.com/svn/asterisk/branches/1.4 ........ r86726 | russell | 2007-10-22 10:43:30 -0500 (Mon, 22 Oct 2007) | 4 lines Update the static mutex initializer to include the initialization of the internal mutex used to protect the lock debugging data. (closes issue ASTERISK-10583, patch suggested by Ivan) ........ ------------------------------------------------------------------------ By: Yuri (ys) 2007-10-22 11:34:04 Applied patch is not resolve problem By: Digium Subversion (svnbot) 2007-10-22 16:15:18 Repository: asterisk Revision: 86836 U branches/1.4/include/asterisk/lock.h ------------------------------------------------------------------------ r86836 | russell | 2007-10-22 16:15:17 -0500 (Mon, 22 Oct 2007) | 9 lines If lock tracking is not enabled, then we can not attempt to log any mutex failures. If so, we could end up in infinite recursion. The only lock that is affected by this is a mutex in astmm.c used when MALLOC_DEBUG is enabled. (closes issue ASTERISK-10583) Reported by: ys Patches: lock.h.diff uploaded by ys (license 281) ------------------------------------------------------------------------ By: Digium Subversion (svnbot) 2007-10-22 16:17:04 Repository: asterisk Revision: 86839 _U trunk/ U trunk/include/asterisk/lock.h ------------------------------------------------------------------------ r86839 | russell | 2007-10-22 16:17:03 -0500 (Mon, 22 Oct 2007) | 17 lines Merged revisions 86836 via svnmerge from https://origsvn.digium.com/svn/asterisk/branches/1.4 ........ r86836 | russell | 2007-10-22 16:36:12 -0500 (Mon, 22 Oct 2007) | 9 lines If lock tracking is not enabled, then we can not attempt to log any mutex failures. If so, we could end up in infinite recursion. The only lock that is affected by this is a mutex in astmm.c used when MALLOC_DEBUG is enabled. (closes issue ASTERISK-10583) Reported by: ys Patches: lock.h.diff uploaded by ys (license 281) ........ ------------------------------------------------------------------------ | ||