Summary:ASTERISK-11621: System crash at format_wav.c:375
Reporter:Eric Dantie (edantie)Labels:
Date Opened:2008-03-12 04:17:17Date Closed:2011-06-07 14:00:49
Versions:Frequency of
Environment:Attachments:( 0) 20080313__bug12192.diff.txt
( 1) valgrind.txt
( 2) valgrind-good.txt
Description:System just crashed at format_wav.c:375


#0  0xb7e706d0 in free () from /lib/libc.so.6
#1  0xb7e6cf63 in _IO_free_backup_area () from /lib/libc.so.6
#2  0xb7e6d75c in __underflow () from /lib/libc.so.6
#3  0xb7e6a35b in ?? () from /lib/libc.so.6
#4  0xb7e6c258 in _IO_sgetn () from /lib/libc.so.6
ASTERISK-1  0xb7e5fdb4 in fread () from /lib/libc.so.6
ASTERISK-2  0xb6f6bc74 in wav_read (s=0xb576cc18, whennext=0xb67fd8d0)
   at format_wav.c:375
ASTERISK-3  0x080a200c in ast_readframe (s=0xb576cc18) at file.c:627
ASTERISK-4  0xb7d873e0 in moh_files_readframe (chan=0xb570fa30)
   at res_musiconhold.c:281
ASTERISK-5  0xb7d87436 in moh_files_generator (chan=0xb570fa30, data=0x8383e00, len=0,
   samples=160) at res_musiconhold.c:296
ASTERISK-6 0x08083164 in generator_force (data=0xb570fa30) at channel.c:1570
ASTERISK-7 0x080846a7 in __ast_read (chan=0xb570fa30, dropaudio=0) at channel.c:1995
ASTERISK-8 0x08085e1e in ast_read (chan=0xb570fa30) at channel.c:2318
ASTERISK-9 0x08076380 in autoservice_run (ign=0x0) at autoservice.c:120
ASTERISK-10 0x080ff132 in dummy_start (data=0x82adba8) at utils.c:865
ASTERISK-11 0xb7fb817b in start_thread () from /lib/libpthread.so.0
ASTERISK-12 0xb7eca09e in clone () from /lib/libc.so.6
Comments:By: destiny6628 (destiny6628) 2008-03-12 06:49:13

Hey mate also check your hard disk utilization , it may be full .I also faced same issue long time ago due to this .

By: Eric Dantie (edantie) 2008-03-12 09:02:38

Hard disk only used a 7%. Sorry.

By: Eric Dantie (edantie) 2008-03-12 11:02:20

Same issue in asterisk 1.6 from SVN 1.6.0 108030

#0  0xb7d0c6b9 in free () from /lib/libc.so.6
#1  0xb7d08f63 in _IO_free_backup_area () from /lib/libc.so.6
#2  0xb7d0975c in __underflow () from /lib/libc.so.6
#3  0xb7d0635b in ?? () from /lib/libc.so.6
#4  0xb7d08258 in _IO_sgetn () from /lib/libc.so.6
ASTERISK-1  0xb7cfbdb4 in fread () from /lib/libc.so.6
ASTERISK-2  0xb6cefc74 in wav_read (s=0xb626fcc0, whennext=0xb66a28b0)
   at format_wav.c:360
ASTERISK-3  0x080bc1c7 in ast_readframe (s=0xb626fcc0) at file.c:637
ASTERISK-4  0xb75b266b in moh_files_readframe (chan=0xb6225fa8)
   at res_musiconhold.c:289
ASTERISK-5  0xb75b26c1 in moh_files_generator (chan=0xb6225fa8, data=0x8265e40, len=0,
   samples=160) at res_musiconhold.c:304
ASTERISK-6 0x0808a109 in generator_force (data=0xb6225fa8) at channel.c:1721
ASTERISK-7 0x0808b757 in __ast_read (chan=0xb6225fa8, dropaudio=0) at channel.c:2363
ASTERISK-8 0x0808d01f in ast_read (chan=0xb6225fa8) at channel.c:2692
ASTERISK-9 0x0807cad5 in autoservice_run (ign=0x0) at autoservice.c:113
ASTERISK-10 0x0812ce77 in dummy_start (data=0x8218388) at utils.c:870
ASTERISK-11 0xb7c8f17b in start_thread () from /lib/libpthread.so.0
ASTERISK-12 0xb7d6609e in clone () from /lib/libc.so.6

By: Tilghman Lesher (tilghman) 2008-03-12 13:05:00

Please follow the instructions in doc/valgrind.txt.

By: Eric Dantie (edantie) 2008-03-12 16:02:01

Sorry I can't reproduce the error with valgrind: I've tried the 1.6 for some time to see if working, but after more or less 300 incoming calls, the crash happened. Like system is in production can't reproduce.

By: Tilghman Lesher (tilghman) 2008-03-12 16:20:09

I know you can't reproduce the crash, but I still need the output from valgrind.  Valgrind generally detects issues and logs them, then prevents the crash from occurring, but the output sometimes gives us enough information to fix the problem.

By: Eric Dantie (edantie) 2008-03-12 17:46:33

I've done the valgrin and wait for more or less 400 incoming calls with no fail. The valgrind file was 30MB so cut the center part with was always the same: invalid file descriptor...

oops there were more errors like:

==25671== Thread 43:
==25671== Invalid read of size 4
==25671==    at 0x49C2700: ??? (res_musiconhold.c:307)
==25671==    by 0x808A108: generator_force (channel.c:1721)
==25671==    by 0x808B756: __ast_read (channel.c:2363)
==25671==    by 0x808D01E: ast_read (channel.c:2692)
==25671==    by 0x807CAD4: autoservice_run (autoservice.c:113)
==25671==    by 0x812CE76: dummy_start (utils.c:870)
==25671==    by 0x42E517A: start_thread (in /lib/libpthread-2.6.1.so)
==25671==    by 0x426C09D: clone (in /lib/libc-2.6.1.so)
==25671==  Address 0x5E9CD4C is 60 bytes inside a block of size 528 free'd
==25671==    at 0x40223E2: free (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
==25671==    by 0x80BCAA8: ast_closestream (file.c:824)
==25671==    by 0x49C66E3: ??? (res_musiconhold.c:1309)
==25671==    by 0x8093E65: ast_moh_stop (channel.c:4753)
==25671==    by 0x5B9CDC0: ctqueue_exec (queue.c:285)
==25671==    by 0x5B98298: app_ctqueue_main (app_ctasterisk.c:148)
==25671==    by 0x80DEEF9: pbx_exec (pbx.c:729)
==25671==    by 0x80E4307: pbx_extension_helper (pbx.c:2706)
==25671==    by 0x80E569E: ast_spawn_extension (pbx.c:3197)
==25671==    by 0x80E5DB0: __ast_pbx_run (pbx.c:3296)
==25671==    by 0x80E6F3D: pbx_thread (pbx.c:3558)
==25671==    by 0x812CE76: dummy_start (utils.c:870)

Will try to upload a better file

By: Eric Dantie (edantie) 2008-03-12 18:03:54

Can remove valgrind.txt. Not correct.
FYI: app_ctasterisk is a home made application which do more or less like app_queue.

the lines of codes of queue.c are:

284:    if (ast_test_flag(chan, AST_FLAG_MOH)) {
285:       ast_moh_stop(chan);
286:       ast_autoservice_stop(chan);
287:    }

Hope will help.
If you need any more information let me know.

By: Tilghman Lesher (tilghman) 2008-03-13 17:57:02

I'm uploading this patch as a courtesy, although we generally do not support 3rd party applications.

By: Eric Dantie (edantie) 2008-03-14 03:48:00

I understand that you don't support 3rd party applications, but I think it's an asterisk bug. Neverless, I thank you.

Looking at res/res_musiconhold.c I saw that this piece of code is reproduced more than once. Shouldn't be added the usleep(1) before every ast_closestream()?