[Home]

Summary:ASTERISK-15251: [patch] Asterisk crashes after receiving fax with 'double free'
Reporter:Vlad M (vlad)Labels:
Date Opened:2009-12-01 06:48:28.000-0600Date Closed:2010-02-09 12:09:38.000-0600
Priority:CriticalRegression?No
Status:Closed/CompleteComponents:Applications/app_fax
Versions:Frequency of
Occurrence
Related
Issues:
Environment:Attachments:( 0) 20100202__issue16361__debug.diff.txt
( 1) 20100208__issue16361.diff.txt
( 2) bt_1.6.11.txt
( 3) debug.log
( 4) valgrind.txt
Description:Version 1.6.1.9 works OK.
Versions 1.6.1.10 and .11 both crashes after receiving fax

Linux 2.6.31.6-134.fc12.i686.PAE #1 SMP Mon Nov 16 20:53:21 EST 2009 i686 i686 i386 GNU/Linux
spandsp-0.0.6-0.1.pre12.fc12.i686


****** ADDITIONAL INFORMATION ******

- Executing [13129245774@default:3] Wait("SIP/isp-primary-00000002", "2") in new stack
[Dec  1 06:09:32] NOTICE[19314]: rtp.c:1796 ast_rtp_read: Unknown RTP codec 100 received from 'IP'
   -- Executing [13129245774@default:4] NVBackgroundDetect("SIP/isp-primary-00000002", "vm-intro|t") in new stack
[Dec  1 06:09:33] WARNING[19322]: pbx.c:956 pbx_exec: The application delimiter is now the comma, not the pipe.  Did you forget to convert your dialplan?  (NVBackgroundDetect(vm-intro|t))
   -- <SIP/isp-primary-00000002> Playing 'vm-intro.ulaw' (language 'en')
[Dec  1 06:09:36] NOTICE[19322]: app_nv_backgrounddetect.c:210 nv_background_detect_exec: Redirecting SIP/isp-primary-00000002 to fax extension
   -- Executing [fax@default:1] Set("SIP/isp-primary-00000002", "FAXFILE=/var/spool/asterisk/fax/1259669368.2") in new stack
   -- Executing [fax@default:2] ReceiveFAX("SIP/isp-primary-00000002", "/var/spool/asterisk/fax/1259669368.2.tif") in new stack
[Dec  1 06:09:36] NOTICE[19322]: channel.c:2932 __ast_read: Dropping incompatible voice frame on SIP/isp-primary-00000002 of format slin since our native format has changed to 0x4 (ulaw)
[Dec  1 06:09:46] WARNING[19322]: app_fax.c:128 span_message: WARNING T.30 ECM carrier not found
   -- Executing [fax@default:3] Set("SIP/isp-primary-00000002", "CALLERIDNAME=Chicago      IL") in new stack
   -- Executing [fax@default:4] Set("SIP/isp-primary-00000002", "CALLERIDNUM=xxxxxxxxxxxx") in new stack
   -- Executing [fax@default:5] System("SIP/isp-primary-00000002", "/usr/local/bin/fax2mail -p -f "/var/spool/asterisk/fax/1259669368.2" --cid-name "Chicago      IL" --cid-number "yyyyyyyyyyyyy" --dest-name Marchenko --dest-email vlad@xxxxxxxxxx.net") in new stack
   -- Executing [fax@default:6] Hangup("SIP/isp-primary-00000002", "") in new stack
 == Spawn extension (default, fax, 6) exited non-zero on 'SIP/isp-primary-00000002'
*** glibc detected *** asterisk: double free or corruption (!prev): 0xb75041f8 ***
Comments:By: Vlad M (vlad) 2009-12-01 06:49:15.000-0600

BackTrace is attached.
By: avalon73 (avalon73) 2009-12-28 08:12:44.000-0600

I'm seeing the same crash at the same time, with the same backtrace, but only in 1.6.1.12... not in any earlier versions.  Also, it doesn't appear to happen if T.38 is used.  If I disable T.38 on the ATA, I get a message from the ReceiveFAX app that negotiation for that protocol was refused, and then the crash as the call is cleaned up.  If I enable T.38, then the fax goes through and Asterisk continues to run.
By: Tilghman Lesher (tilghman) 2009-12-28 09:13:11.000-0600

Please run with the instructions listed in doc/valgrind.txt.
By: Terry Cook (krycek) 2009-12-30 17:50:35.000-0600

I had a working 1.6.1.1 system using SpanDSP to receive faxes. Upgraded to 1.6.1.12; same issue. As soon as SpanDSP would receive a fax, Asterisk would crash. I was able to replicate the issue on both 1.6.1.10, and 1.6.1.11 as well. 1.6.1.9 on the other hand, works fine.
By: Jan Klepal (kenny) 2010-01-04 14:56:25.000-0600

I have same problem. Asterisk 1.6.0.20 segfaults with double free after receiving FAX. I tried to run it with valgrind... but it fails with following message:

FATAL: in suppressions file '/root/asterisk-1.6.0.20/contrib/valgrind.supp': location should start with 'fun:' or 'obj:'

So after I add fun: on line 24 valgrind starts correctly. However Asterisk does not segfault under valgrind. Please see attached valgrind.txt (Asterisk was recompiled with DONT_OPTIMIZE and MALLOC_DEBUG flags).
By: Olivier Krief (okrief) 2010-01-07 05:54:41.000-0600

Judging from Asterisk's changelog, app_fax.c-related differences between 1.6.1.12 and 1.6.1.9 are corrections to issues 16039, 016025 and 16127.
By: Olivier Krief (okrief) 2010-01-08 15:13:29.000-0600

I reverted this morning a running system from 1.6.1.11 to 1.6.1.9.
It used to crash for every fax call received (5 times a day).
Now it doesn't crash anymore.
By: Justin Korkiner (jkorkiner) 2010-02-02 20:27:56.000-0600

I have the same error on 1.6.1.13. I upgraded to 1.6.1.14, same issue. Reverted to 1.6.1.9, and now have stability. The system has not been turned live for production, so if you need a backtrace/valgrind please let me know.
By: Tilghman Lesher (tilghman) 2010-02-02 21:40:46.000-0600

Okay, so this patch is the result of looking at that Valgrind output.  What it should do is output to your log whenever something weird is detected in terms of the frame cache, which is what is being corrupted.
By: Anthony Bloodoff (bloodoff) 2010-02-05 04:25:02.000-0600

asterisk-1.6.1.12, spandsp-0.0.6pre17 on Fedora 9, kernel 2.6.25-14.fc9.i686, glibc-2.8-3.i686 - fax recieving forks fine
but
asterisk-1.6.1.12, spandsp-0.0.6pre17 on Fedora 12, kernel 2.6.31.12-174.2.3.fc12.i686.PAE, glibc-2.11.1-1.i686 - crahes every time on recieving faxes.
By: Jan Klepal (kenny) 2010-02-08 14:03:10.000-0600

Sorry for late reply, I was on vacation. With debug patch applied I got error frame.c: Frame already freed! Not storing.

Tried on Asterisk 1.6.0.20, spandsp 0.0.5~pre4-1 on Debian 5.0, kernel 2.6.26-2-686-bigmem, libc 2.7-18.

Please let me know if you need any other debugging or another version of Asterisk.
By: Tilghman Lesher (tilghman) 2010-02-08 16:23:05.000-0600

Okay, patch uploaded.  This should solve it.  Please test and verify.
By: Anthony Bloodoff (bloodoff) 2010-02-09 03:24:51.000-0600

after applying patch:
asterisk-1.6.1.12, spandsp-0.0.6pre17 on Fedora 12, kernel 2.6.31.12-174.2.3.fc12.i686.PAE, glibc-2.11.1-1.i686
fax receiving success, no error messages


By: Morten Isaksen (misaksen) 2010-02-09 03:36:32.000-0600

The patch works here on Asterisk 1.6.1.12

Lunux 2.6.18-164.11.1.el5xen and spandsp-0.0.6pre12
By: Jan Klepal (kenny) 2010-02-09 03:52:51.000-0600

Yes, that patch solved it (Asterisk 1.6.0.20). No error message even with the debug patch still applied.
By: Digium Subversion (svnbot) 2010-02-09 12:06:31.000-0600

Repository: asterisk
Revision: 245729

U   trunk/apps/app_fax.c

------------------------------------------------------------------------
r245729 | tilghman | 2010-02-09 12:06:30 -0600 (Tue, 09 Feb 2010) | 8 lines

Ensure frames are only freed once.

(closes issue ASTERISK-15251)
Reported by: vlad
Patches:
      20100208__issue16361.diff.txt uploaded by tilghman (license 14)
Tested by: kenny, bloodoff, misaksen

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=245729
By: Digium Subversion (svnbot) 2010-02-09 12:09:17.000-0600

Repository: asterisk
Revision: 245730

_U  branches/1.6.0/
U   branches/1.6.0/apps/app_fax.c

------------------------------------------------------------------------
r245730 | tilghman | 2010-02-09 12:09:17 -0600 (Tue, 09 Feb 2010) | 15 lines

Merged revisions 245729 via svnmerge from
https://origsvn.digium.com/svn/asterisk/trunk

........
 r245729 | tilghman | 2010-02-09 12:06:30 -0600 (Tue, 09 Feb 2010) | 8 lines
 
 Ensure frames are only freed once.
 
 (closes issue ASTERISK-15251)
  Reported by: vlad
  Patches:
        20100208__issue16361.diff.txt uploaded by tilghman (license 14)
  Tested by: kenny, bloodoff, misaksen
........

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=245730
By: Digium Subversion (svnbot) 2010-02-09 12:09:26.000-0600

Repository: asterisk
Revision: 245731

_U  branches/1.6.1/
U   branches/1.6.1/apps/app_fax.c

------------------------------------------------------------------------
r245731 | tilghman | 2010-02-09 12:09:25 -0600 (Tue, 09 Feb 2010) | 15 lines

Merged revisions 245729 via svnmerge from
https://origsvn.digium.com/svn/asterisk/trunk

........
 r245729 | tilghman | 2010-02-09 12:06:30 -0600 (Tue, 09 Feb 2010) | 8 lines
 
 Ensure frames are only freed once.
 
 (closes issue ASTERISK-15251)
  Reported by: vlad
  Patches:
        20100208__issue16361.diff.txt uploaded by tilghman (license 14)
  Tested by: kenny, bloodoff, misaksen
........

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=245731
By: Digium Subversion (svnbot) 2010-02-09 12:09:35.000-0600

Repository: asterisk
Revision: 245732

_U  branches/1.6.2/
U   branches/1.6.2/apps/app_fax.c

------------------------------------------------------------------------
r245732 | tilghman | 2010-02-09 12:09:35 -0600 (Tue, 09 Feb 2010) | 15 lines

Merged revisions 245729 via svnmerge from
https://origsvn.digium.com/svn/asterisk/trunk

........
 r245729 | tilghman | 2010-02-09 12:06:30 -0600 (Tue, 09 Feb 2010) | 8 lines
 
 Ensure frames are only freed once.
 
 (closes issue ASTERISK-15251)
  Reported by: vlad
  Patches:
        20100208__issue16361.diff.txt uploaded by tilghman (license 14)
  Tested by: kenny, bloodoff, misaksen
........

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=245732