[Home]

Summary:ASTERISK-13495: Background leaves files open indefinately
Reporter:Fredrik Liljegren (fiddur)Labels:
Date Opened:2009-02-02 04:57:58.000-0600Date Closed:2009-02-04 13:45:17.000-0600
Priority:BlockerRegression?No
Status:Closed/CompleteComponents:Core/PBX
Versions:Frequency of
Occurrence
Related
Issues:
Environment:Attachments:( 0) 14384.patch
( 1) debugging
Description:After a Background(mysound) the sound file stays open, according to lsof.

This leads after a while to a total crippling of asterisk since it can't open any socket, getting "Too many open files" or "Can't create alert pipe!".

****** ADDITIONAL INFORMATION ******

I set category Core/PBX since BackGround is builtin and is in main/pbx.c.
Comments:By: Fredrik Liljegren (fiddur) 2009-02-03 11:02:23.000-0600

If anyone else is having this problem in a critical environment (yes, I made too few tests before upgrading my production server...), the temporary fix seems to be to raise number of allowed open files and restart asterisk daily or so... I added this to /etc/init.d/asterisk:
       ulimit -Hn 100000
       ulimit -n 100000
By: Mark Michelson (mmichelson) 2009-02-03 16:22:21.000-0600

I know what's wrong here and I have created a fix for this. I'll upload a patch shortly.
By: Mark Michelson (mmichelson) 2009-02-03 16:24:16.000-0600

I've uploaded 14384.patch. The fix is so simple that it almost feels too simple, but I understand why it was needed and why the problem did not manifest itself earlier. Please give it a sanity check to be sure that the error is cleared up. Thanks!
By: Mark Michelson (mmichelson) 2009-02-03 16:33:28.000-0600

I forgot to switch the status to "ready for testing" when I uploaded the patch.
By: Fredrik Liljegren (fiddur) 2009-02-04 00:24:59.000-0600

Yes, it works great!  Thank you VERY much, GREAT work!
By: Mark Michelson (mmichelson) 2009-02-04 09:23:08.000-0600

Thanks for the feedback. I'll get the patch merged ASAP.
By: Digium Subversion (svnbot) 2009-02-04 09:30:13.000-0600

Repository: asterisk
Revision: 173354

U   trunk/main/file.c

------------------------------------------------------------------------
r173354 | mmichelson | 2009-02-04 09:30:13 -0600 (Wed, 04 Feb 2009) | 30 lines

Fix a problem where file playback would cause fds to remain open forever

The problem came from the fact that a frame read from a format interpreter
was not freed. Adding a call to ast_frfree fixed this. The explanation for
why this caused the problem is a bit complex, but here goes:

There was a problem in all versions of Asterisk where the embedded frame
of a filestream structure was referenced after the filestream was freed. This
was fixed by adding reference counting to the filestream structure. The refcount
would increase every time that a filestream's frame pointer was pointing to an
actual frame of data. When the frame was freed, the refcount would decrease. Once
the refcount reached 0, the filestream was freed, and as part of the operation,
the open files were closed as well.

Thus it becomes more clear why a missing ast_frfree would cause a reference leak
and cause the files to not be closed. You may ask then if there was a frame leak
before this patch. The answer to that is actually no! The filestream code was
"smart" enough to know that since the frame we received came from a format interpreter,
the frame had no malloced data and thus didn't need to be freed. Now, however, there
is cleanup that needs to be done when we finish with the frame, so we do need to
call ast_frfree on the frame to be sure that the refcount for the filestream is
decremented appropriately.

(closes issue ASTERISK-13495)
Reported by: fiddur
Patches:
     14384.patch uploaded by putnopvut (license 60)
Tested by: fiddur, putnopvut


------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=173354
By: Digium Subversion (svnbot) 2009-02-04 09:30:55.000-0600

Repository: asterisk
Revision: 173355

_U  branches/1.6.0/
U   branches/1.6.0/main/file.c

------------------------------------------------------------------------
r173355 | mmichelson | 2009-02-04 09:30:55 -0600 (Wed, 04 Feb 2009) | 38 lines

Merged revisions 173354 via svnmerge from
https://origsvn.digium.com/svn/asterisk/trunk

........
r173354 | mmichelson | 2009-02-04 09:30:12 -0600 (Wed, 04 Feb 2009) | 30 lines

Fix a problem where file playback would cause fds to remain open forever

The problem came from the fact that a frame read from a format interpreter
was not freed. Adding a call to ast_frfree fixed this. The explanation for
why this caused the problem is a bit complex, but here goes:

There was a problem in all versions of Asterisk where the embedded frame
of a filestream structure was referenced after the filestream was freed. This
was fixed by adding reference counting to the filestream structure. The refcount
would increase every time that a filestream's frame pointer was pointing to an
actual frame of data. When the frame was freed, the refcount would decrease. Once
the refcount reached 0, the filestream was freed, and as part of the operation,
the open files were closed as well.

Thus it becomes more clear why a missing ast_frfree would cause a reference leak
and cause the files to not be closed. You may ask then if there was a frame leak
before this patch. The answer to that is actually no! The filestream code was
"smart" enough to know that since the frame we received came from a format interpreter,
the frame had no malloced data and thus didn't need to be freed. Now, however, there
is cleanup that needs to be done when we finish with the frame, so we do need to
call ast_frfree on the frame to be sure that the refcount for the filestream is
decremented appropriately.

(closes issue ASTERISK-13495)
Reported by: fiddur
Patches:
     14384.patch uploaded by putnopvut (license 60)
Tested by: fiddur, putnopvut


........

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=173355
By: Digium Subversion (svnbot) 2009-02-04 09:31:26.000-0600

Repository: asterisk
Revision: 173356

_U  branches/1.6.1/
U   branches/1.6.1/main/file.c

------------------------------------------------------------------------
r173356 | mmichelson | 2009-02-04 09:31:26 -0600 (Wed, 04 Feb 2009) | 38 lines

Merged revisions 173354 via svnmerge from
https://origsvn.digium.com/svn/asterisk/trunk

........
r173354 | mmichelson | 2009-02-04 09:30:12 -0600 (Wed, 04 Feb 2009) | 30 lines

Fix a problem where file playback would cause fds to remain open forever

The problem came from the fact that a frame read from a format interpreter
was not freed. Adding a call to ast_frfree fixed this. The explanation for
why this caused the problem is a bit complex, but here goes:

There was a problem in all versions of Asterisk where the embedded frame
of a filestream structure was referenced after the filestream was freed. This
was fixed by adding reference counting to the filestream structure. The refcount
would increase every time that a filestream's frame pointer was pointing to an
actual frame of data. When the frame was freed, the refcount would decrease. Once
the refcount reached 0, the filestream was freed, and as part of the operation,
the open files were closed as well.

Thus it becomes more clear why a missing ast_frfree would cause a reference leak
and cause the files to not be closed. You may ask then if there was a frame leak
before this patch. The answer to that is actually no! The filestream code was
"smart" enough to know that since the frame we received came from a format interpreter,
the frame had no malloced data and thus didn't need to be freed. Now, however, there
is cleanup that needs to be done when we finish with the frame, so we do need to
call ast_frfree on the frame to be sure that the refcount for the filestream is
decremented appropriately.

(closes issue ASTERISK-13495)
Reported by: fiddur
Patches:
     14384.patch uploaded by putnopvut (license 60)
Tested by: fiddur, putnopvut


........

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=173356