Summary:ASTERISK-28567: Problem with ASTERISK-20207: Asterisk should clear out any .lock files in the voice mail directory on startup.
Reporter:Michael (ringo)Labels:
Date Opened:2019-10-05 10:35:24Date Closed:2019-10-10 12:32:01
Versions:13.28.0 16.5.0 Frequency of
is the original version of this clone:ASTERISK-20207 Asterisk should clear out any .lock files in the voice mail directory on startup.
Environment:debian9 realtime odbc mysqlAttachments:
Description:While attempting upgrade from vcertified-13 to v13.28.1 or to v16.5.1, asterisk takes a really long time to start.  It stops after loading app_voicemail.  No logging or debug indicates what is wrong.  While strace'ing the pid, i see asterisk is traversing the voicemail directory.  This lead me to ASTERISK-20207.  Clear out any lock files on startup.  This probably works fine with a small amount of mailboxes, but with 7,500+ mailboxes, it takes up to 30 minutes to start.

after commenting out the call in :
static int load_module(void)
       /* Now that we have a spool directory, clean up old lock files */
       /* cleanup_orphaned_lock_files(VM_SPOOL_DIR); */
and recompiling, asterisk starts up right away.
Comments:By: Asterisk Team (asteriskteam) 2019-10-05 10:35:26.070-0500

Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution.

A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report.

Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].

Please note that once your issue enters an open state it has been accepted. As Asterisk is an open source project there is no guarantee or timeframe on when your issue will be looked into. If you need expedient resolution you will need to find and pay a suitable developer. Asking for an update on your issue will not yield any progress on it and will not result in a response. All updates are posted to the issue when they occur.

By: Sean Bright (seanbright) 2019-10-07 07:43:55.205-0500

I'll take a look. 30 minutes for 7500 mailboxes seems unusually high.

By: Sean Bright (seanbright) 2019-10-07 08:09:15.175-0500

[~ringo], is your voicemail spool directory on some kind of network-attached storage?

By: Michael (ringo) 2019-10-07 09:29:31.293-0500

I could time it to get more accurate time.  30 minutes was more of a guess.  Also, in my setup the mailboxes are across an NFS on a 2nd system.

By: Michael (ringo) 2019-10-07 12:24:12.627-0500

patch applied.  no change.  startup still halts at  Loading app_voicemail.so.

I don't think this is a good option for my setup since I have an active and hot spare setup where the voicemail is on a remote share, via NFS, and accessible to both machines at all times.  Wouldn't a restart of the hot spare system be removing lock files that the active system has created?

By: Michael (ringo) 2019-10-07 12:35:20.617-0500

I timed it this time.  17 minute startup time.

By: Michael (ringo) 2019-10-07 12:46:46.703-0500

would it be a good idea to maybe keep track of all lock files created in the astdb, and clear after lock is removed?  And then upon startup, only remove lock files that are in the astdb?  This sounds like it would save traversing the whole directory tree, and also would prevent an instance of asterisk removing lock files that have been created by another instance.

By: Sean Bright (seanbright) 2019-10-07 12:47:38.216-0500


By: Sean Bright (seanbright) 2019-10-07 12:49:04.065-0500

[~ringo], I don't hate that idea. Is that something you could implement?

By: Michael (ringo) 2019-10-07 12:50:17.665-0500

no sorry. not that familiar with this kind of code.  was just an idea.  I sure would love to help though.

By: Friendly Automation (friendly-automation) 2019-10-10 10:02:48.351-0500

Change 13020 merged by Friendly Automation:
Revert "app_voicemail: Cleanup stale lock files on module load"


By: Friendly Automation (friendly-automation) 2019-10-10 10:08:30.176-0500

Change 13019 merged by Friendly Automation:
Revert "app_voicemail: Cleanup stale lock files on module load"


By: Friendly Automation (friendly-automation) 2019-10-10 10:09:38.606-0500

Change 13018 merged by Friendly Automation:
Revert "app_voicemail: Cleanup stale lock files on module load"


By: Michael (ringo) 2019-10-10 11:49:16.668-0500

could have maybe made it an option to enable/disable?  It was easy to comment out the call for this process as shown in description above.

By: Asterisk Team (asteriskteam) 2019-10-10 11:49:17.050-0500

This issue has been reopened as a result of your commenting on it as the reporter. It will be triaged once again as applicable.

By: Sean Bright (seanbright) 2019-10-10 12:32:01.350-0500

There are too many edge cases to consider that we felt it was better to just remove it altogether.

By: Friendly Automation (friendly-automation) 2019-10-10 13:48:07.652-0500

Change 13017 merged by George Joseph:
Revert "app_voicemail: Cleanup stale lock files on module load"