[Home]

Summary:ASTERISK-10574: Segfault in app_voicemail, inside c-client
Reporter:Daniel White (dwhite)Labels:
Date Opened:2007-10-18 22:25:19Date Closed:2008-01-11 12:43:55.000-0600
Priority:CriticalRegression?No
Status:Closed/CompleteComponents:Applications/app_voicemail
Versions:Frequency of
Occurrence
Related
Issues:
Environment:Attachments:( 0) vms-mailstream.diff
Description:I'm encountering some segfaults while running Asterisk, about 2 times a day on a fairly low use server. It has been statically compiled with c-client.

I'm using the following versions:

Asterisk 1.4.11 (which IMAP_STORE enabled for app_voicemail)
c-client from version imap-2006k.DEV.SNAP-0709111641
I'm connecting to cyrus-imap version 2.3.8 on the same server

I should note that I've modified the app_voicemail.c file slightly, which was necessary for use with cyrus.

After each crash, Asterisk restarts automatically (using asterisk_safe), so the problem doesn't hold over through the restart.

I submitted this report on the imap-uw list, which contained the following discussion, which includes output from a core dump:

http://mailman1.u.washington.edu/pipermail/imap-uw/2007-October/001705.html

I believe there's a missing initialization of vms.mailstream

A small patch is attached.
Comments:By: Mark Michelson (mmichelson) 2007-10-19 09:15:58

Has applying that patch caused the crashes to stop?
By: Daniel White (dwhite) 2007-10-19 09:52:53

> Has applying that patch caused the crashes to stop?

I'll be applying this patch tonight during a maintenance window.

I'll let it run for a few days and let you know how it works.
By: Mark Michelson (mmichelson) 2007-11-02 14:50:01

By the way, I'm curious what you had to modify in app_voicemail.c in order for IMAP storage to work with Cyrus.
By: Daniel White (dwhite) 2007-11-05 10:56:25.000-0600

This patch did not resolve the segfaults I was seeing.

I'll try recompiling without optimization to see if that may be confusing the output from the core dump.
By: Daniel White (dwhite) 2007-11-05 11:09:29.000-0600

Concerning the modifications I've made to work with cyrus... this comment applies to 1.4.11. I'd have to take a closer look to see if it still applied to 1.4.13.

There are a couple of calls to c-client which pass a string that Cyrus does not recognize as a valid mailbox. The quotaroot command, and I believe the section which saves a message into another IMAP mailbox (such as Work).

Here are some telemetry logs from Cyrus:

---------- 9183641498 Thu Sep 20 07:08:29 2007

<1190290109<00000003 SELECT INBOX
>1190290109>* FLAGS (\Answered \Flagged \Draft \Deleted \Seen)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen \*)]  
* 1 EXISTS
* 0 RECENT
* OK [UNSEEN 1]  
* OK [UIDVALIDITY 1160847137]  
* OK [UIDNEXT 199]  
* OK [NOMODSEQ] Sorry, modsequences have not been enabled on this mailbox
* OK [URLMECH INTERNAL]
00000003 OK [READ-WRITE] Completed
<1190290109<00000004 GETQUOTAROOT "{127.0.0.1:143/imap/authuser=umsmaster/novalidate-cert/user=9183641498}INBOX"
>1190290109>00000004 NO Mailbox does not exist
<1190290109<00000005 SEARCH UNDELETED SEEN HEADER X-Asterisk-VM-Extension ""
>1190290109>* SEARCH
00000005 OK Completed (0 msgs in 0.000 secs)

----------------

Asterisk is sending "{127.0.0.1:143/imap/authuser=umsmaster/novalidate-cert/user=9183641498}INBOX" when it should be just sending "INBOX".

Also, in the case where I attempt to save a message in my INBOX into another folder:

...
00000015 OK Completed (0.000 sec)
<1190291630<00000016 COPY 1 "{127.0.0.1:143/imap/authuser=umsmaster/novalidate-cert/user=9183641498}Work"
>1190291630>00000016 NO Mailbox does not exist
>1190293430>* BYE idle for too long

I would offer a patch, but of course I've only tried this with my Cyrus server and I'm afraid of breaking things for others IMAP servers.
By: Daniel White (dwhite) 2007-11-06 12:07:49.000-0600

I apparently already had compiled Asterisk without optimization: my debian/rules build script has 'BUILDFLAGS += MAKECMDGOALS=dont-optimize' (from debian unstable).

I've added an ugly test to check to see if the value of mailstream is 0x101, and to set it to NIL before calling init_mailstream, which may have to be my fix for a while until I can figure out what's going on.

I'll report again in a few days if this is actually working in preventing the segfault.

I'd appreciate any ideas you may have on what may be going on here.
By: Mark Michelson (mmichelson) 2008-01-08 14:36:41.000-0600

I took a few hours to analyze the code and find any potential problem areas that could result in the mailstream parameter of a vm_state to be invalid. I came up with a list of possible factors:

1. Some other thread is deleting the vm_state while we are still using it. I went through the code and made sure that if we are going to access the mailstream component of the vm_state, we make sure to retrieve the persistent (meaning it will not be deleted) vm_state structure instead of a temporary "interactive" one.

2. There was some race condition being experienced during the retrieval of a vm_state from the list of vm_states. This was very possible since, though a lock for the list is defined and used in the insertion and removal of a vm_state from the list, the lock is not used when retrieving a vm_state. I have added locking to this code to be sure that the list is not compromised while we traverse it.

3. The vm_state's mailstream parameter was not properly initialized to zero. While I considered this to be a potential issue, in all cases where a vm_state was declared its memory was either calloc'd or was memset'd to 0.

4. Something in the IMAP c-client is causing the mailstream to be compromised. I can't confirm that this is happening, and even if it is, I can't do anything about it except file a bug report with the c-client developers.

While inspecting the code, I also found that the init_mailstream function returned an incorrect value in one instance, and though I have fixed this, I doubt that it caused the crash you were experiencing.

I am going to commit my changes and close this issue as resolved. If the problem recurs, then please feel free to reopen.