[Home]

Summary:ASTERISK-08350: [patch] Caller Id and Message Waiting Indicator problems
Reporter:meneault (meneault)Labels:
Date Opened:2006-12-14 08:47:22.000-0600Date Closed:2008-06-16 17:24:51
Priority:MinorRegression?No
Status:Closed/CompleteComponents:Channels/chan_zap
Versions:Frequency of
Occurrence
Related
Issues:
Environment:Attachments:( 0) 8587-callerid-c.diff
( 1) 8587-callerid-h.diff
( 2) 8587-mwicall.diff
( 3) 8587-mwicall-2.diff
( 4) 8587-mwicall-2add.diff
( 5) 8587-mwidetect.diff
( 6) callerid_c.patch
( 7) callerid_h.patch
( 8) callerid-c_patch_trunk_rev88490
( 9) callerid-h_patch_trunk_rev88490
(10) chan_zap_c.patch
(11) chan_zap-c_patch_trunk_rev88490
Description:I've got two issues but i didn't open two bugs because they are quite related.

Issue 1) In France callerid signalling is V23 and callerid start is RING. Current callerid works well except when callerid is associated to a "message waiting indicator"(mwi) message. In this case callerid spill is just ignored by chan_zap.
It is due to the fact that callerid's mdmf type is 0x80 and callerid with mwi is 0x82 (mdmf fields are the same). So the patch to callerid_feed is straight forward.

But well when a message waiting indication arrives it should'nt be treated as normal call so my patch makes the fxo channel start at 'mwi' extension if it exist.

Issue 2) Current code try to generate mwi signal to fxs channel when their mailbox state change. It is using ONHOOKTRANSFER to do so and it doesn't work for french phones (based on ETSI standard, so it mail fail for others aswell).
To work, we should ring once to wake up the phone and then transmit mwi data, no rings should follow the transmission.


I added an option to the native format request so as to ring and stop when cidspill has been sent, that's 'm' option.
ex:
Dial(Zap/1m), to ring using default cadence and stop after first ring (ie cidspill is sent).

Dial(Zap/1m1), to ring using custom cadence and stop after cidspill sent (depending on the cadence)

Message waiting indicator will be set if channel's ${HASVOICEMAIL} is set to 1.
Message waiting indicator will be cleared otherwise.


In France :
defining  cadence=250,1500,1500,3000
and then doing Dial(Zap/1m1) works well.

So when mailbox state changed instead of doing an onhooktransfer, we should Dial (Zap/1m1).

The best way I found to do this and keep backward compatibility is to launch the pbx in a specific extension.

I added an option to zapata.conf so as to decide what to do when mailbox state changed for a FXS channel (option called 'onmboxchange').
- Either 'sendmwi', default mode (for backward compatibility), to send data using onhooktransfer.
- Or 'jumptomboxchanged', to start the pbx on 'mboxchanged' extension in the channel's default context. The variable {HASVOICEMAIL} will be set to 1 if mailbox is not empty and set to 0 otherwise. Then the user is free to decide what to do when it's mailbox state changed.

Ok to sum up:
here's is my config:

zapata.conf

cadence=250,1500,1500,3000
onmboxchange=jumptomboxchanged

extensions.conf

[fxschannel] ;=> that's the extension for my Zap/1 FXS channel
; our mailbox state changed, we will send the mwi to our phones Zap/1
exten => mboxchanged,1,NoOp(Status of mailbox changed ${HASVOICEMAIL})
; timeout is 3 seconds, enough time to send the cidspill
exten => mboxchanged,2,Dial(Zap/1m1,3)
exten => mboxchanged,3,Hangup()
exten => mboxchanged,103,Hangup()

[fxochannel]
; we received mwi from telco provider
exten => mwi,1,NoOp(Got Messages Indication from telco ${HASVOICEMAIL})
; here we decide to send it to our phones
; CALLERID should be valid also if we want to check calling number
exten => mwi,2,Dial(Zap/1m1,3)
exten => mwi,3,Hangup()
exten => mwi,103,Hangup()


****** ADDITIONAL INFORMATION ******

I tested my code and it works well.
I tried to maintain backward compatibility as much as possible and normally it is guarenteed.
I tried to follow coding guidelines and respect the overall coding scheme.
Comments:By: meneault (meneault) 2006-12-14 11:55:27.000-0600

Disclaimer faxed
By: Clod Patry (junky) 2007-02-18 00:24:54.000-0600

Any interest in that patch?
By: Serge Vecher (serge-v) 2007-02-19 12:02:05.000-0600

this needs testing results from other people.
By: meneault (meneault) 2007-02-20 12:34:23.000-0600

Something that i didn't mention is that some DECT phones are explicitly designed to be used with a specific telco and won't accept incoming message waiting indication if it does not come from a specific number (some kind of authorization/access control is hardcoded on those phones).

For instance my Siemens only accepts mwi if the caller id is "3103" (that's France Telecom mwi number). Hopefully that's not the case for all French phones but i think that's worth mentioning.

So you should have in your extensions.conf:


[fxschannel] ;=> that's the extension for my Zap/1 FXS channel

; our mailbox state changed, we will send the mwi to our phones Zap/1
exten => mboxchanged,1,NoOp(Status of mailbox changed ${HASVOICEMAIL})

; I must spoof french telco provider because my DECT would deny mwi otherwise
exten => mboxchanged,2,Set(CALLERID(num)=3103)
exten => mboxchanged,3,Set(CALLERID(name)=Top Message)

; same as before send the mwi cidspill to my phones with new state
exten => mboxchanged,4,Dial(Zap/1m1,3)
exten => mboxchanged,5,Hangup()
exten => mboxchanged,105,Hangup()


-----------------------------------------------------------------------

This exception shows the need of having the 'mboxchanged' extension, in that case we are obliged to do a custom handling (on mail box state change).

-----------------------------------------------------------------------

I use my patch since december 2006 and it works nice. I am quite busy now but I can still answer questions if needed.

Finding testers for that patch will probably be hard because this patch focus on a narrow part of Asterisk users.
Mostly home users (as I am) or small companies could be interested in that patch  and they may not know how to apply a patch and test it.
By: Michiel van Baak (mvanbaak) 2007-09-05 11:02:55

those patches wont apply cleanly anymore.
Can you provide updated patches with a valid license ?
By: meneault (meneault) 2007-09-26 05:53:06

I am sorry, but I do not plan to make new patches in a near future. I am on another business right now.
I still use the SVN version 48462 at home since december 2006 and everything works  as well as it should. I do not plan to update my server for the moment.

About the licensing issue, I faxed it in december 2006, but now faxing seems to be a deprecated way. So I'll sign the Open Source Contributor License right now.
And you can fully assume that the Open Source Contributor License is exactly the license applicable to my patches.

If your really need the patch right now, it wouldn't be too difficult for the maintainer of chan_zap.c to look at my patches, read the bug description and to fix them so that they apply cleanly.

I'll get into it as soon as I need to update my home server (maybe not tomorrow). I am sure it will cost me a lot to come back into asterisk source and particularly in chan_zap.c (sorry about saying that but it's handling too much devices at the same time, updating it safely is a huge pain).

However I am open-minded so if patch is really wanted, I may change my mind.

For the moment bug didn't receive much attention, I already explained why in my previous note.
Having said so, I am still convinced that some users are desperately trying to get their message waiting indication working.
By: meneault (meneault) 2007-11-05 04:54:39.000-0600

Ok I have updated my running asterisk to the last revision (from trunk) i.e 88490.
So here are my new patches:
callerid-h_patch_trunk_rev88490 (callerid.h patch)
callerid-c_patch_trunk_rev88490 (callerid.c patch)
chan_zap-c_patch_trunk_rev88490 (chan_zap.c patch)

Please note that this is an improved version of my older patch (better code quality and design).
One example of this is that I moved out the mailbox change code (from do_monitor ) into a separate inline function for code clarity.

As ever my patch is meant to be backward compatible (i.e. no regression).
By: Jason Parker (jparker) 2007-12-04 18:07:32.000-0600

There was something very similar to this added to svn trunk today.  It takes a different approach (it doesn't start a pbx at the mwi exten - it instead runs an external script).

In the simplest case, I believe you can have an fxo and fxs point to the same mailbox, and when mwi is detected on the fxo, the fxs should automatically light up.

To be perfectly honest, I'm very surprised that people haven't tested this - it seems like a very useful feature (which is why Mark wrote something similar - he didn't even know this existed).


Would you be willing to try out the code that was added to trunk, and respond here if there is anything that your patch does that it cannot do (or if you see any issues with it)?  Without looking too far into what was committed, I believe that yours had the ability to send the mwi cid spill, while this doesn't - is that something you think would be useful to have?
By: meneault (meneault) 2007-12-05 08:59:14.000-0600

Thanks for the information qwell (I was not aware that someone else was working on the issue neither :) ).
I will test it as soon as I can. However Please note that I have already overlooked the code and it wouldn't probably work as I need too.

Apparently, the mwi monitoring is made only during pre-ring phase (channel is free) not during ringing phase. A small add-on could however make the trick
(testing flags when callerid_feed is called during ringing and then deciding or not to launch the pbx).

The same problem arise for lighting up an FXS phone, the generation is still made without ringing phase and so the phone won't react and won't light up.
It is only a matter of standard, I should point out ETSI TS 300 659-1 which is an european standard describing on-hook data transmissions. It may occur during a pre-ring phase or during a ringing phase.
In France we need a ring to send on-hook data otherwise it is ignored, and France Telecom is still stricter and states that no ring should follow the data transmission when this is a mwi message.
And some DECT phones are still stricter and only accept them when the caller identification is 'Top Message'-3103.

That's why for the generation part my code is still needed, custom handling is clearly needed to generate the MWI message for FXS (I guess that in the US no ringing is needed to light up phones).

So to sum up:
- for detection part, some fix will have to be added for the ringing case.
- for the generation part, my code is still needed.
By: Jason Parker (jparker) 2007-12-05 14:58:11.000-0600

I just uploaded a patch that I believe might come close to what you need.  It doesn't support callerid in the vmwi message yet, but that's fairly trivial to add.

You mentioned two issues - the detection and the generation of post-ring mwi messages.  I couldn't find anything in your code that modified the detection in the way you specified, but I believe I did get enough of the generation to make it testable.

Basically, it always monitors for new mwi messages, and when it detects one (on or off), it will run a script that you specify in mwimonitornotify.  In that script, you could parse the mailbox, and dial the appropriate fxs port.  In the simplest case, the script could drop a call file which just dialed Zap/1m.  It would see the m, realize that it needs to only send mwi for that call, then hang up afterwards.


This code is entirely untested, but I think most of the logic is there - a lot of which was copied from your patches.

Did I get anywhere near the requirement?
By: meneault (meneault) 2007-12-06 16:11:34.000-0600

> I couldn't find anything in your code that modified the detection in the way >you specified, but I believe I did get enough of the generation to make it >testable.
>
>Basically, it always monitors for new mwi messages

When reading the patch added in trunk, what I see is that mwi is only checked when the channel has no owner. So I fear it won't work for the case of ringing. Because when a ring arrives, do_monitor calls handle_init_event which creates a new asterisk channel owning the FXO channel. Then do_monitor won't listen again for mwi on this channel because it is owned.

To correct this I think we should only check for mwi when usual callerid is expected (as my patch did). And then take the appropriate action when callerid appears to be a mwi message.
See 8587-mwidetect.diff
You'll see what I mean, this is only an example but it should work.
Please note that this patch would be less flexible than my orginal patch. My original patch let the pbx run into 'mwi' extension, this one does not run the pbx when callerid appears to be a mwi message.

>In the simplest case, the script could drop a call file which just dialed >Zap/1m. It would see the m, realize that it needs to only send mwi for that >call, then hang up afterwards.
Yes this is what I need but I still have the need of having distinctive ring on 'm' option too. e.g. Zap/1m2
See 8587-mwicall-2.diff

>Did I get anywhere near the requirement?
We are close but... still two issues that my patch handled also.

- The patch added in trunk does not handle the FXS issue only the FXO one.
When do_monitor notices that a FXS channel has got voicemail it does an on-hook transfer, something that would only work for US phones... (Because no ring is generated).
So the other part of my patch is still needed, an option is needed in zaptel to specify either we want an on-hook transfer or do a custom handling (my original patch launched the pbx on mboxchanged extension but we may change this by a call to a script if it is preferred).

- and adding the proper generation of wmi data of course (callerid information).
By: Jason Parker (jparker) 2007-12-06 17:47:00.000-0600

I've incorporated your 2 new patches.  I also plan on adding callerid to the mwi packet (should I do that in all cases?  will some phones see the callerid info and freak out?)

I didn't understand your last paragraph.  What do you mean the FXS case?  What purpose does the on-hook transfer serve?
By: meneault (meneault) 2007-12-07 15:32:00.000-0600

>I've incorporated your 2 new patches.

This sounds great -- I will try to find time this week-end to test them to be sure everything work as expected -- they look correct from a logical point of view, it shouldn't prevent us to test them though :)
My last patch on revision 88490 work thus this should work as well.

>I also plan on adding callerid to the mwi packet (should I do that in all cases?
>will some phones see the callerid info and freak out?)

This is a good question, I will try to answer with all my knowledge. It depends on international standards, unfortunately I don't know all of them (however if you can point me to some US standards about this I would be happy to study them :)). More than that it depends on telco providers which add some restrictions on these standards and then at the end of the chain are phone manufacturers which still have some freedom on the implementation of these standards.
Another issue is that international standards emerged after telco standards and thus some telco standards are not complying with later standards.

What I know about is that we have two possibilities to send mwi message, either using SDMF format (old Bellcore format) and MDMF format.
Current implementation in callerid.c propose the two, however only MDMF format is apparently used (in chan_zap.c).
So we may forget about the SDMF format, moreover it cannot represent caller identification along with mwi.

For MDMF we may choose which fields to generate for mwi, current implementation   only generate 0x0B parameter which is the visual indicator, ETSI standard says that this is the only mandatory parameter others like Date/Time and Caller Number/Name are optional (0x01, 0x02, 0x07).
France Telecom standard states that Date and Time parameter is required, and my Siemens DECT phone only accepts mwi if Caller Number is present and equals to '3103' and Caller Name is present and equals to 'Top Message'.

Now to answer your question, normally adding caller number/name to each message
would not cause any trouble according to ETSI.
However we cannot be sure that some old standard didn't state that MWI MDMF format would only contain Visual Indicator parameter. I am quite confident that it is not the case because MWI MDMF appeared after CLID MDMF and thus every  phone compliant with the latter should accept MWI with CLID added (even if it only ignores it).

Anyway when you look at my patch (callerid-c_patch...) I decided to let both modes (MDMF FULL with Date/Time/Number/Name/Visual and MDMF with Visual Indicator only).
- When calling with Zap/1m1 then MDMF FULL is chosen.
- When this mwi is generated from do_monitor MDMF is chosen (Moreover from do_monitor thread caller id cannot be fetched and thus cannot be added).

It is up to you to decide what's the best solution, with mine you are sure that we get backward compatibility.

>I didn't understand your last paragraph. What do you mean the FXS case? What >purpose does the on-hook transfer serve?

I will try to better explain the issue.

First the FXO case, you will agree that it detects MWI message on a FXO line and then executes a script -- that's what was added in trunk.
(It could for instance, as you said, light up a FXS phone using Zap/1m1).

But what about the FXS case? a mailbox is usually assigned to a FXS channel, when there are messages in the mailbox and the channel is not used then the phone should be warned of this.
In current implementation, chan_zap.c notices that some messages are in the mailbox, if channel is free then it decides to light up the phone(s) connected to this FXS channel sending mwi data.
The method used is an on hook transfer (from chan_zap.c):
ioctl(p->subs[SUB_REAL].zfd, ZT_ONHOOKTRANSFER, &x);

An on-hook transfer is nothing more than a data communication when the channel is on-hook, in this case mwi message is sent while the phone is on-hook. In theory this is what is needed.

Unfortunately it won't work for me as this hook transfer is not preceded by a ring (and caller id not present...). Thus instead of doing this on-hook transfer I would need some custom generation of mwi (like Zap/1m1).

In my patches I added an option onmboxchange which handles it:
- either sendmwi which is the default mode -- on-hook transfer--
- or jumptomboxchanged which launch the pbx in mboxchanged extension -- for custom generation --
(take a look at handle_mbox_changed in chan_zap-c_patch_trunk_rev88490)

Instead of launching the pbx we could do the same as what was done for FXO and call a script.
In zapata.conf we should have something like

; option for a fxs channel
onmboxchange=sendmwi
(default mode only on-hook transfer)

or

; option for a fxs channel
onmboxchange=path/to/script
(calls this script when the states of its mailbox changed)

Hope this help...
By: Jason Parker (jparker) 2007-12-07 15:48:16.000-0600

Yes, that helps immensely.  I was definitely misunderstanding the issue.

I realize now that you're saying an FXS with a voicemail *in asterisk*.  I was thinking something completely different.

It's a hack, but you can do this now using the externnotify functionality of app_voicemail - you could do basically the same thing I mentioned with the FXO case, of just dropping a call file that dials Zap/1m - but, like I said, it's a hack.  I'll think about ways this can be done better and perhaps more transparently.

As for the MWI with CallerID - I will likely use with the MDMF_FULL method you proposed.
By: meneault (meneault) 2007-12-09 16:27:13.000-0600

I am back again...

Ok so I did some tests today, and it works.
I used:
8587-mwicall-2.diff
8587-mwidetect.diff

And as I needed MWI generation with callerid I added these 3 new patches:
8587-callerid-h.diff
8587-callerid-c.diff
8587-mwicall-2add.diff

So I applied these 5 patches.

Detection works perfectly on my FXO line.
And Generation for my FXS phones works perfectly too.
So the FXO problem with these 5 patches is solved.

For the FXS issue, I tested with externotify of voicemail.conf and it works well too.

However please note that I had to remove the code that does the on-hook transfer from chan_zap.c otherwise it generates some strange events from zaptel (event 160 -- does not even exist AFAIK).


So for the FXS I still think that an option would be good:
onmboxchange=sendmwi|none|script

; only valid if onmboxchange=script
mboxchangenotify=/path/to/script
By: Doug Bailey (dbailey) 2008-05-22 11:17:22

I have added a development branch (http://svn.digium.com/view/asterisk/team/dbailey/mwi_rpas/) that takes many of the changes originally suggested with some changes I have added.

Among the changes I made:
- I moved the sending of MWI messages into its own thread and added a state machine to sending it.
(Previously it was incorporated into the do_monitor thread)

- I changed the ring pulse duration to match the etsi spec.

- I added a configuration element (mwisendtype) that can be set to "rpas" to initiate a ring pulse alert signal
By: meneault (meneault) 2008-06-01 15:53:20

I just checked out mwi_rpas branch.

But it doesn't seem to work => even with mwisendtype=rpas...
My phones don't even react no ring generated.

When reading the code mwi_send_thread, an on-hook transfer is attempted anyway and in my case we shouldn't. I removed this on hook transfer but that doesn't fix the issue.

Does that work for you?
By: Doug Bailey (dbailey) 2008-06-13 15:24:57

I found a missing break in my patch and did a little more testing.  I have recorded output from the second patch and have seen the CID spill on the incoming.
By: meneault (meneault) 2008-06-15 02:43:32

This is it! I just tested it and it appears to be working!
I am just waiting for this branch to be merged in trunk and finally close this issue!
Thanks
By: Digium Subversion (svnbot) 2008-06-16 16:35:59

Repository: asterisk
Revision: 123203

U   trunk/channels/chan_dahdi.c
U   trunk/include/asterisk/callerid.h
U   trunk/main/callerid.c

------------------------------------------------------------------------
r123203 | dbailey | 2008-06-16 16:35:59 -0500 (Mon, 16 Jun 2008) | 8 lines

Clean up code that handles fsk mwi message generation by pulling it from do_monitor and creating its own thread.
Added RP-AS mwi message generation using patches from meneault as a basis.

(closes issue ASTERISK-8350)
Reported by: meneault
Tested by: meneault


------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=123203
By: Digium Subversion (svnbot) 2008-06-16 17:24:51

Repository: asterisk
Revision: 123237

_U  team/group/newcdr/
U   team/group/newcdr/channels/chan_dahdi.c
U   team/group/newcdr/include/asterisk/callerid.h
U   team/group/newcdr/main/callerid.c

------------------------------------------------------------------------
r123237 | murf | 2008-06-16 17:24:50 -0500 (Mon, 16 Jun 2008) | 21 lines

Merged revisions 123201,123203 via svnmerge from
https://origsvn.digium.com/svn/asterisk/trunk

........
r123201 | tilghman | 2008-06-16 15:31:43 -0600 (Mon, 16 Jun 2008) | 2 lines

Oopsie, breakage

........
r123203 | dbailey | 2008-06-16 15:42:46 -0600 (Mon, 16 Jun 2008) | 8 lines

Clean up code that handles fsk mwi message generation by pulling it from do_monitor and creating its own thread.
Added RP-AS mwi message generation using patches from meneault as a basis.

(closes issue ASTERISK-8350)
Reported by: meneault
Tested by: meneault


........

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=123237