Summary:	ASTERISK-01695: [patch] Caller ID support for UK (BT POTS lines)
Reporter:	Tony Hoyle wibble wibble wibble (tonyhoyle)	Labels:
Date Opened:	2004-05-26 10:13:53	Date Closed:	2011-06-07 14:05:07
Priority:	Major	Regression?	No
Status:	Closed/Complete	Components:	Core/General
Versions:		Frequency of Occurrence
Related Issues:
Environment:		Attachments:	( 0) ast-polarity-UK-cid.patch ( 1) ast-polarity-UK-cid-sampleconf.patch ( 2) ast-UK-and-DTMF-pol-CID.diff
Description:	This is an experimental patch to add caller id support for UK POTS lines. The zaptel side of the patch just adds a 2 second rolling buffer that the userspace driver can interrogate.  As the caller ID information in the UK is sent before the first ring the CID burst will be there rather than within the ring data itself as on the Bell system. The asterisk side of the patch just adds v23 support to fskmodem and grabs the saved buffer and passes it through the callerid state machine.   Selection of the UK caller id format is currently by a 'ukcallerid=yes' line.  This should probably be broadened out into a generic CID selection for things like DTMF, etc.   At the moment I'm just encouraging testing... it seems to work for me but haven't had much feedback yet. ****** ADDITIONAL INFORMATION ****** <removed links to patches not handled through electronic license agreement system>
Comments:	By: Mark Spencer (markster) 2004-05-26 10:49:21 I don't like the "history" idea, but I"ll give you an event shortly which will be polarity reversal in zaptel, this is a great start! By: Tony Hoyle wibble wibble wibble (tonyhoyle) 2004-05-26 11:10:51 Mostly I was trying to avoid having to do things like detect guard tones etc.  - to detect 'bogus' polarity reversals you have to watch for a dual 2130/2750hz tone ... not really sure how to do that (although I guess I could work it out if pushed). When I first wrote it I considered it to be a hack, but after using it for a bit it's not that bad... it helps with DTMF systems too as I think they don't have such a reversal at the beginning. By: flavour (flavour) 2004-05-28 03:12:54 FYI: Tony is working on getting the disclaimer sent in By: Tony Hoyle wibble wibble wibble (tonyhoyle) 2004-05-28 17:46:14 Updated patch as it broke distinctive ring detection (which I was unaware was available in the UK).   Several people appear to be using the patch successfully, but the latest changes need testing etc. as I don't have DR to test with (I've confirmed at least that nothing breaks in the existing setup with the new patch). By: thorran (thorran) 2004-05-29 11:47:03 CallerID information is also sent with Call waiting - during a call - and is signalled by the same line reversal, so triggering on a line reversal rather than a buffer preceeding the ring would be the only way to correctly handle this feature. By: cursor_software (cursor_software) 2004-05-29 12:49:35 The patch works for me - including distinctive ring support.  I don't have "call waiting" on the BT line, so I can't test that. The X101P doesn't have hardware detection for line reversal, which is the whole point of the patch and the new "history" feature provided by the Zaptel patch.  The new FXO modules apparently have line reversal detection in hardware, but that'd mean throwing out the X101P (along with the entire server) and buying brand new hardware.  I prefer the hstory approach. :-) By: Tony Hoyle wibble wibble wibble (tonyhoyle) 2004-05-29 13:19:53 There is someone working on DTMF support too, and that requires the history approach as there is no wakeup signal or line reversal in that spec - this means that it's going to have to go in anyway eventually so we might as well use it now. Adding support for line reversal and scanning for CID then is possible but is outside the remit of this patch as it would require a lot of extra code have guard tone detection, plus a possible restructuring of the existing code anyway as you'd be trying to handle CID *during* a call - that's a pretty nontrivial addition.  Not to mention I don't have access to hardware which detects it anyway, so it'd be a different patch by someone else.  For now this works, and adds a much needed feature. By: Mark Spencer (markster) 2004-05-30 01:42:13 This history thing still seems like a terrible idea.  Why aren't we just reading from it?  If you wanted to support DTMF we could still do the zt_read on the channel we're waiting for. Anyway I still owe you a polarity reversal event.  It's coming, really :) By: Tony Hoyle wibble wibble wibble (tonyhoyle) 2004-05-30 21:46:15 chan_zap doesn't seem to begin reading from the channel until it gets an event (if you were reading constantly the CPU usage would probably suck).  If there was somewhere that was called for every single incoming frame then I could benchmark it but I couldn't find anything like that (it might be possible to move the history to userspace though as it's just a store/increment.  Calling fskmodem for every single frame even silence is going to drag the processor badly). By: Mark Spencer (markster) 2004-05-30 23:53:27 So why not allow zaptel to just generate an event when a sample is detected that is above a certain threshold.  i.e. int x = 100; /* Request notification when there is a sample above my threshold */ ioctl(fd, ZT_READNOTIFY, &x); then when zaptel sees a linear "x" where abs(x) >= 100, then it generates an event (just once, until ZT_READNOTIFY is called).  Since the event would be called either exactly when, or somewhat before the buffer was ready for reading, you'd be guaranteed to be able to read that buffer as long as you reacted reasonably quickly. This would require almost no extra memory and very little extra code By: Tony Hoyle wibble wibble wibble (tonyhoyle) 2004-05-31 14:08:39 I've just spent an hour trying to do that and basically given up. Firstly, it breaks ring detection - it seems any event that fires prior to a ring breaks the ring detect routines badly... ring detection just stops working. I also found that even if I forced it to pickup when I knew it was really ringing, the speech became garbled as it's making an extra roundtrip to userspace for every single frame. TBH I'm going to have to leave this to someone else to work out, as I don't have the time to get involved in a major development effort - I've spent more than I wanted to already on this.  I have a patch that works... until someone comes along to do it 'properly' I'll stick with that.  Sorry I can't be more help. By: Mark Spencer (markster) 2004-05-31 15:27:09 No need to be sorry, you've done a great job.  Just be sure we have a disclaimer on file so someone else can pick up where you've left off and we'll get it in place. By: Tony Hoyle wibble wibble wibble (tonyhoyle) 2004-05-31 17:13:47 I faxed a disclaimer yesterday so digium should have it now. By: nkans (nkans) 2004-06-17 11:55:56 I tested this patch with my asterisk with x100p and could not see the caller id. here is the verbose, -- Starting simple switch on 'Zap/1-1' Jun 17 17:42:20 NOTICE[245776]: chan_zap.c:4811 ss_thread: Got event 2 (Ring/Answered)... Jun 17 17:42:23 NOTICE[245776]: chan_zap.c:4811 ss_thread: Got event 2 (Ring/Answered)...    -- Executing MySQLput("Zap/1-1", "cid/cid=") in new stack    -- mysqlput: family=cid, key=cid, value=    -- Executing Dial("Zap/1-1", "SIP/12345|20|tr") in new stack    -- Called 12345    -- SIP/12345-342e is ringing  == Spawn extension (default, s, 2) exited non-zero on 'Zap/1-1'    -- Hungup 'Zap/1-1' the value is empty. Any help on this. Here is my extensions.conf: exten => s,1,MySQLPut(cid/cid=${DNID}${CALLERID}${EXTEN}${CALLERIDNUM}${CALLERIDNAME}) exten => s,2,Dial(SIP/12345,20,tr) Can anyone help plz. By: robpow (robpow) 2004-06-18 19:07:43 I think the correct syntax for zapata.conf is: usecallerid=uk and not ukcallerid=yes Cheers, Matt By: senad (senad) 2004-06-21 22:11:51 is this patch in CVS? By: Mark Spencer (markster) 2004-06-21 23:58:58 No and won't be until several changes are made, but in the mean time it's a good starting place for anyone wanting to continue the work.   I think it's easiest to wait until we have polarity detection on FXO modules and then from there, we'll get the rest. By: cursor_software (cursor_software) 2004-06-22 00:19:07 The ability to detect a polarity reversal on the new FXO modules won't help X100P users who would like to use this facility.  I vote that the changes are added as-is.  If we get a polarity reversal event to play with in the future then it can be merged into the code with a couple of #ifdef OLD_X100P (or whatever) directives. In the mean time, I've had these patch in my system for ages and haven't had to deal with even one CVS collision.  My suggestion would be for others to do the same until there's something similar in CVS. By: Mark Spencer (markster) 2004-06-22 01:10:51 My desire to keep the implementation clean and efficient is stronger than my desire to support the feature on the X100P.  However, we'll have this patch (and presumably a diff agianst the version working with the polarity reversal deteection) for those who really want it. By: pg_king (pg_king) 2004-07-05 05:53:12 What's the status of this? Anyone working on it? I've done something similar (before finding this...) except didn't update the kernel driver. What's the objection to the history? I am maintaining a circular buffer for the devices which need it whilst they are on hook, as soon as ringing starts it stops recording. The overhead of this is actually very low (it's only shifing 16000 bytes per second per device). I'll happily put some effort into getting this into a production ready state. By: twisted (twisted) 2004-07-23 20:43:46 Friendly reminder from housekeeping.... By: dant (dant) 2004-08-05 19:02:47 FYI... If anyone's interested, I've got a bit of a variation of the asterisk patch working for BT CLI using the polarity reversal detection in the TDM400 and the new polarity reversal event in zaptel... I need to clean up the code a little, then I'll post the patch here, it could do with some more checking in it too, but... By: dant (dant) 2004-08-08 17:46:54 OK Folks, I've just attached some patches for UK callerID... these will not work if you have an X100P as it doesn't have the polarity reversal detection... There are two patches, should only be one, but I missed the sample config change, there is a new config option so it really should be there... I'm working on this with a TDM400 with FXO module... It works for me at this point, I don't have distinctive ring, so I've not been able to test that that still works... By: dant (dant) 2004-08-15 18:25:24 Argh... Small issue with the polarity reversal patch... BT's daily line test seems to trigger a polarity reversal which in this case doesn't signify the start of a CID stream... I'll have a look to see if I can make it check for the guard tone on polarity reversal before it accepts it as a call start... By: wrs (wrs) 2004-08-16 08:56:14 I've tested this patch with a TDM400P, two FXO modules and two BT lines (one with ADSL). Seems to work very well on both -- much better than the previous patches did. Not been running for long enough to see if the automatic line tests cause phantom calls. edited on: 09-08-04 06:56 By: dant (dant) 2004-08-31 20:01:53 New patch against current CVS, merged with the DTMF patch (see bug 9) utilising some logic from that to kill the phantom call problem on BT tests that include polarity reversals... Initial testing seems to work OK, but, needs more testing, especially distinctive ring which I hope will work, but have no way of testing... again, this won't be much use to folks with X100Ps... By: wrs (wrs) 2004-09-08 06:56:30 Tested this with my setup. Works excellently, phatom calls caused by automated line testing appear to have completely stopped. Thanks! By: quietlife (quietlife) 2004-09-20 14:36:55 Is there a version of the patch for the X100P cards available against the current CVS head ? By: cursor_software (cursor_software) 2004-09-20 17:42:33 Yes - contact me directly if you want it. By: quietlife (quietlife) 2004-09-22 03:13:01 : cursor_software Yes please - cannot seem to see a way of emailing you from here - please contact me via : mentor_of_arisia (AT) hotmail.com By: Ben Brown (benbrown) 2004-09-25 03:34:35 Now that dant's UK CLID patches are in CVS is it possible for someone to post a patch to get the X100P working.  Cheers. By: cursor_software (cursor_software) 2004-09-26 14:37:19 I have a patch (against the latest Asterisk CVS version) that provides support for the following zapata.conf configuration, and therefore support for the X100P:    usecallerid = yes    cidsignalling = v23    cidstart = history I've been asked to not upload my patches to this bug tracker.  If you want this patch then you can email me at kevin@cursor.biz. By: Brian West (bkw918) 2004-09-26 15:13:32 Why have you been asked not to upload them? By: cursor_software (cursor_software) 2004-09-26 16:15:36 I have not signed a Digium "disclaimer". By: jlawrence (jlawrence) 2004-09-30 14:25:12 Has anyone got a version of this patch that works against 1.0 If so could you post them or email to jon AT lawrence dot org dot uk By: Mark Spencer (markster) 2004-09-30 15:57:36 Isn't this already fixed in CVS? By: jlawrence (jlawrence) 2004-10-03 16:18:30 If the patches are in 1.0 then I can find nothing in the sample conf's saying how to setup x100p callerid. I've tried ukcallerid=yes, usecallerid=uk. So are they or aren't they ? By: Mark Spencer (markster) 2004-10-03 16:22:40 The UK etc callerid *is* in, but does not support X100P, nor am I willing to merge the support for the X100P since it adds otherwise needless bloat to the zaptel side, however this bug will remain available should anyone need it. By: Tzafrir Cohen (tzafrir) 2007-07-27 14:05:52 I'd like to reopen this issue, as the patch for ukcid is still floating around and maintained. It still seems to be useful for various people. I was wondering if there is any way to merge the asterisk and Zaptel patches without adding unnecessary runtime overhead. That is: on a system with an X100P and another device ukcid support detection can be enabled at runtime, but still not affect the performance of other Zaptel devices on the same system. 1. The patch adds an extra definition in the userspace-visible parts of zaptel.h, and hence the Asterisk parts should be safe to merge: the code is only used for channels explicitly marked as cidstart=polarity". Hence the extra code in the asterisk patch should be modified to use #ifdef ZT_GET_HISTORY . 2. The code in the Zaptel patch is used for all Zaptel channels. Hence there is unnecessary copying of the data to the history buffer. How can we make this only happen for specific channels? I see two general ways: a. Give some sort of optional capability to the x100p driver, and thus only x100p channels will have their history copied. Or: b. Asterisk should ask Zaptel to copy history for specific channels (another ioctl?) By: Tzafrir Cohen (tzafrir) 2007-07-27 14:16:58 BTW: the zaptel patces pointed here are invalid links. By: Tilghman Lesher (tilghman) 2007-08-29 14:13:45 Appears to be no interest. By: Tzafrir Cohen (tzafrir) 2009-02-11 11:41:14.000-0600 Reopening, according to a request by the original submitter By: nick_lewis (nick_lewis) 2009-02-12 10:05:36.000-0600 I have inherited an old asterisk source with the history patch and I need the same x100p uk cid functionality on asterisk 1.6 I am not currently familiar with the implementation but if you give me a steer on 2a versus 2b I will try my hand at a solution minus the runtime overhead By: nick_lewis (nick_lewis) 2009-02-12 11:01:01.000-0600 tzafrir I have had a quick nose at the code. Perhaps another way to have a history buffer only for specific channels is through the dahdi.conf signalling parameter e.g. signalling=fxs_ksrb could be "kewl start, reversal blind" By: David Greaves (lbt) 2009-02-12 11:44:04.000-0600 I just 'ported' the patches to Asterisk-1.6.1-rc1 and dahdi-2.1.0.4 sent it to Marc for publication here: http://www.lusyn.com/resources/asterisk/usehist.htm He emailed me today to confirm he's putting them up. I was also going to take a look at the overhead issue as it seems sensible to get this patch into a state to go into the main dahdi driver. Given the tiny patch size it certainly seems that there's minimal 'bloat' at a code level so maybe making the data structures dynamically allocated will address any memory usage bloat concerns. A dynamic approach would also allow the history approach to be selected by those few of us who need it and minimise the overhead for others. (nb It was I who asked tzafrir to reopen on 11-feb, not the OP) By: nick_lewis (nick_lewis) 2009-02-12 11:55:51.000-0600 Cheers lbt Regarding dynamic use of history buffer (i.e. allocated only for x100p channels) how about adding a signalling type fxs_kspb for "kewl start, polarity blind" as this setting unlike cidstart can be channel specific dahdi.conf example ; modern interface card signalling=fxs_ks channel =>1-4 ; x100p type card requiring history buffer in place of polarity detection signalling=fxs_kspb channel =>5 By: Tzafrir Cohen (tzafrir) 2009-02-12 12:14:37.000-0600 I actually meant the overhead in DAHDI. The History buffer of 2^14 samples is allocated for each channel. How can we get it allocated only for channels that have 'cidstart=usehist' in chan_dahdi.conf? By: nick_lewis (nick_lewis) 2009-02-19 11:07:51.000-0600 I may be on the wrong track but there is much reference in dahdi-base.c to chan->sig. If a new sig type of DAHDI_SIG_FXSKSPB could be added would it not be possible for a history buffer to be created in dahdi-base.c only if chan->sig had this value? By: Russell Bryant (russell) 2009-04-20 12:39:42 The latest patch on this issue is from 2004.  It is quite ancient and much of what is in there has since been added to chan_dahdi.  Other parts are available in newer patches on other issues. If someone would like to work on a new patch that adds functionality not available here (or in another pending issue), that would be fine, but let's handle it in a new issue. Thanks!