[Home]

Summary:ASTERISK-04603: [request] ENUM lookup functionality extension
Reporter:John Todd (jtodd)Labels:
Date Opened:2005-07-15 15:45:11Date Closed:2011-06-07 14:02:36
Priority:MajorRegression?No
Status:Closed/CompleteComponents:Core/NewFeature
Versions:Frequency of
Occurrence
Related
Issues:
Environment:Attachments:
Description:I am offering a $100 bounty for someone to re-write ENUMLookup to perform to the specifications I outline below.  The current ENUMLookup is somewhat simplistic in it's ability to query multiple trees (staticly via the enum.conf file) and it also can only return a single response despite ENUM's ability to have multiple answers to the same question.

Edwin Groothuis had an interesting patch in bug ASTERISK-4026  http://bugs.digium.com/view.php?id=4124 but it didn't quite do things the way I thought they would be most efficient and transparent to the dialplan.  Additionally, the ability to specify the zone suffix from within the Dialplan is immediately useful, as there are already 5+ public ENUM zones which may be looked up, which also may depend on other criteria that change on-the-fly.

****** ADDITIONAL INFORMATION ******

To: Edwin Groothuis <edwin@mavetju.org>
From: John Todd <jtodd@loligo.com>
Subject: Fwd: [Asterisk-Dev] ENUM multiple records handling
Cc: asterisk-dev@lists.digium.com

[sorry for top-posting; the original post was fairly long]

Edwin -
 I saw from the Mantis notes on this that it wasn't implemented, because of unfortunate timing (conversion from apps to functions, and Kevin desired this as a function instead of an app.)

 I have some real interest in getting ENUM to work more intelligently than it does now, since I think ENUM holds great promise (though is also fundamentally broken, as well, but that's a different story.)  I'd put $100 towards getting this done as a function, if you (or someone) could re-write it.  I also have another request, as well, if that is to be done: I'd like to see the ENUM zone able to be specified from the function call, so I can customize which zone I'd like to do the lookup in.  This is different than your linked list idea, but it relies more on "hard" requests rather than "soft" looping array pointers which have their count values hidden from the administrator and which may get very confusing.

 This function is different than any of the previous incarnations of EnumLookup (either in CVS or patches that have been submitted) because it does NOT assume a limited array of pointer types, and it also allows the dialplan designer to intelligently select or incrementally cascade through a list of pointer types and pointer preferences,  and by doing so become more in sync with the goals of ENUM in general.  Some of the flaws in my implementation are that I jumble some of the pointer types in the most simplistic methods of use, but there is a second method I show (the "ALL" incremental counter) which allows the administrator to really get their hands dirty and get the full query return values.




Function: EnumLookup(<number>[,pointer_type[,options[,zone_suffix]]])
 Performs an ENUM tree lookup on the specified number/pointer type and optional ordinal offset, and returns one of four different values (NAPTR of one pointer type, count of elements of one pointer type, count of all pointer types, or type of pointer.

<number> = e164 number

pointer_type = tel, sip, h323, iax2, mailto, ...[others], ALL.  Default type is "sip".
    Special name of "ALL" will create a list of pointer_types across all orders, run a "uniq" pass across them, and then put the results in an ordinal list starting with 1.  The <number> specified will then be returned. If ALL is specified and there is no following <integer> or no pointers in the list, the routine returns a null value.  (see Example 6, below.)  The pointer types are not hardcoded in Asterisk except for the default if no other pointer type specified; any valid pointer type may be used except for the string "ALL".

<options> = optional specifiers.
   c = count.  Returns the number of records of this type are returned (regardless of order or priority)
   <integer> = The record in priority/order sequence based on the total count of records passed back by the query.  If a pointer type is specified, all entries of that type will be sorted into an ordinal list starting with 1 (by order first, then priority).
   The default of <options> is "1"

zone_suffix = allows customization of the ENUM zone.  Default is e164.arpa.


EXAMPLE USES:

Let's use this ENUM list as an example (note the slight order/priority changes from the real world, and pardon my poor NAPTR regexp processing if I've made any errors in return values):

9.9.1.6.1.8.3.0.5.6.1.mavetju.org. 3600 IN NAPTR 10 100 "u" "tel+E2U" "!^\\+16503816199$!tel:+5551!" .
9.9.1.6.1.8.3.0.5.6.1.mavetju.org. 3600 IN NAPTR 21 100 "u" "tel+E2U" "!^\\+16503816199$!tel:+5553!" .
9.9.1.6.1.8.3.0.5.6.1.mavetju.org. 3600 IN NAPTR 25 100 "u" "sip+E2U" "!^\\+16503816199$!sip:5554@barnet.com.au!" .
9.9.1.6.1.8.3.0.5.6.1.mavetju.org. 3600 IN NAPTR 25 100 "u" "sip+E2U" "!^\\+16503816199$!sip:5555@barnet.com.au!" .
9.9.1.6.1.8.3.0.5.6.1.mavetju.org. 3600 IN NAPTR 55 100 "u" "mailto+E2U" "!^\\+16503816199$!mailto:5552@mavetju.org!" .

Example 1: Simplest case, using first SIP return (all defaults)
exten => 100,1,Set(foo=EnumLookup(16503816199))
 returns:  ${foo}="5554@barnet.com.au"

Example 2: What is the first "tel" pointer type for this number?  (after sorting by order/preference)
exten => 100,1,Set(foo=EnumLookup(16503816199,tel))
 returns: ${foo}="+5551"

Example 3: How many "sip" pointer type entries are there for this number?
exten => 100,1,Set(foo=EnumLookup(16503816199,sip,c))
 returns: ${foo}=2

Example 4: For all the "tel" pointer type entries, what is the second one in the list? (after sorting by preference)
exten => 100,1,Set(foo=EnumLookup(16503816199,tel,2))
 returns: ${foo}="+5553"

Example 5: What is the first pointer type referenced for this number? (after sorting by order/preference)
exten => 100,1,Set(foo=EnumLookup(16503816199,POINTERS))
 returns: ${foo}="tel"

Example 6: What is the third pointer type referenced for this number? (after sorting by order/preference)
exten => 100,1,Set(foo=EnumLookup(16503816199,POINTERS,3))
 returns: ${foo}="mailto"

Example 7: How many pointer types (tel, sip, mailto, etc.) are in the list for this number?
exten => 100,1,Set(foo=EnumLookup(16503816199,POINTERS,c))
 returns: ${foo}=3

Example 8: Give back the first SIP pointer for the number in the enum.yoydynelabs.com zone (invalid lookup)
exten => 100,1,Set(foo=EnumLookup(16503816199,sip,1,enum.yoyodynelabs.com))
 returns: ${foo}=[null]


With this method, you could discover:
 1) If there were any ENUM entries at all for the channel types you support
 2) If there are entries for more than 1 channel type you support, you can discover what they are, in order of preference
 3) If there is more than one priority for a channel type you support, you can discover what they are, in order of preference

This may take some looping within the dialplan, but it's actually not all that complex.  No more or less complex than writing it into an AGI, and certainly a lot more portable.

Warnings:
 a) If a query is performed of type "count" (let's say you get back 5 records) and then some seconds later a query is made against ASTERISK-1 in the list, it may not be the case that the DNS resolver has the same answers as it did a second or two ago.  The resolver should be the canonical storage location for DNS records, since that is the intent of ENUM.  However, some obscure future cases may have wildly changing NAPTR records within several seconds.  This is a corner case, and probably only worth noting as a very rare circumstance.  (note: I do not object to the dnsmgr method of locally caching DNS replies, but they need to honor the TTL given by the remote zone master.)

 b) The "h323" specifier in the enum.conf file becomes irrelevant, and that whole config file can go away now since we can do sequential zone lookups from within the dialplan by referencing the EnumLookup function iteratively in the dialplan.

 c) It is difficult, if not impossible, to sort a list of ordered URIs that have recurring types out of order (i.e.: 10 tel, 20 sip, 30 tel)

 d) Default behavior (even in event of an error) should be to jump to the next priority.  Leaping to +101 or +53 or whatever, sucks.  Most ENUM lookups are going to be failures.  Anyone trying to get ENUM to work should have somewhat clueful programming skills, so checking the value of the variable should be sufficient to determine if a successful lookup has happened (which is, I believe, one of the whole points for moving to functions in the first place.)


JT



Date: Fri, 11 Feb 2005 02:10:57 +1100
From: Edwin Groothuis <edwin@mavetju.org>
To: Asterisk-Dev@lists.digium.com
Cc: Anton Holleman <anton.holleman@nominum.com>
Subject: [Asterisk-Dev] ENUM multiple records handling
Reply-To: Asterisk Developers Mailing List <asterisk-dev@lists.digium.com>

Hello,

Just tried to play around with EnumLookup() and it seems it has a
funny behaviour when using multiple NAPTR records with different
preferences and priorities.

After some debugging, it seems always to return the first entry the
answer section of the DNS reply.

How do NAPTR records look like:
   order preference flags service regexp replacement

The records should be processed in order sort, then on preference.

Right now if you do a NAPTR lookup for something which has more
than one NAPTR record (for example: 9.9.1.6.1.8.3.0.5.6.1.mavetju.org),
it will always use the first record. Debugging and TCP output has
confirmed this:

-----
15:35:43.801587 202.83.176.1.domain > 10.10.10.4.33429:  52325* 5/2/2 NAPTR , NAPTR , NAPTR , NAPTR , NAPTR  (473)
0x0000   4500 01f5 ead1 0000 4011 ffc3 ca53 b001        E.......@....S..
0x0010   0a0a 0a04 0035 8295 01e1 922a cc65 8580        .....5.....*.e..
0x0020   0001 0005 0002 0002 0139 0139 0131 0136        .........9.9.1.6
0x0030   0131 0138 0133 0130 0135 0136 0131 076d        .1.8.3.0.5.6.1.m
0x0040   6176 6574 6a75 036f 7267 0000 2300 01c0        avetju.org..#...
0x0050   0c00 2300 0100 000e 1000 3200 0a00 6401        ..#.......2...d.
0x0060   7507 7465 6c2b 4532 5522 215e 5c2b 3136        u.tel+E2U"!^\+16
0x0070   3530 3338 3136 3139 3924 2174 656c 3a2b        503816199$!tel:+
0x0080   3631 3239 3532 3733 3532 3721 00c0 0c00        61295273527!....

Feb 10 15:35:43 NOTICE[7231]: app_enumlookup.c:123 enumlookup_exec: tel: ENUM set to "61295273527"
   -- Executing NoOp("SIP/10.10.12.2-0815b2a0", "TEL") in new stack
   -- Executing NoOp("SIP/10.10.12.2-0815b2a0", "61295273527") in new stack

-----
15:36:40.362791 202.83.176.1.domain > 10.10.10.4.33429:  31650* 5/2/2 NAPTR , NAPTR , NAPTR , NAPTR , NAPTR  (473)
0x0000   4500 01f5 2b75 0000 4011 bf20 ca53 b001        E...+u..@....S..
0x0010   0a0a 0a04 0035 8295 01e1 7e52 7ba2 8580        .....5....~R{...
0x0020   0001 0005 0002 0002 0139 0139 0131 0136        .........9.9.1.6
0x0030   0131 0138 0133 0130 0135 0136 0131 076d        .1.8.3.0.5.6.1.m
0x0040   6176 6574 6a75 036f 7267 0000 2300 01c0        avetju.org..#...
0x0050   0c00 2300 0100 000e 1000 3900 1e00 6401        ..#.......9...d.
0x0060   7507 7369 702b 4532 5529 215e 5c2b 3136        u.sip+E2U)!^\+16
0x0070   3530 3338 3136 3139 3924 2173 6970 3a65        503816199$!sip:e
0x0080   6477 696e 4062 6172 6e65 742e 636f 6d2e        dwin@barnet.com.
0x0090   6175 2100 c00c 0023 0001 0000 0e10 0032        au!....#.......2

Feb 10 15:36:40 DEBUG[7270]: enum.c:120 parse_naptr: input='+16503816199', flags='u', services='sip+E2U', regexp='!^\+16503816199$!sip:edwin@barnet.com.au!', repl=''
   -- Executing NoOp("SIP/10.10.12.2-0815b2a0", "SIP") in new stack
   -- Executing NoOp("SIP/10.10.12.2-0815b2a0", "SIP/edwin@barnet.com.au") in new stack


-----
15:40:15.604325 202.83.176.1.domain > 10.10.10.4.33429:  51499* 5/2/2 NAPTR , NAPTR , NAPTR , NAPTR , NAPTR  (473)
0x0000   4500 01f5 3857 0000 4011 b23e ca53 b001        E...8W..@..>.S..
0x0010   0a0a 0a04 0035 8295 01e1 52a7 c92b 8580        .....5....R..+..
0x0020   0001 0005 0002 0002 0139 0139 0131 0136        .........9.9.1.6
0x0030   0131 0138 0133 0130 0135 0136 0131 076d        .1.8.3.0.5.6.1.m
0x0040   6176 6574 6a75 036f 7267 0000 2300 01c0        avetju.org..#...
0x0050   0c00 2300 0100 000e 1000 3f00 1900 6401        ..#.......?...d.
0x0060   7507 7369 702b 4532 552f 215e 5c2b 3136        u.sip+E2U/!^\+16
0x0070   3530 3338 3136 3139 3924 2173 6970 3a36        503816199$!sip:6
0x0080   3132 3933 3335 3330 3135 4062 6172 6e65        1293353015@barne
0x0090   742e 636f 6d2e 6175 2100 c00c 0023 0001        t.com.au!....#..

Feb 10 15:40:15 DEBUG[7739]: enum.c:120 parse_naptr: input='+16503816199', flags='u', services='sip+E2U', regexp='!^\+16503816199$!sip:61293353015@barnet.com.au!', repl=''
   -- Executing NoOp("SIP/10.10.12.2-0815b2a0", "SIP") in new stack
   -- Executing NoOp("SIP/10.10.12.2-0815b2a0", "SIP/61293353015@barnet.com.au") in new stack



I have changed asterisk/enum.c a little bit and it now supports
multiple NAPTR records and takes the order and preference into
account. It works as follows: The first time EnumLookup() is called,
it loads all NAPTR records in a linked list and returns the first
entry (order and preference wise). Every next call to EnumLookup()
returns the next one (order and preference wise).

How long does this linked list exist? Each element in the list has
the PID of the thread which asked for the lookup first and it has
a timer and after 15 minutes they get cleaned up. Is 15 minutes a
lot? Yes/No. Yes because if you do a lot of lookups, it can grow a
bit. No because you have to take timeouts into account, so if you
have four phone numbers in it, you need a timeout of say 4x45 seconds
which already makes 3 minutes. I can probably set it to 5 minutes
or so, it just has to be determined later.


A test program would be: Do a lookup, echo the type, echo the ENUM
variable and jump back to the lookup until it fails.

   exten => 700,1,NoOp(ENUM TEST)
   exten => 700,n,EnumLookup(16503816199)
   exten => 700,n,NoOp(SIP)
   exten => 700,n,NoOp(${ENUM})
   exten => 700,n,GoTo(2)
   exten => 700,53,NoOp(TEL)
   exten => 700,n,NoOp(${ENUM})
   exten => 700,n,GoTo(2)
   exten => 700,103,NoOp(FAIL)
   exten => 700,n,Hangup()


This will print all NAPTR records for (16503816199).mavetju.org,
and in the right order:

-- Executing NoOp("SIP/128.177.195.13-08144708", "ENUM TEST") in new stack
-- Executing EnumLookup("SIP/128.177.195.13-08144708", "16503816199") in new stack
[...]
   ENUM got '1'
   Feb 11 02:03:06 NOTICE[29941]: app_enumlookup.c:124 enumlookup_exec: tel: ENUM set to "61295273527"
-- Executing NoOp("SIP/128.177.195.13-08144708", "TEL") in new stack
-- Executing NoOp("SIP/128.177.195.13-08144708", "61295273527") in new stack
-- Executing Goto("SIP/128.177.195.13-08144708", "2") in new stack
-- Goto (barnet-from-internet,700,2)
-- Executing EnumLookup("SIP/128.177.195.13-08144708", "16503816199") in new stack
   ENUM got '1'
   Feb 11 02:03:06 NOTICE[29941]: app_enumlookup.c:124 enumlookup_exec: tel: ENUM set to "61409227633"
-- Executing NoOp("SIP/128.177.195.13-08144708", "TEL") in new stack
-- Executing NoOp("SIP/128.177.195.13-08144708", "61409227633") in new stack
-- Executing Goto("SIP/128.177.195.13-08144708", "2") in new stack
-- Goto (barnet-from-internet,700,2)
-- Executing EnumLookup("SIP/128.177.195.13-08144708", "16503816199") in new stack
   ENUM got '1'
   Feb 11 02:03:06 NOTICE[29941]: app_enumlookup.c:83 enumlookup_exec: sip: ENUM set to "SIP/61293353015@barnet.com.au"
-- Executing NoOp("SIP/128.177.195.13-08144708", "SIP") in new stack
-- Executing NoOp("SIP/128.177.195.13-08144708", "SIP/61293353015@barnet.com.au") in new stack
-- Executing Goto("SIP/128.177.195.13-08144708", "2") in new stack
-- Goto (barnet-from-internet,700,2)
-- Executing EnumLookup("SIP/128.177.195.13-08144708", "16503816199") in new stack
   ENUM got '1'
   Feb 11 02:03:06 NOTICE[29941]: app_enumlookup.c:83 enumlookup_exec: sip: ENUM set to "SIP/edwin@barnet.com.au"
-- Executing NoOp("SIP/128.177.195.13-08144708", "SIP") in new stack
-- Executing NoOp("SIP/128.177.195.13-08144708", "SIP/edwin@barnet.com.au") in new stack
-- Executing Goto("SIP/128.177.195.13-08144708", "2") in new stack
-- Goto (barnet-from-internet,700,2)
-- Executing EnumLookup("SIP/128.177.195.13-08144708", "16503816199") in new stack
   ENUM got '0'
-- Executing NoOp("SIP/128.177.195.13-08144708", "FAIL") in new stack
-- Executing Hangup("SIP/128.177.195.13-08144708", "") in new stack

(verbose 3 is happy spamming here :-)


I have made patches against the latest HEAD and the full new enum.c:

http://www.mavetju.org/~edwin/patch-enum.c
http://www.mavetju.org/~edwin/enum.c

If you are interested, please test and let me know if it works. I
will test it further on production machines and submit a patch to
the bugs database.

Edwin

--
Edwin Groothuis      |            Personal website: http://www.mavetju.org
edwin@mavetju.org    |          Weblog: http://weblog.barnet.com.au/edwin/
_______________________________________________
Asterisk-Dev mailing list
Asterisk-Dev@lists.digium.com
http://lists.digium.com/mailman/listinfo/asterisk-dev
To UNSUBSCRIBE or update options visit:
  http://lists.digium.com/mailman/listinfo/asterisk-dev
Comments:By: John Todd (jtodd) 2005-07-15 15:46:03

Also: for the bounty to be paid, the patch must be properly disclaimed and accepted into CVS-HEAD.
By: Brian West (bkw918) 2005-07-15 15:49:56

this should really be a function variable it would be nice to do:

Dial(${ENUM(16503816199,sip,1,enum.yoyodynelabs.com)})

Just an idea!
By: Michael Jerris (mikej) 2005-07-15 22:39:30

There is already a partially complete implementation of this in ASTERISK-4026.  We are going to want somthing similar to what was discussed based upon funcitons for this to get into the tree.  Can you please review that bug and modify your specs as necessary along those basic arcitecture decisions.  If you do not want to go that way for any specific reasons, post back to the bug to discuss.
By: John Todd (jtodd) 2005-07-15 22:50:43

If you'll look at the bugnotes, you'll see that my modifications were based on discussion with the author of the patch in 4124.  The methodology I propose is significantly different for several reasons:

1) It is a function-style call (bkw: yep, that's what I meant, sorry if I wasn't clear in my examples)

2) It allows the dialplan administrator to get ordinal counts of each pointer type and ordinal counts of the total number of pointer types returned.

3) It allows the dialplan administrator to request specific elements from within the list of returned values, both within each pointer-sub type or within all types versus a non-specific additive "looping" method (which amounts to just a guess)

4) It allows the addition of per-use modification of domain suffix.
By: Michael Jerris (mikej) 2005-07-15 22:58:17

Let's get some kpfleming arcitecture review on this idea to verify that it is somthing that can make it into the tree.
By: John Todd (jtodd) 2005-07-19 14:51:49

Bounty increased to $150
By: Michael Jerris (mikej) 2005-07-26 21:25:44

It appears this is not getting any traction here.  I would suggest posting the bouty to the wiki if you have not already, or directly contracting this work with a developer.  Feel free to open this back up when progress has been made.