[Home]

Summary:ASTERISK-29486: Hint-like extension value lookup function without device state
Reporter:N A (InterLinked)Labels:
Date Opened:2021-06-21 07:47:25Date Closed:2022-04-27 03:06:49
Priority:MinorRegression?
Status:Closed/CompleteComponents:Functions/NewFeature
Versions:18.4.0 Frequency of
Occurrence
Related
Issues:
Environment:Attachments:
Description:This is a new function that works like the HINT function, only a separate priority is used so that device state overhead is not added. This can provide a mechanism for a fast, efficient, and clean extension value lookup in the dialplan using native core extension pattern matching capabilities to retrieve extension values, including substituting variables.
Comments:By: Asterisk Team (asteriskteam) 2021-06-21 07:47:26.322-0500

Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. Please note that log messages and other files should not be sent to the Sangoma Asterisk Team unless explicitly asked for. All files should be placed on this issue in a sanitized fashion as needed.

A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report.

Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].

Please note that once your issue enters an open state it has been accepted. As Asterisk is an open source project there is no guarantee or timeframe on when your issue will be looked into. If you need expedient resolution you will need to find and pay a suitable developer. Asking for an update on your issue will not yield any progress on it and will not result in a response. All updates are posted to the issue when they occur.

Please note that by submitting data, code, or documentation to Sangoma through JIRA, you accept the Terms of Use present at [https://www.asterisk.org/terms-of-use/|https://www.asterisk.org/terms-of-use/].
By: Rob Grant (robgrant) 2021-07-20 17:06:55.044-0500

I have been reading the dialogue and examples in Gerrit of this proposed function and find it most interesting. It seems versatile (the pattern matching especially) and elegant. I think it would be very useful.  
By: Chaz Antonelli (TandemStacker) 2021-07-26 10:55:24.803-0500

I agree, I think it would be quite useful in the next build.
By: George Joseph (gjoseph) 2021-08-03 13:54:51.743-0500

From your email:

With LOOKUP, one could take code like this:
{code}
same => n,Gosub(billing-telnumber,${CALLERID(num)},1)
same => n,Set(btn=${GOSUB_RETVAL})
same => n,Gosub(feature-group,${CALLERID(num)},1)
same => n,Set(fg=${GOSUB_RETVAL})
same => n,DoSomething(${btn},${fg})
{code}
and replace it with:
{code}
same =>
n,DoSomething(${LOOKUP(${CALLERID(num)}@btn)},${LOOKUP(${CALLERID(num)}@fg})
{code}

I'm very confused.   Exactly what do billing-telnumber and feature-group do?  It appears that they set the btn and fg variables but if so, and the one-liner replaces the 5 lines above it, how do btn and fg get set?

The XML documentation for your function also uses {{extension@context}} but I'm not sure how that fits given that your email example uses variable names (btn and fg) instead of contexts.

Can you _please_ give us a real-world and complete example?   The email example just begs the question...  Why don't the billing-telnumber and feature-group subroutines just set channel variables?  That eliminates the two Sets.




By: N A (InterLinked) 2021-08-03 14:51:06.304-0500

That email was a bit of an early concept and the idea has morphed a bit since then.
The goal is not really set variables, it's to retrieve values, only - the same with DB gets a value from the DB, and you could save that in a variable. Generally, I don't save these in variables, since they're things I'll just look up once and I don't need again, so it can just be passed directly into the application - similar idea to DoSomething(${DB(something/something)}).

GoSub, in contrast, requires that variables are explicitly set, if you are using it to do more than one thing. This is because you can't just do all the lookups in the application itself, to get the value that you need. You need to call them one at a time, then save them temporarily while you do other lookups, then pass them all in at once. That's what the email example demonstrates.

The real point though, is that this function isn't another way of doing what Gosub does - Gosub by itself doesn't have any mechanism to retrieve the values this function does. It could be wrapped up in a Gosub, but the point is that it's able to return a value from the dialplan - similar to what HINT does, but without the device state overhead. Additionally, the value is variable-substituted, allowing for certain logic to be nested nicely. Here's an example:

Say that I want to get the extension length for dialing digits at some sort of prompt. Say in order to do that, I need to first get the customer group that the user belongs to. Perhaps that might look something like this:

[customer-groups]
exten => _55XXXXX,1,1
exten => _6[23]3NXXX,1,17
exten => _652[6-9]XXX,1,${EXTEN:-4:1}

[extension-lengths]
exten => _[12],1,5
exten => 3,1,3

[audio]
exten => _555XXXX,1,custom/default ; not user-changeable - static value for this range
exten => _4XXNXXX,1,${DB(usersettings/audio/${EXTEN})} ; we can call any function in a lookup, to return as a value, depending on the extension
exten => _63[14]XXXX,1,custom/companies/${EXTEN:-7:3}_main ; some variable manipulation
exten => _800XXXX,1,${LOOKUP(${EXTEN:-7:3}@nxx-greetings)} ; notice that this lookup calls another lookup, for this pattern match. This means one lookup could call another one, in dialplan execution, it "comes for free" because it's all evaluated at once for a single application call.

Now I can do:
same => n,Read(myvar,${LOOKUP(${CALLERID(num)}@audio)},${LOOKUP(${LOOKUP(${CALLERID(num)}@customer-groups)}@extension-lengths)})

In this example, we don't need to save the number of digits into a variable, we're just using it once, so we don't need to first save it and then use it. If this were a Gosub, it might look more like this:
same => n,Gosub(customer-groups,${CALLERID(num)},1)
same => n,Gosub(extension-lengths,${GOSUB_RETVAL},1)
same => n,Set(tmpvar=${GOSUB_RETVAL})
same => n,Gosub(audio,${CALLERID(num)},1)
same => n,Read(myvar,${GOSUB_RETVAL},${tmpvar})

This is a simple example with 3 *values* (not variables, per se). As the number of values increases, so does the complexity of trying to hijack Gosub for this purpose. You could write a subroutine to read digits with this data. That's not the goal of the function, which is to do a single thing and do it well, the value of which can be used for any purpose. I could make these same LOOKUP calls in 10 different places for different purposes. I don't need to write ten different subroutines to look up the same information. I just get what I need directly and provide it directly as an argument to whatever it is I'm invoking.
Additionally, Gosub can't return a dialplan value. It can Return(123), or Return(${DB(something/something)}), but it can't actually return the application value of an extension, which is kind of the main idea. For instance:

exten => 5,1,Progress(${DB_DELETE(something/${EXTEN})})
 same => n,Answer(${EXTEN})

Suppose that this was some kind of auto-generated dialplan, and you wanted to retrieve what was at that location - easy:
same => n,NoOp(${LOOKUP(default,5,1)} ${LOOKUP(default,5,2)})

== NoOp(Progress(87564375) Answer(5))

Currently, you can retrieve dialplan values using AMI. There is no way to do this from the dialplan itself though - apart from HINT, which sort of works, but has limitations. So this naturally compliments the DIALPLAN_EXISTS function, hence why DIALPLAN_VALUE could be an equivalently suitable name for this function.

The XML documentation might change. Right now, it's currently extension@context, but I may change that to extension@context,priority or context,extension,priority, so that the priority can be specified in a more elegant way. Initially, I didn't anticipate specifying the priority, but that would likely be something's that needed. I am open to changing the argument format around.

TL;DR -
- LOOKUP allows for retrieving extension values. There is no way to do this currently in the dialplan.
- LOOKUP naturally allows for convenient and compactly providing data to applications. This could be from DB, ODBC_FUNC, a static value, or any function. In such cases, extensions contain the desired dialplan functions and the variable parser takes care of everything. In your application, you don't need to care how the underlying data was retrieved - a single interface (the LOOKUP function) abstracts that away.

As an example of the latter, say that depending on the called number, I use a different API to get the caller ID information:

[caller-id]
exten => _212NXXXXXX,1,${CURL(https://new-york-city-cheap-caller-id.com/api/v1/${EXTEN})}
exten => _555NXXXXXX,1,${DB(cnam/${EXTEN:-5})} ; internal, check AstDB for these
exten => _NNXXXXXXX,1,${CURL(http://bulkcnam.com/${EXTEN})} ; use bulk CNAM for everything else
exten => 2124561414,1,WHITE HOUSE
exten => _NXX8675309,1,JENNY

This example makes it clear that LOOKUP is not replacing any of the other functions, e.g. DB. Rather, it complements them.

Now, all this logic is compactly in one place. You can thus do things like this:
same => n,Set(CALLERID(name)=${LOOKUP(${CALLERID(num)}@caller-id)})
or...
same => n,SayAlpha(${LOOKUP(${CALLERID(num)}@caller-id)})
or...
same => n,Dial(${LOOKUP(${EXTEN}@something)},${LOOKUP(${EXTEN}@ring-times)},m(${LOOKUP(${EXTEN}@music-on-hold-class)}f(${LOOKUP(${EXTEN}@ocd)})

Possibilities really are endless. There are many different directions this could be taken, all depending on what one does with it. LOOKUP itself is designed to be intentionally flexible for this.
By: George Joseph (gjoseph) 2021-08-04 10:49:11.220-0500

OK, that's a _LOT_ clearer but now I need time to digest. :)

By: George Joseph (gjoseph) 2021-08-04 11:55:14.887-0500

I think I get it now.  Would something like this work for you?
{code}
[customer-groups]
exten => _55XXXXX,1,Return(1)
exten => _6[23]3NXXX,1,Return(17)
exten => _652[6-9]XXX,1,Return(${EXTEN:-4:1})

[default]
same = n,DoSomething(${CALL_SUB(customer-groups,${CALLERID(num)},1))})

{code}

CALL_SUB would be a function that calls the GoSub application and _returns_ the value instead of saving it in GOSUB_RETVAL.
Then I think this would solve your problem but also allows users to create custom functions in Lua and AEL and call them easily from the dialplan.

You could implement this in app_stack.c where the Gosub app is implemented to make it even easier.



By: N A (InterLinked) 2021-08-04 12:11:58.563-0500

Hmm... it's certainly nicer than what exists currently, and would address some of the issues, but not everything.

Part of the reason LOOKUP is flexible is that it accesses the actual value from the dialplan. CALL_SUB would allow working with static things a user defined or other functions, but the other aspect of being able to retrieve what is actually an arbitrary priority is missing here, since it locks into the GoSub framework of returning something at priority 1. LOOKUP is closer to HINT than GoSub in design.
Performance-wise, GoSub incurs a relatively large cost, so if you had, say, ten of these passed into one application, it would take ten times longer to evaluate everything than with LOOKUP, since it just does so much other stuff we don't need. Probably less of a consideration than how it actually works, and what it does, but something to keep in mind perhaps.

So, CALL_SUB would solve some of the problems that motivated LOOKUP, but it's not quite as flexible or versatile, in my opinion. As LOOKUP would be just a function, wouldn't that also be equally accessible from Lua/AEL?
By: George Joseph (gjoseph) 2021-08-04 13:33:29.523-0500

bq. but the other aspect of being able to retrieve what is actually an arbitrary priority is missing here, since it locks into the GoSub framework of returning something at priority 1

Not understanding.  Gosub/CALL_SUB will start at any priority you want and stop when it's hits the Return.
{code}
[customer-groups]
exten => _55XXXXX,42,Return(1)
exten => _6[23]3NXXX,1,Return(17)
exten => _652[6-9]XXX,1(somelabel),Return(${EXTEN:-4:1})
exten => _652[6-9]XXX,42(someotherlabel),Return(${EXTEN:-4:1})

[default]
same = n,DoSomething(${CALL_SUB(customer-groups,${CALLERID(num)},42))})
same = n,DoSomething(${CALL_SUB(customer-groups,${CALLERID(num)},someotherlabel))})

{code}

bq.  As LOOKUP would be just a function, wouldn't that also be equally accessible from Lua/AEL?
No because LOOKUP returns a simple value and doesn't actually execute anything.

I do get your points about performance though.
Let me think more and I'll make comments on the review.



By: N A (InterLinked) 2021-08-04 20:13:53.916-0500

> Not understanding. Gosub/CALL_SUB will start at any priority you want and stop when it's hits the Return.

So this would be a function that essentially executes dialplan at that point, no?
I'm not sure what other implications might arise from that, but it's not compartmentalized like LOOKUP is. (Simple explanation: in the CLI, you don't see what LOOKUP is doing, as it's a function, and that's how functions work... but CALL_SUB spawns dialplan, which seems like new territory).

> No because LOOKUP returns a simple value and doesn't actually execute anything.

This isn't quite accurate. LOOKUP does variable substitution, and the PBX core does the actual execution. CURL, SHELL, etc. and other functions all execute when the PBX core parses them and the functions execute. LOOKUP can directly invoke any of these. So it does variable substitution, which indirectly does execute whatever you tell it to.
Hence, I am not sure why calling LOOKUP would be any different for Lua/AEL than any other function, such as calling SHELL or CURL directly.
I guess "execution" could mean different things but LOOKUP is certainly *not* limited to returning "simple values", as the examples above show.

> I do get your points about performance though. Let me think more and I'll make comments on the review.

Thanks, George! I really do appreciate your patience and consideration here ;)
By: George Joseph (gjoseph) 2021-08-05 07:34:17.563-0500

bq. So this would be a function that essentially executes dialplan at that point, no? I'm not sure what other implications might arise from that, but it's not compartmentalized like LOOKUP is. (Simple explanation: in the CLI, you don't see what LOOKUP is doing, as it's a function, and that's how functions work... but CALL_SUB spawns dialplan, which seems like new territory).

Sorry.  I just meant that as an aside.  You'd mentioned "since it locks into the GoSub framework of returning something at priority 1" and I was clarifying that Gosub isn't limited to starting at priority 1 and will return whatever you put in a Return().

bq.  I am not sure why calling LOOKUP would be any different for Lua/AEL than any other function, such as calling SHELL or CURL directly. I guess "execution" could mean different things but LOOKUP is certainly not limited to returning "simple values", as the examples above show.

Lua or AEL _using_ LOOKUP should be just fine.  I was thinking of calling _into_  Lua or AEL from LOOKUP, not the other way around.  This works for AEL but not for Lua since Lua expects to execute commands.  Probably not a big issue.



By: Friendly Automation (friendly-automation) 2022-04-27 03:06:51.375-0500

Change 18359 merged by Friendly Automation:
func_evalexten: Extension evaluation function.

[https://gerrit.asterisk.org/c/asterisk/+/18359|https://gerrit.asterisk.org/c/asterisk/+/18359]
By: Friendly Automation (friendly-automation) 2022-04-27 03:15:53.739-0500

Change 16075 merged by Friendly Automation:
func_evalexten: Extension evaluation function.

[https://gerrit.asterisk.org/c/asterisk/+/16075|https://gerrit.asterisk.org/c/asterisk/+/16075]
By: Friendly Automation (friendly-automation) 2022-04-27 03:29:45.285-0500

Change 18380 merged by George Joseph:
func_evalexten: Extension evaluation function.

[https://gerrit.asterisk.org/c/asterisk/+/18380|https://gerrit.asterisk.org/c/asterisk/+/18380]
By: Friendly Automation (friendly-automation) 2022-04-27 03:29:56.251-0500

Change 18381 merged by George Joseph:
func_evalexten: Extension evaluation function.

[https://gerrit.asterisk.org/c/asterisk/+/18381|https://gerrit.asterisk.org/c/asterisk/+/18381]