Summary:	ASTERISK-08030: macro's appdata can't be longer than 114 chars plus a 3 chars exten
Reporter:	Caio Begotti (caio1982)	Labels:
Date Opened:	2006-10-30 11:19:24.000-0600	Date Closed:	2006-11-01 14:05:54.000-0600
Priority:	Major	Regression?	No
Status:	Closed/Complete	Components:	Core/Configuration
Versions:		Frequency of Occurrence
Related Issues:	is related to ASTERISK-23582 [patch]Inconsistent column length in *odbc
Environment:		Attachments:
Description:	That seems quite weird for me, but after asking about it on IRC and trying to figure out the problem on my own, I'm filling this bug. This is what I got when I ask Asterisk to show my dialplan's Macro() for outbound calls. Please note that I'm using the Realtime Static scheme (with PostgreSQL and ODBC) and also note my "_12" exten pattern. weon*CLI> dialplan show outbound-routes-RotaPadrao '_12' => 1. Macro(dialoutpstn|${TDIRECTCALL-PST}/0${EXTEN}|${E${CALLERID(num)}-OPT}|${TDIRECTCALL-CID}|${TDIRECTCALL-MAX}|RotaPadrao) Whatever I put more in this exten pattern ("_12"), my outbound-routes-RotaPadrao goes crazy. Like that (and yes, I did save and reload my configuration files, just to make sure - but it did not help anyway): weon*CLI> dialplan show outbound-routes-RotaPadrao '_1234' => 1. Macro(dialoutpstn|${TDIRECTCALL-PST}/0${EXTEN}|${E${CALLERID(num) As you can see, my new exten pattern is fine (added the 3 and the 4 numbers), but my Macro()'s appdata has been reduced to a non-working pattern, is lacking lots of arguments (really needed in my "dialoutpstn" mini-macro) and it's not even displaying the full syntax for the last visible argument (the one related to the callerid number). That's really, really weird. Any hints or suggestions for debugging? I've been trying to edit the app_macro.c and pbx_config.c looking for some data length limitation but nothing good has been dugg from it anyway. It's not possible to switch to the real Realtime scheme now, neither it's possible to change my dialplan to a plan old and static extensions.conf text file. I checked my database and everything seesm ok, including the whole Macro()'s appdata, with all that arguments missing from "dialplan show" output.
Comments:	By: Caio Begotti (caio1982) 2006-10-30 11:46:03.000-0600 I just tested it again with all the Macro()'s appdata inside a unique temporary variable and made the Macro application to call only this temp var and it worked, although it's quite weird anyway and looks more like a bug for me, not a undocumented feature... By: Caio Begotti (caio1982) 2006-10-30 13:15:44.000-0600 Don't ask me why, but double checking my syntax I've found extra empty spaces between the priority number and the rest of the line. Removing them fixed the problem. By: Steve Murphy (murf) 2006-10-30 18:51:06.000-0600 Thanks for filing this bug. If this kind of thing bites you, it will eventually bite several others! Question: were the spaces befpre or after the comma following the priority number? By: Caio Begotti (caio1982) 2006-10-31 06:57:27.000-0600 Thanks for commenting, Murf. The blank spaces were before the number priority, after the first comma (sounds like a database counter from 1 to 9999 filling the spaces for me). I noted that the same problem occurs with using "CALLERID(num)" as one Macro() argument. Even escaping the ( and ) with a backslash didn't work. Even escaping the escapes (twice, yeah, to avoid substitution from the DB's internals). So my final advice could be: strip all the spaces between extens, prioritites and applications AND avoid using arguments with ( and ). Until I find some other way to escape things in my dialplan (any hints?). By: Caio Begotti (caio1982) 2006-10-31 07:06:27.000-0600 I've been testing it today at morning and I've found the following situation. Please note that I'm not using parenthesis anymore and the syntaxe looks okay for me, but the lenght of the strings are weird, like being limited in chars: weon*CLI> dialplan show outbound-routes-ComecaComZero  '_0XXXXXXX' =>    1. Macro(dialoutpstn|${TDIRECTCALL-PST}/0${EXTEN}|${E${CALLERIDNUM}-OPT}|${TDIRECTCALL-CID}|${TDIRECTCALL-MAX}|ComecaC)  '_0XXXXXX' =>     1. Macro(dialoutpstn|${TDIRECTCALL-PST}/0${EXTEN}|${E${CALLERIDNUM}-OPT}|${TDIRECTCALL-CID}|${TDIRECTCALL-MAX}|ComecaCo)  '_0XXXXX' =>      1. Macro(dialoutpstn|${TDIRECTCALL-PST}/0${EXTEN}|${E${CALLERIDNUM}-OPT}|${TDIRECTCALL-CID}|${TDIRECTCALL-MAX}|ComecaCom)  '_0XXXX' =>       1. Macro(dialoutpstn|${TDIRECTCALL-PST}/0${EXTEN}|${E${CALLERIDNUM}-OPT}|${TDIRECTCALL-CID}|${TDIRECTCALL-MAX}|ComecaComZ)  '_0XXX' =>        1. Macro(dialoutpstn|${TDIRECTCALL-PST}/0${EXTEN}|${E${CALLERIDNUM}-OPT}|${TDIRECTCALL-CID}|${TDIRECTCALL-MAX}|ComecaComZe) PS: I've removed the useless comments inside [] printed by the Asterisk's CLI. The last field in the Macro() shoule always be "ComecaComZero" ("starting with a zero" in portuguese, and it's just a random test rule anyway). As the extension pattern gets longer, the total ammount of chars in these lines gets reduced. By: Steve Murphy (murf) 2006-10-31 16:19:27.000-0600 caio1982-- I am running on the 1.4 version, and I cannot reproduce this problem at all. I use even bigger strings (160 chars & longer), along with your sample, (I don't have realistic values for the variables you mention in your dialoutpstn macro call). Listing these lines out works fine, too. No chop-off, no strange affects. No signs of any problems... we need to synchronize about your configuration, mayhaps. I've tried, btw, inserting spaces before and after the priority number, and it seems to make no difference. Listing these lines out works fine, too. No chop-off, no strange affects. Also, I've been running that little PIPS loop on 1.4, and no double-running when the repetitions get bigger. I usually run 1,000,000 its on my machine, to give good granularity in the resulting numbers. By: jmls (jmls) 2006-11-01 04:55:48.000-0600 caio1982, wasn't this discussed on -dev recently, and you found it was a postresql error, not asterisk ? By: Caio Begotti (caio1982) 2006-11-01 08:19:56.000-0600 Jmls: yeah, but that seemed to be a related problem, not this one (contrary of what I thought and found). I'm completely sure that blank spaces in the priorities field in the PostgreSQL database make it act strange. I just edited it and removed the spaces out of way and it worked. Murf: have you tested it using the Realtime Static scheme or flat files? Btw, I'm using 1.4beta3 right now, not trunk. Does it make too much difference for debugging? I can upgrade it if needed, no problem. Do you guys think that realistic values for these variables can help? I can provide them too, but I doubt that would change anything. Anyway, I'm gonna test some big exten patterns here but without those long variables. I'll just use plain chars like "lalalala" as the Macro()'s arguments. Is it safe, right? By: Caio Begotti (caio1982) 2006-11-01 08:49:54.000-0600 Using plain arguments, not that variables (through ODBC with PostgreSQL): weon*CLI> dialplan show outbound-routes-ComecaComZero '_0XXXXXXX' => 1. Macro(dialoutpstn|myfirstargument|mysecondargument|mythirdargument|looploop|onemoreargument|almostthere|asterisk|ac) '_0XXXXXX' => 1. Macro(dialoutpstn|myfirstargument|mysecondargument|mythirdargument|looploop|onemoreargument|almostthere|asterisk|act) '_0XXXXX' => 1. Macro(dialoutpstn|myfirstargument|mysecondargument|mythirdargument|looploop|onemoreargument|almostthere|asterisk|acti) '_0XXXX' => 1. Macro(dialoutpstn|myfirstargument|mysecondargument|mythirdargument|looploop|onemoreargument|almostthere|asterisk|actin) '_0XXX' => 1. Macro(dialoutpstn|myfirstargument|mysecondargument|mythirdargument|looploop|onemoreargument|almostthere|asterisk|acting) The arguments should be these: appdata="dialoutpstn,myfirstargument,mysecondargument,mythirdargument,looploop,onemoreargument,almostthere,asterisk,acting,weird,lastargument,notreally,nowitis" Now using the same patterns above, the same arguments, in flat files: From the extensions.conf: [outbound-routes-ComecaComZero] exten => _1XXXXXXX,1,Macro(dialoutpstn|myfirstargument|mysecondargument|mythirdargument|looploop|onemoreargument|almostthere|asterisk|ac) From the CLI: weon*CLI> dialplan show outbound-routes-ComecaComZero '_1XXXXXXX' => 1. Macro(dialoutpstn|myfirstargument|mysecondargument|mythirdargument|looploop|onemoreargument|almostthere|asterisk|ac) As you guys can notice, I only changed the zero to "1", so that we can see the it different with "dialplan show". No really a structural change :) So maybe we can ignore ODBC now, since flat files had the same problem? By: Steve Murphy (murf) 2006-11-01 09:01:19.000-0600 Ah-ha! So, this info is coming from the postgres database, a detail I missed previously. And, getting executed via realtime; I need the snippet of your dialplan where this is performed. It could easily be that your database is the one that is truncating your app call; Perhaps the size of a field is fixed in the db schema. Spaces in the priority slot may be a factor in storing the data in the database; they are NOT a factor in the asterisk config. (Although, spaces in certain places in the dialplan (like Set(x =5) will create a variable called "x " instead of "x", which some folks won't suspect, and will give them problems.) So, if this is all it is, I think we can add a little warning about unnecessary spaces somewhere so others might avoid the same pitfall. By: Steve Murphy (murf) 2006-11-01 09:58:29.000-0600 One more note: You said: From the extensions.conf: [outbound-routes-ComecaComZero] exten => _1XXXXXXX,1,Macro(dialoutpstn|myfirstargument|mysecondargument|mythirdargument|looploop|onemoreargument|almostthere|asterisk|ac) From the CLI: weon*CLI> dialplan show outbound-routes-ComecaComZero '_1XXXXXXX' => 1. Macro(dialoutpstn|myfirstargument|mysecondargument|mythirdargument|looploop|onemoreargument|almostthere|asterisk|ac) I don't see the prob in this case: the end of the macro args is "....|ac)" and the end of the output from the dialplan show command is "....|ac)" -- no truncation... By: Caio Begotti (caio1982) 2006-11-01 11:59:20.000-0600 Sorry... I've pasted it wrong, Murf. That's why there isn't truncation :) I'll check all my schemas. Thanks for pointing this out, I didn't remember that! By: Caio Begotti (caio1982) 2006-11-01 12:45:52.000-0600 Uh, I got it, Murf :-) From the res/res_config_odbc.c file: struct config_odbc_obj {        char *sql;        unsigned long id;        unsigned long cat_metric;        unsigned long var_metric;        unsigned long commented;        char filename[128];        char category[128];        char var_name[128];        char var_val[500];        SQLINTEGER err; }; That var_value had a 128 chars limitation and changing it to 500 for a very quick test made it. It's working now. I believe this var_val should have a default of 1024 (yeah, pretty big... but a safe number). Thanks for your effort on this issue! Do you think that can be fixed until the final release? PS: doesn't seem necessary to file a disclaimer, right? I'd love the credits, but that doesn't worth :-) By: Steve Murphy (murf) 2006-11-01 13:59:03.000-0600 Excellent detective work, caio1982--!!! I don't think you need to file a disclaimer, or will have much copyright issue over such a small, obvious (once it's found) change. I've made the following commits: 1.2 branch: 46803 1.4 branch: 46802 trunk:      46801 Not much credit to spread around, but you'll be spared having to bump the limit every time you upgrade/update the machine, and you should get a small chunk of karma at the very least for the fix to this problem. Good work!