Summary:	ASTERISK-07411: [branch] Standalone AEL2 compiler
Reporter:	Sherwood McGowan (rushowr)	Labels:
Date Opened:	2006-07-28 10:22:07	Date Closed:	2006-08-14 14:51:09
Priority:	Major	Regression?	No
Status:	Closed/Complete	Components:	PBX/pbx_ael
Versions:		Frequency of Occurrence
Related Issues:
Environment:		Attachments:
Description:	First off, let me apologize if this is not where I should make this request. While loading AEL2 files into an asterisk install and then dumping the resulting dialplan with "save dialplan" works for outputting dialplan code from AEL files, it's not the best solution due to the number of steps and the fact that it overwrites the existing extensions.conf file. How difficult would it be to modify the existing apps or create a new app that outputs the parsed and translated code into a file? If aelparse could be modified possibly to have an additional option like "-w outputfile.conf", that would be awesome.
Comments:	By: Steve Murphy (murf) 2006-07-28 12:50:20 Excellent idea, rushowr!!! It would not be hard at all to add some code to generate extensions.conf format output from the compiler. It has most of the format ready when it calls the routines to enter the dial plan directly into asterisk. This would most likely best be done as a command line option in aelparse, but if you see some strange need to access this feature directly from the dialplan itself, say so, and I can maybe add an option/command to do that. Now, as to the reverse, converting extensions.conf to AEL: it is possible, but a lot of work. Think of it this way: what you are asking for is a reverse compiler. Given, for example, an assembly file, produce the equivalent C++ code that might have produced that assembly file. It's a bit easier (relatively) if you have some nice constraints, say you know the assembly file was generated with g++, and only one particular version of it, at that. You could then pattern match certain sequences in the assembler output back into C++ structures with reasonable confidence. Others, you just synthesize something that would work. But, incoming extensions.conf code does not follow such narrow limits.It would not be uncommon to see subtle tricks being used to accomplish some goal, which would be difficult for a human to interpret, let alone a program. That being said, most extensions code **is** pretty simple, and in most cases simple heuristics could cover a great majority of the code. Stuff that it can't grok could simply be copied verbatim into the output for a human to complete. Most of the formatting and easy stuff could be done by the translator and speed up the job of conversion tremendously. I'm all for it. Such a program could help reduce the cost of converting to AEL, and aid the whole community greatly. So, I'll keep this bug open until I have something to release. Here in a bit, I'll associate a development branch with this bug so others who are interested can review/add/complain/modify/etc. Maybe we can split these two issues up into two bugs, and have two separate branches, covering the respective issues... We'll see... By: Sherwood McGowan (rushowr) 2006-07-29 08:41:36 murf, Glad that you like my ideas! That said, I can't wait to check it out. I'll download a fresh copy of the trunk and peek around the code for AEL and see what I can do to possibly help. I'm not a great programmer, but hey if nothing else I'll learn a little while I'm looking. By: Steve Murphy (murf) 2006-08-01 16:38:14 OOps, sorry, I've been kinda splitting off the extensions.conf => AEL converter into bug 7638, and it looks like your compliment got munged. Sorry. Remarks like that may not get you anywhere, but it can always earn you some positive karma! ;^) By: Steve Murphy (murf) 2006-08-01 16:42:05 I've created the branch: http://svn.digium.com/svn/asterisk/team/murf/bug_7606 for this request. By: Steve Murphy (murf) 2006-08-02 16:05:14 OK, I think I've completed work on this one: It outputs the context headers, ignorepats, includes, switches, and eswitches (oops, just remembered, not quite done with the switch stuff yet, but almost). The [global] section is also output at the beginning. labels are included in exten => lines. BUT, I can't commit this stuff back to the branch, until I merge in 7598... or I'll have a mess. I merged in the changes to 7598 into the bug_7606 branch, because I really need to have that in place as a basis. So, rushowr, if you want extensions.conf.aeldump  via a -w switch on aelparse, you have to help me close 7598 first. (murf waves carrot hanging from long stick in front of rushowr). By: Steve Murphy (murf) 2006-08-02 17:07:36 Yes, now I'm pretty done. Switches print properly, contexts without extensions (just includes/switches/ignorepats/etc) are output properly. Looks good now. Very good. Very, very good! (carrot waving!) By: Sherwood McGowan (rushowr) 2006-08-02 22:57:28 Murf, I'll hopefully have some good info for helping to close 7598 tomorrow after I test it. Thanks for the hard work! By: Sherwood McGowan (rushowr) 2006-08-06 01:41:56 I just checked out the aforementioned branch and compiled. I don't see any file writing output.. here's some screen output: # aelparse -n -w /etc/asterisk/test.conf Executed ast_register_file_version(); Executed ast_register_file_version(); Executed ast_register_file_version(); Executed ast_register_file_version(); (You can use the -d option if you want to use the current working directory as the CONFIG_DIR. I will look in this dir for extensions.ael* and its included files) LOG: lev:2 file:../pbx/pbx_ael.c  line:3480 func: pbx_load_module  Starting AEL load process. LOG: lev:2 file:../pbx/pbx_ael.c  line:3487 func: pbx_load_module  AEL load process: calculated config file name '/etc/asterisk/extensions.ael'. LOG: lev:2 file:ael.flex  line:422 func: ael_yylex    --Read in included file /etc/asterisk/voipo/basefuncs.ael, 1014 chars LOG: lev:2 file:ael.flex  line:422 func: ael_yylex    --Read in included file /etc/asterisk/voipo/db.ael, 2776 chars LOG: lev:2 file:ael.flex  line:422 func: ael_yylex    --Read in included file /etc/asterisk/voipo/debug.ael, 835 chars LOG: lev:2 file:../pbx/pbx_ael.c  line:3490 func: pbx_load_module  AEL load process: parsed config file name '/etc/asterisk/extensions.ael'. LOG: lev:2 file:../pbx/pbx_ael.c  line:3493 func: pbx_load_module  AEL load process: checked config file name '/etc/asterisk/extensions.ael'. LOG: lev:2 file:../pbx/pbx_ael.c  line:3495 func: pbx_load_module  AEL load process: compiled config file name '/etc/asterisk/extensions.ael'. LOG: lev:2 file:../pbx/pbx_ael.c  line:3498 func: pbx_load_module  AEL load process: merged config file name '/etc/asterisk/extensions.ael'. LOG: lev:2 file:../pbx/pbx_ael.c  line:3501 func: pbx_load_module  AEL load process: verified config file name '/etc/asterisk/extensions.ael'. LOG: lev:4 file:ael2_parse  line:253 func: main  24 contexts, 39 extensions, 194 priorities am I missing something? By: Steve Murphy (murf) 2006-08-07 12:43:22 Sorry!!! I hadn't yet committed the changes to the branch. But I have now!! It looks good to me, try checking out or merging this branch to your current trunk, and see if it works acceptably to you. The output file will be extensions.conf.aeldump, when you apply -w to aelparse. I've added a regression test, and made sure the other regression tests should PASS. Within the next day or two, I'll commit this to trunk, which Kevin approved since it's not going to change asterisk itself. If you find any bugs, please report them here!!!  Many thanks again! And again! By: Sherwood McGowan (rushowr) 2006-08-09 04:40:15 Murf, bug_7606 isn't there. Where should I be pulling the new code from? By: Steve Murphy (murf) 2006-08-09 08:08:09 Rushowr, It's been folded into the trunk. Sorry, I should have told you, but I wanted to make sure it ended up in 1.4. I'll leave this bug open till Friday night for you for feedback. To get it, just do 'svn update' on the trunk. Good luck!! By: Serge Vecher (serge-v) 2006-08-09 08:35:04 for the archives, the commit was in revision 39430/39431. By: Sherwood McGowan (rushowr) 2006-08-10 17:39:51 Murf, I just did a few test dumps with the -w otpion from the most trunk, gradually building the complexity. I got all the way up to dumping a HUGE dialplan consisting of 55 contexts, 116 extensions, and 1220 priorities. You sir, rock, it all dumped just fine :). Just a heads up though, I'm submitting a bug/improvement suggestion, concerning not changing the extension when building the switches.... I'll get in depth there :) By: Sherwood McGowan (rushowr) 2006-08-14 14:41:09 in case the system is waiting on more feedback from me, everything's good, thanks Murf By: Serge Vecher (serge-v) 2006-08-14 14:51:08 fixed and committed to trunk (see revisions above). Thanks to rushowr for testing and murf for fixing ;)