Summary:	ASTERISK-02438: [patch] Allow app_directory to work with REALTIME
Reporter:	drmac (drmac)	Labels:
Date Opened:	2004-09-20 14:14:36	Date Closed:	2008-01-15 15:26:45.000-0600
Priority:	Major	Regression?	No
Status:	Closed/Complete	Components:	Applications/app_directory
Versions:		Frequency of Occurrence
Related Issues:
Environment:		Attachments:	( 0) app_directory_realtime_1.4.1.patch ( 1) app_directory_realtime_1.4.patch ( 2) app_directory_realtime_1.5.patch ( 3) app_directory_realtime_1.6.1.patch ( 4) app_directory_realtime_1.6.patch ( 5) config.c.patch ( 6) config.h.patch
Description:	mysql-vm-routines.h allow you to have all your voicemail mailboxes defined in a MySQL database. Because of this, they are not listed in voicemail.conf. app_directory gets its information from voicemail.conf. Since there is nothing in it, it cannot function. This patch adds mysql support to app_directory. It will read (each time called) the voicemail information for the specified context. There is no extra configuration necessary if you already have mysql-vm-routines.h running with default tables. If you have changed the table name, check the newly patched app_directory.c file for a #define VOICEMAIL_TABLE and change it accordingly. Just apply the 2 patches, hit make, and copy the new app_directory.so. Enjoy.
Comments:	By: Mark Spencer (markster) 2004-09-20 17:15:33 Can you please confirm you have a disclaimer on file, thanks. By: drmac (drmac) 2004-09-21 10:18:10 Mark, I just faxed the disclaimer at 10:15AM Central time. It was signed by our company president and should apply to myself as I am the only one here who knows anything about C/Linux/Asterisk. -Matthew By: trashlyn (trashlyn) 2004-09-21 14:05:26 Has a problem linking to mysql_store_result upon starting asterisk with the patches. By: drmac (drmac) 2004-09-22 08:14:31 I need more info than that if I want to have any hopes of fixing it. You using latest Asterisk CVS? What version MySQL are you using? What is the exact error you are recieving? Did you uncomment the log_debug lines in the code? If so, send debug info. Need more... By: trashlyn (trashlyn) 2004-09-22 15:06:51 I ma currently using 4.0.20. the voicemail works fine, but the app_directory does not work. I am using the latest head as of 9/21/2004. Everything will compile, but when you load asterisk you will receive an error "mysql_store_result not found" By: drmac (drmac) 2004-09-23 08:58:12 Using MySQL 4.0.21 client libraries connecting to a 3.23 mysql server on another machine. Not a single problem. Try doing 'unload app_directory.so' and then 'load app_directory.so'. Does that produce any error? Is asterisk loading and functioning ok when you get this error? Make a noload line for app_directory and see if you still get the error when asterisk starts up. By: Mark Spencer (markster) 2004-10-07 15:03:10 Do you mind updating this to use the new "realtime" config stuff? By: drmac (drmac) 2004-10-07 20:27:35 fraid I don't understand. Real-time config? I haven't seen that on the -dev list. Got a URL or some source for me to read? Thanks. By: drmac (drmac) 2004-10-07 20:37:33 I just looked at app_realtime.c and I don't see how this is used to get information out of a database and into an ast_config struct that can be then parsed by current/established code. My patch 'makes' app_directory realtime by getting info out of a database each time the app is executed. By: Brian West (bkw918) 2004-10-07 21:09:33 Thats how the realtime stuff works... Look at chan_iax.c and chan_sip.c that have been ported to it... I suspect mark is working on app_voicemail.c's port to realtime API. bkw By: twisted (twisted) 2004-10-27 17:10:15 I'm assuming this is in the works still? Just requesting a status update.. By: drmac (drmac) 2004-10-28 09:00:58 yes, i am making progress. By: drmac (drmac) 2004-11-05 11:17:23.000-0600 tada! app_directory with RealTime support. I need some testers to test it with other RealTime drivers. The patch allows app_directory to use RealTime ability and retrieve directory information out of the voicemail database AND the voicemail.conf file. (YES! It works with both! am i a genius or what?) But wait! There's more! If you use the option 'hidefromdir=yes' you can keep a person(s) from being listed by the app! Wow! (Disclaimer: hidefromdir ability only valid with RealTime) (Sorry for being so giddy but earlier this morning someone quoted me to someone else on the -users list and I've felt so great. So awesome to be quoted by someone to someone else.) By: Gregory Hinton Nietsky (irroot) 2004-11-10 16:23:42.000-0600 i like it ... i went and spent most of my afternoon doing somthing similar hacking app_directory to use the vm mysql info from addons ala app_voicemail ... i did it by rewriting do_directory to look at the sql data and not the ast cfg file ... it was a good exersize pity i only saw this here once i had completed it ... ill post a diff as soon as i generate one for any one intrested but it only looks at SQL data ... By: ruok (ruok) 2004-11-16 16:04:21.000-0600 I attached app-directory-realtime-patch2.txt which cleans up the SQL part a bit, which is better because: 1) it does not assume your backend realtime config engine is SQL based and 2) i would hope at some point the sql drivers will be escaping thier queries, in which case this kind of sql injection will fail. This patch is also friendly to what I think is how the new realtime voicemail stuff works, which is instead of an 'options' column, each option is its own column. I included support for both forms of the 'hidefromdir' option. By: drmac (drmac) 2004-11-17 08:54:41.000-0600 Mr. RUok,  Could you please delete your upload? I can tell that you did not test this patch in all cases. I had originally coded the exact same code as you posted and it will not work for those database entries that have an empty options column. I have not seen an official schema for the RealTime Voicemail table so I could only go off the old schema.  Since I was the original poster of this patch, I feel that it would have been more appropriate for you to post your problems with the original patch or contact me off-list (my email is in the patch file) and let me then modify my patch rather than to completly usurp my patch with your own.  2ndly, RealTime implies an SQL backend. I don't understand your comment "assume your backend realtime config engine is SQL based". That is the entire point of RealTime; to allow Asterisk to interface with an SQL database. If you don't want SQL database, then you can continue to use the flatfile configs. --Bug Marshal-- If he can't delete the upload, could you? By: drmac (drmac) 2004-11-17 09:04:44.000-0600 If someone could post an official voicemail schema for use with RealTime, then I will happily modify my code to work with the new table schema. By: ruok (ruok) 2004-11-17 12:02:18.000-0600 Well, your patch actually didnt even work on my system, so this one at least worked. Its probably related to the way you do the sql modifications. I thought I tested empty options but guess not. Should extconfig return variables with empty values from the database? I would think so. If it did, it would fix this problem. I agree, having a defined schema would help, not only programming this, but for people trying to follow some standard for how voicemail and realtime config should work. I would think if you want voicemail in realtime table, then you are going to also want your options in there too. Mantis will not let me delete the upload file. Sorry for stepping on your bug (and ego). My post clearly described what the new file was, and was just posted for people to try out. Didn't want to open a duplicate bug because I would think it best for comments about the same problem should all be in one place. Sorry for assuming. Just trying to do some collabortive development here. Didn't realize you had the monopoly on app_directory and realtime config. By the way, I have seen reference to someday support NON SQL based realtime config (LDAP?), so thats what I was referring to. By: Mark Spencer (markster) 2004-12-01 15:31:59.000-0600 I've added some things to realtime to make it better.  Take a peek at the new pbx_realtime.c and see if that makes this easier to implement.  Thanks! By: drmac (drmac) 2004-12-02 00:05:48.000-0600 From the stuff I looked at tonight, I think only 1 or 2 lines of code in the patch will change. I will make the changes and post new diff tomorrow. By: twisted (twisted) 2004-12-15 20:47:53.000-0600 The patch never made it. Is this going to be actively persued? By: drmac (drmac) 2004-12-16 16:37:29.000-0600 Uploaded app_directory_realtime_1.3.patch Thanks Mark for modifying RealTime to support extra SQL operators. By: Trevor Peirce (trev) 2004-12-24 17:59:47.000-0600 I've got a two VM boxes for two different people with the same last name.  It seems your patch only lets me hear the first match, then I'm told there are no more compatable entries. This is with the patch applied to latest CVS. By: drmac (drmac) 2004-12-26 10:19:46.000-0600 Are you positive this is an issue with this patch and not app_directory itself? Meaning, it has worked before app_directory was database enabled? By: Trevor Peirce (trev) 2004-12-26 14:02:04.000-0600 I can't say for certain (and it would be a real PITA to test), but I seem to recall having this worked before we started using realtime. By: drmac (drmac) 2004-12-26 20:55:06.000-0600 I just verified that app_directory does indeed work with 2 entries that have same last name using the above patch. Everything worked as expected. Using: /app_voicemail.c/1.187/Mon Dec 27 01:36:46 2004// with "app_directory_realtime_1.3.patch" By: Trevor Peirce (trev) 2004-12-27 17:19:48.000-0600 I will further investigate my issue... I have several VM entries, some of which have no names, some have no recordings (ie. never initialised by user), and some don't even have folders.  I'm going to play with this a bit and will report back here if it is something with realtime. By: Mark Spencer (markster) 2005-01-05 00:37:30.000-0600 I think you might want ast_load_realtime_multientry here? By: drmac (drmac) 2005-01-05 08:18:42.000-0600 an ast_load_realtime_multientry would return an ast_config struct and what is needed is an ast_variable list. its true that a multientry would be better if you where only getting from database, but I had to program it so it worked side-by-side with RealTime and flatfiles because some people store entries in both database and flatfiles. By: Mark Spencer (markster) 2005-01-09 04:06:37.000-0600 What happens with multiple matches though? By: drmac (drmac) 2005-01-10 08:44:12.000-0600 what do you mean by multiple matches? I have verified that people with the same last name get sorted and listed correctly, so no problem there. If you mean a match in RealTime and a match in flatfile, there should be no problem. Don't know why any admin would put entries in both..well..I guess redundancy..but still, if I typed "7-2-6 (SCO) and "Joe Scott" was in both database and flatfile, then app_dir would most likely announce him twice. By: Trevor Peirce (trev) 2005-01-19 13:10:16.000-0600 Looks like Directory is only considering the first result of the realtime data. Basically it seems like something similar to select * from voicemail where context = ? LIMIT 1 is happening.  Only the first match is considered and if that isn't who you are looking to then too bad.  I've added in dummy names to all the accounts that had nothing but none of them work but the one that MySQL (or ODBC) is returning first. By: Olle Johansson (oej) 2005-02-13 13:08:39.000-0600 Any activity here? We need to complete this :-) /Housekeeping By: Mark Spencer (markster) 2005-02-13 16:45:03.000-0600 It is now possible using the new realtime multi result to do this properly I believe. By: drmac (drmac) 2005-02-13 19:50:42.000-0600 The question I need answered is: should i code app_directory to read BOTH flatfile and RealTime? or just one or the other? Reading both is kinda a bi$ch. Why? Because you need to read the flatfile into a ast_config then read the RealTime into an ast_config then merge the two. However, reading from RealTime doesn't give you the same format as reading from flatfile. app_directory's parsing code expects <extension> => <name>,<pass>,... format. You have to format this string before merging it with the flatfile config. I will revisit this code this week. I think that kpfleming's config.c code changes will help out alot. will keep you posted.. By: drmac (drmac) 2005-02-18 18:03:03.000-0600 holy crap. can't believe I did it. be sure to apply all 3 of these files: app_directory_realtime_1.3.patch config.c.patch config.h.patch For some reason or another, there are several functions in config.c but not in config.h. I have only patched the config's relative to what I needed changed for this original patch. I tested this using entries only in RealTime, only in flatfile, and entries in both RealTime and Flatfile. I even tested having the same entry in both RT and FF, and seperate entries having the same last name in both RT and FF. It all worked. By: drmac (drmac) 2005-02-18 18:05:11.000-0600 could someone remove the first 5 files above? just to help remove any confusion. keep the 3 files that were uploaded on 2-18-05. thx. By: drmac (drmac) 2005-02-18 18:08:03.000-0600 son of a'.... added 1.4.1 forgot about my neat-o debug code... By: Kevin P. Fleming (kpfleming) 2005-02-18 18:29:17.000-0600 There are multiple problems with this set of patches: - inherit_category was marked static because it was not intended to be used outside of config.c, and that's why it was not listed in config.h; if it's going to be part of the API, it must be renamed to ast_category_inherit and all existing uses need to be updated - your get_directory_realtime function will leak memory; there are multiple ways that it can return without free'ing ffcfg - you do not need to create a new config object at all; just use the ffcfg that was already created by ast_config_load - you do not need to create a new category for the context you are searching for unless one does not exist in ffcfg; if it does, just use ast_category_get to get a pointer to it, and then append the realtime-generated entries directly to it (this will eliminate the need for using inherit_category) By: Mark Spencer (markster) 2005-02-19 10:49:49.000-0600 We could just search two different contexts right? By: Kevin P. Fleming (kpfleming) 2005-02-19 11:10:58.000-0600 Yes, that would be a relatively easy solution as well. By: drmac (drmac) 2005-02-21 10:12:45.000-0600 kevin's so awesome... hopefully this fixes everything. i removed the need for inherit and hopefully created a much more memory safe patch. By: Kevin P. Fleming (kpfleming) 2005-02-21 10:25:39.000-0600 Sorry to say, rev5 has a few more bugs: - if (!cfg) means that cfg is NULL, you can't call ast_destroy_config on it, you don't have one - you don't need the "needappend" flag, just append the category to the config as soon as you create it, you can still add variables to it later - this may just be me, but please put spaces in if( and while( type constructs; they are not function calls, they shouldn't look like them... - you should not be calling get_directory_realtime in do_directory, you should be calling it in directory_exec, in place of the existing ast_config_load(DIRECTORY_CONFIG). if you don't do that, the config that gets created by get_directory_realtime will not get freed, and DIRECTORY_CONFIG is getting loaded twice unnecessarily By: drmac (drmac) 2005-02-21 11:03:32.000-0600 weeeeeeeeeeeeeeeeeee.... had to move some stuff around in the directory_exec just for you kevin, i added in the spaces By: Kevin P. Fleming (kpfleming) 2005-02-21 11:14:32.000-0600 Looks good now.. there are some left-over comments from code you just removed, but other than that it's clean and simple. Nice work :-) By: drmac (drmac) 2005-02-21 11:28:21.000-0600 reworded some comments. By: Mark Spencer (markster) 2005-03-02 23:34:42.000-0600 Added to CVS, thanks! By: Digium Subversion (svnbot) 2008-01-15 15:26:45.000-0600 Repository: asterisk Revision: 5125 U   trunk/apps/app_directory.c ------------------------------------------------------------------------ r5125 | markster | 2008-01-15 15:26:44 -0600 (Tue, 15 Jan 2008) | 2 lines Add realtime support to app_directory (bug ASTERISK-2438) ------------------------------------------------------------------------ http://svn.digium.com/view/asterisk?view=rev&revision=5125