Summary:ASTERISK-14690: [branch] Add support for distributing device state and MWI via XMPP PubSub
Reporter:Bradley Watkins (marquis)Labels:
Date Opened:2009-08-21 15:42:25Date Closed:2010-06-15 12:06:28
Versions:Frequency of
Environment:Attachments:( 0) distributed_devstate-XMPP.txt
Description:Distributing device state in this way opens up the possibility of having geographically distributed clusters, as well as having integration with XMPP clients and other components that are PubSub-aware.


This feature is being developed in a team branch and can be checked out.

Comments:By: Bradley Watkins (marquis) 2009-08-21 15:43:58

This is definitely not ready to be committed at the present time, but this issue is being opened to track progress.

By: Leif Madsen (lmadsen) 2009-08-21 15:57:40

Assigned to myself for some initial testing and feedback.

By: Bradley Watkins (marquis) 2009-09-01 17:05:03

As an update, this branch is now feature-complete, so heavy testing may commence.  I'll be working on both testing and some real documentation for this feature.

By: phsultan (phsultan) 2009-09-01 17:35:06

This is a great feature!

Do you recommend any PubSub client in particular to work with for further testing?

By: Bradley Watkins (marquis) 2009-09-01 18:41:49

AFAIK, there  is only one client that will do anything particularly useful with the PubSub items being published.  It's a modified version of Pidgin that is capable of setting device states based on user presence, as well as reacting to device state changes and change user presence accordingly.  But it's not readily available at the moment.  I can get you a copy of the code if you'd like, though.

By: Leif Madsen (lmadsen) 2009-10-07 18:22:49

Which server should I be using to distribute these events? Is there any documentation yet for how to set this up?

By: Leif Madsen (lmadsen) 2009-10-07 19:18:54

Well, I got a little further. I have ejabberd setup and Asterisk is now connected to it. However, I get this error when it tries to do the PubSub stuff:

[Oct  7 20:14:00] ERROR[9040]: res_jabber.c:2776 aji_handle_pubsub_error: Error performing operation on PubSub node device_state, 409.

I'll probably talk to you tomorrow on IRC.

By: Leif Madsen (lmadsen) 2009-10-08 07:05:39

Also a note I found in reviewboard states that this was tested against Tigase XMPP server, for anyone else looking to do some testing. I'm working on getting all the java stuff working so Tigase will start correctly. My opinion is that ejabberd is a million times easier to setup, so it'd be nice if we could make this work with that server; however I don't know how good mod_pubsub is in ejabberd compared to Tigase.

By: Leif Madsen (lmadsen) 2009-10-08 18:52:59

I just got thinking that we probably need a better way of distributing the buddies than having to specify all them in the jabber.conf file. The reason this is a problem is when you start to get larger installations, that you have the problem that every time you add a new server, you have to go to all the existing servers and update the jabber.conf file.

By: Bradley Watkins (marquis) 2009-10-08 20:49:26

The problem there is that you're confined by what's allowed in XEP-0060, and unfortunately that means modifying affiliations by individual JID (as opposed to the various subscription access models, which are more flexible).

See here for details:

That's said, I *can* add support for modifying affiliations on the fly via CLI commands (possibly AMI too if it makes sense).  I hadn't really done that simply due to the effort involved, which while not monumental is nontrivial.

But you'll still have to add the individual JIDs in jabber.conf so that on startup Asterisk has a list of JIDs to add to nodes as they are created and their affiliations are configured.  I don't really see any way around that.  But being able to dynamically modify the affiliations would at least save you from the hassle of having to restart Asterisk and, worse still, delete all the existing nodes (much more cumbersome when using XEP-0248 mode which is itself very much preferable if you're using this feature to integrate with appropriate end-user clients).

By: Leif Madsen (lmadsen) 2009-10-09 07:50:34

OK, the initial documentation is submitted for review and use by testers. Please let me know if there is anything that is obviously missing or broken. I will try my best to update the document, as time allows.

Thanks to Russell Bryant for the extremely well written distributed_devstate.txt document. I've re-utilized the outline from that document, and used the 'Testing Distributed Device State' section as that really was the easiest method of testing. No point in solving the same problem twice :)


By: Leif Madsen (lmadsen) 2009-11-03 14:14:03.000-0600

Hmmmm, so I guess there isn't anything wrong with my documentation...

What else can be done here before we think about maybe merging this?

By: Bradley Watkins (marquis) 2009-11-03 15:39:06.000-0600

Based on some of the discussion we had on #asterisk-bugs (or was it -dev?) I think I need to add a way to dynamically add JIDs to the affiliations via CLI and probably AMI as well.

That will make it much easier to add new nodes to the cluster without laborious manual intervention.

I'll see about getting around to it this week and update my branch and the review request appropriately.

By: kael (kael) 2009-11-04 16:03:47.000-0600

I haven't tested the XMPP Pubsub support yet but I'd have some suggestions based on what I've read.

It may be better to use Pubsub-On-Jid as defined by http://xmpp.org/extensions/xep-0060.html#presence.

This would have two advantages :

- It'd make possible to define pseudo-standard Pubsub nodes that could hence be automatically discovered by all clients (ideally, defining such nodes in a XEP would be better). Whereas using classic Pubsub doesn't make easy to automatically discover nodes ;

- Clients could easily subscribe and unsubscribe for events notifications by using filtered notifications with '+notify' as defined by http://xmpp.org/extensions/xep-0060.html#filtered-notifications. This would be a way to temporarily suspend notifications.

There could be for example the following node 'http://asterisk.org/notification/voicemail' and clients wishing to receive notifications for such events would simply have to publish 'http://asterisk.org/notification/voicemail+notify' in their capability features.

BTW, which format do use in the payload ? Do you use PIDF ? I'd be curious to know what Pubsub messages look like.

By: Bradley Watkins (marquis) 2009-11-04 17:35:26.000-0600

Are you aware of any XMPP servers that actually support Pubsub-on-a-JID?  Responses to disco#info queries on at least three types of servers I have running (Openfire, ejabberd, Tigase) yield no responses with that feature and a quick grep of their source reveals no strings with pubsub-on-a-jid (case-insensitive).

I'm also not entirely sure it works the way one would want a cluster of Asterisk systems to be publishing device states and MWI events, but perhaps I don't fully understand the feature.

As for the XML format of the nodes themselves, it's a very straightforward serialization of Asterisk's internal representation of the events.  I considered PIDF for the device state events, at least, but decided against it due to the increased complexity and the fact that as far as I could tell there were no standard representations that would adequately fit Asterisk's device state model.

I'm not entirely averse to changing the XML format for device states if there is an existing PIDF extension that fits well, but if it requires a new extension anyway I'd just as soon keep the current, simpler format at least for the initial implementation.

By: Leif Madsen (lmadsen) 2010-01-19 13:53:57.000-0600

Pinging this issue as I'm quite interested in moving this forward (somehow) so we can ideally get it into trunk before it gets branched to 1.8. I think this is an important feature, so would like to hear thoughts on what needs to be done to make this usable in the real world. I'm sure there are some nice things we still would like to have, but those can be added as additional features in the future.

What is the minimum frame work that is needed to make this usable? I think we're probably almost there (if not all the way there) as I tested this and wrote up some documentation. It seems to work as advertised.

By: Bradley Watkins (marquis) 2010-01-20 11:59:39.000-0600

It's definitely in a usable state as-is, since we use it in production already. ;)

But insofar as what's needed before a merge, I guess there is a judgement call (not mine) to be made about whether a command to easily add new JIDs to the affiliations of existing nodes is a requirement or merely a nice-to-have.

Other than that, I think the code just generally needs a review from more sets of eyes and (once any spotted issues are fixed, of course) it's ready to go.

By: Leif Madsen (lmadsen) 2010-01-21 11:01:59.000-0600

I can't remember how hard it was to add them before -- I think I documented it, but I haven't tried this out again since my documentation of the feature.

I'd say anything we can do to make things easier for the user is something we should try and do, although it sounds like 1.8 may be relaxed a bit in terms of new features being added as we go, so it *may* be possible to add the functionality later on.

However, if you were willing to add it now, then I'm certainly not going to object, and I'll even test again on a fresh install and update the documentation :)

By: Bradley Watkins (marquis) 2010-01-21 18:09:30.000-0600

At this point, the easiest way by far is to temporarily shut down the existing hosts, delete the PubSub nodes, and bring it back up with the new member system in place.

You *could* manually make the necessary changes, but depending on the number of mailboxes and/or devices that might well take a very long time and would unquestionably be tedious.  It's the very kind of repetition for which computers are uniquely suited. :)

Even deleting the nodes if you're using XEP-0248 mode rather than XEP-0060 would take a while on a large system if you use the built-in commands.  If you're storing your PubSub nodes in a database then a not-particularly-clever DELETE FROM will solve all that ails you in that regard.

Unfortunately, I don't have any spare cycles at the moment to make this happen.  So it might need to wait if it's a "must have".

I don't think that it is, personally, but I don't regularly add new hosts.  So I'm probably a little biased.

The code to do it wouldn't be even remotely invasive, so if the policy for 1.8 and features is even a little relaxed then adding the necessary commands in a 1.8.x is certainly an option.

By: Bradley Watkins (marquis) 2010-01-21 18:45:21.000-0600

After thinking about this a little more, I think I can distill the question down to its most basic form:

Is online insertion/removal of a new host into the distributed eventing system a requirement for this to be merged?

If it is, then it will have to wait.
If not, then it's good as-is.

By: Leif Madsen (lmadsen) 2010-01-22 10:41:03.000-0600

When you say new host, do you mean new server, or new end-point? If you could define 'host' that might help a lot. Also, what is the work around in the clearest form so I can update the documentation?

I've somewhat lost what feature we're talking about here :)

By: Bradley Watkins (marquis) 2010-01-26 18:26:06.000-0600

Basically, I mean anything that the Asterisk instances are expected to add to the affiliations list as an owner.  More often than not, that will mean new Asterisk instances.  But strictly speaking that's not necessary.  For example, you could have an outside process that needs publish access to the PubSub nodes.  And whatever JID that process is using would need to be in the affiliations list.

The way to add a new JID to the affiliations is:
1) Bring down all but one of the Asterisk instances
2) Use the built-in commands to delete the nodes
3) Stop the last instance
4) Make the addition(s) to the buddy list in jabber.conf
5) Restart all Asterisk instances

You could also bring them all down and run an external app/script to delete the nodes (or change them on the fly, for that matter).

The same would go for removing a JID from the affiliations list.

Is that clear enough?

By: Leif Madsen (lmadsen) 2010-01-28 10:44:14.000-0600

I think that gives me enough information -- let me see if I understand correctly though.

1) When you say Asterisk instance, you mean, "I'm adding a new server to the cluster".

2) What is the built-in command to delete the nodes? Are these in Asterisk, or in the Tigase XMPP server?

3) Stop the last running Asterisk server?

4) Add the new buddies in the jabber.conf file across all servers (can this be done via a database using the static database configuration from extconfig.conf so that they will all be loaded across the Asterisk server upon restart?)

5) Bring up all Asterisk servers?

I think I'm getting you now -- just double checking what you mean in all the steps. I went back and re-read through all the comments, so I think I'm more caught up to speed now.

By: Bradley Watkins (marquis) 2010-01-28 11:53:11.000-0600

1) Yes

2) jabber delete node <jabber connection name> <pubsub node name>
This is an Asterisk CLI command.

3) Yes

4) Yes.  Using a static database configuration would be advantageous here as you suggest, but I haven't actually tested that.  res_jabber uses the standard Asterisk config system, though, so I don't foresee any problems there.

5) Yes

By: Leif Madsen (lmadsen) 2010-02-24 09:39:38.000-0600

Note there is a bug which I've marked as a child of this that needs to be resolved prior to this going into trunk. It appears to be a minor bug though.

By: Marcello Ceschia (marcelloceschia) 2010-02-24 13:46:14.000-0600

We did some tests this week.
Publication works well, also the subscription.
But in some case the new status will not be set.

I can see that the status is being published and the remote server gets a notification with the new status. But asterisk will not set this.

So i will do some debug to find out where the problem is.

By: Tilghman Lesher (tilghman) 2010-04-09 10:55:33

Marquis asked:
Is online insertion/removal of a new host into the distributed eventing system a requirement for this to be merged?

My answer is no, as long as documentation makes it clear the steps necessary to do this (add/remove/change nodes).  I would suggest a Perl/Python/whatever script for the necessary maintenance to the node list, to be added in the contrib/scripts/ directory.

Also, the issue identified by marcelloceschia in the previous note needs to be verified and fixed, before this can be committed to trunk.  Distributed devicestate needs to be reliable.

By: Marcello Ceschia (marcelloceschia) 2010-04-09 15:18:59


The last issue I mentioned was no issue with the asterisk code, it was something with my jabber server (openfire).
I fixed this, and since than the distributed devicestate works very well.
I would vote for this in trunk.

By: Tilghman Lesher (tilghman) 2010-04-14 17:20:26

At this point, what this needs is a document describing how to set this up, add/change/remove nodes, and anything else that you may decide is pertinent.  Following the creation of such a document, this can be committed.

By: Leif Madsen (lmadsen) 2010-04-15 12:38:51

Well, part of the document is already done -- it's attached to this issue.

By: Bradley Watkins (marquis) 2010-04-26 14:49:09

marcelloceschia:  What was necessary to make this work with Openfire?  i.e., was it just a config problem or were there specific tweaks needed?

By: Marcello Ceschia (marcelloceschia) 2010-04-27 09:11:29

I had to patch the openfire.jar.
I can attach the modified jar file (openfire version 3.6.4)

The trunk of openfire fixed this issue.

By: Leif Madsen (lmadsen) 2010-04-27 11:05:12

marcelloceschia: I think if you just let us know what revision of OpenFire fixed the issue then we can state you need a version newer than that. Is there a version out yet with the fixes in it? Would it be included in 3.6.5 of OpenFire?

By: Marcello Ceschia (marcelloceschia) 2010-04-28 02:31:08

lmadsen: the trunk of openfire fixed this issue, but I dont konw if there will be a 3.6.5 - the last release was in May 2009

By: Leif Madsen (lmadsen) 2010-04-28 09:43:08

marcelloceschia: I guess that's why I was asking for the revision number it was resolved in so we could point people to that revision (or I guess we could just say "fixed in versions newer than 3.6.4")

By: Tilghman Lesher (tilghman) 2010-05-24 21:02:02

Just as an FYI, I've just updated the branch to current SVN, to get it one step closer to merging.  Please verify that my update did not break the functionality of this branch.

By: Bradley Watkins (marquis) 2010-06-03 16:51:00

Thanks for updating the branch, that's been needed for some time.  I just noticed when trying to test this, though, that there is a superfluous '/' that snuck in on *my* last commit.

Yay for compile-testing before a commit, eh? :S

Anyway, that also brought to the fore the fact that my cert has expired and so I'm working with jpeeler on getting that resolved.  In the meantime, I'm perfectly capable of using vim to remove the offending character and continue testing, so I should have some results either later today or tomorrow.

By: Digium Subversion (svnbot) 2010-06-15 12:06:23

Repository: asterisk
Revision: 270519

U   trunk/CHANGES
U   trunk/configs/jabber.conf.sample
A   trunk/doc/distributed_devstate-XMPP.txt
U   trunk/include/asterisk/jabber.h
U   trunk/res/res_jabber.c

r270519 | tilghman | 2010-06-15 12:06:23 -0500 (Tue, 15 Jun 2010) | 10 lines

Add distributed devicestate via the XMPP protocol.

(closes issue ASTERISK-14690)
Reported by: Marquis
      distributed_devstate-XMPP.txt uploaded by lmadsen (license 10)
Tested by: Marquis, lmadsen, marcelloceschia

Review: https://reviewboard.asterisk.org/r/351/