Monday, November 15, 2010

ServiceMix 4 Example Project

I've just pushed a new version of the "Getting Started with ServiceMix 4" example webinar project to GitHub ( My goal is that this example shows best practices for creating ServiceMix 4 projects with Camel. I'm also experimenting with GitHub…

This project is an example of using OSGi, Camel, and ServiceMix together, specifically ServiceMix 4's NMR component. A number of things are shown within this project:
* use of the Camel-NMR component for inter OSGi bundle communication
* dynamic routing combining Camel's recipient list and the OSGi Service Registry
* use of Camel's Content Based Router
* multiple front-end proxies (WS and batch file)
* bridging one way (fire and forget) messaging with request-response
* and much more...

The scenario is a payment transfer service where transfer requests can be made either through a WS (SOAP/HTTP) interface or through batch files. These transfer requests are routed to banking services that can come and go at runtime (i.e. new banks can be added and removed at runtime).

This solution is a bit over-engineered, but the goal of this effort is to provide examples of best practices in creating applications using these technologies.

My plan is over the next few days to post some blog entries on aspects of this project to explain things like: how to communicate between OSGi bundles with Camel, dynamic routing to OSGi bundles that are deployed at runtime, etc. Ultimately I'll get to updating the webinar to talk through this updated code.

I'd suggest downloading the 1.0.0 version of this project ( and give it a try with ServiceMix 4.3.0-fuse-02-00 ( Please feel free to submit ideas for how I could enhance this example; I'm thinking adding in transaction support spanning two bank services would be cool, for example.

Tuesday, September 28, 2010

ActiveMQ Message Groups

The JMS specification does not provide a lot of guidance about use of Message Groups.

Property Name Type Set By Use
JMSXGroupID String Client The identity of the message group this message is part of
JMSXGroupSeq int Client The sequence number of this message within the group; the first message is 1, the second 2,…
JMSXGroupID and JMSXGroupSeq are standard properties clients should use if they want to group messages. All providers must support them.
and that's about it… clear, right???

This post is about how to use Message Groups within ActiveMQ. The documentation on using Message Groups within ActiveMQ has gotten much better -, but there are still some nuances that users should be aware of.

First of, why should you care about Message Groups…

Message Groups are used to ensure that one and only one message consumer is processing messages, in order, from a queue (point to point). The message producer controls which messages are in what group, if any, by setting the JMSXGroupID message property.

When message order matters, its much easier to ensure correct processing if only one message consumer is receiving the messages. The problem is that this does not scale; if you have more messages than a single consumer can process in a timely fashion, you need to add additional consumers. When you have multiple consumers receiving messages, you lose the easy ability to process messages in order; messages will be dispatched to available consumers based on many different criteria: round robin, least load, and just the fastest consumer gets more messages. So using Message Groups allows you to specify that for all the messages in that group should be processed by one and only one consumer, while all the other messages (not in groups or in different groups) can be load balanced to other consumers.

The classic example for Message Groups is a stock feed. For any given stock (IBM, MSFT, ORCL), a queue might receive messages for quotes, buy orders, sell orders, bids, etc. If you're trading a given stock, it would be important that your trading application see all of the messages for that stock in order so that you can make the best trade (not get out bid, not pay too much, buy when some one sells cheap, etc.). The challenge is that there are a lot of stocks, and a lot of messages per stock so one message consumer can not possible handle the load. Enter Message Groups. So if each message is put into a message group by stock ticker (e.g. Message Producer sets "JMSXGroupID = APPL" in the message's properties), then ActiveMQ will ensure that only one consumer will get all the messages for that group in order. Pretty cool…

So how does this work for ActiveMQ…

Message Groups are controlled by the Message Producer, and are as easy as simple setting a message property

Message message = session.createTextMessage("hey");
  message.setStringProperty("JMSXGroupID", "IBM_NASDAQ_20/4/05");

That's it. Now only one message consumer will get all messages in that group (i.e. JMSXGroupID = IBM_NASDAQ_20/4/05)

Other things you can do with Message Groups…

The other producer controlled feature of Message Groups is setting the Message Group sequence id - JMSXGroupSeq. ActiveMQ requires that valid sequence numbers must be greater than 0 (i.e. start with 1). ActiveMQ takes no special action based on the sequence number, and does not enforce that they increase, are unique, etc. Sequence numbers are there as a convenience for you the developer, and it is your responsibility to make them meaningful to your application.

The only time ActiveMQ cares about sequence numbers is when they are negative (less than 0). If you set JMSXGroupSeq = −1 then ActiveMQ will close the Message Group. What does it mean for the group to be closed? Closing a Message Group means that any future messages in that Message Group could be dispatched to a different consumer. That is, its as though this was the first use of that Message Group, and ActiveMQ will assign the now re-opened (message in group after JMSXGroupSeq=-1 was sent) to one and only one consumer.

So when could a Message Group get assigned to a different consumer…

There are only two instances where a Message Group would get re-assigned to a different consumer:
  • Message Group is explicitly closed (JMSXGroupSeq = −1)
  • the original consumer goes away (consumer.close(), loss of network connectivity, consumer process dies, …)

In either of these two scenarios, ActiveMQ will reassign all future messages in that Message Group to a new consumer which will exclusively get all future messages… even if the original consumer comes back on line…

Note: there is no way to force ActiveMQ to use a specific consumer for a specific Message Group (short of there only being one consumer for that queue). So if a Message Group consumer, for example, loses network connectivity temporarily, then ActiveMQ will re-assign future messages in that group to one of the surviving consumers.

When ActiveMQ assigns (or reassigns) a Message Group, the first message in that group delivered to a consumer will have a special Boolean header set - JMSXGroupFirstForConsumer = true. This is a special flag to let a consumer know that this is the first message of a group that has been assigned to that consumer. It can allow the consumer to flush internal caches or do initial setup work that might be need to process the messages in that group.

Tips and Tricks…

Since ActiveMQ will assign Message Groups to consumers based on which consumers are available, it can be helpful to tell ActiveMQ to wait for all the consumers to come on line before ActiveMQ starts dispatching messages, and making Message Group assignments (even load balancing). To do this, you can set a couple of Destination Policy entries for that queue: consumersBeforeDispathStarts and timeBeforeDispathStarts. Both of these destination properties can into existence as of ActiveMQ 5.3.

        <policyEntry queue=">" consumersBeforeDispatchStarts="2" timeBeforeDispatchStarts="2000"/>

This policy tells ActiveMQ to wait either 2 seconds (2000ms) or until 2 consumers connect. You can set either option, both, or neither per destination, or use ActiveMQ destination wildcards like in the above example which will include all queue.

The other thing to be aware of is managing a large number of Message Groups. By default, ActiveMQ uses a Hash map of Message Groups (MessageGroupHashBucketFactory) that is limited to less than 1024 unique Message Group names. If you plan on using more than 1024 Message Group IDs (JMSXGroupID), than you need to configure ActiveMQ to use a different implementation to manage the larger number of Message Group IDs.

        <policyEntry queue=">">

Setting the above Destination Policy will allow ActiveMQ to manage a larger set of group ids at the expense of a little more memory consumption. If you do not make this configuration (i.e. use the default), then with a large number of ids (more than 1024), you will start to get hash map conflicts which can result in ActiveMQ reassigning groups to new consumers unexpectedly, which is not why you were using Message Groups in the first place.

So that's what I've got on ActiveMQ Message Groups. I hope you find them as useful as I have.

Wednesday, July 14, 2010

Getting Started with FUSE ESB 4.2 Webinar

I'm doing the Getting Started with FUSE ESB 4.2 Webinar tomorrow - 15-July

As always, I'm updating the sample code to reference the latest and greatest version of FUSE. One of the big changes I'm doing this time is making the shared WSDL file a real shared resource. Its always bugged me that I've had two+ copies of the WSDL file in the project, since it needs to be the same for everything to work correctly.

I found a great article on how to do shared resources in Maven here. So we'll see how this goes…

I figure (hope) its helpful to people that this project could be used as a starter template for future projects. It does make the Maven POMs more complicated than they need to be for this simple project, but it reflects the best practices that I've been able to find on the web. My hope is that if this is used as a template for more complicated projects, then all this extra effort will pay off.

Hope to see you tomorrow at the webinar…

Thursday, April 8, 2010

Evaluating Open Source Integration Software

I've been helping a number of companies evaluate Open Source solutions to their integration problems, and I wanted to share some thoughts (and hopefully get some feedback) on things I've seen work and not work.

The biggest challenge that I've seen companies have is that they don't get the same level of help and support (for free) in evaluating Open Source versus traditional close sourced products. So the result is they underestimate the level of effort to evaluate the Open Source solution, leading to lots of frustration.

Closed source companies have a product license fee that helps offset the cost of their technical field resource(s) (generally SEs) help a customer with their evaluation. This generally includes answering RFI / RFP questions, and implementing a POC scenario. SEs are used to doing many, many POCs a year, so they can provide a lot of help (though naturally biased in their product's favor) to the evaluator in best testing the integration solution and creating a presentation to their management about the results.

With Open Source, there is no product license fee to help a company afford to expend the same level of free pre-sales support. This means that the evaluator is on their own to download, and implement a POC scenario to see if the Open Source solution meets their requirements. This is both good and bad. Good in that the evaluator has a really good sense at the end of the evaluation of the fit for the Open Source solution. The bad is the evaluator generally does not know how to use (or use well) the Open source product, so they have to ramp a steep learning curve quickly, and will most likely have a frustrating experience implementing the POC scenario. To get a true sense of the real fit of the Open source solution, either a great deal more time needs to be allocated to its evaluation (allow more time to ramp the learning curve), OR pay money to a knowledgeable consultant to help in your evaluation.

What have your experiences been?