ServiceMix 4 Example Project

I've just pushed a new version of the "Getting Started with ServiceMix 4" example webinar project to GitHub ( My goal is that this example shows best practices for creating ServiceMix 4 projects with Camel. I'm also experimenting with GitHub…

This project is an example of using OSGi, Camel, and ServiceMix together, specifically ServiceMix 4's NMR component. A number of things are shown within this project:
* use of the Camel-NMR component for inter OSGi bundle communication
* dynamic routing combining Camel's recipient list and the OSGi Service Registry
* use of Camel's Content Based Router
* multiple front-end proxies (WS and batch file)
* bridging one way (fire and forget) messaging with request-response
* and much more...

The scenario is a payment transfer service where transfer requests can be made either through a WS (SOAP/HTTP) interface or through batch files. These transfer requests are routed to banking services that can come and go at runtime (i.e. new banks can be added and removed at runtime).

This solution is a bit over-engineered, but the goal of this effort is to provide examples of best practices in creating applications using these technologies.

My plan is over the next few days to post some blog entries on aspects of this project to explain things like: how to communicate between OSGi bundles with Camel, dynamic routing to OSGi bundles that are deployed at runtime, etc. Ultimately I'll get to updating the webinar to talk through this updated code.

I'd suggest downloading the 1.0.0 version of this project ( and give it a try with ServiceMix 4.3.0-fuse-02-00 ( Please feel free to submit ideas for how I could enhance this example; I'm thinking adding in transaction support spanning two bank services would be cool, for example.

ActiveMQ Message Groups

The JMS specification does not provide a lot of guidance about use of Message Groups.

Property Name Type Set By Use
JMSXGroupID String Client The identity of the message group this message is part of
JMSXGroupSeq int Client The sequence number of this message within the group; the first message is 1, the second 2,…
JMSXGroupID and JMSXGroupSeq are standard properties clients should use if they want to group messages. All providers must support them.
and that's about it… clear, right???

This post is about how to use Message Groups within ActiveMQ. The documentation on using Message Groups within ActiveMQ has gotten much better -, but there are still some nuances that users should be aware of.

First of, why should you care about Message Groups…

Message Groups are used to ensure that one and only one message consumer is processing messages, in order, from a queue (point to point). The message producer controls which messages are in what group, if any, by setting the JMSXGroupID message property.

When message order matters, its much easier to ensure correct processing if only one message consumer is receiving the messages. The problem is that this does not scale; if you have more messages than a single consumer can process in a timely fashion, you need to add additional consumers. When you have multiple consumers receiving messages, you lose the easy ability to process messages in order; messages will be dispatched to available consumers based on many different criteria: round robin, least load, and just the fastest consumer gets more messages. So using Message Groups allows you to specify that for all the messages in that group should be processed by one and only one consumer, while all the other messages (not in groups or in different groups) can be load balanced to other consumers.

The classic example for Message Groups is a stock feed. For any given stock (IBM, MSFT, ORCL), a queue might receive messages for quotes, buy orders, sell orders, bids, etc. If you're trading a given stock, it would be important that your trading application see all of the messages for that stock in order so that you can make the best trade (not get out bid, not pay too much, buy when some one sells cheap, etc.). The challenge is that there are a lot of stocks, and a lot of messages per stock so one message consumer can not possible handle the load. Enter Message Groups. So if each message is put into a message group by stock ticker (e.g. Message Producer sets "JMSXGroupID = APPL" in the message's properties), then ActiveMQ will ensure that only one consumer will get all the messages for that group in order. Pretty cool…

So how does this work for ActiveMQ…

Message Groups are controlled by the Message Producer, and are as easy as simple setting a message property

Message message = session.createTextMessage("hey");
message.setStringProperty("JMSXGroupID", "IBM_NASDAQ_20/4/05");

That's it. Now only one message consumer will get all messages in that group (i.e. JMSXGroupID = IBM_NASDAQ_20/4/05)

Other things you can do with Message Groups…

The other producer controlled feature of Message Groups is setting the Message Group sequence id - JMSXGroupSeq. ActiveMQ requires that valid sequence numbers must be greater than 0 (i.e. start with 1). ActiveMQ takes no special action based on the sequence number, and does not enforce that they increase, are unique, etc. Sequence numbers are there as a convenience for you the developer, and it is your responsibility to make them meaningful to your application.

The only time ActiveMQ cares about sequence numbers is when they are negative (less than 0). If you set JMSXGroupSeq = −1 then ActiveMQ will close the Message Group. What does it mean for the group to be closed? Closing a Message Group means that any future messages in that Message Group could be dispatched to a different consumer. That is, its as though this was the first use of that Message Group, and ActiveMQ will assign the now re-opened (message in group after JMSXGroupSeq=-1 was sent) to one and only one consumer.

So when could a Message Group get assigned to a different consumer…

There are only two instances where a Message Group would get re-assigned to a different consumer:
  • Message Group is explicitly closed (JMSXGroupSeq = −1)
  • the original consumer goes away (consumer.close(), loss of network connectivity, consumer process dies, …)

In either of these two scenarios, ActiveMQ will reassign all future messages in that Message Group to a new consumer which will exclusively get all future messages… even if the original consumer comes back on line…

Note: there is no way to force ActiveMQ to use a specific consumer for a specific Message Group (short of there only being one consumer for that queue). So if a Message Group consumer, for example, loses network connectivity temporarily, then ActiveMQ will re-assign future messages in that group to one of the surviving consumers.

When ActiveMQ assigns (or reassigns) a Message Group, the first message in that group delivered to a consumer will have a special Boolean header set - JMSXGroupFirstForConsumer = true. This is a special flag to let a consumer know that this is the first message of a group that has been assigned to that consumer. It can allow the consumer to flush internal caches or do initial setup work that might be need to process the messages in that group.

Tips and Tricks…

Since ActiveMQ will assign Message Groups to consumers based on which consumers are available, it can be helpful to tell ActiveMQ to wait for all the consumers to come on line before ActiveMQ starts dispatching messages, and making Message Group assignments (even load balancing). To do this, you can set a couple of Destination Policy entries for that queue: consumersBeforeDispathStarts and timeBeforeDispathStarts. Both of these destination properties can into existence as of ActiveMQ 5.3.

<policyEntry queue=">" consumersBeforeDispatchStarts="2" timeBeforeDispatchStarts="2000"/>

This policy tells ActiveMQ to wait either 2 seconds (2000ms) or until 2 consumers connect. You can set either option, both, or neither per destination, or use ActiveMQ destination wildcards like in the above example which will include all queue.

The other thing to be aware of is managing a large number of Message Groups. By default, ActiveMQ uses a Hash map of Message Groups (MessageGroupHashBucketFactory) that is limited to less than 1024 unique Message Group names. If you plan on using more than 1024 Message Group IDs (JMSXGroupID), than you need to configure ActiveMQ to use a different implementation to manage the larger number of Message Group IDs.

<policyEntry queue=">">

Setting the above Destination Policy will allow ActiveMQ to manage a larger set of group ids at the expense of a little more memory consumption. If you do not make this configuration (i.e. use the default), then with a large number of ids (more than 1024), you will start to get hash map conflicts which can result in ActiveMQ reassigning groups to new consumers unexpectedly, which is not why you were using Message Groups in the first place.

So that's what I've got on ActiveMQ Message Groups. I hope you find them as useful as I have.

Getting Started with FUSE ESB 4.2 Webinar

I'm doing the Getting Started with FUSE ESB 4.2 Webinar tomorrow - 15-July

As always, I'm updating the sample code to reference the latest and greatest version of FUSE. One of the big changes I'm doing this time is making the shared WSDL file a real shared resource. Its always bugged me that I've had two+ copies of the WSDL file in the project, since it needs to be the same for everything to work correctly.

I found a great article on how to do shared resources in Maven here. So we'll see how this goes…

I figure (hope) its helpful to people that this project could be used as a starter template for future projects. It does make the Maven POMs more complicated than they need to be for this simple project, but it reflects the best practices that I've been able to find on the web. My hope is that if this is used as a template for more complicated projects, then all this extra effort will pay off.

Hope to see you tomorrow at the webinar…

Evaluating Open Source Integration Software

I've been helping a number of companies evaluate Open Source solutions to their integration problems, and I wanted to share some thoughts (and hopefully get some feedback) on things I've seen work and not work.

The biggest challenge that I've seen companies have is that they don't get the same level of help and support (for free) in evaluating Open Source versus traditional close sourced products. So the result is they underestimate the level of effort to evaluate the Open Source solution, leading to lots of frustration.

Closed source companies have a product license fee that helps offset the cost of their technical field resource(s) (generally SEs) help a customer with their evaluation. This generally includes answering RFI / RFP questions, and implementing a POC scenario. SEs are used to doing many, many POCs a year, so they can provide a lot of help (though naturally biased in their product's favor) to the evaluator in best testing the integration solution and creating a presentation to their management about the results.

With Open Source, there is no product license fee to help a company afford to expend the same level of free pre-sales support. This means that the evaluator is on their own to download, and implement a POC scenario to see if the Open Source solution meets their requirements. This is both good and bad. Good in that the evaluator has a really good sense at the end of the evaluation of the fit for the Open Source solution. The bad is the evaluator generally does not know how to use (or use well) the Open source product, so they have to ramp a steep learning curve quickly, and will most likely have a frustrating experience implementing the POC scenario. To get a true sense of the real fit of the Open source solution, either a great deal more time needs to be allocated to its evaluation (allow more time to ramp the learning curve), OR pay money to a knowledgeable consultant to help in your evaluation.

What have your experiences been?

Looking at JBI and Camel

I've been distracted by doing a couple of webinars on Camel so I haven't had a lot of time to spend on digging into my original blog post on Camel, CXF, and ServiceMix / JBI working well together. From looking at the JBI spec it seems that for WSDL 1.x, JBI does 'mandate' using the JBI XML wrapper. My sense is that since I'm having the CXF components work with / generate from a WSDL 1.1, it seems reasonable that would wrap the XML message. The other ServiceMix components are magically generating the NMR / WSDL binding for me so apparently its up for debate if the must use the JBI wrapper element - if they generate WSDL 2.0, the wrapper element is optional.

From looking at the JMS and HTTP components, its clear that there is a framework in place using Camel interceptors to add the JBI or SOAP envelopes when needed.

At this point, I'm going to let the dust settle with Camel 2.0, ServiceMix 4.x, etc. before I dig in more on how to get Camel's jbi endpoint to do the right thing with that jbi message wrapper….

Setting up Nexus as a local Maven Repository Manager

I do a lot with Apache Maven, and I kept hearing about how Nexus helps with better managing your local Maven repository, especially if you go offline a lot, which I do. I was pleasantly surprised as how easy it was to setup and that it did actually seems to speed up my Maven project builds. Sad to get excited about a product that actually does what it claims...

My needs are pretty modest in regards to Maven, and most of the Maven projects I deal with are small, so normally dealing with annoyances of offline and/or configuring m2eclipse to know about the 2 other Maven repos I use isn't that bad. However, when I started trying to build all of ServiceMix (24+ Maven projects) and started to see lots and lots of waiting on downloading various dependencies, I decided maybe I'd see if Nexus could help.

The install of the Nexus Open Source is very straightforward - untaring a tar ball - and starting the Nexus server was as easy as bin/nexus start. Sonatype provides a great (free) on-line book on Nexus - Repository Management with Nexus - that provides good instructions on installation and initial configuration. There isn't much configuration by default except for modifying your Apache Maven ~/.m2/settings.xml file to use the Nexus server versus looking on the Internet for dependencies.

The biggest challenge I had was since I was dealing with a large project which references ~12 other Maven Repositories (Nexus ships with Maven Central and Apache Repos configured) I had to add in those other repositories into Nexus so it could proxy them. Again the Nexus book provides a great section on how to know your missing a repo entry and how to add custom repos into Nexus. After I did a couple, it was very straightforward to figure out the other 10 or so custom repos that ServiceMix requires.

What I saw was that now with Nexus it felt that the ServiceMix built went much faster - maybe 10-20 % - I guessing mostly from just quick all local machine lookups on dependencies. It also felt that Nexus was a little smarter about how it downloaded dependencies from the Internet, not positive on that. Regardless, I noticed a build speed up and a dramatic reduction in network traffic during the build.

I also saw that using m2eclipse was much easier, which makes sense since both Nexus and m2eclipse are developed by the folks at Sonatype. m2eclipse will use you personal Maven ~/.m2/settings.xml file, which in my case is now configured to use Nexus, so I automagically get access to all those custom repositories I already configured in Nexus without having to painfully add into Eclipse. This is nice as I can now lookup archetypes in non-Central based repos, and do the "so what dependency do I need to add for this Class" search as well on non Maven Central hosted dependencies.

Overall a positive experience. Nice work Sonatype!

Eclipse Galileo, 64-bit Java, Java 5, and Snow Leopard...

I thought I'd provide some details on the adventures I had in getting Eclipse Galileo (3.5) working on my Mac running Snow Leopard.

As detailed in this great DZone article by Zviki Cohen, Snow Leopard only ships with a 64-bit version of Java 6. In fact, if you upgrade from Leopard to Snow Leopard, the installer will actually uninstall Java 5 and replace it with symbolic links to Java 6.

This proved to by a challenge for my Eclipse needs as 1) you need to make a decision about which flavor (32 bit or 64 bit; Carbon or Cocoa) of Eclipse to install on the Mac, and 2) Eclipse gets confused by the symbolic link to Java 5 and thinks you really do have a real Java 5 JDK installed.

After reading the Dzone article, I decided to go with the 64-bit Cocoa version as that seems in line with where Java and Mac OS are going. Also, as I was going through my installation Eclipse Galileo SR 1 (3.5 SR 1) shipped which provided 64-bit Cocoa versions for all Eclipse profiles (JEE, Java, etc.). The SR 1 release eliminated many of the hoops Zviki talks about in this article, making it a much easier path to choose. After using this version of Eclipse for about a week, I'd say I don't notice a big difference from the 32-bit version, but it certainly seems stable and having it do builds with really big projects (ServiceMix) it seems relatively fast. I do notice my Laptop fan turning on and both cores maxed out, which overall I assume is a good thing as that means that its taking full advantage of my computer during a large complex build...

The next big task was fixing the Java 5 issue. This was an especial challenge / need for me since 1) most FUSE projects still support Java 5 and 2) m2eclipse (Apache Maven support in Eclipse) gets confused importing Maven projects that use Java 5 with Eclipse thinking its got a JDK 5 installed but really its Java 6 (see above note about Snow Leopard linking JDK 5 dir to JDK 6). This post at OneSwarm details how to hack a real JDK 5 onto your Snow Leopard OS. The highlight being

In the terminal:
Get the java 5 that was included in 10.5 "leopard" and unpack
cd /tmp/
tar -xvzf java.1.5.0-leopard.tar.gz
Move it to your System java folder (password needed)
sudo mv 1.5.0 /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0-leopard

Tell OS X that java 5 actually is java 5
cd /System/Library/Frameworks/JavaVM.framework/Versions/
sudo rm 1.5.0
sudo ln -s 1.5.0-leopard 1.5.0

I also updated the /System/Library/Frameworks/JavaVM.framework/Versions/1.5 link to point to the 1.5.0 link for completeness.

After doing this I updated Eclipse to use the real JDK 5, and m2eclipse with my ServiceMix project built much better. Now I can do both Java 5 and 6 development on 64-bit Cocoa version of Eclipse - all nice and shiny new :-)

Now onto some real coding... :-)

First Try at Blogging - Digging into ServiceMix and Camel Code

Well, here's the classic "Its my first blog post" statement, so let's see how this goes :-)

I've been working with ServiceMix and Camel since Feb 2009 when I joined the Progress FUSE technical field. Now I'm looking to take the next step and start digging much deeper into the code. So I thought this experience of digging into the code would also be a code this to start blogging about.

The first thing I was going to look into is how some of the ServiceMix components handle messages with or without a JBI Message wrapper. Specifically, the CXF BC and SE components, by default, expect an XML message to be wrapped. The challenge is that messages sent from Camel via the JBI endpoint do not / can not be wrapped automatically (i.e. I'd need to code in the wrapping myself). Putting in a simple XSLT transform to wrap my XML messages in the minimal JBI message elements isn't a big deal, but its something I'd expect Camel to do for me (I am a lazy developer at heart :-) ).

Right now I'm getting my development environment setup so I'm working off the ServiceMix and Camel Subversion repos. I've also taken this opportunity to setup the Nexus Open Source version to help better manage interactions with the various Maven repos, and Nexus does appear to help speed up a full ServiceMix build.

Next steps are to find the JBI wrapper code in the CXF components to understand how they work. A big question for me is, it looks like the JBI Message wrapper is optional in the JBI spec, so shouldn't the servicemix-cxf components just handle the presence, or not, of the JBI wrapper automatically versus, by default, requiring other JBI components to do it? I understand that I can disable the JBI wrapper by setting useJBIWrapper=false on the CXF components, its just annoying that I have to disable something for many / most other components to interact with it.