Evaluating Open Source Integration Software

I've been helping a number of companies evaluate Open Source solutions to their integration problems, and I wanted to share some thoughts (and hopefully get some feedback) on things I've seen work and not work.

The biggest challenge that I've seen companies have is that they don't get the same level of help and support (for free) in evaluating Open Source versus traditional close sourced products. So the result is they underestimate the level of effort to evaluate the Open Source solution, leading to lots of frustration.

Closed source companies have a product license fee that helps offset the cost of their technical field resource(s) (generally SEs) help a customer with their evaluation. This generally includes answering RFI / RFP questions, and implementing a POC scenario. SEs are used to doing many, many POCs a year, so they can provide a lot of help (though naturally biased in their product's favor) to the evaluator in best testing the integration solution and creating a presentation to their management about the results.

With Open Source, there is no product license fee to help a company afford to expend the same level of free pre-sales support. This means that the evaluator is on their own to download, and implement a POC scenario to see if the Open Source solution meets their requirements. This is both good and bad. Good in that the evaluator has a really good sense at the end of the evaluation of the fit for the Open Source solution. The bad is the evaluator generally does not know how to use (or use well) the Open source product, so they have to ramp a steep learning curve quickly, and will most likely have a frustrating experience implementing the POC scenario. To get a true sense of the real fit of the Open source solution, either a great deal more time needs to be allocated to its evaluation (allow more time to ramp the learning curve), OR pay money to a knowledgeable consultant to help in your evaluation.

What have your experiences been?