Yesterday I gave a talk at the Ruby Users of Minnesota (RUM) meeting where I presented some slides and gave a demo on some of the current Continuous Integration (CI) tools that are available for Ruby and Ruby on Rails.
The technologies reviewed were Continuous Builder, Cerberus, CruiseControl, and CruiseControl.rb. I also compiled and presented a comparison matrix much like Aslak Hellesoy has done on the DamageControl web site.
All the artifacts that I used for the demo can be found on the Docs page.
Ross Niemi's Musings
Ross Niemi's Musings 
Hey Ross,
Great work. I have a few corrections for cruisecontrol.rb, though.
* We support any builder that returns 1 on fail and 0 on pass, this should include ant, nant, etc. You can see we currently have a java build using ant on http://cruisecontrolrb.thoughtworks.com/
* We also provide access to build artifacts. You simply have to put them in a special directory for us. If you look at the demo page, cruise has rcov artifacts. (I think this is the type of thing you meant) – this isn’t built in, you just specify to copy html files to cruise in your build file.
* Rspec should work, in a similar fashion, though if you mean do we do special formatting for it, we don’t
* We do have special formatting for Test::Unit failures. Find yourself a failing build on the demo site and see if this is what you meant.
Getting it to install as a gem is coming.
Also a feature that you left out that people seem to like a lot is clickable errors. Any path with a line number in the logs will become a link to the source code. It’s pretty sweet.
It’s hard though, I don’t think we particularly want to compete on a feature matrix. CC.rb’s selling point is that it’s super easy to install.
Thanks again for the great post!
Hi Jeremy,
Thanks very much for the feedback. I’ve updated what I have missed or reported in error. I understand that some of the features are pretty gray and confusing. I’m hoping to improve on it.
I’m trying to mimic the strategy that Aslak did for his matrix: Give credit for a feature if the tool has “direct” support for it. In those cases where a feature could be achieved through configuration or third party tool, I would give credit and provide an explanation.
What I want to avoid is to blanket “Yes” for the catch alls. For example, if a tool offers support for executing builders via the command line, I’m leery to say “well, that means this tool can support any builder” . This is a somewhat true argument, but when I’m evaluating a CI tool, I’m looking to see what conveniences does the tool give the user so they don’t have to resort to maintaining custom platform-specific scripts.
I’m not saying that a command-line builder is bad and there are situations when I will use it; however, if the user experience can be improved (i.e. sharing of information between the CI tool) and the builder and the CI tool team addresses it, I think it should be credited.
Formatted output is another good example where credit should be given.
[…] Gestern gefunden, heute im Rahmen unseres Forschungstages installiert: CruiseControl.rb ist nach Java und .Net die […]
Is your review limited only to free tools? I am asking because our Parabuild work with RoR just fine.
Regards,
Slava Imeshev
Slava,
My review is not limited to free tools. I have never used Parabuild before, but if you wish to use the matrix I created as a template, please fill it in with your product information, email it back to me, and I’ll update what I have.
Also, please highlight any additional Ruby / Rails features that Parabuild may offer.
My contact information is on the “About” page.
Hey Ross,
How can I reach comparison matrix page? I clicked on it and returned a blank page
Hi Yi,
I just tried it now and I’m able to pull up the comparison matrix on the document page. If the problem persists then please ping me again.
Hi! I just published a very simple Continuous Integration App for RubyOnRails + SVN/GIT. Maybe you should give it a try:
http://github.com/felipegiotto/Inotegration/tree/master
I took some ideas of measurement and testing tools and, after trying some CI tools and not liking anyone of them, I decided to build my own, without needing to build big XML files or any other configuration. Just the way Rails was made to be.
If you like, please send me some feedback..
PS: Sorry for using your blog to send this, but I just want to make a good CI tool for the Rails Community.
Best regards,
Felipe Giotto.
Hi Ross.
Thanks for the great work you have done. How about TeamCity(http://www.jetbrains.com/teamcity/) . if you ever review CI tools for rails again, I would appreciate if you would also include TeamCity in your evaluation.
Thanks, Jarl
We developed a web service which provides continuous integration for ruby apps in the cloud called Railsonfire. It’s really simple to setup. Just connect your GitHub account. We are still in beta and would really appreciate to get some feedback.
=> http://www.railsonfire.com
Cheers,
Mo