CI Feature Matrix

There are many Continuous Integration systems available. This page is an attempt to keep an unbiased comparison of as many as possible of them (see another comparison on Wikipedia). The goals are:

  • Make it easier to choose an appropriate CI tool for your project.
  • "Healthy competition aid" for the people involved in the development of these various CI systems.

Note: If you want a new feature in one of these, please use the respective project's issue trackers. Don't add feature requests here.

Note: This is starting as a straight copy of the document at codehaus since that word is that it will be archived (see comments on the page at codehaus).

Legend:
The feature exists
The feature exists, but is buggy or not well tested
The feature is planned for the near future
The feature doesn't exist

Project info CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed easyCIS RedJack
ElectricCommander Project info
Project origin ThoughtWorks ThoughtWorks ThoughtWorks ThoughtWorks Jay Flowers Timpani Software Mozilla Project Brian Warner Urbancode Urbancode Atlassian
PMEase PMEase Apache Gump Apache CSH Consult OpenMake Software
 OpenMake Software
 Viewtier Systems John Keiser Zutubi JetBrains java.net VSoft Technologies Hericus Software Vaclav Zahradnik Michal Nowikowski
 Electric Cloud
 Project origin
Open Source
Source provided to customers
- core

- selected plugins
Open Source
Implementation language Java C# Ruby Java C# C# Perl Python Java Java Java Java Java Java
Python
Java C# Java, C++, C, C#, JSP
Java, C++, C, C#, JSP
Java Perl Java Java Java C# Java C# Python Mostly Java
 Implementation language
Free Free edition available
Free 2-user version
Free for Open Source Projects
Free for Open Source Projects and Community
Free for Open Source Projects
Free for Open Source Projects
Free for Small Teams and Open Source Projects
Free Professional edition
Free Community Edition 		Free
Issue tracker JIRA JIRA Lighthouse Mingle Google ? Bugzilla Trac private JIRA JIRA
Supports Javaforge JIRA JIRA JIRA JIRA Official support Bugzilla Jira JetBrains Tracker java.net private private   private Issue tracker
Online demo here here here here here here Working implementation here Request or
Five Minute Intro 		Building Open Source projects
 here Use demo/domo ? ? here Eval download Online Demo
 Online Demo
 Working implementation
Online Demo here Live Demo Live Demo here   Online demo
 Online demo
Number of active developers 5 4
5+ ? 1+ 5 ? 5+ ?
-
4+
? 3 ? 4 1 5-10 5-10 ? ? 2 12 5-10 (core)
20+ (plugins)
? ? 1 1 ? Number of active developers
SCM support CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed easyCIS RedJack ElectricCommander SCM support
AccuRev
plugin 		?
plugin
plugin 		AccuRev
AlienBrain ? ? ? ? ? ? ? ? ? AlienBrain
Bazaar ? ? ? ? ? ? ? ?
plugin
plugin
Bazaar
BitKeeper ? ? ? ? ? ? ? ? ?
plugin 		BitKeeper
ClearCase
plugin 		?
plugin 		  ClearCase 
CA Harvest ? ? ? ? ? ? ? ? ?
plugin 		CA Harvest
CM Synergy   ? ? ? ? ? ? ? ? ?
plugin 		CM Synergy
CVS CVS
Dimensions ? ? ? ? ? ? ?
? ? Dimensions
"File system SCM" ?
plugin
"File system SCM"
Git ? ?
plugin 
? ? ? ? ? ?

plugin 		Git
HTTP file ? ?

plugin
(via process trigger)

HTTP file
Mercurial ? ? ?
plugin 		? ? ? ? ? ? ?

plugin 		Mercurial
MKS ? ? MKS 
Perforce (p4) ?
plugin 		  Perforce (p4)
PVCS ? ?
plugin 		PVCS
SourceGear Vault ? ?


plugin 		SourceGear Vault
StarTeam ?
plugin
StarTeam 
Subversion Subversion 
Surround ? ?


plugin 		  Surround 
Team Foundation Server
(via plugin) 		? ? ? ? ? ? ? ? ? ?
plugin 		Team Foundation Server
VSS ?
plugin 		VSS
VSS Journal ? ? ? VSS Journal
SCM related support CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed easyCIS RedJack ElectricCommander SCM related support
SCM filtering ? ? ?   SCM filtering
Multi-SCM
? about ? ?
using project dependencies 		Multi-SCM
Can create new SCM repositories ? ? ?
via FinalBuilder 		Can create new SCM repositories
Footprints the built binaries back to SCM managed source versions
? 
? 
? 
? 
? 
? 

 
Uses built in artifact repo instead.  		? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
  ? Footprints the built binaries back to SCM managed source versions
Audits source files used in binaries but not stored in a SCM repository
? 
? 
? 
? 
? 
? 
? 

 
  		? 
? 
? 
? 
? 
? ? 
? 
? 
? 
? 
? 
? 
? 
? 
  ? Audits source files used in binaries but not stored in a SCM repository
Build management CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed easyCIS RedJack ElectricCommander Build management
Parallel builds (ability to build several projects simultaneously) ? Parallel builds (ability to build several projects simultaneously)
Distributed builds
via contrib 		 
 
?
Distributes the workload of the pre and post build activities across multiple agents
Distributes the workload of the pre and post build activities across multiple agents 		Distributed builds
Runs compilers and linkers with Build Avoidance (incremental builds)
? 
? 
? 
? 
? 
? 
? 

 
  		? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
 
 
(for Maven projects)
? 
? 
Runs compilers and linkers with Build Avoidance (incremental builds)
Accelerates calls to the compilers and linkers through multi-threaded processing
? 
? 
? 
? 
? 
? 
? 

 
  		? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
Accelerates calls to the compilers and linkers through multi-threaded processing
Agents' code auto-updated ? ? ? ?
? ? ? ? ? ? ?   Agents' code auto-updated
Performs source code dependency scanning, impact analysis and reporting
? 
? 
? ? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
Yes, with ElectricAccelerator
 Performs source code dependency scanning, impact analysis and reporting
Auto-gens build.xml based on IDE project files.
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
?
for IntelliJ IDEA 		? 
? 
? 
Auto-gens build.xml based on IDE project files.
Manually force builds
Email 		?
Not needed
Manually force builds
X-platform and X-language compile management
? 
? 
? 
? 
? 
? 

  		? 
? 
? 
? 
? 
? 
? 
? 
? 
? ? 
? 
? 
? 
? 
X-platform and X-language compile management
SCM trigged builds
(if supported by SCM) 		  ?
plugin
SCM trigged builds
SCM poll based builds
Tinderbox2 		?
SCM poll based builds
Temporal build scheduling   ? ?   Temporal build scheduling
Builds promotion ?   ? about ?
plugin 		  Builds promotion
Interproject Dependencies ? ?   ? ? ? ? ? ? ? ?
Interproject Dependencies
Builds deletion ?   ? ? ? ?
automatic

manual or automatic 		  Builds deletion
Reproduce history builds ?   ? ?
  Reproduce history builds
Proactive (can prevent build breakages) ?   ? ? ? ? ? ?
  Proactive (can prevent build breakages)
Detect new failing tests while build ? ?   ? ? ?
? ? ? ? ? ? ?
? ? Detect new failing tests while build
Notify when first test in build fails ? ?   ? ? ? ? ? ? ? ? ? ? ? ?
? Notify when first test in build fails
Security CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed easyCIS RedJack ElectricCommander Security
User authentication  
For management 		?
For management 		  User authentication
User authorization schemes   ? ?
Database
configurable
  User authorization schemes
LDAP Integration   AD
? ? ? ?   LDAP Integration
Kerberos ?   ? ? ? ? ? ? ? ? ? ? ? ? ?
Kerebos
Single Sign On ?   ? ? ?
via Crowd 
? ? ? ? ? ? ? ? ?   Single Sign On
Custom JAAS ?   ? ? ?
via Crowd 
? ? ? ? ?

? ? ? ? Custom JAAS
Publishing CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed easyCIS RedJack ElectricCommander Publishing
Confluence   ?
plugin 		? ?


plugin 		Confluence
Email
using script 		Email
Run executable   ?
using script 		Run executable
FTP  
plugin 		?
plugin
using script 		  FTP
IRC   ? ?
plugin
using script 		  IRC
Jabber   ? ?
plugin

using script 		  Jabber
Lotus Sametime  
via XMPP 		?
plugin

Lotus Sametime
NetSend   ?


using script
NetSend 
ProjectStart   ? ?

ProjectStart 
RSS   ?
RSS
SCP   ? ? ?
plugin

using script 		  SCP 
Windows System Tray
plugin 
?
plugin
Windows System Tray
Formatted Logging
XML
XML 		 
(In Relational Db)
Xml, Html 		?
?
XML, saved in DB 		  Formatted Logging
Yahoo Messenger   ? ? ?



using script
Yahoo Messenger
MSN Messenger ? ?   ? ? ?
using script 		MSN Messenger
X10  
via XMPP 		? ?



using script 		X10 
Web interface CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed easyCIS RedJack ElectricCommander Web interface
View changesets Bonsai ?
 

 
Bonsai

View changesets
Add new projects  
"Tree" only 		?
 

 

"Tree" only
Add new projects
Clone projects   ? ?
 

 


branching 		  Clone projects
Delete projects   ?
 

 
  Delete projects
Modify projects   ?
 

 

  Modify projects
Kill builds   ?
 

 
  Kill builds
Pause builds   ?
 

 
  Pause builds
Access to build artifacts   ?
 

 

  Access to build artifacts
Browse CI's working copy ? ?
 

 


n/a Browse CI's working copy
Delete CI's working copy   ? ?
 

 

  n/a Delete CI's working copy
Search in builds   ? ?
 

 

  Search in builds
Historic graphs   ?

  Historic graphs
Self-updating web page
?


Ajax 		  Self-updating web page
Multi project support  


Multi project support
Multi project view   ?


Multi project view
Add/remove agent machines (for distributed builds) ? ?   ? ? ? ? ? ?


  Add/remove agent machines (for distributed builds)
Directly supported build tools CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed easyCIS RedJack ElectricCommander
 Directly supported build tools
Shell / command script  
Via command line
?


via FinalBuilder

via other tool
Shell / command script
Ant  
Via command line

via FinalBuilder
Via command line
via other tool
Ant
Groovy   Via command line
Via command line
Grails Plugin


plugin 		Via command line
via other tool 		  Groovy
OpenMake Meister

         
Via command line
Via command line
                via command line
   
via other tool 		    OpenMake Meister
Maven   Via command line
Via command line


Via command line
via other tool 		  Maven
Maven2   Via command line
Via command line

Via command line
via other tool 		  Maven2
Make   Via command line
Via command line
Via command line
? ?

via command line
or plugin
Via command line
via other tool 		  Make
MsBuild ?   Via command line ? ?
Via command line 		? ? ? ? ?

?
plugin
via FinalBuilder 		Via command line   MsBuild
NAnt  
Via command line 		?


plugin
via FinalBuilder 		Via command line   NAnt
Rake (Ruby) ? ? ? ? ?
Via command line
Via command line
Via command line
? ? ? ? ?

? ?

plugin 		Via command line
via other tool 		  Rake (Ruby)
Visual Studio ('devenv')   Via command line
Via command line 		? ?


via FinalBuilder 		Via command line   Visual Studio ('devenv')
FinalBuilder   Via command line
Via command line
Via command line

Via command line



Via command line
via other tool 		  ? FinalBuilder
Issue & PM integrations CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory
Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister
OpenMake Mojo
Parabuild Tinderbox3 Pulse TeamCity Hudson FinalBuilder Server
Zed easyCIS RedJack ElectricCommander Issue & PM integrations
Bugzilla   ?  

plugin
  Bugzilla
ClearQuest ? ?   ? ? ? ? ? ? ? ? ?

? ? ?
ClearQuest
Confluence   ?



Confluence 
JIRA   ?  


plugin
JIRA
Mingle  
Mingle
QualityCenter
?
?
  ? ?
?
?

  		?
?
?
?
?
?

?
?
?

?
?

QualityCenter
Rally
?
  ? ?
?

 
?
?
?
?
?
?


?
?
?
?

Rally
Rubyforge.org   ? ? ? ? ?


Rubyforge.org
Scarab   ? ? ?


Scarab
Sourceforge.net   ? ? ? ? ?


Sourceforge.net
Trac   ? ? ? ?


plugin
Trac
VersionOne

?
  ? ?
?
?

 
?
?
? ?
?
?


?
?
?

plugin
 ?
?
VersionOne
Test Tool integrations CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed easyCIS RedJack ElectricCommander
 Test Tool integrations
Agitar ?
  ?
?
?
?
?
?
?
?
?


?
?
?
?
?
?
?   Agitar
CppUnit result rendering ? ?   ? ? ? ? ? ? ?

?
plugin
plugin 		? ?   CppUnit result rendering
JUnit result rendering   ?


?
JUnit result rendering
NUnit result rendering   ?


plugin
?   NUnit result rendering
QualityCenter test rendering ?
?
  ?
?
?

  		?
?
?
?
?
?


?
?
?
?
?
?
?   QualityCenter test rendering
PHPUnit result rendering
? ?   ? ? ? ? ? ? ? ?

? ? ?
plugin 		? ?   PHPUnit result rendering
PMD result rendering
? ?   ? ? ?
plugin 		? ? ? ? ?
? ?
plugin 		? ?   PMD result rendering
Clover result rendering ? ?   ? ? ? ? ? ? ?

? ?
about
plugin
?   Clover result rendering
Selenium result rendering ? ?   ? ? ? ? ? ? ? ?

? ?
about
plugin
?   Selenium result rendering
SilkCentral
?
?
  ?
?
?

  		?
?
?
?
?
?


?
?
?
?

plugin 		?
?   SilkCentral
MSTest result rendering
?
?
?
?
?
?
?
?

  		?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
  MSTest result rendering
IDE Integrations CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed easyCIS   ElectricCommander IDE Integrations
Eclipse Plugin  
Atlassian Eclipse Connector 		?


about

Eclipse Plugin
VS 2005 Plugin ? ?   ? ? ? ? ? ? ? ?

? ? ?
VS 2005 Plugin
IntelliJ Plugin   ?
Atlassian IntelliJ Connector 		?


about
IntelliJ Plugin
Source Inspection integration CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed easyCIS RedJack ElectricCommander Source Inspection integration
Bonsai ? ?   ? ? ? ? ?

?

  Bonsai
ViewVC   ? ?


about

  ViewCVS
Fisheye   ?


about


  Fisheye
Remote Management API CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed easyCIS RedJack ElectricCommander Remote Management API
REST ? ?   ? ?
? 		? ? ? ? ? ?

? ? ?
details 		?
REST
SOAP  
Asynchronous Messaging 		? ?



SOAP
XML-RPC   n/a ?



XML-RPC
JMX N/A   n/a ?


plugin
JMX
Hessian ?   n/a ?



Hessian
Client Library ? ?   .Net Remoting
? ? about ? ? ? ? ? ?

? ? ?


? Client Library
Installation and Configuration CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed easyCIS RedJack ElectricCommander Installation and Configuration
Windows installer   ?


Windows installer
Self contained distribution (except SCM clients) ?


Self contained distribution (except SCM clients)
Additional dependencies JRE, SCM client .NET Runtime, IIS, SCM client   SCM client .NET Runtime, IIS, SCM client, depends on the packages you choose
.NET Runtime, SQL Server (MSDE Included in distribution) Perl, CVS Python, Twisted JRE, SCM client JDK, Servlet container, SCM client JRE
jdk, SCM client jdk,tomcat,SCM client ? JRE, SCM client Subversion, .NET runtime/IIS or Mono/XSP JRE, Perl, SCM Client
JRE, Perl, SCM Client
Perl, CVS, SCP JDK, some SCM clients JRE, some SCM clients JRE .Net, FinalBuilder, IIS, ASP.NET AJAX Ext. JRE .NET, MS chart controls, IIS, SQL server Python, SCM clients
JRE Additional dependencies
Execution platform JVM Windows, .NET, Mono   JVM (bundled) Windows .Net
Windows, .NET, Mono Perl Any (Python) JVM JVM JVM
JVM JVM ? JVM Windows, Linux Windows, AIX, Solaris, HP-UX, Linux, z/OS Windows, AIX, Solaris, HP-UX, Linux, z/OS Windows, Unix, Linux, Mac OS X, Solaris, HP UX Perl JVM JVM JVM Windows JVM Windows Any with Python
Any Execution platform
Project platform (what it can build) Java + anything Ant/Maven/NAnt can build .NET or anything with a command line interface build   anything with a command line interface .Net, Java, C++, VB6
Any language Any language Any language Any language Any language with supplied ant wrapper Any language
Any language Any language with supplied ant wrapper ? Any language Any language Control Compile and Linker calls for any language or executes pre-written build script
Execute pre-written compile/link script for any language
Any language
Any language 		Any Language Any Language Any language Any Language
Any language
Visual Studio solutions, Anything else Any command line
Any language
 Project platform (what it can build)
Preferred build tool Ant, Maven Nant   Ant, Nant, Rake MSBuild, Ant, devenv NAnt Make Any build tool Any build tool Ant Any build tool
Any build tool Ant, Maven ? Maven 2 Any build tool Openmake Meister
Any build tool
Any build tool Make Any build tool Any Any FinalBuilder Any MSBuild, NAnt Any Any Preferred build tool
No need to  modify build scripts   ?

  No need to  modify build scripts
Supports multiple projects   ?

Supports multiple projects
Automatic configuration from build script   ? N/A, it is automatic, there is no need for you to provide a build script

Maven dependency import & export (via pom.xml)

Maven only


IntelliJ IDEA projects, Maven

Maven only
  ? Automatic configuration from build script
Text file configuration
XML
XML
YML
XML
XML 		Stored in SQL Db - XML input/export coming soon
XML import/export.
Database backend
(Oracle, MySQL, Derby, Postgres, DB2 or SQL Server) 		?


XML 		XML
via FinalBuilder

XML, import/export
Yaml

XML
 Text file configuration
  CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory
Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister 
OpenMake Mojo
Parabuild Tinderbox3 Pulse TeamCity Hudson FinalBuilder Server
Zed easyCIS RedJack ElectricCommander