Continuous Integration

Colonel Panic

I use Jenkins in a modest C++/ObjC personal project with CMake generating project files and GoogleTest for unit tests and Mercurial for source control, but need would like to retrospectively do something similar for a product 10x the size in work and was wondering what the best approach should be here.

The app is a Windows only mix of C++ and C# code with dependencies on two external UI libraries in C++ land and a dozen .Net libraries. We're using Team Foundation Server.

I'm trying to get an approach sorted in my head. I think the first port of call is to get CMake working properly with the C++ projects but what about the .Net stuff?

Furthermore, our C++ code isn't exactly unit testable at the moment, although the .Net stuff is. Is continuous integration still worthwhile with a relatively low level of code coverage or should I hold off on CI and just do nightly/weekly builds for now?

Also, has anyone got experience with CruiseControl.net? I think Jenkins is okay, but I don't want to marry it or anything and would be open to alternatives.

srsly78

I use Jenkins with a c++/c# mixed project. Can just run msbuild/mstest off the commandline as jenkins postbuild steps. Haven't had much luck getting it to run static code analysis or coverage, which we do have on the pure c++ linux compilable portion of the codebase.

We use cmake for the pure c++ portion of our project, but have great difficulties using it with c#. Thus we just use normal "handmade" visual studio project files for the c# stuff. Subversion for source control, builds triggered 10 mins after a checkin.

This would be a pretty big project, and Jenkins works pretty well. However we are missing the fancy microsoft specific tools for coverage with the c# stuff coz we don't use team server or cruise control.net - this seems to be what you are most interested in and might be a better way to go for c# stuff. It's very strange being able to have all this with c++ but not with c#

The c++ stuff builds on linux using gcc with coverage settings, then runs a custom script to move files around from the cmake out-of-tree build structure to what gcov tool expects. Then another script runs to convert coverage output to xml, and jenkins cobertura plugin presents this. All of this could be done on windows as well with cygwin I think, just a bit more awkward. Since we have master/slave setup for jenkins we don't bother running coverage on any windows slaves, only the linux master.

Giblet

For .NET CruiseControl is your only man, easy to set up and have multiple configurations for it. MSTest works fine with it to, with customisable outputs.

Colonel Panic

Thanks guys, I really want to use .Net unit tests and fxcop et al so I think CruiseControl.net might be worth a look.

I have another .Net only project coming up in work that I could use as the guinea pig for it and I could tidy up the other project's build process in the mean time even if we don't go to using CI on it in the end.

kayos

If you are using TFS why not use TFS for the CI side of things? It will do it all for you Test, Static Analysis, Lab Deployment etc. If you are using TFS 2010 you even get gated checkin's. Think CI builds kicked off on checkin, but on a private branch. If the CI fails then your changes are booted out and never touch the main code base.

Jenkins and the coverage results from .NET dont play nice as srsly78 has said. I've previously had to write a converter to get the binary data from the coverage results in .NET into Emma XML format so I could get the code coverage into Jenkins. This was a custom MSBuild extension run as a step and then the output was pulled into jenkins wrote it once and have it deployed in many, many builds. (@srsly78 drop me a pm if you want about this). FxCop Static Analysis runs fine in Jenkins again need a plug-in. Using MSBuild and MSTest again both work fine and there is yet more plugin's to help/allow them work correctly.

But seeing as you have TFS for source control I would really think its the better option... Anything you can do in Jenkins with the aid of custom code, tricky configurations and a number of plugin's will all just work in TFS . All depends on what else you need to do that jenkins does easily and you might have trouble with in TFS. Or of course if you must use Jenkins because its what they use in work and just don't see the value in using the best tool for the job.

BTW I've only been using Jenkins in my current job, previously used an earlier version of TFS but not with anger as they say.

Colonel Panic

I didn't realise TFS could do all of that! We only recently switched to it and my experience with Mercurial and Jenkins on C++ projects isn't really helping here!

Will install TFS on my test system at the weekend and see if I can get CI up and running on it for a dummy project.

kayos

Colonel Panic wrote: »

I didn't realise TFS could do all of that! We only recently switched to it and my experience with Mercurial and Jenkins on C++ projects isn't really helping here!

Will install TFS on my test system at the weekend and see if I can get CI up and running on it for a dummy project.

That and more the lab management is meant to be pretty swanky.

markpb

I used CruiseControl here for a while here and grew to hate it. I'm a developer, not a release engineer but setting up new builds and diagnosing problems with builds was taking so long that I was beginning to wonder.

Jetbrains TeamCity does exactly the same and is far easier to setup and configure. It's free version should be good enough for most people, the limits are pretty loose. If nothing else, bear in mind that if you ever leave, someone after you will have to figure out CC's scripts. Like the others have said, the TFS stuff is supposed to be pretty swanky these days but I haven't used it myself.