This is a rant against the Cobertura maven integration - and to a certain extent on Cobertura also - and hopefully an objective one ;). Hopefully this’ll help someone select between the two big (open source) coverage tools out there.
We’ve been using Cobertura at work till now and its done its job nicely - the reports look great and the maven 1.x integration, while not neat, is functional. While we knew that someday we were going to have to merge coverage data and have a single report across multiple test methods (junit, selenium tests and manual), cobertura documentation stated that this was possible and so we weren’t really bothered.
Thought I’d give it a whirl and set it up - and that’s when the trouble started. Atleast, with the maven integration.
First of all - there’s no way to just instrument code. You can generate the report (which will instrument classes and run the tests) but if you just want to instrument classes so that the final deployable contains instrumented classes, its a no go with the maven plugin tools.
Obviously, no point giving it up there - so thought I’d just include the ant tasks and go the ant way in my maven goal. Turns out that there’s no 'plugin init' kind of goal that can be called post build:start that will set up dependencies and import the cobertura ant tasks. You have to do it all yourself. Fine - went that way too - so now my maven.xml uses the cobertura ant tasks and finally I’m able to generate an instrumented build. YIPPPPPPPEEEEEE…. or wait…lets' just make sure that this thing works…
Does it?
Turns out - no - it doesnt - so I dropped the WAR into tomcat and accessed the login page of the application and then shut down tomcat nicely. There’s even a cobertura.ser created in the tomcat bin folder and I’m thinking that probably this will all work together finally…
So I go ahead, tweak my Maven.xml further with a coverage task that will merge the data from the junit runs and servlet container runs. Turn the switch on… and lo and behold…Exception reading the merged data file. Back to google and after hunting around for sometime found this Bug while merging reports
So finally I’m ready to give up cobertura and give Emma a try…and it couldnt have been better…
-
Goals are nicely setup
-
You can init the emma system with the emma:init goal and then use ant tasks if you want flexibility for doing things like merging reports.
-
The merging works :))
One sticky issue that I did come up with was that the for the same source and test cases the coverage reported by cobertura and emma differ widely. With Cobertura, we were at 40% coverage while with EMMA, the number’s up to 60% coverage - and while EMMA has some literature on how it does things - I’d be glad if someone did explain why or how the reported numbers could be so different for the same base code and unit test suite?
