Adding full support for SonarQube

[jeremfg]

With the sonar-cxx plugin (Community C++) we can get test reports inside SonarQube.

This works pretty well as long as you don't try to get more than metrics by using the sonar.cxx.xunit.provideDetails=True configuration. Keeping it basic will still give you number of failed/successful tests but that's pretty much it.

However, it would be pretty easy to add support for more than metrics, as explained here:
https://github.com/wenns/sonar-cxx/wiki/Get-test-execution-metrics
We only need to add a filename attribute to each testcase tag in the XML file.

I would have tried contributing myself to cmockery2... It seems we only need to add an extra attribute of type SourceLocation to the XunitTestCase structure but I cannot figure out where/how to obtain this information during the execution of a test case.
Correct me if I'm wrong, but there are no way currently to tie together the SourceLocation of a XUnitTestCase, right?

[lpabon]

I think we could add a new parameter to run_tests() which would use __FILE__ and pass it to _run_test(). Then _run_test() could add it as an entry to XunitTestCase. I think that would work.

[jeremfg]

I don't think this would achieve the intended result...

Granted, this would provide a filename, but what we want is the filename where the function under test resides, like we have inside each "assert_xxx" function".

Your suggestion, if I understand the layout of cmockery2 correctly, would provide only the file where the main function resides...

[lpabon]

Ah, I understand now. Good question, if I come up with an answer I will let you know. Also, FYI, you may want to check out Cmocka. They are actively continuing development of cmockery.

[jeremfg]

Wow, a lot has happened in the last few months with Cmocka... Making it a lot more attractive than it used to be.

We've evaluated it back in January and partly discarded it as an option for lack of xUnit-style xml output support. This was a requirement to fit with our development process and integration with our continuous integration setups.

I'll see if it's not too late to switch over to Cmocka at this point, but I fear it might be. We have a few subcontracted companies already using cmockery2 per previous agreement when we decided to unify our unit-testing strategies.

The support of a detailed report inside sonar is only a "Nice-to-have" as we have good reporting elsewhere by other tools already. So for us cmockery2 is still a good option :)

I'll keep an eye out in case you come up with a clever idea.

Thank you!

[lpabon]

This is great news. I didn't think people were using it. I will keep cmockery2 alive and let you know if/when I figure out how to enable your request.

[jeremfg]

I don't want to get your hopes up too high :)... Although we have committed to use Cmockery2 a couple of months ago and are very happy with it, I'm still very much interested in investigating the possibility of a switch to CMocka.

I was especially attracted to it after you've shown me it's obvious return to active development with the 1.0 release, increase in quality and larger adoption it seemed to have gained lately... All good points for increased confidence over the long term.

And they finally added the feature of XML reporting it was sadly missing up to last February.

I'll keep you informed on what we choose to do...

[jeremfg]

We've discussed this a little internally, and since there is no compelling argument to move we'll be sticking with CMockery2.

[lpabon]

Cool! I'll let you know if I come up with a method to satisfy your request.