Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[XML] Inaccurate coverage for files with same name, but from different source sets #279

Open
IgnatBeresnev opened this issue Nov 21, 2022 · 6 comments
Assignees
Labels
Bug Bug issue type Kover Coverage Tool S: in progress Status: implementing or design in process

Comments

@IgnatBeresnev
Copy link
Member

Describe the bug

XML report generates inaccurate coverage for files with same name, but from different source sets.

Take the following project structure as an example:

2022-11-21_13-38-18

where KoverFile.kt will contain:

  • KoverCommonClass for commonMain source set
  • KoverJvmClass for jvmMain source set.

Generated XML report will look the following way:

<?xml version="1.0" ?>
<report name="Intellij Coverage Report">
    <package name="kover/reproducer">
        <class name="kover/reproducer/KoverCommonClass" sourcefilename="KoverFile.kt">
            ...
        </class>
        <class name="kover/reproducer/KoverJvmClass" sourcefilename="KoverFile.kt">
            ...
        </class>
        <sourcefile name="KoverFile.kt">
            <line nr="3" mi="0" ci="2" mb="0" cb="0"/>
            <line nr="3" mi="0" ci="2" mb="0" cb="0"/>
            <line nr="5" mi="0" ci="4" mb="0" cb="0"/>
            <line nr="5" mi="0" ci="4" mb="0" cb="0"/>
            <counter type="INSTRUCTION" missed="0" covered="12"/>
            <counter type="BRANCH" missed="0" covered="0"/>
            <counter type="LINE" missed="0" covered="4"/>
        </sourcefile>
        ...
    </package>
    ...
</report>

As you can see, both classes lead to the same source file, the description for which duplicates line numbers. Because the code is the same in both classes, the duplicated lines are the same, but I imagine if the code is vastly different, it'll be a mess.

This leads to codecov.io ignoring such files altogether, presumably because it doesn't know how to map it to project's git structure. See codecov report. This, in turn, affects calculated coverage. Umbrella issue: #16

This problem does not exists in HTML reports as it doesn't offer file-based coverage.

(additional files CommonFile and JvmFile are added to demonstrate that codecov sees Kotlin files otherwise)

Expected behavior

Not sure what could be done in this situation as it seems to be a limitation of XML report's structure, but maybe some additional kover-specific tags could be added, or some other way to differentiate between source sets.

Reproducer

Project (notice the branch): https://github.com/IgnatBeresnev/kover-reproducers/tree/mpp-same-filename

There are instructions in the README on how the report can be generated and uploaded to codecov.io

Reports
Gist with XML report: https://gist.github.com/IgnatBeresnev/418ab0514b699fea7473cb2d4d275480

Environment

  • Kover Gradle Plugin version: 0.6.1
  • Gradle version: 7.4.2
  • Kotlin project type: Kotlin/Multiplatform with Common and JVM source set
@IgnatBeresnev IgnatBeresnev added Bug Bug issue type Kover Coverage Tool S: untriaged Status: issue reported but unprocessed labels Nov 21, 2022
@shanshin shanshin added S: in progress Status: implementing or design in process and removed S: untriaged Status: issue reported but unprocessed labels Nov 21, 2022
@shanshin
Copy link
Collaborator

We will do as in JaCoCo, for summing up the values for different classes. Since the XML format does not assume that there are duplicates of files with the same name, then there is no single correct solution that other tools will work with.

@IgnatBeresnev
Copy link
Member Author

IgnatBeresnev commented Nov 21, 2022

To add context to the message above, source files will have a path relative to the source set roots, similar to how it's done for classes:

<class name="kover/reproducer/JvmClass" sourcefilename="JvmFile.kt">
    ...
</class>

This will solve the problem of duplicated sourcefile lines and allow to differentiate between files from different locations. However, if both files share the same relative path, they will still be indistinguishable

@zuevmaxim
Copy link

The 'sum up' behavior is done in agent version 1.0.688

@shanshin
Copy link
Collaborator

shanshin commented Nov 22, 2022

To use custom engine you need to configure it in all projects where Kover is used

for Kotlin Gradle script

kover {
    engine.set(kotlinx.kover.api.IntellijEngine("1.0.688"))
}

for Groovy

kover {
    engine = new kotlinx.kover.api.IntellijEngine("1.0.688")
}

@shanshin
Copy link
Collaborator

Relates codecov/uploader#913

@rfermontero
Copy link

Any news on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Bug issue type Kover Coverage Tool S: in progress Status: implementing or design in process
Projects
None yet
Development

No branches or pull requests

4 participants