Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

<Temp don't merge> #819

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

<Temp don't merge> #819

wants to merge 1 commit into from

Conversation

RulaKhaled
Copy link
Contributor

Two main changes:

  • Updated the copy of the footer of the PR comment.
  • Updated the format so it displays colored code, using language of the repository.

Issues:

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. In 2022 this entity acquired Codecov and as result Sentry is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

Copy link

codecov bot commented Oct 24, 2024

❌ 3 Tests Failed:

Tests completed Failed Passed Skipped
1700 3 1697 0
View the top 3 failed tests by shortest run time
services.tests.test_test_results test_generate_failure_info
Stack Traces | 0.028s run time
def test_generate_failure_info():
        flags_hash = generate_flags_hash([])
        test_id = generate_test_id(1, "testsuite", "testname", flags_hash)
        fail = TestResultsNotificationFailure(
            "hello world",
            "testname",
            [],
            test_id,
            1.0,
            "https://example.com/build_url",
        )
    
&gt;       res = generate_failure_info(fail)
E       TypeError: generate_failure_info() missing 1 required positional argument: 'commit'

services/tests/test_test_results.py:84: TypeError
services.tests.test_test_results test_build_message
Stack Traces | 0.031s run time
def test_build_message():
        flags_hash = generate_flags_hash([])
        test_id = generate_test_id(1, "testsuite", "testname", flags_hash)
        fail = TestResultsNotificationFailure(
            "hello world",
            "testname",
            [],
            test_id,
            1.0,
            "https://example.com/build_url",
        )
        payload = TestResultsNotificationPayload(1, 2, 3, [fail], dict())
        commit = CommitFactory(branch="thing/thing")
        tn = TestResultsNotifier(commit, None, None, None, payload)
        res = tn.build_message()
    
&gt;       assert (
            res
            == f"""### :x: 1 Tests Failed:
    | Tests completed | Failed | Passed | Skipped |
    |---|---|---|---|
    | 3 | 1 | 2 | 3 |
    &lt;details&gt;&lt;summary&gt;View the top 1 failed tests by shortest run time&lt;/summary&gt;
    
    &gt;
    &gt; ```
    &gt; testname
    &gt; ```
    &gt;
    &gt; &lt;details&gt;&lt;summary&gt;Stack Traces | 1s run time&lt;/summary&gt;
    &gt;
    &gt; &gt;
    &gt; &gt; ```
    &gt; &gt; hello world
    &gt; &gt; ```
    &gt; &gt;
    &gt; &gt; [View](https://example.com/build_url) the CI Build
    &gt;
    &gt; &lt;/details&gt;
    
    &lt;/details&gt;
    
    To view individual test run time comparison to the main branch, go to the [Test Analytics Dashboard](https://app.codecov.io/{services_short_dict.get(commit.repository.service)}/{commit.repository.owner.username}/{commit.repository.name}/tests/thing%2Fthing)"""
        )
E       AssertionError: assert '### :x: 1 Te...dback/issues)' == '### :x: 1 Te...hing%2Fthing)'
E         
E         Skipping 390 identical leading characters in diff, use -v to show
E           /details&gt;
E         - 
E         - To view individual test run time comparison to the main branch, go to the [Test Analytics Dashboard](https://app.codecov..../exist-parent-ago/tests/thing%2Fthing)
E         ?         ^^^^^^^^^^      ^^ ^  ---  --------       -----------------------
E         + \To view more test analytics go to the [Test Analytics Dashboard](https://app.codecov..../exist-parent-ago/tests/thing%2Fthing)...
E         
E         ...Full output truncated (2 lines hidden), use '-vv' to show

services/tests/test_test_results.py:113: AssertionError
services.tests.test_test_results test_build_message_with_flake
Stack Traces | 0.031s run time
def test_build_message_with_flake():
        flags_hash = generate_flags_hash([])
        test_id = generate_test_id(1, "testsuite", "testname", flags_hash)
        fail = TestResultsNotificationFailure(
            "hello world",
            "testname",
            [],
            test_id,
            1.0,
            "https://example.com/build_url",
        )
    
        payload = TestResultsNotificationPayload(
            1, 2, 3, [fail], {test_id: FlakeInfo(1, 3)}
        )
        commit = CommitFactory(branch="test_branch")
        tn = TestResultsNotifier(commit, None, None, None, payload)
        res = tn.build_message()
    
&gt;       assert (
            res
            == f"""### :x: 1 Tests Failed:
    | Tests completed | Failed | Passed | Skipped |
    |---|---|---|---|
    | 3 | 1 | 2 | 3 |
    &lt;details&gt;&lt;summary&gt;View the full list of 1 :snowflake: flaky tests&lt;/summary&gt;
    
    &gt;
    &gt; ```
    &gt; testname
    &gt; ```
    &gt;
    &gt; **Flake rate in main:** 33.33% (Passed 2 times, Failed 1 times)
    &gt; &lt;details&gt;&lt;summary&gt;Stack Traces | 1s run time&lt;/summary&gt;
    &gt;
    &gt; &gt;
    &gt; &gt; ```
    &gt; &gt; hello world
    &gt; &gt; ```
    &gt; &gt;
    &gt; &gt; [View](https://example.com/build_url) the CI Build
    &gt;
    &gt; &lt;/details&gt;
    
    &lt;/details&gt;
    
    To view individual test run time comparison to the main branch, go to the [Test Analytics Dashboard](https://app.codecov.io/{services_short_dict.get(commit.repository.service)}/{commit.repository.owner.username}/{commit.repository.name}/tests/{commit.branch})"""
        )
E       AssertionError: assert '### :x: 1 Te...dback/issues)' == '### :x: 1 Te.../test_branch)'
E         
E         Skipping 455 identical leading characters in diff, use -v to show
E           /details&gt;
E         - 
E         - To view individual test run time comparison to the main branch, go to the [Test Analytics Dashboard](https://app.codecov..../everybody-improve/tests/test_branch)
E         ?         ^^^^^^^^^^      ^^ ^  ---  --------       -----------------------
E         + \To view more test analytics go to the [Test Analytics Dashboard](https://app.codecov..../everybody-improve/tests/test_branch)...
E         
E         ...Full output truncated (2 lines hidden), use '-vv' to show

services/tests/test_test_results.py:162: AssertionError

To view individual test run time comparison to the main branch, go to the Test Analytics Dashboard

@codecov-staging
Copy link

❌ 3 Tests Failed:

Tests completed Failed Passed Skipped
1700 3 1697 0
View the top 3 failed tests by shortest run time
services.tests.test_test_results test_generate_failure_info
Stack Traces | 0.028s run time
def test_generate_failure_info():
        flags_hash = generate_flags_hash([])
        test_id = generate_test_id(1, "testsuite", "testname", flags_hash)
        fail = TestResultsNotificationFailure(
            "hello world",
            "testname",
            [],
            test_id,
            1.0,
            "https://example.com/build_url",
        )
    
&gt;       res = generate_failure_info(fail)
E       TypeError: generate_failure_info() missing 1 required positional argument: 'commit'

services/tests/test_test_results.py:84: TypeError
services.tests.test_test_results test_build_message
Stack Traces | 0.031s run time
def test_build_message():
        flags_hash = generate_flags_hash([])
        test_id = generate_test_id(1, "testsuite", "testname", flags_hash)
        fail = TestResultsNotificationFailure(
            "hello world",
            "testname",
            [],
            test_id,
            1.0,
            "https://example.com/build_url",
        )
        payload = TestResultsNotificationPayload(1, 2, 3, [fail], dict())
        commit = CommitFactory(branch="thing/thing")
        tn = TestResultsNotifier(commit, None, None, None, payload)
        res = tn.build_message()
    
&gt;       assert (
            res
            == f"""### :x: 1 Tests Failed:
    | Tests completed | Failed | Passed | Skipped |
    |---|---|---|---|
    | 3 | 1 | 2 | 3 |
    &lt;details&gt;&lt;summary&gt;View the top 1 failed tests by shortest run time&lt;/summary&gt;
    
    &gt;
    &gt; ```
    &gt; testname
    &gt; ```
    &gt;
    &gt; &lt;details&gt;&lt;summary&gt;Stack Traces | 1s run time&lt;/summary&gt;
    &gt;
    &gt; &gt;
    &gt; &gt; ```
    &gt; &gt; hello world
    &gt; &gt; ```
    &gt; &gt;
    &gt; &gt; [View](https://example.com/build_url) the CI Build
    &gt;
    &gt; &lt;/details&gt;
    
    &lt;/details&gt;
    
    To view individual test run time comparison to the main branch, go to the [Test Analytics Dashboard](https://app.codecov.io/{services_short_dict.get(commit.repository.service)}/{commit.repository.owner.username}/{commit.repository.name}/tests/thing%2Fthing)"""
        )
E       AssertionError: assert '### :x: 1 Te...dback/issues)' == '### :x: 1 Te...hing%2Fthing)'
E         
E         Skipping 390 identical leading characters in diff, use -v to show
E           /details&gt;
E         - 
E         - To view individual test run time comparison to the main branch, go to the [Test Analytics Dashboard](https://app.codecov..../exist-parent-ago/tests/thing%2Fthing)
E         ?         ^^^^^^^^^^      ^^ ^  ---  --------       -----------------------
E         + \To view more test analytics go to the [Test Analytics Dashboard](https://app.codecov..../exist-parent-ago/tests/thing%2Fthing)...
E         
E         ...Full output truncated (2 lines hidden), use '-vv' to show

services/tests/test_test_results.py:113: AssertionError
services.tests.test_test_results test_build_message_with_flake
Stack Traces | 0.031s run time
def test_build_message_with_flake():
        flags_hash = generate_flags_hash([])
        test_id = generate_test_id(1, "testsuite", "testname", flags_hash)
        fail = TestResultsNotificationFailure(
            "hello world",
            "testname",
            [],
            test_id,
            1.0,
            "https://example.com/build_url",
        )
    
        payload = TestResultsNotificationPayload(
            1, 2, 3, [fail], {test_id: FlakeInfo(1, 3)}
        )
        commit = CommitFactory(branch="test_branch")
        tn = TestResultsNotifier(commit, None, None, None, payload)
        res = tn.build_message()
    
&gt;       assert (
            res
            == f"""### :x: 1 Tests Failed:
    | Tests completed | Failed | Passed | Skipped |
    |---|---|---|---|
    | 3 | 1 | 2 | 3 |
    &lt;details&gt;&lt;summary&gt;View the full list of 1 :snowflake: flaky tests&lt;/summary&gt;
    
    &gt;
    &gt; ```
    &gt; testname
    &gt; ```
    &gt;
    &gt; **Flake rate in main:** 33.33% (Passed 2 times, Failed 1 times)
    &gt; &lt;details&gt;&lt;summary&gt;Stack Traces | 1s run time&lt;/summary&gt;
    &gt;
    &gt; &gt;
    &gt; &gt; ```
    &gt; &gt; hello world
    &gt; &gt; ```
    &gt; &gt;
    &gt; &gt; [View](https://example.com/build_url) the CI Build
    &gt;
    &gt; &lt;/details&gt;
    
    &lt;/details&gt;
    
    To view individual test run time comparison to the main branch, go to the [Test Analytics Dashboard](https://app.codecov.io/{services_short_dict.get(commit.repository.service)}/{commit.repository.owner.username}/{commit.repository.name}/tests/{commit.branch})"""
        )
E       AssertionError: assert '### :x: 1 Te...dback/issues)' == '### :x: 1 Te.../test_branch)'
E         
E         Skipping 455 identical leading characters in diff, use -v to show
E           /details&gt;
E         - 
E         - To view individual test run time comparison to the main branch, go to the [Test Analytics Dashboard](https://app.codecov..../everybody-improve/tests/test_branch)
E         ?         ^^^^^^^^^^      ^^ ^  ---  --------       -----------------------
E         + \To view more test analytics go to the [Test Analytics Dashboard](https://app.codecov..../everybody-improve/tests/test_branch)...
E         
E         ...Full output truncated (2 lines hidden), use '-vv' to show

services/tests/test_test_results.py:162: AssertionError

To view individual test run time comparison to the main branch, go to the Test Analytics Dashboard

@codecov-qa
Copy link

codecov-qa bot commented Oct 24, 2024

❌ 3 Tests Failed:

Tests completed Failed Passed Skipped
1700 3 1697 0
View the top 3 failed tests by shortest run time
services.tests.test_test_results test_generate_failure_info
Stack Traces | 0.028s run time
def test_generate_failure_info():
        flags_hash = generate_flags_hash([])
        test_id = generate_test_id(1, "testsuite", "testname", flags_hash)
        fail = TestResultsNotificationFailure(
            "hello world",
            "testname",
            [],
            test_id,
            1.0,
            "https://example.com/build_url",
        )
    
&gt;       res = generate_failure_info(fail)
E       TypeError: generate_failure_info() missing 1 required positional argument: 'commit'

services/tests/test_test_results.py:84: TypeError
services.tests.test_test_results test_build_message
Stack Traces | 0.031s run time
def test_build_message():
        flags_hash = generate_flags_hash([])
        test_id = generate_test_id(1, "testsuite", "testname", flags_hash)
        fail = TestResultsNotificationFailure(
            "hello world",
            "testname",
            [],
            test_id,
            1.0,
            "https://example.com/build_url",
        )
        payload = TestResultsNotificationPayload(1, 2, 3, [fail], dict())
        commit = CommitFactory(branch="thing/thing")
        tn = TestResultsNotifier(commit, None, None, None, payload)
        res = tn.build_message()
    
&gt;       assert (
            res
            == f"""### :x: 1 Tests Failed:
    | Tests completed | Failed | Passed | Skipped |
    |---|---|---|---|
    | 3 | 1 | 2 | 3 |
    &lt;details&gt;&lt;summary&gt;View the top 1 failed tests by shortest run time&lt;/summary&gt;
    
    &gt;
    &gt; ```
    &gt; testname
    &gt; ```
    &gt;
    &gt; &lt;details&gt;&lt;summary&gt;Stack Traces | 1s run time&lt;/summary&gt;
    &gt;
    &gt; &gt;
    &gt; &gt; ```
    &gt; &gt; hello world
    &gt; &gt; ```
    &gt; &gt;
    &gt; &gt; [View](https://example.com/build_url) the CI Build
    &gt;
    &gt; &lt;/details&gt;
    
    &lt;/details&gt;
    
    To view individual test run time comparison to the main branch, go to the [Test Analytics Dashboard](https://app.codecov.io/{services_short_dict.get(commit.repository.service)}/{commit.repository.owner.username}/{commit.repository.name}/tests/thing%2Fthing)"""
        )
E       AssertionError: assert '### :x: 1 Te...dback/issues)' == '### :x: 1 Te...hing%2Fthing)'
E         
E         Skipping 390 identical leading characters in diff, use -v to show
E           /details&gt;
E         - 
E         - To view individual test run time comparison to the main branch, go to the [Test Analytics Dashboard](https://app.codecov..../exist-parent-ago/tests/thing%2Fthing)
E         ?         ^^^^^^^^^^      ^^ ^  ---  --------       -----------------------
E         + \To view more test analytics go to the [Test Analytics Dashboard](https://app.codecov..../exist-parent-ago/tests/thing%2Fthing)...
E         
E         ...Full output truncated (2 lines hidden), use '-vv' to show

services/tests/test_test_results.py:113: AssertionError
services.tests.test_test_results test_build_message_with_flake
Stack Traces | 0.031s run time
def test_build_message_with_flake():
        flags_hash = generate_flags_hash([])
        test_id = generate_test_id(1, "testsuite", "testname", flags_hash)
        fail = TestResultsNotificationFailure(
            "hello world",
            "testname",
            [],
            test_id,
            1.0,
            "https://example.com/build_url",
        )
    
        payload = TestResultsNotificationPayload(
            1, 2, 3, [fail], {test_id: FlakeInfo(1, 3)}
        )
        commit = CommitFactory(branch="test_branch")
        tn = TestResultsNotifier(commit, None, None, None, payload)
        res = tn.build_message()
    
&gt;       assert (
            res
            == f"""### :x: 1 Tests Failed:
    | Tests completed | Failed | Passed | Skipped |
    |---|---|---|---|
    | 3 | 1 | 2 | 3 |
    &lt;details&gt;&lt;summary&gt;View the full list of 1 :snowflake: flaky tests&lt;/summary&gt;
    
    &gt;
    &gt; ```
    &gt; testname
    &gt; ```
    &gt;
    &gt; **Flake rate in main:** 33.33% (Passed 2 times, Failed 1 times)
    &gt; &lt;details&gt;&lt;summary&gt;Stack Traces | 1s run time&lt;/summary&gt;
    &gt;
    &gt; &gt;
    &gt; &gt; ```
    &gt; &gt; hello world
    &gt; &gt; ```
    &gt; &gt;
    &gt; &gt; [View](https://example.com/build_url) the CI Build
    &gt;
    &gt; &lt;/details&gt;
    
    &lt;/details&gt;
    
    To view individual test run time comparison to the main branch, go to the [Test Analytics Dashboard](https://app.codecov.io/{services_short_dict.get(commit.repository.service)}/{commit.repository.owner.username}/{commit.repository.name}/tests/{commit.branch})"""
        )
E       AssertionError: assert '### :x: 1 Te...dback/issues)' == '### :x: 1 Te.../test_branch)'
E         
E         Skipping 455 identical leading characters in diff, use -v to show
E           /details&gt;
E         - 
E         - To view individual test run time comparison to the main branch, go to the [Test Analytics Dashboard](https://app.codecov..../everybody-improve/tests/test_branch)
E         ?         ^^^^^^^^^^      ^^ ^  ---  --------       -----------------------
E         + \To view more test analytics go to the [Test Analytics Dashboard](https://app.codecov..../everybody-improve/tests/test_branch)...
E         
E         ...Full output truncated (2 lines hidden), use '-vv' to show

services/tests/test_test_results.py:162: AssertionError

To view individual test run time comparison to the main branch, go to the Test Analytics Dashboard

Copy link

Test Failures Detected: Due to failing tests, we cannot provide coverage reports at this time.

❌ Failed Test Results:

Completed 1700 tests with 3 failed, 1697 passed and 0 skipped.

View the full list of failed tests

pytest

  • Class name: services.tests.test_test_results
    Test name: test_build_message

    def test_build_message():
    flags_hash = generate_flags_hash([])
    test_id = generate_test_id(1, "testsuite", "testname", flags_hash)
    fail = TestResultsNotificationFailure(
    "hello world",
    "testname",
    [],
    test_id,
    1.0,
    "https://example.com/build_url",
    )
    payload = TestResultsNotificationPayload(1, 2, 3, [fail], dict())
    commit = CommitFactory(branch="thing/thing")
    tn = TestResultsNotifier(commit, None, None, None, payload)
    res = tn.build_message()

    > assert (
    res
    == f"""### :x: 1 Tests Failed:
    | Tests completed | Failed | Passed | Skipped |
    |---|---|---|---|
    | 3 | 1 | 2 | 3 |
    <details><summary>View the top 1 failed tests by shortest run time</summary>

    >
    > ```
    > testname
    > ```
    >
    > <details><summary>Stack Traces | 1s run time</summary>
    >
    > >
    > > ```
    > > hello world
    > > ```
    > >
    > > [View](https://example.com/build_url) the CI Build
    >
    > </details>

    </details>

    To view individual test run time comparison to the main branch, go to the [Test Analytics Dashboard](https://app.codecov.io/{services_short_dict.get(commit.repository.service)}/{commit.repository.owner.username}/{commit.repository.name}/tests/thing%2Fthing)"""
    )
    E AssertionError: assert '### :x: 1 Te...dback/issues)' == '### :x: 1 Te...hing%2Fthing)'
    E
    E Skipping 390 identical leading characters in diff, use -v to show
    E /details>
    E -
    E - To view individual test run time comparison to the main branch, go to the [Test Analytics Dashboard](https://app.codecov..../exist-parent-ago/tests/thing%2Fthing)
    E & ^^^^^^^^^^ ^^ ^ --- -------- -----------------------
    E + \To view more test analytics go to the [Test Analytics Dashboard](https://app.codecov..../exist-parent-ago/tests/thing%2Fthing)...
    E
    E ...Full output truncated (2 lines hidden), use '-vv' to show

    services/tests/test_test_results.py:113: AssertionError
  • Class name: services.tests.test_test_results
    Test name: test_build_message_with_flake

    def test_build_message_with_flake():
    flags_hash = generate_flags_hash([])
    test_id = generate_test_id(1, "testsuite", "testname", flags_hash)
    fail = TestResultsNotificationFailure(
    "hello world",
    "testname",
    [],
    test_id,
    1.0,
    "https://example.com/build_url",
    )

    payload = TestResultsNotificationPayload(
    1, 2, 3, [fail], {test_id: FlakeInfo(1, 3)}
    )
    commit = CommitFactory(branch="test_branch")
    tn = TestResultsNotifier(commit, None, None, None, payload)
    res = tn.build_message()

    > assert (
    res
    == f"""### :x: 1 Tests Failed:
    | Tests completed | Failed | Passed | Skipped |
    |---|---|---|---|
    | 3 | 1 | 2 | 3 |
    <details><summary>View the full list of 1 :snowflake: flaky tests</summary>

    >
    > ```
    > testname
    > ```
    >
    > **Flake rate in main:** 33.33% (Passed 2 times, Failed 1 times)
    > <details><summary>Stack Traces | 1s run time</summary>
    >
    > >
    > > ```
    > > hello world
    > > ```
    > >
    > > [View](https://example.com/build_url) the CI Build
    >
    > </details>

    </details>

    To view individual test run time comparison to the main branch, go to the [Test Analytics Dashboard](https://app.codecov.io/{services_short_dict.get(commit.repository.service)}/{commit.repository.owner.username}/{commit.repository.name}/tests/{commit.branch})"""
    )
    E AssertionError: assert '### :x: 1 Te...dback/issues)' == '### :x: 1 Te.../test_branch)'
    E
    E Skipping 455 identical leading characters in diff, use -v to show
    E /details>
    E -
    E - To view individual test run time comparison to the main branch, go to the [Test Analytics Dashboard](https://app.codecov..../everybody-improve/tests/test_branch)
    E & ^^^^^^^^^^ ^^ ^ --- -------- -----------------------
    E + \To view more test analytics go to the [Test Analytics Dashboard](https://app.codecov..../everybody-improve/tests/test_branch)...
    E
    E ...Full output truncated (2 lines hidden), use '-vv' to show

    services/tests/test_test_results.py:162: AssertionError
  • Class name: services.tests.test_test_results
    Test name: test_generate_failure_info

    def test_generate_failure_info():
    flags_hash = generate_flags_hash([])
    test_id = generate_test_id(1, "testsuite", "testname", flags_hash)
    fail = TestResultsNotificationFailure(
    "hello world",
    "testname",
    [],
    test_id,
    1.0,
    "https://example.com/build_url",
    )

    > res = generate_failure_info(fail)
    E TypeError: generate_failure_info() missing 1 required positional argument: 'commit'

    services/tests/test_test_results.py:84: TypeError

@RulaKhaled RulaKhaled changed the title feat: Tests results PR comment update <Temp don't merge> Nov 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant