Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests: Add ugly integration test for Reports\Full #664

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

MatmaRex
Copy link
Contributor

The class does not really have individually testable components. Provide a big ugly real report data as the input, and a big ugly real generated report as the output. This is not great, but it's something.

The Full report ignores the $phpcsFile argument, so it's easy to mock. Other reports would require a more complicated setup to test.

The test data has been generated from a file simple.php as follows:

<?php

    // This is a space indented line.
    $a = 10;
	// This is a tab indented line.

…using the following command for the output:

phpcs simple.php --basepath=. --report=full --tab-width=4 --no-colors --standard=generic

…and adding this snippet in generateFileReport() for the input:

var_export($report);

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
    • This change is only breaking for integrators, not for external standards or end-users.
  • Documentation improvement

PR checklist

  • I have checked there is no other PR open for the same change.
  • I have read the Contribution Guidelines.
  • I grant the project the right to include and distribute the code under the BSD-3-Clause license (and I have the right to grant these rights).
  • I have added tests to cover my changes.
  • I have verified that the code complies with the projects coding standards.
  • [Required for new sniffs] I have added XML documentation for the sniff.

@MatmaRex
Copy link
Contributor Author

I could add some more test cases, but I thought I'd submit this first to see what you think about this kind of test.

The class does not really have individually testable components.
Provide a big ugly real report data as the input, and a big ugly real
generated report as the output. This is not great, but it's something.

The Full report ignores the $phpcsFile argument, so it's easy to mock.
Other reports would require a more complicated setup to test.

The test data has been generated from a file `simple.php` as follows:
```
<?php

    // This is a space indented line.
    $a = 10;
	// This is a tab indented line.
```
…using the following command for the output:
```
phpcs simple.php --basepath=. --report=full --tab-width=4 --no-colors --standard=generic
```
…and adding this snippet in generateFileReport() for the input:
```
echo var_export($report);
```
Copy link
Member

@jrfnl jrfnl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @MatmaRex, sorry for my slow response on this PR, but I wanted to think this over a little.

I don't think this is the way to go for these tests. Mocking the input is going to make maintaining the tests (and creating additional tests) very fiddly.
I'm also concerned that if something in the File class would change and inadvertently create a breaking issue for the reports, these tests wouldn't catch it.

I think the Report tests will need to follow a pattern similar to the tests which I recently created for the Generator classes, which combines fixtures (specially crafted input files to run the reports on to cover all sorts of situations which need to be handled in the reports and potentially some custom sniffs just to create specific error messages for the reports) with output expectations.

Once I'm finished with the work on the Ruleset class, I think I'll have a go at this myself if you don't mind.


Just in case you want to continue with this in the mean time anyway, here are some things to consider:

  • Please use the ConfigDouble instead of the Config class. That ensures that the tests are not influenced by settings in a CodeSniffer.conf file of the person running the tests.
  • The test as-is is unstable as the --report-width is not set, which - in combination with using the Config class instead of the ConfigDouble - would make the report width dependent on the screen width of the user running the tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants