Improve tests

[jonasraoni]

Problems

• The tests take too long to run
• Running the tests locally is painful
• If a test fails, it forces us to run from beginning (or adopt a database/file backup strategy)
• Creating data for the tests can be challenging
• Frontend tests are tightly dependent on the backend
• Unable to paralelize tests due to race conditions
• Unable to test specific features on the e2e tests (it's possible on PHPUnit)

[jonasraoni]

Ideas

• Installing the database on a memory drive/ramdisk should improve the performance (it's needed to check if it fits the GitHub limits).
• Tools to reset/seed the database: mass create default submissions, users, reviews, metrics, etc.
  • Aren't the APIs enough to create data? What are they missing? There are UI tests that create a bunch of users, this type of thing could be moved to API calls.
  • Dumping/restoring a seeded database is pretty slow, but resetting a dockerized database is fast.
  • It's also needed to run tests with an empty dataset (e.g. installation, empty lists, etc)
• Make the frontend tests more granular/less dependent on each other.
• A faker package might be helpful on the PHP side, as well as on the JavaScript side.
• The frontend can have a switch to make use of mocked data to speed up tests (good for local tests/development time) and use the backend when there's intention to merge (probably will lead to maintenance burden).
• Spinning off more database instances might be enough to paralelize the tests, but GitHub limits might be a blocker.