You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently all our integration tests are executed on the same 3-node cluster.
That means we are not testing topology changes at all.
Side note: I think that we are also not adequately testing schema changes - there are 5 occurences of ALTER statement and 6 occurences of DROP statement in tests.
There are ideas like #1078 but it doesn't fully address the issue:
Any SCT run takes a loot of time and resources and is not easy to set up. That means it can't be used locally by developers - tests to run locally must finish in reasonable time and be easy to run.
It uses cql-stress: it won't t thoroughly test drivers APIs correctness like in-repo tests can.
Being out of repo means it is harder to edit / fix: you need to makes changes here, in cql stress and in sct. That's a lot of friction.
For those reasons I mostly view SCT tests as benchmarks (which is something we severely miss!) and as a guard against some regressions.
What I propose to have in-repo topology tests, in steps:
Move all tests that require running Scylla/C* from unit tests to integration tests.
1.5 Step 1 may require exposing more private stuff in test_utils module. To avoid having it compiled in user code (and to prevent users from accidentaly modifying it) we should move it behind a feature - or even better, a cfg, like Tokio does: https://docs.rs/tokio/latest/tokio/#unstable-features
Create an API for tests to choose the required cluster configuration and other parameters - more on that later.
For that I'd see a macro that describes the cluster. It would look a bit like this:
#[scylla::integration_test(cluster=Cluster(nodes=3, tablets=off), ks=( parameters for created keyspace), table=(parameters for created table), session = (parameters for created session))]
test_fn(session, ks_name, table_name) {
}
Currently all our integration tests are executed on the same 3-node cluster.
That means we are not testing topology changes at all.
Side note: I think that we are also not adequately testing schema changes - there are 5 occurences of
ALTER
statement and 6 occurences ofDROP
statement in tests.There are ideas like #1078 but it doesn't fully address the issue:
cql-stress
: it won't t thoroughly test drivers APIs correctness like in-repo tests can.For those reasons I mostly view SCT tests as benchmarks (which is something we severely miss!) and as a guard against some regressions.
What I propose to have in-repo topology tests, in steps:
1.5 Step 1 may require exposing more private stuff in test_utils module. To avoid having it compiled in user code (and to prevent users from accidentaly modifying it) we should move it behind a feature - or even better, a
cfg
, like Tokio does: https://docs.rs/tokio/latest/tokio/#unstable-featuresFor that I'd see a macro that describes the cluster. It would look a bit like this:
It would use ccm underneath for cluster provisioning. It would also create a keyspace(s) and table(s) required by the test and clean them up after.
Relevant links:
https://www.infinyon.com/blog/2021/04/rust-custom-test-harness/
https://crates.io/crates/inventory
https://github.com/infinyon/fluvio/tree/master/crates/fluvio-test/src
https://github.com/infinyon/fluvio/blob/6635b8aae20df796604d60d2706289984bcc8da4/tests/runner/src/fluvio-integration-derive/src/lib.rs
The text was updated successfully, but these errors were encountered: