change to not depend on equality after seed in stoch tests #868
Conversation
| @@ -313,6 +309,66 @@ end | |||
|
|
|||
| ### Additional Correctness Tests ### | |||
|
|
|||
| # Tests that events are properly triggered for SDEs. | |||
There was a problem hiding this comment.
Makes stochastic models where all types of events should trigger. Trigger updates a parameter, which is used to check that they were triggered.
| sprob_int_ref = SDEProblem(rs_ref_int, u0_dec, tspan_stoch, ps_int) | ||
| ssol_int = solve(sprob_int, ImplicitEM(); seed) | ||
| ssol_int_ref = solve(sprob_int_ref, ImplicitEM(); seed) | ||
| @test mean(ssol_int[:X1]) ≈ mean(ssol_int_ref[:X1]) atol = 2*1e0 |
There was a problem hiding this comment.
Checks that stochastic model means are approximately the same (tolerance set so this holds over as many simulations I could bother trialing)
There was a problem hiding this comment.
What is the expected value of the mean here? Is it comparable to 2? (I'm trying to understand what accuracy you are requesting and if the test is worthwhile with such a large atol...)
Same for below.
| @@ -16,12 +16,17 @@ include("../test_networks.jl") | |||
|
|
|||
| # Compares jump simulations generated through hCatalyst, and manually created systems. | |||
| let | |||
| # Manually declares jumps to compare Catalyst-generated jump simulations to. | |||
| # Declares vectors to store information about each network in. Initial conditions are approximately | |||
There was a problem hiding this comment.
I have added manually checked parameter sets and initial conditions, and for each checked that the mean are reproducibly similar across very many simualtions.
|
|
||
| sol_dsl = solve(SDEProblem(rn_dsl, u0, tspan, ps), ImplicitEM(); seed = 1234) | ||
| sol_prog = solve(SDEProblem(rn_prog, u0, tspan, ps), ImplicitEM(); seed = 1234) | ||
| @test sol_dsl == sol_prog |
There was a problem hiding this comment.
These are not strictly necessary. I mostly added them because I figured they could catch some underlying issue in MTK (which I never will exclude). Hopefully it won't happen, and if it does, we try SDE simulations all over the tests, so one of those might catch it.
| jsol_int = solve(jprob_int, SSAStepper(); seed) | ||
| jsol_int_ref = solve(jprob_int_ref, SSAStepper(); seed) |
There was a problem hiding this comment.
Just be careful about how many jumps are occurring as this is saving for every jump as configured...
| sprob_int_ref = SDEProblem(rs_ref_int, u0_dec, tspan_stoch, ps_int) | ||
| ssol_int = solve(sprob_int, ImplicitEM(); seed) | ||
| ssol_int_ref = solve(sprob_int_ref, ImplicitEM(); seed) | ||
| @test mean(ssol_int[:X1]) ≈ mean(ssol_int_ref[:X1]) atol = 2*1e0 |
There was a problem hiding this comment.
What is the expected value of the mean here? Is it comparable to 2? (I'm trying to understand what accuracy you are requesting and if the test is worthwhile with such a large atol...)
Same for below.
|
The typical error in those situations are about 0.1, I have seen it go up to 0.3 (over maybe 50 simulations?). I had it at 1e1, but then added a 2 at the end just to be thoroughly sure. |
TorkelE
force-pushed
the
change_stoch_sol_tests
branch
from
9e18f42
to
96eefbc
Compare
No description provided.