-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WIP: docs: Add high-level QA framework #9303
base: main
Are you sure you want to change the base?
Conversation
…ent_algo # Conflicts: # spec/consensus/light-client.md
Move light specs to their own dir, add readme and diagram
Update specification of light client algorithm to align with the code
lite->light
./light -> ./light-client
Fix link in readme
* Update the secret connection spec with the use of merlin to eliminte handshake malleability * Update spec/p2p/peer.md Co-Authored-By: Anton Kaliaev <[email protected]> * Update spec/p2p/peer.md Co-Authored-By: Anton Kaliaev <[email protected]> * Update spec/p2p/peer.md Co-Authored-By: Anton Kaliaev <[email protected]> Co-authored-by: Anton Kaliaev <[email protected]>
- cmn was remvoed in favor of sub pkgs. cmn.kvpair is now kv.pair Signed-off-by: Marko Baricevic <[email protected]>
* evidence: Add time to evidence params - this pr is grouped together with #4254, once that PR is merged then this one can be as well. Signed-off-by: Marko Baricevic <[email protected]> * remove note Signed-off-by: Marko Baricevic <[email protected]> * Apply suggestions from code review Co-Authored-By: Anton Kaliaev <[email protected]> Co-authored-by: Anton Kaliaev <[email protected]>
update link to the pex reactor
Merge `main` into `feature/abci++ppp`
* Typo fix in README.md (#350) * Updated Apalache type annotations (#395) Co-authored-by: Prajjwol Gautam <[email protected]> Co-authored-by: Kukovec <[email protected]>
This RFC lays out a proposed set of changes for consolidating the set of information that may be included in the block structure. {{ [rendered](https://github.com/tendermint/tendermint/blob/wb/rfc-block-structure/docs/rfc/rfc-020-block-structure-consolidation.md) }}
…lity (#9263) * update Apalache type annotations and split evidence into 3 variables * remove the duplicate of AllPrevotes, due to merge
Partial backport of #8480
This is largely a cherry pick of #6755 with some additional fixups added where detected. This change moves the blockchain package to a package called blocksync. Additionally, it renames the relevant uses of the term `fastsync` to `blocksync`. closes: #9227 #### PR checklist - [ ] Tests written/updated, or no tests needed - [x] `CHANGELOG_PENDING.md` updated, or no changelog entry needed - [x] Updated relevant documentation (`docs/`) and code comments, or no documentation updates needed
* [cherrypicked] abci-cli: added `PrepareProposal` command to cli (#8656) * Prepare prosal cli * [cherrypicked + fixes] abci-cli: Add `process_proposal` command to abci-cli (#8901) * Add `process_proposal` command to abci-cli * Added process proposal to the 'tutorial' examples * Added entry in CHANGELOG_PENDING.md * Allow empty blocks in PrepareProposal, ProcessProposal, and FinalizeBlock * Fix minimum arguments * Add tests for empty block * Updated abci-cli doc Co-authored-by: Sergio Mena <[email protected]> Co-authored-by: Jasmina Malicevic <[email protected]> * Addressed @williambanfield's comment Co-authored-by: Jasmina Malicevic <[email protected]> Co-authored-by: Hernán Vanzetto <[email protected]>
Signed-off-by: Thane Thomson <[email protected]> Signed-off-by: Thane Thomson <[email protected]>
* Update docs references from master to main Signed-off-by: Thane Thomson <[email protected]> * Update DOCS_README to reflect current reality Signed-off-by: Thane Thomson <[email protected]> * Update vuepress config with current versions and updated discussions link Signed-off-by: Thane Thomson <[email protected]> * Update generated docs versions Signed-off-by: Thane Thomson <[email protected]> * Update docs build to use temp folder instead of home Signed-off-by: Thane Thomson <[email protected]> * Document build-docs Makefile target Signed-off-by: Thane Thomson <[email protected]> * Add serve-docs Makefile target to serve local build of docs Signed-off-by: Thane Thomson <[email protected]> * Ensure 404 page is copied during docs build Signed-off-by: Thane Thomson <[email protected]> * Redirect /master/ to /main/ Signed-off-by: Thane Thomson <[email protected]> * Attempt to resolve #7908 Signed-off-by: Thane Thomson <[email protected]> * Update OpenAPI references from master to main Signed-off-by: Thane Thomson <[email protected]> * Update CHANGELOG references from master to main Signed-off-by: Thane Thomson <[email protected]> * Update Docker readme references from master to main Signed-off-by: Thane Thomson <[email protected]> * Update UPGRADING references from master to main Signed-off-by: Thane Thomson <[email protected]> * Update package-specific documentation references from master to main Signed-off-by: Thane Thomson <[email protected]> * Update spec references from master to main Signed-off-by: Thane Thomson <[email protected]> * Update all code comment references to docs site from master to main Signed-off-by: Thane Thomson <[email protected]> * Build v0.34.x as "latest" Signed-off-by: Thane Thomson <[email protected]> * Explicitly mark v0.34 docs as latest in version selector Signed-off-by: Thane Thomson <[email protected]> * Update all links from docs.tendermint.com/main to docs.tendermint.com/latest Signed-off-by: Thane Thomson <[email protected]> * ci: Redeploy docs on pushes to v0.34.x Signed-off-by: Thane Thomson <[email protected]> * Temporarily copy spec directory into docs while building Signed-off-by: Thane Thomson <[email protected]> * Add nav link to main and clearly mark as unstable Signed-off-by: Thane Thomson <[email protected]> * Revert to only publishing docs in nav for v0.34 and v0.33 with no latest Signed-off-by: Thane Thomson <[email protected]> * Link to docs.tendermint.com/v0.34 from RFCs Signed-off-by: Thane Thomson <[email protected]> * Rather just use main for all docs.tendermint.com references on main branch Signed-off-by: Thane Thomson <[email protected]> * Rename GitHub tree links from master to main Signed-off-by: Thane Thomson <[email protected]> * Update link for ABCI Rust client Signed-off-by: Thane Thomson <[email protected]> * Update github links from master to main Signed-off-by: Thane Thomson <[email protected]> * Update badges in root readme Signed-off-by: Thane Thomson <[email protected]> * Remove codecov badge since we do not use it any more Signed-off-by: Thane Thomson <[email protected]> * Remove Java and Kotlin tutorials Signed-off-by: Thane Thomson <[email protected]> * Remove specs from docs build Signed-off-by: Thane Thomson <[email protected]> * Migrate spec links to GitHub repo from docs site Signed-off-by: Thane Thomson <[email protected]> * Remove references to non-existent PEX reactor spec Signed-off-by: Thane Thomson <[email protected]> * Fix linting badge in README Signed-off-by: Thane Thomson <[email protected]> Signed-off-by: Thane Thomson <[email protected]>
* Backport of sam/abci-responses (#9090) (#9159) Signed-off-by: Thane Thomson <[email protected]> Co-authored-by: William Banfield <[email protected]> Co-authored-by: Thane Thomson <[email protected]>
Signed-off-by: Thane Thomson <[email protected]>
Signed-off-by: Thane Thomson <[email protected]>
Signed-off-by: Thane Thomson <[email protected]>
| - Grammar | | | | | | | | | | ||
| - Parameters of API functions | | | | | | | | | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These two are quite specific for ABCI++. I would replace them in all tables by "Implementation ensures safety and liveness as given in the specification". Or perhaps something more catchy.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have the impression that this suggestion can be merged with "Robustness" below
| Security | | | | | | | | ||
| Memory leaks | | | | | | | | ||
| Storage leaks | | | | | | | | ||
| Error handling | | | | | | | | ||
| Serialization / deserialization | | | | | | | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know that this view of the problem leads to an effective prioritization of methodology, or gives us particular confidence in any of these dimensions.
While I think the best approach to this would be to start from stability/operational/performance requirements and work backwards from that, I think it'd be useful to frame the components that are the highest priority (e.g. memory leaks in testnet/e2e testing, error handling in static analysis, etc.) rather than divide our time attempting to cover everything at every level.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The point is not to cover every concern with every method but to eventually figure out what is the most favorable methods for what concern.
In the beginning, let's just record what methods we are using and what concerns we believe we can address with them.
Eventually, we should also identify bottlenecks. E.g., if the only method we are using to find memory leaks are testnets (which seem super expensive), it might make sense to figure out whether a different (cheaper) methods will give us similar confidence. Such an overview allows us to identify such bottlenecks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While I think the best approach to this would be to start from stability/operational/performance requirements and work backwards from that
This is implicit in the correctness evaluation for all of the specific protocols that Tendermint implements.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @thanethomson for this.
| End-to-end (E2E) testing | High | Low | | ||
| Testnet testing | High | Moderate/High | | ||
| Protocol audits | | High | | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider adding an entry here with something like "Protocol/Interfate Specification" (setup cost high, "per-execution cost" empty). Or, if you don't consider it an "approach", this info should be somewhere in this document.
This is needed for, at least, MBT and protocol audits mentioned above. Although, theoretically, it would also be needed to be able to say is a test (at whatever level) is passing or not.
- A description of the QA effort. | ||
- References: | ||
- [Link 1] | ||
- [Link 2] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add a field to this template with the outcome: "PASS, FAIL, INCONCLUSIVE".
The "INCONCLUSIVE" field is typically used when you try to run a test case (e.g. using a complex testnet script), and the test cannot be run because of problems outside the scope of the UUT.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the "outcome" idea. I am not sure whether we should give a list to choose from. For instance, if the item describes an audit, then we could say "We opened the following issues"
| - Grammar | | | | | | | | | | ||
| - Parameters of API functions | | | | | | | | | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have the impression that this suggestion can be merged with "Robustness" below
- [Crash/recovery](#crash-recovery) | ||
- [Upgrading](#upgrading) | ||
|
||
### Correct software engineering |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Clarifying question: what is the "type" of the cells? Is it a boolean to say we're already doing it? Is it something like {"N/A", "planned", "done", "in progress", "nice to have"}?
Closes #9020
📖 Rendered
@josef-widder and @sergio-mena, this is my first attempt at trying to codify Josef's QA framework in a way that's relevant to Tendermint.
Where I need some assistance at present is in answering the following questions and filling in some missing information:
What's left to be done here before this PR can be considered "ready for review":
PR checklist
CHANGELOG_PENDING.md
updated, or no changelog entry neededdocs/
) and code comments, or nodocumentation updates needed