Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Monitor performance #50

Open
letmaik opened this issue Dec 19, 2022 · 0 comments
Open

Monitor performance #50

letmaik opened this issue Dec 19, 2022 · 0 comments
Labels
ci Continuous Integration enhancement New feature or request testing Anything related to testing
Milestone
Backlog

Comments

@letmaik
Copy link
Member

letmaik commented Dec 19, 2022

We should define some performance metrics and monitor them similar to how CCF does in their repository.

Metrics could be:

  • Throughput on POST /entries
    • did:web using cached resolution
    • X.509
  • Throughput on GET /entries/<txid>/receipt
    • Cached historical transaction
  • Latency as time to receipt following POST /entries
    • Uncached did:web, requires at minimum two calls to POST /entries, followed by one or more GET /entries/<txid>/receipt until receipt is ready
    • Cached did:web, requires one call to POST /entries, followed by one or more GET /entries/<txid>/receipt
    • X.509, same as cached did:web
    • Note: Measuring latency is tricky as it relies on some parameters like CCF's tree signing interval (configurable) and the client delay between retries both for POST /entries in the case of uncached did:web and for receipt fetching.
  • Memory usage after inserting a large number of entries
    • Single issuer
    • Many issuers
    • With/without prefix tree

Using a small claim payload size for now is probably fine, like 1 KB.
@letmaik letmaik added enhancement New feature or request testing Anything related to testing ci Continuous Integration labels Dec 19, 2022
@letmaik letmaik mentioned this issue Jan 11, 2023
Merged
letmaik added a commit that referenced this issue Jan 16, 2023
@letmaik
Add latency perf test (#69)
67fe437 
This implements some of #50, in particular the latency for `POST
/entries` as well as time-to-receipt which involves further GET calls.
It is implemented as a pytest that's disabled by default. It writes a
`perf.json` to the current folder (added to `.gitignore`) which gets
uploaded as pipeline artifact in CI.
@ivarprudnikov ivarprudnikov added this to the Backlog milestone Jan 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ci Continuous Integration enhancement New feature or request testing Anything related to testing
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
2 participants
@letmaik @ivarprudnikov