Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature/of-504-user-metrics-in-subgraph #78

Merged
merged 12 commits into from
Jan 14, 2025

Conversation

george-openformat
Copy link
Contributor

@george-openformat george-openformat commented Dec 11, 2024

Description

This PR adds aggregation stats to be able to create these graphs:

  • Unique users over time (line graph)
  • Total issued rewards over time (line graph)
  • Breakdown of rewards by id (pie chart)

Changes

Entities:

UserReward - Created to solve the Unique users rewarded problem. The idea of this entity is to keep track of what users have received a reward on which app. Because we can only query an entity id from the db in our handlers, this UserReward saves the relationship ${user_id}-${app_id} in its ID the first time (and only the first time) a user receives a reward in the app.

AppRewardId - Created to save RewardId for each app. Entity Reward and timeseries RewardData already have this data, we need this new entity because we can not create a Graphql resolver to add a distinct or group by to the queries.

Timeseries:

UserRewardData - Created to count the amount of new unique users per app. When we save a UserReward we also save a new UserRewardData, the result is we only create a new stat when a new user is rewarded in an app.

RewardData - To create aggregations over all Rewards regardless of type (token, badge).

RewardTokenData - To create aggregation over only Token rewards.

RewardBadgeData - To create aggregation over only Badges rewards.

Note: In all timeseries entities, the relations in the form app: App where changed to appId: String. The previous form was throwing occasional errors: ERROR: functions in index expression must be marked IMMUTABLE

Current stats:

  • UserRewardAppStat, source: UserRewardData - New users by app.
  • RewardAppStat - Count and total rewards per app.
  • RewardAppRewardIdStat
  • RewardAppRewardTypeStat
  • RewardAppRewardIdRewardTypeStat
  • RewardAppUserStat
  • RewardAppUserRewardTypeStat
  • RewardAppUserRewardIdStat
  • RewardAppUserRewardIdRewardTypeStat
  • RewardAppTokenTypeStat, source: RewardTokenData - Stats for all tokens (and only tokens) on each app.
  • RewardAppTokenTypeUserStat, source: RewardTokenData - Stats for all tokens (and only tokens) on each app and user.
  • RewardAppTokenStat, source: RewardTokenData - Stats for specific tokens and apps
  • RewardAppTokenUserStat, source: RewardTokenData
  • RewardAppBadgeTypeStat, source: RewardBadgeData - Stats for all badges (and only badges) on each app.
  • RewardAppBadgeTypeUserStat, source: RewardBadgeData
  • RewardAppBadgeStat, source: RewardBadgeData - Stats for specific badges and apps
  • RewardAppBadgeUserStat, source: RewardBadgeData

Tests

Tests are failing because matchstick library does not have support yet for Aggregations.

@george-openformat george-openformat force-pushed the feature/of-504-user-metrics-in-subgraph branch from 3a05333 to 74737b0 Compare December 17, 2024 14:13
@nup9151f nup9151f self-assigned this Dec 17, 2024
@nup9151f nup9151f requested a review from tinypell3ts December 19, 2024 09:22
Base automatically changed from staging to main January 13, 2025 09:19
Copy link
Contributor Author

@george-openformat george-openformat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for not looking at this for a while, wanted to make sure give it some good use before reviewing.

Proposed a couple of small changes to the shema. But I think this is basically ready to go.

The main blocker looks like the test suite not supporting aggregations. @nup9151f are there any ways around this so we can still test the rest of the subgraph?

schema.graphql Outdated
tokenAmount: BigInt!

badgeId: String
badgeCount: Int!
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

badgeCount should be badgeTokenCount
Reason being in the schema badge refers to NFT contract and badgeToken refers to the individual NFT

schema.graphql Outdated
transactionHash: String!

badgeId: String
badgeCount: Int!
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here badgeCount should be badgeTokenCount

schema.graphql Outdated
Comment on lines 302 to 317
type RewardAppTokenTypeStat @aggregation(intervals: ["hour", "day"], source: "RewardTokenData") {
id: Int8!
timestamp: Timestamp!
appId: String!
count: Int8! @aggregate(fn: "count")
totalCount: Int8! @aggregate(fn: "count", cumulative: true)
}

type RewardAppTokenTypeUserStat @aggregation(intervals: ["hour", "day"], source: "RewardTokenData") {
id: Int8!
timestamp: Timestamp!
appId: String!
userId: String!
count: Int8! @aggregate(fn: "count")
totalCount: Int8! @aggregate(fn: "count", cumulative: true)
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After coming back to this I think we should remove stat types that contain TokenType as I am not totally convinced it's all that useful to an end user. So remove RewardAppTokenTypeStat and RewardAppTokenTypeUserStat

It shows us the amount of rewards issued that contained a token for a given app. This can be constructed from querying all the tokens used by that app, with the RewardAppTokenStat.

I would rather keep it out for now and add in later if highly requested, but more than happy to be convinced otherwise.

schema.graphql Outdated
Comment on lines 342 to 361
type RewardAppBadgeTypeStat @aggregation(intervals: ["hour", "day"], source: "RewardBadgeData") {
id: Int8!
timestamp: Timestamp!
appId: String!
count: Int8! @aggregate(fn: "count")
totalCount: Int8! @aggregate(fn: "count", cumulative: true)
badgeAmount: BigInt! @aggregate(fn: "sum", arg: "badgeCount")
totalBadgeAmount: BigInt! @aggregate(fn: "sum", arg: "badgeCount", cumulative: true)
}

type RewardAppBadgeTypeUserStat @aggregation(intervals: ["hour", "day"], source: "RewardBadgeData") {
id: Int8!
timestamp: Timestamp!
appId: String!
userId: String!
count: Int8! @aggregate(fn: "count")
totalCount: Int8! @aggregate(fn: "count", cumulative: true)
badgeAmount: BigInt! @aggregate(fn: "sum", arg: "badgeCount")
totalBadgeAmount: BigInt! @aggregate(fn: "sum", arg: "badgeCount", cumulative: true)
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here as TokenType I think we should remove RewardAppBadgeTypeStat and RewardAppBadgeTypeUserStat.

We can get this results by summing the results of RewardAppBadgeStat or RewardAppBadgeUserStat

schema.graphql Outdated
Comment on lines 370 to 371
badgeAmount: BigInt! @aggregate(fn: "sum", arg: "badgeCount")
totalBadgeAmount: BigInt! @aggregate(fn: "sum", arg: "badgeCount", cumulative: true)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

badgeAmount should be badgeTokenAmount
totalBadgeAmount should be totalBadgeTokenAmount

schema.graphql Outdated
Comment on lines 382 to 383
badgeAmount: BigInt! @aggregate(fn: "sum", arg: "badgeCount")
totalBadgeAmount: BigInt! @aggregate(fn: "sum", arg: "badgeCount", cumulative: true)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

badgeAmount should be badgeTokenAmount
totalBadgeAmount should be totalBadgeTokenAmount

@nup9151f
Copy link
Collaborator

The main blocker looks like the test suite not supporting aggregations. @nup9151f are there any ways around this so we can still test the rest of the subgraph?

Hey @george-openformat:
Only mechanism I found is to comment RewardsFacet tests.
Problem is that every time we fire a reward event we need to store the associated timeseries which throws the error in the test library.

@george-openformat
Copy link
Contributor Author

Gotcha, thanks for explaining that.
I had a play around and have a working solution that is somewhat of a hacked feature flag.
see #82

@nup9151f nup9151f marked this pull request as ready for review January 14, 2025 17:19
@nup9151f
Copy link
Collaborator

Hey @george-openformat:
Just made this ready for review, can not add you as Reviewer.

@george-openformat george-openformat changed the title Sketch add aggregations feature/of-504-user-metrics-in-subgraph Jan 14, 2025
Copy link
Collaborator

@nup9151f nup9151f left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@george-openformat george-openformat changed the base branch from main to staging January 14, 2025 17:42
@george-openformat
Copy link
Contributor Author

Awesome

@george-openformat george-openformat merged commit 5a19e56 into staging Jan 14, 2025
1 check passed
@george-openformat george-openformat deleted the feature/of-504-user-metrics-in-subgraph branch January 14, 2025 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants