Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

clarify range of replication of excluded group feed #15

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

clarify range of replication of excluded group feed #15

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 22 additions & 13 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,8 +173,9 @@ graph TB;
zero--"a excludes c"-->one[H: a,b,d]
```

Let `Ga` be `a`'s subfeed dedicated for publishing messages for epoch `G`. To
perform member exclusion, the following steps SHOULD be taken in order:
Let `Ga` be `a`'s subfeed dedicated for publishing messages for epoch `G`, and
similarly `Gc` be `c`'s subfeed. To perform member exclusion, the following
steps SHOULD be taken in order:

* 4.1.1. `a` MUST create a new symmetric group key `H` (also known as the epoch
key) which MUST have at least 32 bytes of cryptographically secure random data
Expand All @@ -190,9 +191,12 @@ Section 3.2.2.
* 4.1.4. `a` SHOULD publish an encrypted `group/exclude` message on `Ga` with
the following fields in the message `content`:
* 4.1.4.A. `type` equals the string `group/exclude`
* 4.1.4.B. `excludes` is an array of group member IDs (their root metafeed
IDs) excluded from `G`. In this case `c` is the only excluded member, but
Section 4.1. supports excluding multiple members at once.
* 4.1.4.B. `excludes` is an array of objects with the shape
`{id, groupFeedId, sequence}` where `id` is the root metafeed ID of `c`,
`groupFeedId` is `Gc`'s ID, and `sequence` is the sequence number of the last
message `a` possesses from `Gc`. In this case `c` is the only excluded
member, so we only have one `{id, groupFeedId, sequence}` object but Section
4.1. also supports excluding multiple members at once.
Comment on lines +194 to +199
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should be able to (i think?) calculate this technically by looking at the group tangle but I guess this way is both clearer and more efficient.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A really fun bug would be if the excluder starts creating this message, but then inbetween that and creating the tangle for the message, receives another message from the excluded person, pointing the group tangle at that message in the tangles.group.previous. Then the announce of this sequence and the group tangle would disagree on which messages to include.

I guess a way of preventing that would be for the excluder to stop replicating the excluded person before starting to create this message.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minus the tangle thing, I wouldn't see a problem. The admin saw "the last straw" from Oscar and decided that Oscar has to leave. It doesn't matter if Oscar keeps on posting because "the last straw" message is the one that the admin decided is the last one.

But since you mentioned tangles and their linking, and since I don't understand tangles (I thought I did), I'm like 🤷 and need someone to explain like I'm five. Maybe @mixmix ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah but if we later on add e.g. tangle sync, then people using that would end up with a different set of messages than people syncing whole feeds.

either way this is just in case an implementation has what's essentially a bug.

maybe just add a warning, something like

"implementers SHOULD (must?) be careful to not include newer messages than sequence in any tangle."

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@staltz did you get to understanding tangles? did my scuttelbutt write-up help?

Hmm... if the tangle thing is gonna be used/ useful we have an interesting edge case:

  • Staltz excludes { id: Oscar, groupFeedId: OscarG, sequence: 234 }
  • Oscar published message in OscarG with sequence: 235
  • Mix replicates Oscar, and extends the group tangle off of that message of Oscars, then replicates Staltz and is like "oops I included a message after the exclusion in the group tangle").

We aren't using the tangle ordering (tangles.group.previous) thing yet for anything... so this is quite hard to design for. We mainly use tangles.group.root to find messages in a particular group....

Gut feeling is say "fuck it" and just continue on.... but if people receive Staltz's exclusion message first they will stop replicating and then have a gap in their tangle... hmmm.
Hard

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, in this situation

Mix replicates Oscar, and extends the group tangle off of that message of Oscars, then replicates Staltz and is like "oops I included a message after the exclusion in the group tangle").

even without tangles, Mix might be replying to Oscar's latest comment. But Staltz would never see that comment since he doesn't want to go beyond the sequence he announce before. This would break discussion threads. Tangles in this case doesn't create the problem, it's a method of dealing with the problem that is eventual consistency.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#19

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mix might be replying to Oscar's latest comment. But Staltz would never see that comment

I could see Mix's comment, I just won't see what did Oscar say that Mix is replying to. I think that's an okay compromise to make.

By allowing Oscar to be "eventually excluded", we open up a few abuse channels too, because Oscar knows he's excluded but still has his final words to make, and at that point he might not give a shit about the code of conduct anymore. I think that's a worse outcome than discussion threads looking slightly punched with holes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already don't make much of/any effort to deal with bad actors once they're inside a group. And not accomodating for eventual consistency would break a core design choice of scuttlebutt. It's not necessarily "slightly punched with holes", it might as well be long discussions. It'd also break i think one of our design principles for groups, that everyone should have the same set of messages etc. That's e.g. the reason we ignore blocks in groups.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mixmix @arj03 Help us out here? We're stuck

* 4.1.4.C. `recps` is an array containing a single string: the group ID for
`G`, signalling that this message should be box2-encrypted for the group `G`
* 4.1.5. `a` MUST publish a `group/add-member` message on `Ga`, to add remaining
Expand Down Expand Up @@ -414,14 +418,19 @@ Assume that replicating an SSB feed involves two operations: "fetching" new
updates (new messages published) from that feed, and "serving" any messages in
that feed to an interested peer.

As soon as a peer `a` has discovered their most preferred epoch `H`:

* 4.8.2.A. `a` SHOULD cease fetching messages from group feeds of other epochs
`X` belonging to a member that was excluded from `X`, but should continue to
fetch messages from group feeds belonging to any remaining member of `X`. See
figure 9 as an example.
* 4.8.2.B. `a` SHOULD continue to serve messages from group feeds of any epoch
`X` belonging to **any** member of `X`.
As soon as a peer `a` has discovered their most preferred epoch `H`, then for
every epoch `X` with lower preference than `H`:

* 4.8.2.A. (Fetching excluded members) `a` SHOULD fetch messages from group
feeds `Xc` belonging to every peer `c` that was excluded from `X`, up until the
sequence number defined in 4.1.4.B., after which `a` SHOULD cease fetching `Xc`.
In other words, `a` SHOULD NOT fetch messages from `Xc` with sequence numbers
greater than the sequence number defined in 4.1.4.B.
* 4.8.2.B. (Fetching remaining members) `a` SHOULD continue to fetch messages
from group feeds `Xb` belonging to every remaining member `b` in `X`. See figure
9 as an example.
* 4.8.2.C. (Serving all members) `a` SHOULD continue to serve messages from
group feeds of any epoch `X` belonging to **any** member of `X`.

```mermaid
---
Expand Down