Friendica with C2S - AndStatus is really greedy and confusing

[annando]

I'm currently experimenting with C2S on Friendica. This implementation is so new, that there isn't even a PR for it. it is currently only running on my develop machine pirati.ca. I'm currently experiencing some weird behaviour which I'm not sure if I made a mistake while coding C2S or if this is some AndStatus functionality.

At first: AndStatus is really greedy. It seems to try to download my full timeline several days in the past:

Can this behaviour be changed? I don't really see a sense in that.

Also it seems as if there is no real order in the timeline. It currently starts with 5 hours ago and decreases, while at the end it is 28 days ago.

Next is: I'm not really sure about the meaning of the different timelines:

Is there any explanation, what they are used for? At first I thought "Public" was the public timeline of the instance. But what is "Everything" then for? "Interactions" (at least on my system) displays a bunch of post that I don't think I interacted with. Does "Private notes" display all posts in my timeline that aren't public?

I guess that a lot of stuff is caused due to bugs in my implementation, although I mostly looked at how Pleroma did and also I had a glance at several discussion threads here.

P.S.: If you want to try out the implementation: It is currently read only. Posting is not yet supported.

[yvolk]

Hi @annando This looks more like a blog post than as an issue to be resolved. And you know it :-)
Let's move step-by-step and discuss problems separately.
What's the most important problem? Please create a separate issue here for it.

[annando]

The topic of this post is: "I don't know if I implemented C2S correctly or simply don't understand AndStatus". 😬

I see a bunch of posts in the different timelines that I don't really fit to what I think the timelines are thought for - and the items inside don't appear to be in a specific order. I guess it is mostly some: "What have I done wrong with the implementation".

[yvolk]

Answering to the general question: why AndStatus is "greedy".
From the User's point of view this means that on an account creation AndStatus is getting User's timelines' history to some depth. So the User doesn't feel like joining the social network just now :-). The user has friends and followers, the User posted, favorited and "announced" something... And AndStatus tries to load some latest part of that on the account creation.

As I see, currently 200 latest notes are downloaded for each new timeline. For active people this is a few days of history. Is this too much?

[yvolk]

Regarding timelines.
There is no strict definition of what should be in any of them. They are mostly different filters of "Everything" timeline. "Everything" is for one Social network, not for an Account (for an "Origin" in AndStatus internal terms).
I think this "TimelineType" definition in AndStatus with its comments will clear meaning of the timelines a bit: https://github.com/andstatus/andstatus/blob/master/app/src/main/kotlin/org/andstatus/app/timeline/meta/TimelineType.kt
That definition is universal for all types of Social networks.

[yvolk]

In ActivityPub C2S we have fewer "endpoints" than in older APIs. For example several "legacy" timeline types map to the INBOX. This could really cause AndStatus requesting the same content for "different" timelines.
Please see https://github.com/andstatus/andstatus/blob/master/app/src/main/kotlin/org/andstatus/app/net/social/ActorEndpointType.kt for mappings between timelines that we are accustomed to and "Actor's endpoints" that could be different for each Actor.

I'm sure there are lots of bugs, not fully implemented things or discussable decisions in current AndStatus ActivityPub implementation simply because there were too few users and their reports so far. I'm glad that I'm starting receiving them now :-)

[annando]

The first time I set up the connection, AndStatus took about half an hour downloading posts and images. (The performance is currently 5-10 seconds per page). I now found the setting where I can restrict the amount of posts to fetch. BTW: I just had a look, I'm receiving nearly 2,000 posts and comments a day. Since I found really old stuff there, AndStatus possibly downloaded more than 50,000 posts and comments - which is quite a lot, I think 😁

Concerning the timelines at AndStatus: Yeah, I guess I remember that in the past I already had got problems understanding them as well. Most apps simply map timelines with a specific endpoint, while on AndStatus the posts seems to be fetched from various endpoints and then are displayed according to some specific filters. This is no criticism, but more a lack of understand on my side :-)

But to the main problem: why is the content unsorted? Can it be that although I deleted all data during my tests, that there is some cache files still present?

[annando]

Concerning possible bugs: Of course I can't occupy your time, but it would be great if you could test on my system with your test account. I'm happy to fix all bugs that will appear.

[yvolk]

Regarding activities sorting in timelines.
The "Unread notifications" timeline is sorted by the date when the Activities were inserted into AndStatus database. I.e. the order of downloading. So you know what is really new for you. Some old activities may be downloaded just now, and you need to see them...

All other timelines are sorted by "updated date" of an Activity ( "updated" JSON field in ActivityPub)

[yvolk]

Concerning possible bugs: Of course I can't occupy your time, but it would be great if you could test on my system with your test account. I'm happy to fix all bugs that will appear.

Thank you @annando! I will do tests and report to you. Right now I'm investigating correctness and completeness of ActivityPub C2S implementation using microblog.pub, and I do fix AndStatus bugs one by one. That fixes may help you also, e.g. recently I removed one of sources of unneeded server requests (requests for Actor info with invalid ID).
The latest AndStatus build is here: #456

[yvolk]

Hello @annando!
I started testing. Where should I post Bug Reports / Issues?

I logged into pirati.ca as an ActivityPub type of Social Network.

1. I see that you use Mastodon flavor of OAuth 2.0 for client registration and logging in. Please note that recently we implemented a much more OAuth 2.0-compliant API, please see starting with Support /.well-known/oauth-authorization-server for ActivityPub #561
2. I see empty inbox.
3. I do see my own older messages sent via "GnuSocial" API in the "Sent" timeline.
4. Posting to outbox fails: it is not implemented yet, as I understand?! I got HTML page instead (time in UTC below)

02-05 09:38:24.604 D/HttpReadResult: getJsonObject; ; statusCode:OK (200); url:'https://pirati.ca/outbox/andstatus'; response:'<!DOCTYPE html…'

5. Didn't notice any "greedy" behavior so far: I have everything logged in files... But I'm not following anyone yet...

[annando]

You can open an issue at our repository, I guess that this is the best idea. Posting to the outbox is currently not implemented.

Concerning the empty inbox: When you directly call the inbox, what do you see then?

[yvolk]

Concerning the empty inbox: When you directly call the inbox, what do you see then?

Calling https://pirati.ca/inbox/andstatus from a Browser I see the same JSON as via API: it has no activities:

    "id": "https://pirati.ca/inbox/andstatus?page=1",
    "type": "OrderedCollectionPage",
    "partOf": "https://pirati.ca/inbox/andstatus",
    "orderedItems": []

Looking via Web UI on the "My posts and conversations" page I do see recent reply to me from mastodon.social. It should have been put into inbox, I think: