Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add discussions support #319

Open
wants to merge 21 commits into
base: v4
Choose a base branch
from

Conversation

3urobeat
Copy link
Contributor

Hey!
This PR adds support for scraping information and comments from discussions, as well as posting & deleting comments, subscribing & unsubscribing to/from discussions and setting the amount of comments shown per page.

I have added a new class which represents an existing discussion supplied via URL and scrapes various information from the DOM:

  • id (string) - ID of the discussion, is used in the URL
  • type (EDiscussionType) - Where the discussion is associated to (app, group or the Steam forum)
  • appID (string) - The appID this discussion is associated to, if of type EDiscussionType.App
  • forumID (string) - ID of the forum the discussion is in, is used in the URL
  • gidforum (string) - ID used in post requests when interacting with discussions, not sure why
  • topicOwner (string) - ID used in post requests when interacting with discussions, not sure why
  • author (SteamID) - SteamID object of the author of this discussion
  • postedDate (Date) - When this discussion was created
  • title (string) - Title of the discussion
  • content (string) - Content of the discussion
  • commentsAmount (number) - Amount of comments present in this discussion
  • answerCommentIndex (number) - If this discussion has an answer, this is the index of the comment

The functions already mentioned above are implemented in components/discussions.js.
Beside the usual functions for commenting and subscribing, it includes another scraper for retrieving a range of comments. Each comment includes the following information:

  • index (number) - Index of this comment in the comment chain of this discussion
  • commentId (number) - ID of this comment, issued by Steam
  • commentLink (string) - Link to this specific comment, will highlight it
  • authorLink (string) - Profile link of the author of this comment. I did not convert this one to a SteamID to reduce the large amount of extra requests needed to resolve each author link when requesting a large-ish range
  • postedDate (Date) - When this comment was posted
  • content (string) - Content of all comments quoted in this comment and the comment itself

I have not added support for posting discussions as a class only represents one discussion, not a forum.

Test code to get 3 comments from the provided discussion and see the nested quotes support:

community.getSteamDiscussion("https://steamcommunity.com/app/739630/discussions/0/1750150652078713439/", (err, res) => {
    res.getComments(35, 37, (err, comments) => {
        console.log(comments);
    });
});

Note: I have encountered an error with the scraper yesterday at discussion.js line 39 getting children of undefined, which I was sadly unable to reproduce again. I just wanna note that the scraper might not be completely failsafe, this is however the same case with the other scrapers as well. If I find any bugs I'll ofc look into fixing them and open another PR, just like previously.

Thanks for reading and I hope you find this feature useful.
Feel free to use the text from this PR in any documentation page if you wish.
Have a nice day :)

@3urobeat
Copy link
Contributor Author

Oh and while we are at it, I think I mixed both of these a bit.
Not terrible, however a uniform usage of either of those would probably be better

@Revadike
Copy link
Contributor

You prob want to merge it into #230

@3urobeat
Copy link
Contributor Author

Yeah I thought about that but ultimately decided against it as DoctorMcKay merged my previous PRs into v4 afterwards anyway and the v4 branch seems a bit stale.
I can open another PR/rebase this one though if you wish

@3urobeat 3urobeat changed the base branch from master to v4 October 1, 2023 17:03
@DoctorMcKay
Copy link
Owner

DoctorMcKay commented Oct 2, 2023

I'm cool with scraping discussions to retrieve data, but posting threads and comments isn't something I'm comfortable with adding due to the potential for abuse/spam. I'm aware that it's not really difficult for spammers to post spam on their own without the help of the lib, but still. I like to try to maintain at least some plausible deniability for enabling bad people to do bad things.

Also, if you're going to target v4 (which seems appropriate), please make sure you're addressing the new lint rules. There are also a bunch of other changes, for example the removal of httpRequestGet.

@Revadike
Copy link
Contributor

Revadike commented Oct 2, 2023

I'm cool with scraping discussions to retrieve data, but posting threads and comments isn't something I'm comfortable with adding due to the potential for abuse/spam. I'm aware that it's not really difficult for spammers to post spam on their own without the help of the lib, but still. I like to try to maintain at least some plausible deniability for enabling bad people to do bad things.

Valve wouldn't know it came from your library, though. Still, you can combat it somewhat by implementing some guards that align with the steam discussion rules, like a blocklist and URL detection. Let's be real, anything can be abused, I hate to see a feature gone just because of theoretical bad actors.

@3urobeat
Copy link
Contributor Author

3urobeat commented Oct 2, 2023

I can understand leaving out voting in the sharedfile PR from a while ago but when the lib already supports posting comments to profiles, groups and sharedfiles then I don't really see where posting comments to discussions is much different.
In the end it is the user's account that is displayed as the comment author and is responsible for its content. If one wants to post malicious content they could do the same manually - we are not circumventing anything.

Also, if you're going to target v4 (which seems appropriate), please make sure you're addressing the new lint rules

Yeah, I saw the check yesterday and wanted to address those today

@3urobeat
Copy link
Contributor Author

3urobeat commented Oct 2, 2023

I have fixed all eslint errors/warnings. The check is still failing because of other files not relevant to this PR.
I'm not yet sure how the new httpRequest() usage should look like, so I might have to revisit that some time later.

Edit:
I have updated the httpRequest() usage - I hope it is correct like this. I briefly tested the methods and they seem to work as expected, I however don't know if I need to do more checking if the request failed (previously the httpRequestPost helper returned an err param besides the body itself) as I'm currently blindly using the resulting body.
Or maybe I'm overthinking this and the httpRequest helper already takes care of it?

I have also renamed the class method postComment to comment to follow the same naming scheme as the comment methods of the other classes. Not sure why I named it this way in the first place.

@DoctorMcKay
Copy link
Owner

I have fixed all eslint errors/warnings. The check is still failing because of other files not relevant to this PR.

Sounds good, I should probably try to fix all those errors in the v4 branch shortly.

Edit: I have updated the httpRequest() usage - I hope it is correct like this. I briefly tested the methods and they seem to work as expected, I however don't know if I need to do more checking if the request failed (previously the httpRequestPost helper returned an err param besides the body itself) as I'm currently blindly using the resulting body. Or maybe I'm overthinking this and the httpRequest helper already takes care of it?

It's not final yet, but I think the helper is going to take care of all this like it did previously, and reject in cases where it would return an error previously.

Is this ready for review?

@3urobeat
Copy link
Contributor Author

3urobeat commented Oct 2, 2023

Yeah, I think it is ready for review.
If I notice any oversights or issues I'll of course look into them and open another PR.
As of now all functions seem work as expected

@3urobeat
Copy link
Contributor Author

3urobeat commented Mar 2, 2024

I've added support for eventcomments too, this is another type of discussion, usually used to post patch notes or make event announcements.
I have also added a accountCanComment prop, because devs can choose if limited accounts are allowed to comment on these posts (for example CS2 & TF2 allow limited accounts, PUBG doesn't - all three are free games).

I have also noticed that testing the changes made in this PR just by checking out this branch does not work because it was rebased, and therefore httpRequestGet still has the old implementation. Just for your information when reviewing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants