Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Headless module's browser on custom sources doesn't get closed on each chapter download #171

Open
luevano opened this issue May 30, 2023 · 1 comment
Labels
bug Something isn't working

Comments

@luevano
Copy link

luevano commented May 30, 2023

Describe the bug

For each chapter download there is one browser that gets piled up and causes a lot of CPU and Memory usage overtime which leads to soft-lock of the system (stops responding and needs manual intervention).

Noticed this while downloading 180+ chapter mangas using the Mangasee custom source (which uses headless). I couldn't see any exposed close() (or similar) exposed function for the browser, as shown in the examples (not exactly the same, but I assume there is a way to manually close one).

Currently my solution is to let it download ~30 chapters then Ctrl + c out of the command (either TUI or inline), then run again.

Reproduction steps

  1. Select any source that uses headless (Mangasee, for example).
  2. Search for any manga with a lot of chapters (Solo Leveling, Overgeared, for example).
  3. Start downloading all chapters.
  4. Wait for a few minutes, then monitor the processes.
  5. There will be a lot of chrome processes piled up, and on each new chapter downloading more will pile up.
  6. Ctrl + c out of the download, all chrome processes gone.

Expected behavior

After each thread downloading a chapter, the headless browser should be closed.

Additional context

Just showing a couple of screenshots for reference, I don't know how else to show this behavior. Showing 3 pairs of screenshots for mangal output and chrome processes (through gotop), these are not taken at the exact same time, so there is a bit of discrepancy of amount of processes, but I think the effect can be shown.

Fresh start of a download:
Screenshot 2023-05-30 051826

Currently piled up chrome processes:
Screenshot 2023-05-30 051820

After a bit more chapters downloaded:
Screenshot 2023-05-30 051925

More chrome processes:
Screenshot 2023-05-30 051919

Cancelling the command:
Screenshot 2023-05-30 051954

All chrome processes gone:
Screenshot 2023-05-30 052000

Operating System

Linux

@luevano luevano added the bug Something isn't working label May 30, 2023
@luevano
Copy link
Author

luevano commented Jun 1, 2023

I fixed this at luevano/mangal-lua-libs@97fba97, I'm not doing PR anymore as these repos look inactive. And also need to include the browser:close() on the corresponding part of the code for the lua scrapers, also found at luevano/mangal-scrapers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant