Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add channel to worker for done work #187

Open
sa7mon opened this issue Aug 20, 2023 · 0 comments
Open

Add channel to worker for done work #187

sa7mon opened this issue Aug 20, 2023 · 0 comments
Labels
refactor Code cleanup / optimization
Milestone

Comments

@sa7mon
Copy link
Owner

sa7mon commented Aug 20, 2023

Instead of the worker having to handle writing to the database or outputting logs to the terminal, the worker should only:

  • take in buckets from input channel
  • scan them
  • push them into output channel

then s3scanner.Run() should wait for buckets to be pushed into the output channel and handle writing to the database and writing logs.

I think this is what "clean code" is referring to.

We'll need to add tests to verify the output channel is closed at appropriate times and the program exits gracefully. Once we make this change, we should be able to combine the 2 worker classes into a unified worker.

@sa7mon sa7mon added the refactor Code cleanup / optimization label Aug 20, 2023
@sa7mon sa7mon added this to the Future milestone Aug 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
refactor Code cleanup / optimization
Projects
None yet
Development

When branches are created from issues, their pull requests are automatically linked.

1 participant