Update network function to use a single dispatcher and buffered channel #97
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1. Change the main Listen routine to use a single go routine with a buffered channel so that a blocked api.Writer won't build up an unlimited number of go routines eventually exhausting the stack and OOMing the process 2. Added an optional log.Logger member to the server so that the library isn't just spitting error messages at stdout. If the logger is not specified the old behavior is maintained.
kris-watts-gravwell
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR does two things:
Some additional context:
RE point 1. We have found that when using the library and the current method of kicking off a go routine for every message is susceptible to stack and memory exhaustion if the api.Writer blocks for any reason. I believe that a buffered channel is a better methodology as it will eventually just block and stop consuming stack and memory space if the api.Writer cannot service the messages.
RE point 2. Errors in parsing are currently just tossed to stdout with no way for a caller to capture or route them. This just provides the option for a caller to hand in a log.Logger and route/capture those errors.