Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unclear if client.Close() actually sends the queued messages #166

Open
mikeparker opened this issue Aug 17, 2020 · 0 comments
Open

Unclear if client.Close() actually sends the queued messages #166

mikeparker opened this issue Aug 17, 2020 · 0 comments

Comments

@mikeparker
Copy link

mikeparker commented Aug 17, 2020

From docs on Batching: (https://segment.com/docs/connections/sources/catalog/libraries/server/go/#batching)

Batching
... If batch messages are not arriving in your debugger and no error is being thrown you may want to slow the speed of your scipt down. This is because we run a message batching loop in a go-routine so if the script runs too fast it won’t execute on the network calls before it exits the loop.
By default, our library will flush:
every 20 messages (control with FlushAt)
if 5 seconds has passed since the last flush (control with FlushAfter)
There is a maximum of 500KB per batch request and 32KB per call.
Sometimes you might not want batching (eg. when debugging, or in short-lived programs). You can turn off batching by setting the FlushAt argument to 1, and your requests will always be sent right away.

I am running in AWS Lambda which could be fairly short lived (instances are reused for multiple executions, which means it may be long lived) but if i defer client.Close() from the code it is supposed to flush the queue and send all messages, right?

The docs make it sound like i need to set my batch size to 1 for short lived programs or scripts but I don't understand why I need to do that if client.Close() sends all queued messages anyway?

This issue raised #157 also confuses me, if client.Close() sends all queued messages then there is no need to check first?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant