Handle campaigns with a large number of emails #64
Conversation
|
Will be verifying these changes later today but looks promising! |
killion
changed the title
Running update on a large campaign will time out. There is a separate update option already. Linting also changed the file a lot.
|
Thanks, I added a few things to handle large campaigns besides chunking so I renamed the PR. When I set the RATE_LIMIT to something higher I would often run into a race condition that resulted in the same email being sent more than once. So now there is a lock in Redis. |
I worry that the 15 minutes isn't long enough if a lot of emails are going at once.
|
Hey, is this PR ready for review? |
I think so, I left some logging in there because it is only a few lines and I found it to be generally helpful. |
|
Looks great, could you set the Once that is done, it is ready to be merged in! |
|
just seen that this PR is stale, but it seems quite useful. |
I ran into the Prisma limit with more than 32,767 contacts in a campaign. I saw this chunk method had been used in other parts of this controller so I reused it.
const chunkSize = 500;gets set 3 times now. Maybe it can be an env var?I'm also thinking the 80 sends per batch could be an env var too. I'll try to make that a PR.