Sending mail with large attachement consumes lots of memory

[jurajw]

Observed on 3.5.0.

Sending a mail with a single 3_000_000 bytes attachment requires ~ 27_000_000 bytes of heap. Based on my observation following is happening:

1. An AttachmentPart instance is created for the attachment. In the constructor the buffer is base64 encoded into a ~ 4_000_000 characters long string. On JDK8 this is ~ 8_000_000 bytes in the heap.

2. A MultiPart instance is created for the AttachmentPart.
   a. The AttachmentPart header is concatenated with the base64 data.
   b. The copy is appended into a StringBuilder.
   c. StringBuilder#toString is called and the result is set to MultiPart#part.

3. MultiPart#headers is set.

4. MultiPart#toString is called.

In 2a, 2b, 2c and 4 the base64 data is copied.

[jurajw]

I have been prototyping changes such that:

1. base64 encoding is performed lazily line-by-line (by exposing an Iterator).
2. No copies are created (iterators are concatenated).

Is there interest in a patch?

[pmlopes]

@jurajw I think it would be interesting to have such encoder not just for mail but for json in Vertx too. Since we support a few extra RFC for the json implementation which cover temporal types and base64 I can imagine that such encoder would reduce the memory usage there too.

Perhaps you could provide a pull request and we could continue the conversation there?

[vietj]

there has been contributions recently in vertx-web-client for this, perhaps some of its classes should be moved to vertx-core ?

[jurajw]

@vietj Can you point me to the relevant vertx-web-client changes?

[vietj]

see https://github.com/vert-x3/vertx-web/commits/master/vertx-web-client <https://github.com/vert-x3/vertx-web/commits/master/vertx-web-client> the 10 last commits

…

On 3 May 2018, at 17:37, Juraj Wagner ***@***.***> wrote: @vietj <https://github.com/vietj> Can you point me to the relevant vertx-web-client changes? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#93 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AANxiiydL-D7WJJvYfpPi0WSGPJ8ioZ9ks5tuyQfgaJpZM4S_goD>.