Less memory allocations

[MiloszKrajewski]

Hi,

I just ran some log running test with lot of log messages and memory analyser and it reported three potential improvements in LokiBatchFormatter.

TL;DR: I would be happy to look at those cases and submit PR, I have some experience in optimizing towards allocation-free code, just asking before-hand if you would accept such PR?

• StringBuilder.ToString() (this one can be the easiest one)

Small Object Heap: code that allocates a lot of memory in SOH
Allocated object type: String
Last observation: 2023-06-12 11:19 Demo.Grafana.Api.exe
  Allocated size: 101.9 MB

at StringBuilder.ToString()
at LokiBatchFormatter.GenerateEntry(LokiLogEvent, ITextFormatter, LokiStream) in /_/src/Serilog.Sinks.Grafana.Loki/LokiBatchFormatter.cs:line 182 column 9
at LokiBatchFormatter.Format(IReadOnlyCollection, ITextFormatter, TextWriter) in /_/src/Serilog.Sinks.Grafana.Loki/LokiBatchFormatter.cs:line 137 column 17
at LokiSink.OnTick() in /_/src/Serilog.Sinks.Grafana.Loki/LokiSink.cs:line 111 column 29

• MemoryStream.Write(...) (maybe not stream? just linked list of pooled arrays?)

Small Object Heap: code that allocates a lot of memory in SOH
Allocated object type: Byte[]
Last observation: 2023-06-12 11:19 Demo.Grafana.Api.exe
  Allocated size: 162.9 MB

at MemoryStream.set_Capacity(int)
at MemoryStream.EnsureCapacity(int)
at MemoryStream.Write(ReadOnlySpan)
at StreamWriter.Flush(bool, bool)
at StreamWriter.Write(String)
at LokiBatchFormatter.Format(IReadOnlyCollection, ITextFormatter, TextWriter) in /_/src/Serilog.Sinks.Grafana.Loki/LokiBatchFormatter.cs:line 146 column 13
at LokiSink.OnTick() in /_/src/Serilog.Sinks.Grafana.Loki/LokiSink.cs:line 111 column 29

• JsonSerializer.WriteString (might be complex with current design, as it builds string AND then writes, to stream, maybe it could write to stream directly?)

Small Object Heap: code that allocates a lot of memory in SOH
Allocated object type: String
Last observation: 2023-06-12 11:19 Demo.Grafana.Api.exe
  Allocated size: 140.5 MB

at JsonReaderHelper.TranscodeHelper(ReadOnlySpan)
at JsonSerializer.WriteString(ref , JsonTypeInfo)
at LokiBatchFormatter.Format(IReadOnlyCollection, ITextFormatter, TextWriter) in /_/src/Serilog.Sinks.Grafana.Loki/LokiBatchFormatter.cs:line 146 column 13
at LokiSink.OnTick() in /_/src/Serilog.Sinks.Grafana.Loki/LokiSink.cs:line 111 column 29

Anyway, I would be happy to look at those cases and submit PR, just asking before hand if you would accept such PR?

Cheers,
Milosz Krajewski

[mishamyte]

Hi @MiloszKrajewski,

Thank you for that proposals.
I think it would be very valuable to have it, so I would be glad to accept it.

Also important to keep in mind I have a plan to rebuild v9 version from scratch, using F#, so it could be different from existing major version.

But from my PoV is would be nice to eliminate a places with high alloc, because even in v9 will be done in other way, some things could be kept in mind.

So I would be happy to have such kind of PR with optimizations