[DOCS-7980] Remove buffer in OP docs #23270
Conversation
| @@ -51,7 +51,7 @@ Observability Pipelines Worker runs on modern CPU architectures. X86_64 architec | |||
|
|
|||
| ### Memory sizing | |||
|
|
|||
| Due to Observability Pipelines Worker's affine type system, memory is rarely constrained for Observability Pipelines Worker workloads. Therefore, Datadog recommends ≥2 GiB of memory per vCPU minimum. Memory usage increases with the number of sinks due to the in-memory buffering and batching. If you have a lot of sinks, consider increasing the memory or switching to disk buffers. | |||
| Due to Observability Pipelines Worker's affine type system, memory is rarely constrained for Observability Pipelines Worker workloads. Therefore, Datadog recommends ≥2 GiB of memory per vCPU minimum. Memory usage increases with the number of sinks due to the in-memory buffering and batching. If you have a lot of sinks, consider increasing the memory. | |||
|
|
|||
| ### Disk sizing | |||
There was a problem hiding this comment.
@ckelner do we want to remove this "Disk Sizing" section as well?
There was a problem hiding this comment.
Per https://vector.dev/docs/setup/going-to-prod/sizing/#disks it states:
Sizing disks is only relevant if you’re using Vector’s disk buffers.
But I feel like we should have some kind of guidance here, even w/o disk buffers.
Let me ask the OP backroom, I have a thread on this topic.
There was a problem hiding this comment.
@maycmlee I think for "Disk Sizing" we just say "30GB minimum" and leave it at that.
There was a problem hiding this comment.
@ckelner could you let me know if the update below is what you are suggesting? (the commit) I'm assuming the example caclulation is still correct?
For example, an 8 vCPU machine would require 288 GiB of disk space (10 MiB * 60 seconds * 60 minutes * 8 vCPUs).
There was a problem hiding this comment.
Chatted on zoom. Remove the entire section and just add the minimum required sentence.
Preview links (active after the
|
There was a problem hiding this comment.
Just commenting for now, lets wait on OP Eng to comment on this: https://github.com/DataDog/documentation/pull/23270/files#r1607013668
| @@ -89,29 +88,6 @@ Data processing is shifted off your nodes and onto remote aggregator nodes. Remo | |||
|
|
|||
| See [Aggregator Architecture][5] for more details. | |||
|
|
|||
| ## Buffering data | |||
There was a problem hiding this comment.
I think removing this for now is good, but we should write up something about the "built in default" buffering (and remove language about being able to configure it).
We can probably pull something together from here: https://vector.dev/docs/about/under-the-hood/architecture/buffering-model/ then likely have eng review it.
I can draft something in a google doc if you like
There was a problem hiding this comment.
Sure, if you can draft something that'd be great. Thanks!
There was a problem hiding this comment.
EDIT: Disregard - old docs
Check it out here May: https://docs.google.com/document/d/1UKspJYeXbvY5tFdlJUJgjvJ2ngLfg36MAkKVz6YCQvo/edit
Want to post in the #obs-pipelines-backroom channel for a look?~
There was a problem hiding this comment.
Oh jeez! These are the "legacy" 1.0 docs. We can disregard. Leave this as-is.
There was a problem hiding this comment.
Sorry that was my fault, I thought I had stuck to the current docs. I reverted the change.
| @@ -89,29 +88,6 @@ Data processing is shifted off your nodes and onto remote aggregator nodes. Remo | |||
|
|
|||
| See [Aggregator Architecture][5] for more details. | |||
|
|
|||
| ## Buffering data | |||
There was a problem hiding this comment.
Oh jeez! These are the "legacy" 1.0 docs. We can disregard. Leave this as-is.
| @@ -51,7 +51,7 @@ Observability Pipelines Worker runs on modern CPU architectures. X86_64 architec | |||
|
|
|||
| ### Memory sizing | |||
|
|
|||
| Due to Observability Pipelines Worker's affine type system, memory is rarely constrained for Observability Pipelines Worker workloads. Therefore, Datadog recommends ≥2 GiB of memory per vCPU minimum. Memory usage increases with the number of sinks due to the in-memory buffering and batching. If you have a lot of sinks, consider increasing the memory or switching to disk buffers. | |||
| Due to Observability Pipelines Worker's affine type system, memory is rarely constrained for Observability Pipelines Worker workloads. Therefore, Datadog recommends ≥2 GiB of memory per vCPU minimum. Memory usage increases with the number of sinks due to the in-memory buffering and batching. If you have a lot of sinks, consider increasing the memory. | |||
|
|
|||
| ### Disk sizing | |||
There was a problem hiding this comment.
@maycmlee I think for "Disk Sizing" we just say "30GB minimum" and leave it at that.
content/en/observability_pipelines/best_practices_for_scaling_observability_pipelines.md
Outdated
Show resolved
Hide resolved
…observability_pipelines.md Co-authored-by: Chris Kelner <[email protected]>
What does this PR do? What is the motivation?
There isn't a buffer option for OP 2.0 so removing it from the docs.
DOCS-7980
Merge instructions
Additional notes