Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding gzip compression support in browser case #4493

Open
wants to merge 23 commits into
base: main
Choose a base branch
from

Conversation

scheler
Copy link
Contributor

@scheler scheler commented Feb 19, 2024

Which problem is this PR solving?

This introduces gzip compression support to the exporters in the browser case. The change use CompressionStream API available in the modern browsers and does not introduce any additional 3rd party dependency.

Note that this support exists for the node case already.

Short description of the changes

  • Updated sendWithXhr function to take additional parameter for compression. This parameter is not added at the end, so this could potentially qualify as a breaking change. however, I have also modified its only usage in otlp-proto-exporter-base so I'm not sure if this still qualifies to be a breaking change.
  • I've used Copilot to generate the code for compressContent, let me know if you find anything odd.

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

  • I tested compressContent function separately, and was able to uncompress the content using gunzip command line
  • I also tested examples/opentelemetry-web/xml-http-request enabling gzip compression in the OTLPTraceExporter

Checklist:

  • Followed the style guidelines of this project
  • Unit tests have been added
  • Documentation has been updated

@scheler scheler requested a review from a team February 19, 2024 23:53
Copy link

codecov bot commented Feb 20, 2024

Codecov Report

Attention: Patch coverage is 87.75510% with 6 lines in your changes are missing coverage. Please review.

Project coverage is 91.26%. Comparing base (2610122) to head (ca734e5).
Report is 35 commits behind head on main.

Current head ca734e5 differs from pull request most recent head ca10067

Please upload reports for the commit ca10067 to get more accurate results.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4493      +/-   ##
==========================================
+ Coverage   90.77%   91.26%   +0.48%     
==========================================
  Files          90      233     +143     
  Lines        1930     6352    +4422     
  Branches      417     1401     +984     
==========================================
+ Hits         1752     5797    +4045     
- Misses        178      555     +377     
Files Coverage Δ
...ter-base/src/platform/node/OTLPExporterNodeBase.ts 50.00% <ø> (ø)
...kages/otlp-exporter-base/src/platform/node/util.ts 63.80% <100.00%> (ø)
...erimental/packages/otlp-exporter-base/src/types.ts 44.44% <100.00%> (ø)
...es/otlp-exporter-base/src/platform/browser/util.ts 54.36% <86.66%> (ø)

... and 145 files with indirect coverage changes

CHANGELOG.md Outdated Show resolved Hide resolved
@martinkuba
Copy link
Contributor

I do have a question about browser support -

CompressionStream is supported on majority of browsers, but not all. On browsers where this is not supported, the export will fail. We could do a feature detection, and if CompressionStream is not available, then default to export without compression. IMO, this would be a better experience, specifically because getting data about browser usage is a key metric in browser monitoring. (This might be a topic for a broader discussion)

Alternatively, we should at last document that compression is not available on all browsers.

@scheler
Copy link
Contributor Author

scheler commented Mar 28, 2024

I do have a question about browser support -

CompressionStream is supported on majority of browsers, but not all. On browsers where this is not supported, the export will fail. We could do a feature detection, and if CompressionStream is not available, then default to export without compression. IMO, this would be a better experience, specifically because getting data about browser usage is a key metric in browser monitoring. (This might be a topic for a broader discussion)

Alternatively, we should at last document that compression is not available on all browsers.

@martinkuba I now check if CompressionStream is supported by the browser and proceed with sending without compression if it is not supported.

@scheler
Copy link
Contributor Author

scheler commented Apr 13, 2024

@pichlermarc all aspects of this PR are resolved, can you take a look?
@martinkuba @MSNev can you approve this PR when you get a chance?

@pichlermarc pichlermarc dismissed their stale review April 17, 2024 14:15

comments are addressed, need to do another pass on the PR

@pichlermarc pichlermarc self-requested a review April 17, 2024 14:16
Copy link

linux-foundation-easycla bot commented May 20, 2024

CLA Signed

The committers listed above are authorized under a signed CLA.

@scheler
Copy link
Contributor Author

scheler commented May 21, 2024

@pichlermarc can you review this PR when you get a chance? There is a commit 8a2e225 from you that needs your attention.

@pichlermarc
Copy link
Member

/easycla

Copy link
Member

@pichlermarc pichlermarc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO looks good, I'll let @martinkuba and @MSNev have the last word on this PR as they're more knowledgeable about this than I am.

Sorry about that CLA issue - no idea what that was about. Must've been a EasyCLA hiccup.

@@ -0,0 +1,78 @@
/*
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs a rebase as this package does not exist anymore (http and json now use a shared base).

content: string | Blob,
compressionAlgorithm: string
): Promise<Uint8Array> {
const compressionStream = new CompressionStream(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As CompressionStream was added in 2023, it (may) not exist in all environments and this would likely not be handled nicely.

We should have a check (somewhere) which not only check that the CompressionStream exists, but anything else that may not exist in the current runtime so that it can gracefully fallback to not use compression when it's not available rather than just assume the application has configured it correctly and all of their uses are using their supported runtime.

https://caniuse.com/?search=CompressionStream

Copy link
Contributor Author

@scheler scheler Jul 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MSNev CompressionStream api became baseline stable in 2023.
From https://developer.mozilla.org/en-US/docs/Web/API/CompressionStream -

"Since May 2023, this feature works across the latest devices and browser versions. This feature might not work in older devices or browsers."

and at https://caniuse.com/?search=CompressionStream if you go to the tab "Usage relative", you can see that the browsers that do not support this API is in minority.

Can you tell me more about your suggestion to check for more?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but as the current browser support is undefined (we have not said only from this version), we need to deal with cases of code running on older browsers. like older end-user devices that may not be using newer versions (ipads, android devices, iphones, etc) otherwise if we don't we could cause the application which is using OTel to fail.

Additionally, there are browser "like" environments (they pretend to be a browser and look and act like one) that don't always include all of the API's

This is part of what I'm trying to get consensus on with this issue so that for OTel v2.0 we can just say -- hey the supported browsers are blah, and if your using some older / other environment that doesn't support at least this level then we don't support that. And as you can see from the discussions while I was trying to push for ES2020 some want to go back to ES2018 (which means that the Compression API would not exist in either of these).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MSNev the Compression Streams API isn't tied to a specific ECMAScript version but rather individual browser versions. Let me know if I am missing something.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR avoids using CompressionStream API if it's not supported, even if it is configured on the exporter. It falls back gracefully to send without any compression. The only time it would fail is when a browser like environment claims support but implements it incorrectly. I can try and see that case is handled well. However, beyond that if there are any possibility of failure let me know.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR avoids using CompressionStream API if it's not supported

Can you identify where this is as I missed seeing the check, which is all I was talking about with my initial comment on this thread.

the Compression Streams API isn't tied to a specific ECMAScript version but rather individual browser versions. Let me know if I am missing something.

Understand that, but it is indirectly linked (in time) based on what "runtimes" (and therefore versions for browsers) in that if some runtime is only targeting ESXXX support then it may not necessarily support the current environments of browsers.... eg. something like Electron is not a browser, but it provides a browser like runtime so you can write the code (mostly) once and host it on the web or locally, but the Electron container won't support everything the browser does... (Note: I'm not an electron developer so I don't specifically know if it supports the Compression API or what it does or does not support -- so some of this is hand-wavy and generic comments about detecting support at runtime.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you identify where this is as I missed seeing the check

Oh! Yes, I had to put the check outside of compressContent function so that I can pass the headers appropriately. Please see: https://github.com/open-telemetry/opentelemetry-js/pull/4493/files#diff-2aacfc9dbdc5a63485aad503e0de3cd057da36801b4d109232cf233fcceddcadR165

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants