-
Notifications
You must be signed in to change notification settings - Fork 487
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[release-v2.5] [DOC] Draft release notes for 2.5 (#3735)
* [DOC] Draft release notes for 2.5 (#3733) * Draft release notes for 2.5 * Apply suggestions from code review * Apply suggestions from code review * Update docs/sources/tempo/release-notes/v2-5.md * Apply suggestions from code review (cherry picked from commit e35faa6) * Apply suggestions from code review --------- Co-authored-by: Kim Nylander <[email protected]>
- Loading branch information
1 parent
e3e6c47
commit 61d0cc6
Showing
2 changed files
with
255 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,182 @@ | ||
| --- | ||
| title: Version 2.5 release notes | ||
| menuTitle: V2.5 | ||
| description: Release notes for Grafana Tempo 2.5 | ||
| weight: 35 | ||
| --- | ||
|
|
||
| # Version 2.5 release notes | ||
|
|
||
| The Tempo team is pleased to announce the release of Tempo 2.5. | ||
|
|
||
| This release gives you: | ||
|
|
||
| * New features, including more TraceQL metrics, gRPC streaming endpoints, and new experimental vParquet4 block format | ||
| * TraceQL enhancements and performance improvements | ||
| * Performance and stability enhancements | ||
|
|
||
| As part of this release, we’ve updated ownership for `/var/tempo` from `root:root` to a new `tempo:tempo` user with a UUID of `10001`. Learn about this breaking change in Upgrade considerations. | ||
|
|
||
| <!--Read the [Tempo 2.5 blog post](https://grafana.com/blog/2024/06/03/grafana-tempo-2.5-release-vparquet4-streaming-endpoints-and-more-metrics/) for more examples and details about these improvements.--> | ||
|
|
||
| These release notes highlight the most important features and bugfixes. For a complete list, refer to the [Tempo changelog](https://github.com/grafana/tempo/releases). | ||
|
|
||
| <!-- {{< youtube id="c4gW9fwkLhc" >}} --> | ||
|
|
||
| ## Features and enhancements | ||
|
|
||
| The most important features and enhancements in Tempo 2.5 are highlighted below. | ||
| ### Additional TraceQL metrics (experimental) | ||
|
|
||
| In this release, we’ve added several [TraceQL metrics](https://grafana.com/docs/tempo/latest/operations/traceql-metrics/). Tempo 2.4 introduced the `rate()` function to view rates of spans. For this release, we’ve added `quantile_over_time` and `histogram_over_time`. [PR 3605](https://github.com/grafana/tempo/pull/3605), [PR 3633](https://github.com/grafana/tempo/pull/3633), [PR 3644](https://github.com/grafana/tempo/pull/3644)] | ||
|
|
||
| You can use `quantiles_over_time` allowing users to aggregate numerical values, such as the all-important span duration. Notice that you can specify multiple quantiles in the same query. | ||
|
|
||
|  | ||
|
|
||
| You can use `historgram_over_time` to display the metrics as a heatmap. | ||
|
|
||
|  | ||
|
|
||
| Note that using TraceQL metrics may require additional system resources. | ||
|
|
||
| For more information, refer to the [TraceQL metrics queries](https://grafana.com/docs/tempo/latest/traceql/metrics-queries) and [Configure TraceQL metrics](https://grafana.com/docs/tempo/latest/operations/traceql-metrics/). | ||
|
|
||
| ### gRPC streaming endpoints for Tempo APIs | ||
|
|
||
| With this feature, you can now see partial query results as they come in, so you no longer have to wait for the whole query to finish. | ||
| This is perfect for big queries that take a long time to return a response. | ||
|
|
||
| The Tempo API endpoints now support gRPC streaming for tag queries and metrics. | ||
| We’ve added new streaming endpoints for: | ||
|
|
||
| * `SearchTags` | ||
| * `SearchTagsV2` | ||
| * `SearchTagValues` | ||
| * `SearchTagValuesV2` | ||
| * `MetricsQueryRange` | ||
|
|
||
| These new streaming endpoints are not supported by Grafana yet. However, they are supported by the Tempo CLI tool. | ||
|
|
||
| In the Tempo CLI, you can use the `--use-grpc` option to enable GRPC streaming. | ||
|
|
||
| To learn more, refer to the [Tempo gRPC API](https://grafana.com/docs/tempo/latest/api_docs/#tempo-grpc-api) and [Tempo CLI](https://grafana.com/docs/tempo/latest/operations/tempo_cli/#search) documentation. [PR 3460](https://github.com/grafana/tempo/pull/3460) [[PR #3584](https://github.com/grafana/tempo/pull/3584)] | ||
|
|
||
| {{< admonition type="note" >}} | ||
| NOTE: Streaming over HTTP requires the `stream_over_http_enabled` flag to be set. For more information, refer to [Tempo GRPC API documentation](https://grafana.com/docs/tempo/latest/api_docs/#tempo-grpc-api). | ||
| {{< /admonition >}} | ||
|
|
||
| In addition, we’ve reduced memory consumption in the frontend for large traces. [[PR 3522](https://github.com/grafana/tempo/pull/3522)] | ||
|
|
||
| ### New vParquet4 block format (experimental) | ||
|
|
||
| New in Tempo 2.5, the vParquet4 block format is required for querying links, events, and arrays and improves query performance relative to previous formats. | ||
| [[PR 3368](https://github.com/grafana/tempo/pull/3368)] | ||
|
|
||
| In addition, we’ve updated the OTLP schema to add attributes to instrumentation scope in vParquet4.[[PR 3649](https://github.com/grafana/tempo/pull/3649)] | ||
|
|
||
| While you can use vParquet4, keep in mind that it's experimental. If you choose to use vParquet4 and then opt to revert to vParquet3, any vParquet4 blocks would not be readable by vParquet3. | ||
|
|
||
| To try vParquet4, refer to [Choose a block format](https://grafana.com/docs/tempo/latest/configuration/parquet/#choose-a-different-block-format). | ||
|
|
||
| ### TraceQL | ||
|
|
||
| Unique to Tempo, TraceQL is a query language that lets you perform custom queries into your tracing data. | ||
| To learn more about the TraceQL syntax, see the [TraceQL documentation](https://grafana.com/docs/tempo/latest/traceql/). | ||
|
|
||
| For information on planned future extensions to the TraceQL language, refer to [future work](https://github.com/grafana/tempo/blob/main/docs/design-proposals/2023-11%20TraceQL%20Extensions.md). | ||
|
|
||
| We’ve made the following improvements to TraceQL: | ||
|
|
||
| * Add support for scoped intrinsics using a colon (`:`). The available scoped intrinsics are trace:duration, trace:rootName, trace:rootService, span:duration, span:kind, span:name, span:status, span:statusMessage. [[PR [3629](https://github.com/grafana/tempo/pull/3629)] | ||
| * Performance improvements on TraceQL and tag value search. [[PR 3650](https://github.com/grafana/tempo/pull/3650),[PR 3667](https://github.com/grafana/tempo/pull/3667)] | ||
| * Add `nestedSetLeft/Right/Parent` intrinsics to TraceQL. [[PR 3497](https://github.com/grafana/tempo/pull/3497)] | ||
| * Reduce memory usage and improve performance when working with large datasets with many repeated strings. [[PR 3411](https://github.com/grafana/tempo/pull/3411)] | ||
| * Add support querying by `trace:id` and `span:id` [[PR 3670](https://github.com/grafana/tempo/pull/3670)] | ||
|
|
||
| ### Other enhancements and improvements | ||
|
|
||
| * Add TLS support for Memcached Client [[PR 3585](https://github.com/grafana/tempo/pull/3585)] | ||
| * Remove hardcoded delay in distributor shutdown [[PR 3687](https://github.com/grafana/tempo/pull/3687)] | ||
| * Surface new labels for uninstrumented services and systems [[PR 3543](https://github.com/grafana/tempo/pull/3543)] | ||
| * Add messaging-system latency histogram to service-graph [[PR 3453](https://github.com/grafana/tempo/pull/3453)] | ||
| * Add support for sharded ingester queries [[PR 3574](https://github.com/grafana/tempo/pull/3574)] | ||
| * Better compaction throughput and memory usage [[PR 3579](https://github.com/grafana/tempo/pull/3579)] | ||
| * Return a less confusing error message to the client when refusing spans due to ingestion rates. [[PR 3485](https://github.com/grafana/tempo/pull/3485)] | ||
| * Clean Metrics Generator's Prometheus WAL before creating instance [[PR 3548](https://github.com/grafana/tempo/pull/3548)] | ||
| * Delete any remaining objects for empty tenants after a configurable duration, requires config enable [PR 3611](https://github.com/grafana/tempo/pull/3611)] | ||
|
|
||
| ## Upgrade considerations | ||
|
|
||
| When [upgrading](https://grafana.com/docs/tempo/latest/setup/upgrade/) to Tempo 2.5, be aware of these considerations and breaking changes. | ||
|
|
||
| ### Docker image runs as new UID 10001 | ||
|
|
||
| The Tempo process in the [official Docker image](https://hub.docker.com/r/grafana/tempo/tags) used to run as `root`. The Tempo process now runs as UID `10001` in the Docker image. | ||
|
|
||
| Components such as ingesters and metrics generators that maintain files on disk won't come up cleanly without intervention. | ||
| The new user `10001` won't have access to the old files created by `root`. | ||
|
|
||
| The ownership of `/var/tempo` changed from `root:root` to `tempo:tempo` with the UID/GID of `10001`. | ||
|
|
||
| The `ingester` and `metrics-generator` statefulsets may need to [run chown](https://opensource.com/article/19/8/linux-chown-command) to change ownership to start properly. | ||
|
|
||
| Refer to [PR 2265](https://github.com/grafana/tempo/pull/2265) to see a Jsonnet example of an `init` container. | ||
|
|
||
| This change doesn’t impact you if you used the Helm chart with the default security context set in the chart. | ||
| All data should be owned by the `tempo` user already. | ||
| The UID won’t impact Helm chart users. | ||
|
|
||
| ### Support for vParquet format removed | ||
|
|
||
| The original vParquet format [has been removed](https://github.com/grafana/tempo/pull/3663) from Tempo 2.5. | ||
| Direct upgrades from Tempo 2.1 to Tempo 2.5 are not possible. | ||
| You will need to upgrade to an intermediate version and wait for the old vParquet blocks to fall out of retention before upgrading to 2.5. [PR 3663](https://github.com/grafana/tempo/pull/3663)] | ||
|
|
||
| vParquet(1) won't be recognized as a valid encoding and any remaining vParquet(1) blocks won't be readable. | ||
| Installations running with historical defaults should not require any changes as the default has been migrated for several releases. | ||
| Installations with storage settings pinned to vParquet must run a previous release configured for vParquet2 or higher until all existing vParquet(1) blocks have expired and been deleted from the backend, or else will encounter read errors after upgrading to this release. | ||
|
|
||
| ### **Updated, removed, or renamed configuration parameters** | ||
|
|
||
| <table> | ||
| <tr> | ||
| <td><strong>Parameter</strong> | ||
| </td> | ||
| <td><strong>Comments</strong> | ||
| </td> | ||
| </tr> | ||
| <tr> | ||
| <td>`[hedge_requests_at: <duration> | default = 2s ]` | ||
| <p> | ||
| `[hedge_requests_up_to: <int> | default = 2 ]` | ||
| </td> | ||
| <td>Removed options from the configuration. [PR <a href="https://github.com/grafana/tempo/pull/3522">#3522</a>] | ||
| </td> | ||
| </tr> | ||
| </table> | ||
|
|
||
| ### Additional considerations | ||
|
|
||
| * Updating to OTLP 1.3.0 removes the deprecated `InstrumentationLibrary` and `InstrumentationLibrarySpan` from the OTLP receivers. [PR 3649](https://github.com/grafana/tempo/pull/3649)] | ||
| * Removes the addition of a tenant in multitenant trace id lookup. [PR 3522](https://github.com/grafana/tempo/pull/3522)] | ||
|
|
||
| ## Bugfixes | ||
|
|
||
| For a complete list, refer to the [Tempo changelog](https://github.com/grafana/tempo/releases). | ||
|
|
||
| * Fix handling of regex matchers in autocomplete endpoints [PR 3641](https://github.com/grafana/tempo/pull/3641) | ||
| * Return unfiltered results when a bad TraceQL query is provided in autocomplete. [PR 3426](https://github.com/grafana/tempo/pull/3426) | ||
| * Add support for dashes, quotes and spaces in attribute names in autocomplete [PR 3458](https://github.com/grafana/tempo/pull/3458) | ||
| * Fix metrics query results when filtering and rating on the same attribute [PR 3428](https://github.com/grafana/tempo/issues/3428) | ||
| * Fix metrics query results when series contain empty strings or nil values [PR 3429](https://github.com/grafana/tempo/issues/3429) | ||
| * Fix metrics query duration check, add per-tenant override for max metrics query duration [PR 3479](https://github.com/grafana/tempo/issues/3479) | ||
| * Fix metrics query panic "index out of range [-1]" when a trace has zero-length ID [PR 3668](https://github.com/grafana/tempo/pull/3668) | ||
| * Correctly handle 429s in GRPC search streaming. [PR 3469](https://github.com/grafana/tempo/pull/3469) | ||
| * Correctly cancel GRPC and HTTP contexts in the frontend to prevent having to rely on http write timeout. [PR 3443](https://github.com/grafana/tempo/pull/3443) | ||
| * Add spss and limit to the frontend cache key to prevent the return of incorrect results. [PR 3557](https://github.com/grafana/tempo/pull/3557) | ||
| * Use OS path separator to split blocks path. [PR 3552](https://github.com/grafana/tempo/issues/3552) | ||
| * Correctly parse TraceQL queries with > 1024 character attribute names or static values. [PR 3571](https://github.com/grafana/tempo/issues/3571) | ||
| * Fix span-metrics' subprocessors bug that applied wrong configurations when running multiple tenants. [PR 3612](https://github.com/grafana/tempo/pull/3612) | ||
| * Fix panic in query-frontend when combining results [PR 3683](https://github.com/grafana/tempo/pull/3683) | ||
| * Fix TraceQL queries involving non boolean operations between statics and attributes. [PR 3698](https://github.com/grafana/tempo/pull/3698) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters