v2.0.0

Summary

** NEW MAJOR VERSION **
!! BREAKING CHANGES !!
Configuration file formats have changed and some functionality has changed.
Use the new converter tool to convert existing configuration and rules files. Binaries are available as part of the release.
For more information, see the release notes.

Features

• feat: Rewrite config code without Viper (#654) | Kent Quirk
• feat: Add Warn() to Logger interface (#656) | Kent Quirk
• feat: Config conversion and validation code from one data file (#677) | Kent Quirk
• feat: Refactoring of rules configuration and rules converter (#681) | Kent Quirk
• feat: Remove trace key params -> add meta sample_key (#685) | Kent Quirk
• feat: Use the new configuration system (#690) | Kent Quirk
• feat: Metrics cleanup (#692) | Kent Quirk
• feat: Validation integration, part 1 (#700) | Kent Quirk
• feat: More validation add rules metadata and rules validation (#701) | Kent Quirk
• feat: Integrate validation into the executable (#706) | Kent Quirk
• feat: Write out parsed configs (#707) | Kent Quirk
• feat: Add EMAThroughput sampler support (#708) | Kent Quirk
• feat: Add WindowedThroughput sampler (#709) | Kent Quirk
• feat: Support MaxKeys in configs and add default of 500 (#710) | Kent Quirk
• feat: Add sampler metrics (#714) | Kent Quirk
• feat: Preregister libhoney metrics (#716) | Kent Quirk
• feat: Warn about samplers that might need adjustment (#718) | Kent Quirk
• feat: Allow suffixes on memory size in config (#719) | Kent Quirk
• feat: MaxAlloc improvements (#721) | Kent Quirk
• feat: Allow disabling reload monitoring (#730) | Tyler Helmuth
• feat: Add ability to convert a helm chart (#736) | Kent Quirk
• feat: Enable pyroscope deltaprof (#747) | Liz Fong-Jones

Fixes

• fix: Check apikeys for otlp requests too (+tests) (#672) | Kent Quirk
• fix: Various config fixes (#684) | Kent Quirk
• fix: Send a small amount of data to peers during stress relief (#688) | Kent Quirk
• fix: Add stress relief reason; fix JSON unmarshal bug (#698) | Kent Quirk
• fix: Log reason for MinimumStartupDuration (#703) | Terra Field
• fix: Rework APIKeys logic (#712) | Kent Quirk
• fix: Update dynamic samplers to count spans instead of traces (#717) | Kent Quirk
• fix: Try a few slots when there's a buffer overrun. (#722) | Kent Quirk
• fix: Make refinery run from minimal config (#724) | Kent Quirk
• fix: Correct dependency injection instability (#741) | Kent Quirk
• fix: Query auth bug fix; add tests (#753) | Kent Quirk
• fix: Improve drop cache performance (#757) | Kent Quirk

Maintenance

• perf: preallocate outbound libhoney attribute map (#754) | Liz Fong-Jones
• docs: Generated Refinery docs for docs site improvements (#752) | Mary J
• docs: Update config_complete.yaml (#751) | Kent Quirk
• docs: Regenerate docs from recent changes (#750) | Kent Quirk
• docs: Update APIKeys reference in config (#748) | Mary J
• maint: Update dynsampler-go to latest to fix bug (#746) | Kent Quirk
• docs: Fix up docs, especially envvar and cmdline (#737) | Kent Quirk
• docs: Fix convert help and docs (#744) | Kent Quirk
• maint: README updates -- round 1 (#742) | Phillip Carter
• maint(deps): Bump github.com/klauspost/compress from 1.16.4 to 1.16.5 (#675) | dependabot[bot]
• maint(deps): Bump github.com/prometheus/client_golang from 1.14.0 to 1.15.1 (#676) | dependabot[bot]
• refactor: Rename fields for clarity in an E&S world (#680) | Kent Quirk
• maint: Update dependencies (#699) | Kent Quirk
• docs: Improve generated documentation (#711) | Kent Quirk
• maint: Generate docs better suited to docs team prefs (#713) | Kent Quirk
• maint: Remove remaining references to obsolete fields (#720) | Kent Quirk
• docs: Refinery Rules copyediting (#731) | Mary J
• ci: Update build_binaries to build convert (#732) | Tyler Helmuth
• docs: Add generator for website docs (#733) | Kent Quirk
• docs: Refinery Config copyediting (#734) | Mary J
• chore: Switch to temp credentials for CI (#735) | Nathan Lincoln
• docs: Fix up docs, especially envvar and cmdline (#737) | Kent Quirk
• maint: Clarify 1.x configs for 1.x folks, remove old stuff (#739) | Phillip Carter

Additional contributions by:

• Documentation fix | Kevan Carstensen
• WindowedThroughput sampler (in dynsamplers-go) | Yi Zhao

v1.21.0

Summary

Adds many fixes for existing features such as meta fields for use with stress relief mode. Adds ability to annotate sample rates that
were already set upstream before refinery sampling for debugging purposes.

Enhancements

• feat: annotate incoming sample rate (#658) | Faith Chikwekwe

Bug Fixes

• fix: Replace incorrectly used Systemd Alias directive with a WantedBy (#657) | Irving Popovetsky
• fix: add hostname to span during stress relief mode (#666) | Faith Chikwekwe
• fix: only decorate late spans when configured to do so (#665) | Faith Chikwekwe
• fix: validate cache overrun strategy for stress relief mode (#664) | Faith Chikwekwe
• docs: update doc to remove deprecated field name (#659) | Faith Chikwekwe
• fix: Remove Stop() function from stress relief (#645) | Kent Quirk
• chore: Spelling (#644) | Josh Soref
• fix: tweak timeouts (#647) | Faith Chikwekwe
• fix: correct syntax error in config_complete.toml (#639) | Hazel Weakly

Maintenance

• chore: Update MetricsReportingInterval in config_complete.toml (#653) | Davin
• maint: switch dependabot to collection (#660) | [Vera Reynolds](https://github.com/Vera Reynolds)
• maint(deps): bump google.golang.org/protobuf from 1.28.1 to 1.30.0 (#663) | dependabot[bot]
• maint(deps): bump github.com/honeycombio/husky from 0.21.0 to 0.22.2 (#662) | dependabot[bot]
• maint(deps): bump github.com/klauspost/compress from 1.16.3 to 1.16.4 (#661) | dependabot[bot]
• maint(deps): bump go.uber.org/automaxprocs from 1.5.1 to 1.5.2 (#650) | dependabot[bot]
• maint(deps): bump github.com/honeycombio/dynsampler-go from 0.3.0 to 0.4.0 (#649) | dependabot[bot]
• maint(deps): bump google.golang.org/grpc from 1.52.3 to 1.54.0 (#652) | dependabot[bot]
• maint(deps): bump github.com/honeycombio/husky from 0.21.0 to 0.22.2 (#651) | dependabot[bot]
• maint(deps): bump github.com/klauspost/compress from 1.16.0 to 1.16.3 (#648) | dependabot[bot]
• maint: Add labels to docker image (#640) | Tyler Helmuth
• maint: Add LICENSES dir (#638) | Tyler Helmuth

v1.20.0

Summary

This is a significant new release of Refinery, with several features designed to help when operating Refinery at scale:

For details on all of the new features, please see the new Release Notes document.
New features must be enabled by adjusting configuration.

Enhancements

• feat: Add configuration for trace and parent ID field names (#630) | Davin Taddeo
• feat: allow ability to add new attributes to refinery data (#621) | Faith Chikwekwe
• feat: Add ability to set Redis database and prefix in config (#614) | Kent Quirk
• perf: Improve performance of stress relief (#604) | Kent Quirk
• feat: Stress Relief system (#594) | Kent Quirk
• feat: extend and unify metrics system (#593) | Kent Quirk
• feat: allow user to convert datatype if valid (#585) | Faith Chikwekwe
• feat: Implement alternative sharding using rendezvous hash to improve dynamic scalability (#570) | Kent Quirk
• feat: On shutdown, remove ourself from the peers list (#569) | Kent Quirk
• feat: Add cuckoo-based drop cache (#567) | Kent Quirk
• feat: Extract Sent Cache to an interface for future expansion (#561) | Kent Quirk

Bug fixes

• fix: do not send sample rate in dry run (#611) | Faith Chikwekwe
• fix: Remove API key logging (#606) | Tyler Helmuth
• fix: Fix flaky tests, clean up logic on rules (#596) | Kent Quirk
• fix: Add missing done channel to fix build (#573) | Kent Quirk

Maintenance

• chore: publish should only happen on main (#627) | Kent Quirk
• chore: Publish every build to honeycomb's ecr (#613) | Kent Quirk
• docs: update FieldList (#591) | Tyler Helmuth
• docs: add environment variables (#589) | Tyler Helmuth
• chore: Update CODEOWNERS (#588) | Tyler Helmuth
• chore: Change workflow to use Collections board (#587) | Kent Quirk
• chore: update dependabot (#583) | Kent Quirk
• chore: update validate PR title workflow (#572) | Purvi Kanal
• chore: validate PR title (#571) | Purvi Kanal
• refactor: Change Router to use TraceServer (#607) | Tyler Helmuth
• maint(deps): bump golang.org/x/net from 0.4.0 to 0.7.0 (#628) | dependabot[bot]
• maint(deps): bump github.com/pelletier/go-toml/v2 from 2.0.6 to 2.0.7 (#620) | dependabot[bot]
• maint(deps): bump github.com/honeycombio/husky from 0.19.0 to 0.21.0 (#619) | dependabot[bot]
• maint(deps): bump github.com/klauspost/compress from 1.15.15 to 1.16.0 (#618) | dependabot[bot]
• maint(deps): bump github.com/stretchr/testify from 1.8.1 to 1.8.2 (#616) | dependabot[bot]
• maint(deps): bump github.com/honeycombio/husky from 0.17.0 to 0.19.0 (#603) | dependabot[bot]
• maint(deps): bump github.com/hashicorp/golang-lru from 0.5.4 to 1.0.1 (#602) | dependabot[bot]
• maint(deps): bump github.com/klauspost/compress from 1.15.12 to 1.15.15 (#601) | dependabot[bot]
• maint(deps): bump github.com/honeycombio/dynsampler-go from 0.2.1 to 0.3.0 (#600) | dependabot[bot]
• maint(deps): bump grpc to 1.52.3 (#599) | Kent Quirk
• maint(deps): bump github.com/spf13/viper from 1.13.0 to 1.15.0 (#597) | dependabot[bot]
• maint(deps): Bump github.com/prometheus/client_golang from 1.13.0 to 1.14.0 (#576) | dependabot[bot]
• maint(deps): Bump github.com/tidwall/gjson from 1.14.3 to 1.14.4 (#575) | dependabot[bot]
• maint(deps): Bump github.com/hashicorp/golang-lru from 0.5.4 to 1.0.1 (#574) | dependabot[bot]

v1.19.0

1.19.0 2022-11-09

Adds a new query command to retrieve configuration metadata, and also allows for a new (optional) cache management strategy that should be more effective at preventing OOM crashes in situations where memory is under pressure.

Enhancements

• Add command to query config metadata (#556) | @kentquirk
• New cache management strategy (#547) | @kentquirk

Fixes

• Set content-type on marshalToFormat (#548) | @kentquirk

Maintenance

• Bump google.golang.org/grpc from 1.50.0 to 1.50.1 (#553)
• Bump github.com/fsnotify/fsnotify from 1.5.4 to 1.6.0 (#552)
• Bump github.com/stretchr/testify from 1.8.0 to 1.8.1 (#551)
• Bump github.com/honeycombio/libhoney-go from 1.16.0 to 1.18.0 (#550)
• Bump github.com/klauspost/compress from 1.15.11 to 1.15.12 (#549)

v1.18.0

1.18.0 2022-10-12

Enhancements

• Track span count and optionally add it to root (#532) | @kentquirk @kentquirk
• Add support for metrics api key env var (#535) | @TylerHelmuth

Fixes

• RedisIdentifier now operates properly in more circumstances (#521) | @Baliedge
• Properly set metadata to values that will work. (#523) | @kentquirk

Maintenance

• maint: add new project workflow (#537) | @vreynolds
• Bump go version to 1.19 (#534) | @TylerHelmuth
• Bump github.com/klauspost/compress from 1.15.9 to 1.15.11 (#531)
• Bump github.com/honeycombio/husky from 0.15.0 to 0.16.1 (#529)
• Bump github.com/prometheus/client_golang from 1.12.2 to 1.13.0 (#528)
• Bump github.com/spf13/viper from 1.12.0 to 1.13.0 (#527)
• Bump Husky to v0.17.0 (#538) | @kentquirk

New Contributors

• @Baliedge made their first contribution in #521
• @TylerHelmuth made their first contribution in #534

Full Changelog: v1.17.0...v1.18.0

v1.17.0

Enhancements

• Allow adding extra fields to error logs (#514) | @kentquirk
• Allow BatchTimeout to be overriden on the libhoney Transmission (#509) | @leviwilson

Fixes

• Consolidate honeycomb metrics to use single lock & fix concurrent read/write (#511)| @MikeGoldsmith
• Fix variable shadowing bug (#519)| @kentquirk

v1.16.0

1.16.0 2022-09-09

This release contains a number of small new features to assist in running refinery more effectively:

• Adds new endpoints to help in debugging refinery rules (see README.md)
• Fixes issues with SampleRate
• Adds some new configuration parameters (see the *_complete.toml files for more)
• Conforms to the GRPC standard for health probes
• Accepts OTLP/JSON traces and conforms to the most recent OTLP trace specification

Enhancements

• Add /query endpoints to help debug refinery rules (#500, #502) | kentquirk
• Implement grpc-health-probe (#498) | abatilo
• Make gRPC ServerParameters configurable (#499) | abatilo
• Fix sample rate for late spans (#504) | kentquirk
• Optionally record why a sample decision was made (#503) | kentquirk
• Added PeerManagement.Timeout config option (#491) | thrawn01
• Add 'meta.refinery.original_sample_rate' (#508) | epvanhouten

Maintenance

• maint: improvements to GitHub operation (#474, #477, #478) | JamieDanielson, vreynolds

Dependencies

• Bump github.com/stretchr/testify from 1.7.2 to 1.8.0 (#472) | dependabot
• Bump github.com/sirupsen/logrus from 1.8.1 to 1.9.0 (#484) | dependabot
• Bump google.golang.org/grpc from 1.46.2 to 1.49.0 (#485, 494) | dependabot
• Bump github.com/honeycombio/libhoney-go from 1.15.8 to 1.16.0 (#487) | dependabot
• Bump github.com/gomodule/redigo from 1.8.8 to 1.8.9 (#488) | dependabot
• Bump github.com/klauspost/compress from 1.15.7 to 1.15.9 (#495) | dependabot
• Bump github.com/tidwall/gjson from 1.14.1 to 1.14.3 (#497) | dependabot
• Update github.com/honeycombio/husky to latest and fix breaking changes (#505) | kentquirk
• Go mod tidy (#507) | kentquirk

New Contributors

• @abatilo made their first contribution in #498
• @thrawn01 made their first contribution in #491
• @epvanhouten made their first contribution in #508

Full Changelog: v1.15.0...v1.16.0

v1.15.0

Enhancements

• Add rule Scope configuration option to rules-based sampler (#440) | isnotajoke
• Replace hand-rolled binary.BigEndian.Uint32 with the real deal (#459) | toshok
• Validate successful span scoped rules test (#465) | MikeGoldsmith
• Create helm-chart issue on release (#458) | MikeGoldsmith
• github_token needs underscore not hyphen (#464) | @JamieDanielson

Maintenance

• Replace legacy with classic in readme (#457) | MikeGoldsmith

Dependencies

• Bump github.com/spf13/viper from 1.10.1 to 1.12.0 (#461)
• Bump github.com/stretchr/testify from 1.7.1 to 1.7.2 (#467)
• Bump github.com/honeycombio/husky from 0.10.5 to 0.10.6 (#460)
• Bump github.com/klauspost/compress from 1.15.4 to 1.15.6 (#466)
• Bump github.com/prometheus/client_golang from 1.12.1 to 1.12.2 (#463)

v1.14.1

1.14.1 2022-05-16

Fixes

• Fix crash bug related to sharding (#455) | @kentquirk

Maintenance

• bump husky to 0.10.5 (#450) | @MikeGoldsmith
• Bump github.com/klauspost/compress from 1.15.2 to 1.15.4 (#451) | dependabot
• Bump github.com/tidwall/gjson from 1.14.0 to 1.14.1 (#444) | dependabot
• Bump github.com/fsnotify/fsnotify from 1.5.1 to 1.5.4 (#441) | dependabot

Documentation

• add a note about reloading the configuration when running within docker (#448) | @leviwilson
• README: remove incorrect mention of sending SIGUSR1 to trigger a configuration reload (#447) | @jharley

v1.14.0

Enhancements

• Add support for environment and dataset rules with same names (#438) | @MikeGoldsmith

Maintenance

• Update otlp to v0.11.0 (#437) | @MikeGoldsmith
• Update go to 1.18 (#430) | @MikeGoldsmith