Bump apache-beam[gcp] from 2.54.0 to 2.60.0

[dependabot]

Bumps apache-beam[gcp] from 2.54.0 to 2.60.0.

Release notes

Sourced from apache-beam[gcp]'s releases.

Beam 2.60.0 release

We are happy to present the new 2.60.0 release of Beam. This release includes both improvements and new functionality.

For more information on changes in 2.60.0, check out the detailed release notes.

Highlights

• Added support for using vLLM in the RunInference transform (Python) (#32528)
• [Managed Iceberg] Added support for streaming writes (#32451)
• [Managed Iceberg] Added auto-sharding for streaming writes (#32612)
• [Managed Iceberg] Added support for writing to dynamic destinations (#32565)

New Features / Improvements

• Dataflow worker can install packages from Google Artifact Registry Python repositories (Python) (#32123).
• Added support for Zstd codec in SerializableAvroCodecFactory (Java) (#32349)
• Added support for using vLLM in the RunInference transform (Python) (#32528)
• Prism release binaries and container bootloaders are now being built with the latest Go 1.23 patch. (#32575)
• Prism
  • Prism now supports Bundle Finalization. (#32425)
• Significantly improved performance of Kafka IO reads that enable commitOffsetsInFinalize by removing the data reshuffle from SDF implementation. (#31682).
• Added support for dynamic writing in MqttIO (Java) (#19376)
• Optimized Spark Runner parDo transform evaluator (Java) (#32537)
• [Managed Iceberg] More efficient manifest file writes/commits (#32666)

Breaking Changes

• In Python, assert_that now throws if it is not in a pipeline context instead of silently succeeding (#30771)
• In Python and YAML, ReadFromJson now override the dtype from None to an explicit False. Most notably, string values like "123" are preserved as strings rather than silently coerced (and possibly truncated) to numeric values. To retain the old behavior, pass dtype=True (or any other value accepted by pandas.read_json).
• Users of KafkaIO Read transform that enable commitOffsetsInFinalize might encounter pipeline graph compatibility issues when updating the pipeline. To mitigate, set the updateCompatibilityVersion option to the SDK version used for the original pipeline, example --updateCompatabilityVersion=2.58.1

Deprecations

• Python 3.8 is reaching EOL and support is being removed in Beam 2.61.0. The 2.60.0 release will warn users when running on 3.8. (#31192)

Bugfixes

• (Java) Fixed custom delimiter issues in TextIO (#32249, #32251).
• (Java, Python, Go) Fixed PeriodicSequence backlog bytes reporting, which was preventing Dataflow Runner autoscaling from functioning properly (#32506).
• (Java) Fix improper decoding of rows with schemas containing nullable fields when encoded with a schema with equal encoding positions but modified field order. (#32388).

Known Issues

N/A

... (truncated)

Changelog

Sourced from apache-beam[gcp]'s changelog.

[2.60.0] - Unreleased

Highlights

• Added support for using vLLM in the RunInference transform (Python) (#32528)
• [Managed Iceberg] Added support for streaming writes (#32451)
• [Managed Iceberg] Added auto-sharding for streaming writes (#32612)
• [Managed Iceberg] Added support for writing to dynamic destinations (#32565)

I/Os

• PubsubIO can validate that the Pub/Sub topic exists before running the Read/Write pipeline (Java) (#32465)

New Features / Improvements

• Dataflow worker can install packages from Google Artifact Registry Python repositories (Python) (#32123).
• Added support for Zstd codec in SerializableAvroCodecFactory (Java) (#32349)
• Added support for using vLLM in the RunInference transform (Python) (#32528)
• Prism release binaries and container bootloaders are now being built with the latest Go 1.23 patch. (#32575)
• Prism
  • Prism now supports Bundle Finalization. (#32425)
• Significantly improved performance of Kafka IO reads that enable commitOffsetsInFinalize by removing the data reshuffle from SDF implementation. (#31682).
• Added support for dynamic writing in MqttIO (Java) (#19376)
• Optimized Spark Runner parDo transform evaluator (Java) (#32537)
• [Managed Iceberg] More efficient manifest file writes/commits (#32666)

Breaking Changes

• In Python, assert_that now throws if it is not in a pipeline context instead of silently succeeding (#30771)
• In Python and YAML, ReadFromJson now override the dtype from None to an explicit False. Most notably, string values like "123" are preserved as strings rather than silently coerced (and possibly truncated) to numeric values. To retain the old behavior, pass dtype=True (or any other value accepted by pandas.read_json).
• Users of KafkaIO Read transform that enable commitOffsetsInFinalize might encounter pipeline graph compatibility issues when updating the pipeline. To mitigate, set the updateCompatibilityVersion option to the SDK version used for the original pipeline, example --updateCompatabilityVersion=2.58.1

Deprecations

• Python 3.8 is reaching EOL and support is being removed in Beam 2.61.0. The 2.60.0 release will warn users when running on 3.8. (#31192)

Bugfixes

• (Java) Fixed custom delimiter issues in TextIO (#32249, #32251).
• (Java, Python, Go) Fixed PeriodicSequence backlog bytes reporting, which was preventing Dataflow Runner autoscaling from functioning properly (#32506).
• (Java) Fix improper decoding of rows with schemas containing nullable fields when encoded with a schema with equal encoding positions but modified field order. (#32388).

[2.59.0] - 2024-09-11

Highlights

... (truncated)

Commits

• 02208fc Set version for 2.60.0 RC2
• 83a883d fix: skip close on bundles (#32661) (#32759)
• 30c109f Merge pull request #32747 from apache/pydocUpdate (#32762)
• 7d0d9c6 Fix build_wheels and build_release_candidate (#32725) (#32761)
• 20a45bb Fix the FlinkRunner for Dataproc (#32742)
• 75422cb Force kafka 3.1.2 for expansion-service jar (#32703) (#32708)
• f61c8e1 Enforce a size limit on StringSetData (#32650) (#32707)
• 96a96da Report File Lineage on directory (#32662) (#32706)
• 64854bb Revert "Set setFailIfPoolExhausted in SessionPoolOptions for SpannerAccessor"...
• 8eef4a3 [Managed Iceberg] add GiB autosharding (#32612) (#32663)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)