[Microbatch] Leverage static partitioning in dbt-bigquery materialization strategy #538
Labels
feature:microbatch
Issues related to the microbatch incremental strategy
pkg:dbt-bigquery
Issue affects dbt-bigquery
In dbt-labs/dbt-bigquery#1334, we introduced the microbatch strategy which leverages the dynamic partition insert/overwrite mechanism.
However, static partitioning would be significantly more performant. Benchmarking here:
https://discourse.getdbt.com/t/benchmarking-incremental-strategies-on-bigquery/981
Let's:
partitions
prior to calling the insert_overwrite_sql as part of the microbatch strategyGENERATE_TIMESTAMP_ARRAY
(for 'hour' batch_size) orGENERATE_DATE_ARRAY
(for all others) +__dbt_internal_microbatch_event_time_start
and__dbt_internal_microbatch_event_time_end
jinja context vars.The text was updated successfully, but these errors were encountered: