Nightly builds

list of all night builds

• 20241226 b13d5e977b6882ff02d18233c3157bf9cf1c01b0
• 20241225 c40eb045c91e819a9282a641b716427c520089f1
• 20241224 86c3933ff3f14ef5bf226a004d7ac6a9fd72c60e
• 20241223 001ac816136e643593894a4ceff1b19905e73d02
• 20241222 5423dee87068c12b473181731862751dbccbb769
• 20241219 adb8651f78e1bcb6cf9cf73db73111d86a009c34
• 20241218 94ccd00bc21a21de6b82b523f089f2bdd2c9979a
• 20241217 456181340e7f76e2ff4caa198d842c2a1768968f
• 20241216 492e2d7d03ccfb530692649503a3e74f8624844c
• 20241215 680c0982fc56445394a8eef558f4dd7dceab06be
• 20241214 a520c1d493959bb79681e0af3d8000e879534821
• 20241213 bfef38ef5d490165f57b28defecef670c20897df
• 20241212 8e1d60bb7d4ded2264760c8e7202301045e7420d
• 20241211 9292b6f38bc070d54bfcf2f2c6f4e63b77f75c1e
• 20241210 2b819a39c9a52e764b9a98026bd5d4f93c04b6ae
• 20241209 4da6bdacf3876713f806569922a1443e69b3b6df
• 20241208 fcac3111a1524e2c2f5eca2045260a7baa20b8e4
• 20241206 733342764be327921741748db443ff23286cab63
• 20241205 59fa5f882377629798ade81fab64ad54eb82a086
• 20241204 a38c4d111ce0514356c89043f1e373e707c73ad8
• 20241203 e3337876903cbfa53231407f39a58f1ec53b03d5
• 20241202 396099459bb6e456539bb94def7a68aff856db15
• 20241201 22b11ef413b357ab736a7986907de541c1cd710b
• 20241128 4b1cb92e5ce37c72139f933b525ed9c41aec575f
• 20241127 c45327bae82869699328a47e70919d38e783df2d
• 20241126 5d6aaf47710b449cde54081e987ba8d457adb6c6
• 20241125 9bc0b1fb017d0ee0d04249dc01a5f3c5c428c27a
• 20241124 b4bca5724e7acb9c39919aaa188ed699a10eb1f4
• 20241122 bc06ffdb782ea5a9f11d19825f0f61c1723991c1
• 20241121 cfa521dcccfdb2763da050ed6a7f79cb39f63f9f
• 20241120 fe69ce3093b33e16e8f1682eddeac28789190764
• 20241119 72e5ad3e0672687439c8a33688c7acc6ded540fa
• 20241118 0b223b4c2f569e749413620287d1655410041d06
• 20241117 0519b3fdd5ab0b94d39ee6989050272e6f3e5c78
• 20241116 f29c971146bc25cc567cedaf6bd4ae7e0228a2bf
• 20241114 c92069481d2aba5a77ffb673b48c7e27b5a698a6
• 20241113 1aed314c2d8944da22b6d9c3c4d07c7683c2a98b
• 20241112 98aa20b886a63a58e602cda41c1c161a66a9fc06
• 20241111 3c2e81b53a8411451a5f999ca3313654d98c9586
• 20241110 82b782547c0cefe48263d0fc69047b2fcb71cac3
• 20241108 c65834074c8334d835a03333362c0f2eb67a0802
• 20241107 e6e1ca9eb987e6abb9a7f7e8f53163f9f02b5397
• 20241106 9a32e75f227f837a6f16fb3afe159c1f8af2ad5b
• 20241105 a945f52de4633a6f87f62309aa3cdbbebaff8d11
• 20241104 bb0d786b39c9eef9dbf1e16814e8d12e46b8a676
• 20241103 e7e4a2c9d9f288f6fe89c92e3fcdbfe3d6d36ca6
• 20241101 b39c9af88fd1bbe05df7e1ece506741eb489471b
• 20241031 de7b7cd2b66ebe8bacf654f3e488cab2f3c241e8
• 20241030 a36c3172230d2aaf5632f60a4a31f0958d11f98f
• 20241029 98baacd19f521cd75ec7dc9cbdde903e7ef01d12
• 20241028 0d013eab1bba8cd86c575057e165586262362636
• 20241027 a176aced3c047d780adc8f3981a4921faa526255
• 20241026 32666f2183f49585c69d8690afec796e14e5ab78
• 20241025 d5d0283650fafef0eb2592c273a1d0a95a04f35c
• 20241024 65ed113a3c52c0fe079c9fa2751fda48d92ead49
• 20241023 e6f830b68f42022b8ee1d9e4f68836cddcb3086a
• 20241022 87b7164b8029e779c7ccd5f6a475376c491ad0dc
• 20241021 ff61c885fd873de3686111d2db3aaa61b99c0e75
• 20241020 96aeb52372543bf1cc5ab814e50210cb3fb1d898
• 20241018 14f67e9f6c07e03de114bd8b1f3cfb5b66ef8c65
• 20241017 79ee46fdfab686486875713e1705d58cf0bd2da0
• 20241016 360fe8adc5b65fda09ff2a87388b7102326ea190
• 20241015 2af00e81d826ed4f5198e429b5803bfa3dc29652
• 20241014 909b7157978ae5c6fe33d08de955cc4987bc455d
• 20241013 f50535978d243e8158e2647430ba408c7b895008
• 20241011 9ef40ba54a3cfce9f4946440ef78c8020b1b72f5
• 20241010 b12460d08b5fadef15d43495cf63b6fe21dd4ee8
• 20241009 b8c3fcb52964ff87d743cb9349e7097f77047c21
• 20241008 c359bd9662d45fb90f0957101be20d658cd1fde3
• 20241007 ba761f2362886ab3b73602a3423b73c97a9f5b26
• 20241006 51c5e3e0bef991767f48a0c3bf6814426711b5f2
• 20241002 66890dbf054bd2c9980e4817dac7affe974286d0
• 20241001 ef8c50f5f56c04374cda3c8b8204afc10ee6a88f
• 20240930 17ea32610a22d697ccf4499ddd1cc6f90c5b1492
• 20240929 83e3c4b4e35ed1dea80cee4ba49cbb4c49d6e267
• 20240928 b4ac5aba12d40df880931821ee68fde45f1d4683
• 20240927 d8774819c47930d0da158792983188fbeff4eb59
• 20240926 75080b765e5914c07e906de4eff7f16146aa207d
• 20240925 0679b2fbb5f54e4c9e9daf6f8ab5018da9ebb9fa
• 20240924 29adeb97370b6a77a4dcc857ac55249016dc3be9
• 20240923 a898dcc8f42001b5a4f4dcd965cd0bde8e67b45a
• 20240922 bc65ffbf0cf58eadcff87930bc69cb5a8bbcf56d
• 20240921 98a2d410ab40f58d2bc24b296aee6f644e380229
• 20240920 2b8f6a6be6a283b8cfc0f4b348092d897573cc5a
• 20240919 97a329350a14d747e0eae31e4e1c86ff3d4d4ea6
• 20240918 5b625f81b06217c4350a57b8457006e6b9bfdd8d
• 20240917 1c04a5ce3ab2cc605abd6cf4367017433b63c7d8
• 20240916 9413b28b5a836bc577a9c50fa8cd223cd137618b
• 20240915 ac966f1c897544afb00a1e71c14bf01d32866388
• 20240913 72396b2c3bfe00e13bef0653fe68547b93160f8a
• 20240912 b9ac1ac1f606a3ed1c894db2a69de134660a2924
• 20240911 23410f09511f3f75cbd2dbe4813fe766081e6656
• 20240910 596138e0364cc98e02224ea387d6644d9bf9e8cb
• 20240909 29d2e1ef33fad45e8348af9d0de251bbd7ddd194
• 20240908 becb896acd442d32eab4c14dccf558a4b346c643
• 20240907 71753f11223cd0b4d67ba6baf7ac636fa246e7fc
• 20240905 79ba86964acf0616693a8534c9eac1a67eb2889d
• 20240904 0dd06ffa9a34b0cb584393990f8de8a41267718d
• 20240903 c0ce8a8a6b4d51e463a605fb307fa4b014e0f609
• 20240902 b2eea6b799d31aae8240e41cf3f0e0d29e542255
• 20240901 e41b348f2d7d800b45e4102192200fb1698037ca
• 20240829 655ce2a8cdd83f2338b2de23bef1133cf4e1916a
• 20240828 befb04f1e433ecbb564a059f34b4299ba4952bed
• 20240827 63b4d47ea9f54f567983e097e15d8dc9b0384849
• 20240826 ce7a5afc0ecc86adbf5fb7c49cb4337d5d4df983
• 20240825 5c1af4c21215f8887133233a5978402019c3d3fa
• 20240824 d33623accf6e932592f4a945af436c18ced812fd
• 20240823 5cae57827c8ca56868b9cc0de0686d6f14e68510
• 20240822 9c4984d4ec2d17351a741cab8ac512b39dd2261a
• 20240821 dd6515625d980443ae1a71cf6332438cdbd49e86
• 20240820 e6e1f485cc0f778d4f67e33a362d67359e4fa736
• 20240819 11f0acfe548ff0f317c979e51570041058955b9e
• 20240818 7cb75e565470570fd191fac062e91864d4582c98
• 20240816 f5f09a6c15b1291623a1fccc9638035cf19002c0

Commit history

Tip

A release's SHA is the first commit's SHA listed in its section.

Unreleased

• 302c3caa9728a348fa476b43bd9e6f226c9043b0 fix: support string to number type in JSONB_POPULATE_RECORD (#19937)

nightly-20241226

compare changes with previous nightly build

• b13d5e977b6882ff02d18233c3157bf9cf1c01b0 test: add a complex nested type test (#19940)
• 3f76da8614e7eba3c8f6155981482dae75f68ac4 feat(catalog): introduce clean_watermark_index_in_pk for state cleaning (#19938)
• ef74a63682dbe3ea69952a40d026a2afa8ed4bf8 build: further prune unused dependencies and check in CI (#19926)
• 62818d32bb7a0ab59a9187c34545ec29b391520a refactor(frontend): merge steps for resolving on-conflict behavior in create table handler (#19916)
• ad95d1026ada8a332fb932a60838714fb66ff13a feat(dashboard): improve relation & fragment graph (#19872)
• fa5e442bddbf513b4c5aece5c36ac5f6656c2c2c feat: migrate from icelake to iceberg-rust (#19887)
• 764fed7ed58f4df9afc2ea7894ea13617375a456 chore: remove file_cache_bench and s3_bench (#19927)
• 1dee3accdd5baf4cafbff451798fdd7bb2381152 feat(expr): support IGNORE NULLS for first_value/last_value (#19847)
• 2f4008ceff378c88312a6d93fb2f28495a2ee043 fix(iceberg): fix hidden row id for iceberg engine table (#19901)

nightly-20241225

compare changes with previous nightly build

• c40eb045c91e819a9282a641b716427c520089f1 feat(storage): lazily remove instance data in uploader to avoid data loss (#19931)
• d706db6ca770f249221f0c58dd87f41390c6e109 feat: support pg_stat_user_tables and key_column_usage (#19739)
• ae48d9d2beae0c7903ffa6ff5269a5ac39ab2976 feat(snapshot): persist snapshot backfill epoch in fragment stream node (#19751)
• ef5835e6b3b633b86d9eeaa98a034b59c9cc18c8 feat(pb): box stream NodeBody to reduce stack memory usage (#19911)
• 3bdf84dcf62569c6e8ff4861e8ae7ea6185c86ab perf(stream): optimize native pg sink (#19688)
• d7f8b4e52306cb4075175b9231fa3f6ad076963e fix(chore): fix uint gauge vec macro (#19909)
• 3431eabc3ef3c4f6431ba39546707c94a1d5b375 feat(frontend): Supports cut OR condition and push down to storage (#19812)

nightly-20241224

compare changes with previous nightly build

• 86c3933ff3f14ef5bf226a004d7ac6a9fd72c60e fix(metric): include table change log when calculating Hummock version size (#19920)
• 81f651f871db375313e1611558269e3e6f757b57 chore(deps): Bump http from 1.1.0 to 1.2.0 (#19904)
• 1b9b5a797396ce19e08b55c0f111e6e30310fb89 fix(sink): fix mogodb write error handling (#19869)
• af666704444506b48f3ae6e6e0869ee99bd53bed chore(deps): Bump sqllogictest from 0.23.0 to 0.24.0 (#19903)
• 16c7fcd2e483cb6b154ba3ab78fde37f060ad5b4 fix(meta): skip unnecessary delta persistence (#19878)
• 6098f65dd3d628204c1ac59714ea657f7e3a19b6 refactor(frontend): extract arguments for create table handler (#19898)
• e70eb591f6f505bf99a18d116b3b4373f406f424 build: prune unused dependencies (#19885)
• a175600c5f1501dbe2282f57fac23840e4f2ab69 fix(frontend): fix row filter with pk diff order type (#19797)

nightly-20241223

compare changes with previous nightly build

• 001ac816136e643593894a4ceff1b19905e73d02 fix: change JNI sink normal logs to debug level (#19758)
• c5eadcc24fd2ccab3ea02e47e1962f814c934d88 refactor(metrics): introduce UintGauge to keep consistency (#19896)
• 3831db7a21cb49ee4c243a960bb0a8c583e9e347 feat(dashboard): show current epoch and color for fragments & relations (#19829)
• 26805d95beead9f855f100d6b17f7dfd8ab0dea6 chore: update docker image to v2.1.0 (#19888)
• 4aab01730c3b3c24944a2833e63179b6d208fa61 feat(sink): support sink rate limit for external sink (#19660)
• eabb2d06fef2cdb98c097465e063949d88a2aff2 fix: disable telemetry report with madsim (#19890)
• fcf0af32fda55c7903ae3ff1633538916285967a fix(frontend): check data type in column id generator (#19828)
• 6d4f966d9632b0c6503d5236c9dfe8578f0aea12 fix(meta): fix & enable e2e tests with MySQL meta backend (#19846)
• 02b53098350aba3eb1025b8c889e5f88b97ab53b refactor(optimizer): use Cell instead of RefCell for interior mutability (#19883)
• 3cd7241696a939274c709aa59508ad4d66b0ce0f fix(iceberg): fix commit_checkpoint_interval for iceberg table with connector (#19788)

nightly-20241222

compare changes with previous nightly build

• 5423dee87068c12b473181731862751dbccbb769 fix: use ring as default provider (#19874)
• 709894d8a44025bdba7835e9d9c352a2a607ba84 fix: Fix the panic issue with the parallelism() call (#19849)
• 9dff1ac52450380ee5f8208b43841d5e228fb023 feat: add graph check in risectl for sql meta store (#19764)
• de6eebc6cb562f8a9c316762e497b6afecfe5f0d feat(sink): add bigquery sink json string credentials (#19798)
• d6341b74be3f1913cc93993a95c147999df1ff74 refactor: split catalog to smaller files (#19870)
• c022a36fadf9017ef71ed378314398b1cc1dff5f feat(sink): support deafult aws credentials for deltalake (#19557)

nightly-20241219

compare changes with previous nightly build

• adb8651f78e1bcb6cf9cf73db73111d86a009c34 refactor(source): rename some source parser related types and methods (#19863)
• f3d8e0d58f759e51397ac1006d34f8405c119d8d refactor(source): respect max chunk size in source connector parser (#19698)
• 29d697fd7c292417ca8ebe7991e0fecf6b1ffd5a chore(ci): migrate the qa service (#19862)
• 34c93df5b62c88560a7a17a8c3756c54691e170c chore(deps): Bump next from 14.2.12 to 14.2.20 in /dashboard (#19844)
• 5cffd1723b8ac3067770f7cbcb060a846c4f77f9 chore: fix typos in func && doc && comment (#19796)
• c2389e25e86ca2be5c70cbfdd70dbbc3ed91599c chore(deps): Bump quick-xml from 0.36.1 to 0.37.1 (#19692)
• 6ba4565b242949f30a65ebe29284900473648d8c chore(deps): Bump self_cell from 1.0.2 to 1.1.0 (#19767)
• 5e66c86f2d4bf447bb2143e0f8365e6622a0321d chore(deps): Bump twox-hash from 2.0.1 to 2.1.0 (#19861)

nightly-20241218

compare changes with previous nightly build

• 94ccd00bc21a21de6b82b523f089f2bdd2c9979a chore(deps): Bump twox-hash from 1.6.3 to 2.0.1 (#19258)
• fe84f69e1611dad37f0ea97ff9dcdcbbaa972b63 feat(source): create source stream with retry during recovery (#19805)
• 3cbc231b002eeef2e370ae15d067e4518b0f57dd refactor(meta): filter out SSTs for time travel metadata (#19694)
• 1804c80e5bafb1022d35bf480945b1c54d9e513a feat: jdbc sink user password (#19822)
• 569f9f9ad9cdb3aa8470ef076c8e6b2bdca56b32 fix(ci): fix source test dependency issue (#19840)
• a74508342b03fd66a302f754bc23ee6063277996 fix(ci): use correct image version for ci-standard-env (#19839)

nightly-20241217

compare changes with previous nightly build

• 456181340e7f76e2ff4caa198d842c2a1768968f fix(connector): file scan use correct path (#19793)
• 6f14e7937f3524a29ede0f7da4a3e71d8b65a3b9 chore: update package version to 2.3.0-alpha (#19823)
• 7a35a6a2b94afcdca37b646ccb649d5c59a4ca47 feat(error): include gRPC call path in error report (#19809)
• ee9f5005e7997c0921d6499968bba4e375f6ef9c chore(deps): Bump apache/skywalking-eyes from 775fe1ffda59b7e100aa144d0ef8d7beae17f97d to 3ea9df11bb3a5a85665377d1fd10c02edecf2c40 (#19819)
• cb6c013139ef93358ceb30f84acc13031ac46483 chore(deps): Bump peter-evans/create-pull-request from 6 to 7 (#19818)
• 123c5c01372f8f9a230d14bfe35b37957b045349 chore(deps): Bump mozilla-actions/sccache-action from 0.0.6 to 0.0.7 (#19817)
• a5c8c974676062fe36508fdf75592a17eedba7bb chore(deps): Bump crate-ci/typos from 1.28.2 to 1.28.3 (#19816)
• 4ed93eb9fcd75ddc67b77dcce2cc5b33c959c446 fix(log): print java exception stack trace in logger (#19814)

nightly-20241216

compare changes with previous nightly build

• 492e2d7d03ccfb530692649503a3e74f8624844c fix: add mview_definition in scaled actors (#19784)
• 0b1c0e70b7f5b318471557d157d9b7b94d6d013f fix: session variable should not affect shared source (#19807)
• 594ef891d30067f99d48ce5e515a3134a1df7f9a feat: replace icelake Transform with iceberg (#18625)
• 45fcf8f33c3a5eb71dcf31c7f33d1ee80f7a832d refactor(frontend): split create_source into smaller mods (#19803)
• 1e24ca482c29c08ba3aeb31ee44f672e3eaaef05 fix(meta): fix order when restoring metadata (#19791)
• 6b12555285f77aa4a4862b5c437ee5afd1abb4a6 feat(ci): more meta backend agnostic e2e tests in main-cron (#19595)
• 6759a081c8e789f30ce24747218320a06356d9a8 chore(deps): Bump sysinfo from 0.32.0 to 0.33.0 (#19801)
• 7bcb908039547d7b762d73b7d63f3d02e9b00a7b fix: alter shared source fresh schema will make it non-shared (#19802)

nightly-20241215

compare changes with previous nightly build

• 680c0982fc56445394a8eef558f4dd7dceab06be feat: allow generated column in cdc table (#19112)

nightly-20241214

compare changes with previous nightly build

• a520c1d493959bb79681e0af3d8000e879534821 feat(barrier): support database failure isolation (part 2, local) (#19579)
• 5afda5e7427debd35062cf656f479951d08fda17 feat(catalog): add pg_my_temp_schema and pg_get_viewdef(int4, bool… (#19750)

nightly-20241213

compare changes with previous nightly build

• bfef38ef5d490165f57b28defecef670c20897df chore(deps): Bump tempfile, rustix, libc (#19339)
• 6b800b4c0320f554eee2226b818c13652894cf23 feat: add actor info to diagnose & dump diagnose on CI failure (#19787)
• 5a85b55a1946ff44d1f112a131b01ec038f87d6a feat: include subject in NATS source (#19708)
• a474f5e797e9ea18a963d978fb88497be6f52adb chore(deps): Bump nanoid from 3.3.6 to 3.3.8 in /dashboard (#19734)
• f1ac7661234bb890acc7a018e87249d3277468f6 refactor(source): DRY decimal parsing (#19782)
• 9cd0453e7cb79a4e1709671719758643a468c3b5 feat(common): support switching from pg jdbc to pg native sinks (#19703)

nightly-20241212

compare changes with previous nightly build

• 8e1d60bb7d4ded2264760c8e7202301045e7420d fix(source): revert create source reader with retry (#19754)
• 9f929d98ccf5af62d37fed987ca0f63ab03897b7 chore(deps): Bump golang.org/x/crypto from 0.17.0 to 0.31.0 in /integration_tests/client-library/go (#19768)
• b3f8dbcb807cd7c3d6180949e0150c9d2f385c7c chore(deps): Bump actions/checkout from 1 to 4 (#19778)
• a201c5c66108d302b98e36f181f00df785c917ff chore(deps): Bump actions/setup-java from 3 to 4 (#19776)
• 10f26f7da75047b5365986d67bf26c516952c37c chore(deps): Bump 8BitJonny/gh-get-current-pr from 2.2.0 to 3.0.0 (#19780)
• 81c5221c33c89d97dc7dba4342f0479975e68102 chore(deps): Bump mozilla-actions/sccache-action from 0.0.5 to 0.0.6 (#19777)
• 93f720f95f9634884a30cc38d4336ce48ed475a0 chore(deps): Bump thehanimo/pr-title-checker from 1.4.2 to 1.4.3 (#19779)
• ca7dd29647f6d38ffc4261ac6424f7d510ee5d36 ci: add dependabot for GHA & bump buildkite plugin versions (#19774)
• 7301b413cb7654ccf7c796c16e0787b39a16e38c refactor(batch): split executors into a separate crate (#19742)
• 1678ab1e956b3f3f423d6a94d1dedf6cd02a88f4 fix(source): resolve avro Ref during access without hack (#19746)
• 58adff73d872245e5248f677ddc78e8bf39ee453 fix(streaming): fix compatibility upgrading from v2.1.0 for singleton vnode count (#19760)
• 0e436c23884e63a2f3c5a9feea820bf72f86d166 refactor(batch): register-style executor builder (#19741)
• 013725dab5d9511c152d8576ae368e6d0a9531c9 fix(snapshot-backfill): fix incorrect recv barrier and add test for recover (#19756)
• dcf33600d1240aa6a9f6e503bc8a8493de3d6ba1 fix(meta): ensure order when restoring metadata (#19763)
• 8d3015a3c342a3a35200173e02d25ac5076aac17 chore: update PR template (#19762)
• 572404dd561f2f7735103445ed905b04a3c8fd36 chore: fix typo in comment (#19759)

nightly-20241211

compare changes with previous nightly build

• 9292b6f38bc070d54bfcf2f2c6f4e63b77f75c1e feat(iceberg): make commit checkpoint interval default 60 for iceberg engine table (#19738)
• ffe63f1b20755b49f824949bab21964081a20b43 feat(frontend): support alter source refresh schema for shared source (#19740)
• 61d4a7918e1578a86404ee87d02ebc217a114fd1 feat(barrier): support database failure isolation (part 1, meta) (#19664)
• f945379f2f9cd7052c279e7e6212f1a223f3b1ee fix(iceberg): fix iceberg engine table time travel (#19747)
• 57773adbec3d454ac0b4bda78c802b678d66f274 chore(deps): Bump indexmap from 2.6.0 to 2.7.0 (#19748)
• 0b60cc83314bc85be38218a926c390cb224f4d5b chore(deps): Bump anstream from 0.6.4 to 0.6.18 in /integration_tests/feature-store/server (#19718)
• 11dac77ab7c3aa07ef16aa48d3198cf8048d0147 feat(frontend): support alter add column for shared source (#19649)
• 3bc2ccbc8e92595bb2d6df0ddfd6a4a2379e1788 chore: bump h2 to address hanging on long headers (#19752)
• 119e17eee482b6add5d5cb6ceb3b84326bf5ce68 refactor: add 2 pedantic clippy lints for strings (#19614)
• 5954c98171639b6be7bf92d86e6812188b7eafdd fix(meta): do not release connection to in-memory SQLite for meta store (#19605)
• 67491ab7b9e9f5817ba1b289602b0a9492e607de feat(storage): avoid copy key slice in storage table iter when unnecessary (#19717)
• 00ee6663785fc64633d987d65a709597370bcaa0 test: workaround a flaky source test (#19729)
• c269f84cb9da1a50a5adb5f628f2fc648b843894 chore(deps): Bump tower from 0.4.13 to 0.5.0 (#19663)
• 4545b6e98c9ccc9a56b571feeac5ff37f3c00c44 feat: add connector props and format options in rw_catalog (#19689)
• 87d0bb42b192943f628839afa10a044c714c2257 fix(iceberg): fix iceberg config load (#19733)

nightly-20241210

compare changes with previous nightly build

• 2b819a39c9a52e764b9a98026bd5d4f93c04b6ae feat(sink): add sink throughput( mb/s) metrics (#19587)
• 152cab20424d883722d62a2ace5a10280a35197b refactor(batch): simplify batch executor builder (#19731)
• d2bb1db634d2ee62da167bcff1199e86266810aa feat(meta): support replace source (#19650)
• 5d4dbd4fdef95fd505a18eefdcee2a2a17a8eecf feat(meta): ensure recovery is finished after recover command returns (#19737)
• f31e762794a509cb41efd40aada8e083e4caff67 chore(deps): Bump url from 2.5.0 to 2.5.4 (idna from 0.5.0 to 1.0.3) (#19730)
• 6eadbde93d182b5d9df49fd3750a2859a4beb7c9 feat(iceberg): refine iceberg table column datatype and name. (#19690)
• a2e53a556c4624bbdc31312a325682a7bdacc825 fix: double RUST_MIN_STACK to resolve stack overflow (#19695)
• bd82fe328de30796f641971489e4e0f26ded4b5c perf(stream): add hash join memory benchmarking for cache refill (#19712)
• 5585f51c34d9efa02b78b0040816365bd6e32f0d fix: missing dependency info of sql UDFs for streaming job ddl (#19716)
• 2042dd4dd64cc103977b175b43d6edbd88c5a60e fix(file source): fix message offset of opendal source (#19721)

nightly-20241209

compare changes with previous nightly build

• 4da6bdacf3876713f806569922a1443e69b3b6df feat(storage): avoid using box dyn for storage table merge sort (#19713)
• 430fbb9f39dc2dc15193b8b13fb735fede42518f chore: fix some typos (#19710)
• 9b26a969ac78805121cd6416dca67e32b0cefb14 refactor(meta): extract code for connecting sql meta store (#19603)
• 7005c0591e7f2f58c28d8615c87d7aab3d604f8f feat(storage): support online cache resize via risectl (#19677)
• f3ed1de67ace232cb09b196ee50e06e6f9f74bd2 refactor(source): extract avro inner schema precisely (#19701)

nightly-20241208

compare changes with previous nightly build

• fcac3111a1524e2c2f5eca2045260a7baa20b8e4 refactor(cdc source): merge cdc heartbeat chunk builder & data chunk builder (#19671)

nightly-20241206

compare changes with previous nightly build

• 733342764be327921741748db443ff23286cab63 refactor(connector): re-arrange the connector::parser module for better readibility (#19697)
• faf660dde20582bd17e0c1ceaa5a7e24904ce108 feat(frontend): Add struct scan and simply optimise struct filter. (#19425)

nightly-20241205

compare changes with previous nightly build

• 59fa5f882377629798ade81fab64ad54eb82a086 feat(iceberg): support iceberg engine table (in local env) (#19577)
• 45d266424ef4040795d97c6f744e021b2a6a2b8a feat: connection for schema registry (#19685)
• 5ba9a73192262d2d417b1ecc86e33297fc6b9289 fix(storage): fix backward compact (#19669)
• 80f39766c1b8825846e008aef39f279f93d83052 refactor(file source): emit source messages line by line directly from opendal reader (#19654)
• f0a91dfa5bef8474aa6f93ed4b71ef6b37d39327 feat: add support for dml_rate_limit (#19679)
• 623a796005cc96437cd3b7d3a6d6f4bef8891ab0 chore(deps): Bump linkme from 0.3.22 to 0.3.24 (#19682)

nightly-20241204

compare changes with previous nightly build

• a38c4d111ce0514356c89043f1e373e707c73ad8 fix(iceberg): bypass iceberg partition table optimization (#19655)
• 206b10bf70c7ab918817821643da5d9d23beece5 fix(snapshot-backfill): do mapping_chunk with output indices to all chunks (#19665)
• 0b7c43cfde74429c62305144259abe8efe5c35fe chore(deps): Bump governor from 0.6.0 to 0.6.3 (#19662)
• ccd46203faa8b102f65120a99a695c5958560f40 refactor(meta): insert time travel metadata in batch (#19657)
• 404998e422ff7e4727042306dceebf862d8dc11e feat(stream): create cdc table reader and source data stream with retry (#19467)

nightly-20241203

compare changes with previous nightly build

• e3337876903cbfa53231407f39a58f1ec53b03d5 feat(storage): Provide table id filter for sstable_iterator (#19582)
• c2ee827eeed2912a1c94ceca657d3fa36da43851 refactor(meta): generalize replace table to replace job (#18935)
• 70d0275561601f5ad020e2e4e373b8675c923d06 feat: remove flink mock sink from Java connector codebase (#19653)
• d60649262c4fa4ce841124e8ee75b24175923b5e fix(storage): fix the atomicity problem after split/merge operation (#19625)
• 78e39a8b517561a9054a5d422b74a46df5c85ca4 chore(dep): switch foyer to v0.13.0 release version (#19648)
• e9e3b24442cc55755c826ec9264fff909e553ef7 chore: set recursion_limit for sink bench crate (#19651)
• a41866fcd3b6e722e19355c60a839b76319f6301 feat(metrics): add internal latency of actors, MVs and sinks (#19639)
• c3750ebbbe80cd979ff236d621acb48e3c8f7e16 chore(deps): Bump pprof from 0.13.0 to 0.14.0 (#19340)
• 84566dc586c8c2e4d03971180a77427ab0b512c7 feat: make Nats jetstream Source work in parallel (#19529)
• 563c70fda37b83d44415dc448416ecaaca85058f refactor(meta): unify DdlType and StreamingJobType (#19630)
• 2c0d286ddb2ef25e722dc330550eec938676d2fd fix: bind under schema path for function, connection, subscription to avoid misjudgment of not found error (#19640)
• e9c15eb55cc246097eff55b7624124d45167343b chore(deps): Bump bytes from 1.7.2 to 1.9.0 (#19626)
• 69158c47d16d88569a3ef3adf061cfa2f0721211 feat: telemetry for connection create and ref (#19638)
• 6f2d0c33848871a5f24b2537e0c713fc66afcf96 feat(frontend): support alter source pause/resume (#19636)
• 845ed68183de5cb63525417063eb4b30064b9707 feat: add rw_internal_table_info to identity which streaming job the internal table belongs (#19642)

nightly-20241202

compare changes with previous nightly build

• 396099459bb6e456539bb94def7a68aff856db15 feat(meta): support reload runtime info of single database (#19597)
• e2a140e7d1a7d5035ca0d6af19c72d427ade6c3d refactor(meta): extract common method to inject initial barrier (#19622)
• 303ab11cff3f0b0559fa5fe48a2abd788b601935 fix(hummock): fix hummock_trace panic with log store (#19606)
• 4dddbac1fd76ec5fc6eb709daf23083a778d7f53 refactor(iceberg): support eq delete merge on read with hash join (#19126)
• 32e98c00f967d578e94958c4de5380af0782edca feat(ctl): migrate legacy object with new prefix strategy (#19593)
• a6cd45fbcf9ced630bee4c7a3ba959ed0382fdda doc(streaming): Add docs for degree table in hash join (#19628)

nightly-20241201

compare changes with previous nightly build

• 22b11ef413b357ab736a7986907de541c1cd710b fix(meta): limit max orm param (#19617)
• d6cf971164d27296e2aec08eedee3fcdc023a520 refactor: further clarify types of tables (#19539)
• 3b3a1c5f1dd016485bd2ecc1cc9b0669ca46b146 refactor(barrier): explicitly maintain database barrier state separately in local barrier manager (#19556)
• ea2f775f5036b8bbe221d51953b57307465435b0 chore: update docker image to v2.1.0-rc.2 (#19618)
• 4fab947575fab3a37f4bd6363bf0f01bfa648646 chore(deps): Bump quick-xml from 0.35.0 to 0.36.1 (#19420)
• fbfa35a0aeb1668955f8b7019e0481f35ce0908d fix(optimizer): LogicalDedup distinct columns were not mapped when pruning columns (#19611)
• 2ae70563bd23667d72eb0f15aacd8d151f8b42a0 fix: consider schema when commenting on a column or table (#19612)
• 275f10d78eb53f735dc7e732f9c5e27f4a6af513 fix: bypass cluster limits by default in debug build (#19610)
• 01b703dc4b996799d1565f408a57ddea2159c4d5 feat: Connection for Kafka source & sink (#19270)
• 5bc4d0c844a7d4a0e6b00159ce8fd39339f0eaad feat(frontend): support explain with graphviz visualization in dashboard (#19558)

nightly-20241128

compare changes with previous nightly build

• 4b1cb92e5ce37c72139f933b525ed9c41aec575f refactor(storage): use Arc in HummockVersion (#19535)
• c00fe35fbf1515d243c5d7c233c68b4b1638b7bd fix(source): resolve avro Ref during avro_schema_to_column_descs without hack (#19601)
• b8c70c156bb3f224293edd273a03849bb6468834 feat(frontend): support dot output for distsql (#19552)
• 521f67493e192c6c11e6eb1d241ac7b3b140a862 refactor(ci): introduce meta backend agnostic e2e tests (#19568)
• 32c44babc511f3dd3c3678c9cbe511d8108dceba feat(source): support webhook source table (#19272)
• 15ba09dcb6808fd75a025a05603efc265c86b7a6 feat(connector): support postgres_sink in rust (#19328)
• 42f1e1b1698c2f2a8bf617c824325293a6bb1f41 test: add tpch/nexmark tests for shared kafka source (#19589)
• ba225b21dbdf6f24d3f3086c03a18859b9949d73 fix(planner): correctly handle hidden columns for SourceBackfill (#19578)
• 9643ff9097fd7ea6a1d0ed47c1f0ce15d2f9e9f0 fix(secret): the name of a secret in the catalog should not contains the schema name (#19584)

nightly-20241127

compare changes with previous nightly build

• c45327bae82869699328a47e70919d38e783df2d fix(meta): locate upstream source fragment correctly for SourceBackfill (#19564)
• 200ed05c35ce57a178c30def0380eba7d4e5dc4a feat(meta): add a rpc to fetch meta store endpoint (#19594)
• ac5cb4036ed5d1d28483ab296a3a8811f6addc90 feat(barrier): maintain per database scheduled command queue (#19592)
• c121fa7ff0d51b668616c19638db028b1f56e2fa refactor(metrics): adapt to the new foyer metrics framework (#19580)
• 3faa0bf346665379bfa2e1295c71aee9f0e29c21 refactor(common): consolidate StructType constructors (Part 2/2) (#19570)
• 066978338838a0a1f98f93bc440d8ac30ab46126 feat: add builtin function timezone (#19524)
• ba76431f6bee416ee05ed144df58eefe516c9792 feat(batch): add batch_expr_strict_mode to ignore expression error in batch query (#19562)
• d17b4027ccbcf18304a9b9ee78fb819a514ca80a feat(connector): support more parquet data types (#19561)
• 641cf65c4218d632cf17ac155eb91e65cfb9ff17 fix: add RW_HEAP_PROFILING_DIR for frontend (#19323)
• 21ea2bc4d92714d867d44d5d7c6efb37a6e4c3d4 feat(risedev): support configuring meta store by url from env var (#19560)
• 507085657ca54769cb83aafe2e5d6278b9a985eb feat(secret): alter secret in catalog (#19495)

nightly-20241126

compare changes with previous nightly build

• 5d6aaf47710b449cde54081e987ba8d457adb6c6 feat(sink): use mysql dialect for jdbc url start with mariadb (#19523)
• 759ea19eda7cd72afcacb186f79e4e17ed603669 feat(cdc): init default value for cdc table columns (#19354)
• fc639feff477c8b97fee13afcf1de2687a4032b0 perf(cache): reduce exclusive lock usage while dropping in-memory cache entry (#19436)
• 4752db6d820a1af4af0a188f07933504dd9625a0 fix: minor bug of to_binary_with_type of List (#19571)
• a3c2648f6fe751d2eb17be8a56e6b524c2310a6b chore(deps): Bump the all group across 1 directory with 52 updates (#19565)
• 184900ad9f4adad8504dafbfa86dbfd943dcc678 chore(sqlsmith): ignore sqrt with negative input error (#19566)
• 519c95db4c18a3741e3f6ab4334af99c6e552531 refactor(common): consolidate StructType constructors (Part 1/2: trivial ones) (#19569)
• 96c5431ed3866483217d3ff54d027ee2ed18bd72 refactor(meta): rename replace_table to replace_stream_job where appropriate (#19537)
• a58b142d23502e38a902256d3bc7800909e2bca8 chore: bump iceberg version (java) (#19423)
• b34f4fdf4c711c614a5985ce3aa413a4bd65d2e8 refactor(frontend): better cast error message for UPDATE setting to subquery (#19555)
• 6cae4d3fcffb931aa799764494a3f8f96e66c9e8 refactor(meta): provide better message for meta store migration failure (#19551)
• fcea1bd2200b1bc1de5157fc6b8465e50839a6cb chore(ci): increase test timeout for deterministic tests (#19532)

nightly-20241125

compare changes with previous nightly build

• 9bc0b1fb017d0ee0d04249dc01a5f3c5c428c27a feat: introduce node label (#19153)
• 49398f602e8f01c82b8106017647bcdaa2cc5bbc feat(storage): reclaim table throughput statistics (#19135)
• 420b60b591f4da6a6de1eea4966833667f010cd3 test(pgwire): complete the test cases to cover all data types (#19520)
• 9abd62a7a63446c345310c1aeeba994c86cf20e8 refactor(optimizer): record error contexts when casting composite types (#19449)

nightly-20241124

compare changes with previous nightly build

• b4bca5724e7acb9c39919aaa188ed699a10eb1f4 doc(dev-guide): note good first issue (#19543)
• 236e7617aebd55bd61b9ceb43cb2e1fd9b6d59e8 fix: upgrade setuptools because of vulnerablility (#19540)

nightly-20241122

compare changes with previous nightly build

• bc06ffdb782ea5a9f11d19825f0f61c1723991c1 feat(compaction): Limit the size of the new overlapping level (#19277)
• fd5d0912fdebd031873651041866df6a2b8651b6 fix: when target is table, the sink downstream pk is not set (#19515)
• 2064b5b145cda3c0b04061a3e819bf68b32d0333 fix: memory allocation of standalone mode (#19477)
• 09a8ff3a59650df3ec1afa08c7b8579d84a1700b refactor: clarify the meaning of table in TableCatalog and TableFragments (#19510)
• ca5f13a49a54fe861ad499e05bf3a7c260c76ca8 fix(sqlparser): array of struct of struct (#19483)
• 37e108c772ab1a6b3acd1e7e015b33191f3130d9 feat(meta): dependency check for drop function (#19399)

nightly-20241121

compare changes with previous nightly build

• cfa521dcccfdb2763da050ed6a7f79cb39f63f9f test: re-enable backwards compat tests (#19521)
• e544337a477234656f7c580ddb20f40a5c44ecf8 feat(iceberg): support rest authentication (#19406)
• a771dab7465202acd7dab99f5de441a71a82eecd feat(frontend): output explain result as graphviz dot format (#19446)
• 075c90d7e3b2953283e188bd636b0ea05b268314 refactor: rename dummy table id to tmp table id (#19509)
• f00385991226042ace31eff9bf7bc3ba6d3a0366 fix(compactor): Fix an unexpected termination of shared compactor (#19505)
• ca38177324864c9f82372189ef8265d30e7b9aec perf(topn): compact the (Group)TopN output to avoid amplification (#19451)
• 8d3ef48fb0f92b96bc2e0a66c89eaaa36df80815 chore: don't treat subscription as streaming job in system catalog (#19476)
• d5fd260882d56f902d848d5f33a8a1fe5ade7f5e refactor: update configuration (#19501)
• bd61bc6e1795b703ca5a45473161804f77955ffe chore: add telemetry reminder if meeting type/feature (#19359)
• bae490aab2ae895b4a76d1d2a7c5b0cc63169e99 refactor(meta): use correct type for the pb counterpart (#19500)
• ea3e90932f8b8f523f183140ed878e69971e462e feat(streaming): decouple memory manager tick interval with barrier (#19494)
• 1eb4ce04d47e4332d1f0c098306790b9ef2d0e8a feat(expr): add rw_epoch_to_ts (#19473)
• ad1d929a3298a4825c548117db4b48e425005de1 refactor(meta): proactively GC more stale objects. (#19474)
• 3fdd6a5cc0c0789118ab726cbb776bfefb41a557 feat: add rw_rate_limit system catalog (#19466)

nightly-20241120

compare changes with previous nightly build

• fe69ce3093b33e16e8f1682eddeac28789190764 feat: support backfill_rate_limit for source backfill (#19445)
• b9c3f709cc927682738ffc6fee1528be04012cd6 fix: fix potential data loss for shared source (#19443)
• 34bb3cb0a19edddd629a9a9eadc3ed7f57256dd3 fix(iceberg): fix iceberg parquet file size in bytes statistics (#19471)
• a0b65fd618483e38ed351a4d691639ce65c8fa6c feat(pgwire): support struct type in extended mode (#19450)
• c1e8f9a2d177d8a9c9733e06f1c9ce77517582d3 fix(source): REFRESH SCHEMA shall keep INCLUDE pk for UPSERT (#19384)
• 5f1a59b000b92a2c81163c520564827a0a798ff5 refactor(frontend): rework UPDATE to support subqueries (#19402)
• c1162ab701d3df0b8a0a3f0bc8564508b0090330 test: fix test cases of batch/types (#19441)
• c8e96b9634dc38bcd5bbea74e23aa7ef1974922c refactor(barrier): decouple barrier collect and sync in local barrier manager (#19393)
• b37f048e33722fc922d8bd3adce6c45fdeae42af fix: Ensure non-negative variance in stddev calculations (#19448)
• 185243618b23fea634237a70a438e3c3c7da7e2d chore: more logs for offline scaling (#19407)
• c325f42060679bff0dfde4aa5bd63529311d1128 fix(optimizer): enforce input of BatchMaxOneRow to be singleton (#19452)
• 5b224fab428f33e41146e7df157b863402ce607c chore(deps): Bump async-nats from 0.37.0 to 0.38.0 (#19465)
• 5b9cf248a15ce1af4381809d0a3bdc464c4d3d31 fix(meta): remove custom serde (#19463)
• 1c29c810748296f65db0b3c7a03ab221f6312a3a fix(iceberg): Bump Icelake version to fix null metadata_log (#19397)
• a54a04507b4d6f300f15d8dd36519257b6e4ce1c feat: Remove cycle check for sink into table in frontend (#19442)

nightly-20241119

compare changes with previous nightly build

• 72e5ad3e0672687439c8a33688c7acc6ded540fa refactor(connector): extract the common logic of file_scan tvf and parquet source (#19398)
• a05cf34aea19fb407b6c3ad0b5faeb450c7cf233 fix: alter FS source rate_limit is not handled correctly (#19438)
• f89ae893522c3211dbda0eaefcff3dec21b6dec8 feat(optimizer): allow returning error if a rule fails unrecoverably (#19381)
• 9e27e6270c36bda30d61257f84ddc9a939ee51a1 fix: request for topic spec metadata to narrow privilege required (#19421)
• 8f552d859af9f6993354fd1b04b74c15baa32c4a refactor: allow modification of the query mode (#19424)
• e26292b558584b4a66614625e5c60ef294cdee34 feat(pgwire): support 1-dimension array in extended mode (#19432)
• f12f4e51146cf6b42ce88f5fab3d2b7a8ab09998 feat(storage): support sync on multiple partial graph (#19431)
• 7ba6650c926b80c8f50f52837e33fa8f76373329 feat(batch): support extended declare query cursor (#19043)
• f5537d955f102e382da5456a44911519c978ade6 refactor: remove unexpected memory watermark update (#19366)
• 7a50fec7bc0b1a743da1c829d958c046edcfbd59 test: add more planner tests for explain xml, json, yaml (#19430)

nightly-20241118

compare changes with previous nightly build

• 0b223b4c2f569e749413620287d1655410041d06 fix(test): incorrect flaky madsim initial sink parallelism (#19419)
• d2c64465386e19d777952f53677d64816a7de6bb chore(deps): Bump sqllogictest from 0.22.0 to 0.23.0 (#19417)
• 88f9efa58c1366dbce901749c53febf583f0c0b1 fix(starrocks): fix starrocks coordinator incorrect try_join_all on join handle (#19412)
• 084fff883b2c2470754c2745b14afaa6500e6c46 feat(batch): use get_keyed_row to implement point get for _rw_timestamp (#19415)
• a4d96ec2edf7d05fa25fb567ff463b6045fc389f chore: enable dependabot for java (#19212)
• ca54fff9b8a18f953a3b570d6688166c24c12cad test: vnode count chaos test in e2e madsim (#19250)
• 9ab21987eb25c9afee660180d08137481d458b9e fix: approximately update table stats for vnode watermark reclaim (#19370)
• 43730aa0734f54aef96888af03c60fe72c9ca2dd fix(direct-cdc): improve type match for pg-cdc (#19409)
• 94154c03ff7d80076b9e4e10da416ce85a620ab4 refactor(udf): remove function_type field previously used by Deno UDF (#19404)

nightly-20241117

compare changes with previous nightly build

• 0519b3fdd5ab0b94d39ee6989050272e6f3e5c78 chore: bump java dependencies (#19208)
• 7a9679bb310b06d6c9b9be8b779ed4c01ef53898 feat(frontend): support xml, yaml format in explain output (#19400)

nightly-20241116

compare changes with previous nightly build

• f29c971146bc25cc567cedaf6bd4ae7e0228a2bf feat: return full key in state store get_keyed_row interface (#19345)
• 4085f560d672cbbaa125a0f715a17aa381224a8c feat(memory): Separate total memory configurations for FE and Compactor (#19372)
• ac6cb38ec55212d65fa37c59b5687d32ce88b6f4 fix(streaming): correctly skip problematic table-function input (#19353)
• c7a4a3943691b2f9b48eddd63741f561a6dcb145 feat(batch): support system column _rw_timestamp for tables (#19232)
• 310b1c3bc7c0edc1c0ff30f81f4fe282c64c5baf chore(deps): Bump governor from 0.6.0 to 0.6.3 (#19186)
• 4d3f6339a53f6977c12be27d529ab0408a24dee5 chore: split java connector tests (#19382)
• df07ffbce4a77f7f0a6335acfae99c0b7e7cccf0 feat(dashboard): show cluster version (#19388)

nightly-20241114

compare changes with previous nightly build

• c92069481d2aba5a77ffb673b48c7e27b5a698a6 refactor(barrier): add control request to explicitly create partial graph (#19383)
• daed1f243da61ad8132f6f14a702dce6936b825d feat(iceberg): make wrehouse.path optional for iceberg rest catalog (#19380)
• c1435dd7bc1f5281bee14a2845e925c9f5e76347 fix: fix system acl column type to support \l command (#19379)
• fa99969ebbfbd46476455d7dc33a553aa18ab68a fix(pg-cdc): write transactional WAL message as heartbeat action (#19385)
• 1a97b4cccf266594477e487c063de76f2a085944 feat(frontend): support iceberg predicate pushdown (#19228)
• 9aded7178aefb5436009b4cebd5e8f46b59429b2 refactor(meta): merge drop and cancel streaming job command (#19369)
• c5c2119901fa0ff17fd56a878293359f770b1f14 fix(ci): delete unused python install before embedded UDF tests (#19386)
• a0ff192290dd767d9f8677709c3b218a4fad2773 chore(deps): Bump cookie and express in /dashboard (#19324)

nightly-20241113

compare changes with previous nightly build

• 1aed314c2d8944da22b6d9c3c4d07c7683c2a98b feat(storage): support clear the shared buffer of a subset of tables (#19293)
• c626c59da464b98f3d498030060741dcde2497c9 chore(iceberg): suppress java S3FileIO warning (#19371)
• 0ddb6eb17b514aef65f1625ec46087ca212711c1 refactor(barrier): not cache actor failure in local barrier worker (#19367)
• c21a7710fe014607a728c0a94d489291874e3162 fix(agg): fix first_value and last_value to not ignore NULLs (#19332)
• c93b92bab4c3c5d6ae2491de384dda685e9df88f feat(compaction): Introducing configuration for compactor preload SstableMeta (#19218)
• c0b5e641f0a569a8551fa2f13e1e51c7daee1fd2 fix(binder): report error on update query with subquery on the set clause (#19305)
• f0ece4520ff33d07a7ae55af9621d8af69415a4d fix(iceberg): change iceberg rest catalog default implementation to java (#19357)

nightly-20241112

compare changes with previous nightly build

• 98aa20b886a63a58e602cda41c1c161a66a9fc06 fix: background mview without backfilling should not tracked in barrier runtime info (#19355)
• 337235d98785f5c512b39889e87e2e7b1535536e refactor(stream): add more assertion on update op handling in hash dispatcher (#19347)
• 022c0a40c7063a40f42755f61d73b4ef5405ab42 refactor(meta): reorganize code of global barrier manager (part 2) (#19351)
• fe6550975e1b471eb37bb37b70427876a1d1aa1c feat(stream): wait committed epoch in state table init_epoch (#19223)
• 4dea583711844885cdc8abfd3e19c401119d29d4 fix: table with fs connector cannot work with rate limit (#19338)
• 12fbf121003e2d69feb36a0da3d02edd1944771b chore: enable parallel frontend for rustc (#19331)
• b14188179bef6b2b9fee26f987111264a0f6ddc1 refactor(source): enhance parquet file source (#19221)
• 056ab8b5fecf918253fbf413d29e153a9e8ba8a2 chore(deps): Bump serial_test from 3.1.0 to 3.2.0 (#19341)
• 73108d527fd0c0a834a83009d865709183fd4398 feat(meta): extend streaming scheduler to be aware of vnode count (#19267)

nightly-20241111

compare changes with previous nightly build

• 3c2e81b53a8411451a5f999ca3313654d98c9586 refactor(meta): reorganize code of global barrier manager (part 1) (#19334)
• b689615512b773485244a130f22e771142461d9b refactor(meta): list and delete SST objects in meta node (#19329)
• 71146a6bf5931e68d5f51917ac72b7070ad0df3f feat(meta): support database checkpoint isolation (#19173)
• 2c4a0002f7b812cc6b3ec057654401e88db5284f chore(ci): serial-run source_inline/kafka/avro/glue.slt (#19318)

nightly-20241110

compare changes with previous nightly build

• 82b782547c0cefe48263d0fc69047b2fcb71cac3 chore(deps): Bump rustls-native-certs from 0.7.0 to 0.8.0 (#18782)
• 55c4644b70c4a74145b7379bf68ecba21ca596b5 chore(deps): Bump fs-err from 2.11.0 to 3.0.0 (#19235)
• c673db66486e5415d2ea2b124c6195217c08caa7 chore(deps): Bump bytes from 1.7.2 to 1.8.0 (#19185)

nightly-20241108

compare changes with previous nightly build

• c65834074c8334d835a03333362c0f2eb67a0802 refactor: some CDC table's code (#19255)
• 99703d17a207cb7cf925fa24edf47a149e0d74e2 test: fix flaky test two_phase_approx_percentile_merge_stateless_agg.slt due to rounding (#19292)
• 82a80631aa94b45284ff68c07e61c474a5c6132d fix(pg-cdc): fix real/float data type mapping (#19300)
• cc085ebd98a4c224c28277c722914af9e8650ffd fix: Single fragment scaling is too conservative (#19259)
• 31d4406f1c18fb9794b6aa3218d4078511226248 fix(meta): set auto_increment when restoring metadata (#19303)
• ea01a37b3e64c38ffb2d6114f4621159d176d4de feat: support key encode bytes for sink (#19243)

nightly-20241107

compare changes with previous nightly build

• e6e1ca9eb987e6abb9a7f7e8f53163f9f02b5397 refactor(sink): Use s3 sink to replace the original snowflake backend implementation (#18996)
• 218e63e30821fbb500cdaa4edf8ff50078f7f763 fix: fix grant and revoke privileges for internal table (#19249)

nightly-20241106

compare changes with previous nightly build

• 9a32e75f227f837a6f16fb3afe159c1f8af2ad5b feat(compaction): default new compaction group for new table (#19080)

nightly-20241105

compare changes with previous nightly build

• a945f52de4633a6f87f62309aa3cdbbebaff8d11 fix: none-shared source to secret dependence (#19263)
• 0f78e6a3c3ae5eba074209a7b2a8b3d6fb4fd63f fix: support Kinesis with include timestamp (#19211)
• 9d83354629bde178ba31e5fbea4ebc51c814fecf chore(deps): Bump speedate from 0.14.0 to 0.15.0 (#19257)
• 46ad59877c7d5115d5285e3e24ac6590b1478b28 fix(meta): record delta for new compaction group (#19253)
• b796d0ddb02112ddedf2a4ae755924865fbf0ba5 refactor(common): simplify chunk iteration with filter_by_bitmap (#19254)

nightly-20241104

compare changes with previous nightly build

• bb0d786b39c9eef9dbf1e16814e8d12e46b8a676 fix(connector): handle backwards compat parsing for boolean datatype (#19251)
• 946b50089673b026720905e5cccece0485192553 fix: fix index create privilege check and ensure consistency between the owner and its associated table (#19252)
• a9f8945658dc2044f8e2f4eef5037ec6b7f42330 refactor(meta): simplify stream job table/source id assignment (#19171)
• 7690f674c48a7cb824dcd44fe3940a03d42e54ed chore(deps): update sqlx to 0.8.2, sea-orm to 1.1, sea-orm-migration to 1.1 (#19145)
• 6b10d92ecbacec65f361823799ef6470630ee316 fix(meta): address bugs for mysql backend & enable e2e tests (#19156)
• f3e9a3be1952c5f0e7fcb9dc595a99a25a6528be refactor(test): reorganize file connector CI tests (#19230)
• d3d2969036c1f423c04dc76488040d8d01364dc3 feat: support swap rename syntax for table/mview/view/source/sink and subscription (#19172)
• bb8f94dbdf354b2e866d92025ee62e93dd479167 fix(pg-cdc): fix ssl.mode value to debezium (#19239)
• f559d3468a16f0a15e429dfb55276b1a07c814e3 chore(meta): bump sea-orm to 1.0 (#19207)
• 271faac7e8eb71c756746897b1d1db6e86d22ad3 fix(sink): Fix excessive number of starrocks txn (#19217)

nightly-20241103

compare changes with previous nightly build

• e7e4a2c9d9f288f6fe89c92e3fcdbfe3d6d36ca6 feat: share kafka client on meta (#19058)
• 3a8d5a8427b1d0a3a746c302527fb51e8815a1ee fix(mysql-cdc): fix mysql enum type default value (#19192)
• d9ee9b450de59dba6c0310e9eb3cd4d73b701f40 feat(frontend): plan nested loop temporal join (#19201)

nightly-20241101

compare changes with previous nightly build

• b39c9af88fd1bbe05df7e1ece506741eb489471b perf(over window): shortcut implementation for first_value and last_value in over window (#19233)
• 56ab1fac042c1f42430c6febdc32168104bd13f9 feat(iceberg): bump iceberg-rust (#19206)
• ca845040c98ef949d0fd91574b5e262306fc923d feat(meta): ensure each command is applied to exactly one database (#19076)
• 9dd7fba8671193ec9fa9ae8ffee6dfeff1ce87ba chore: fix nextest warning (#19231)
• 051fbad84228006fd6e76d9737ad1e06a3367d9e refactor: move server.id for MySQL CDC from meta to fe (#19169)
• c053c2a2cf6b7b53c5de13f9989adaebd50c15c2 fix(watermark): stop generating watermark messages when stream is paused (#19199)
• 3b8b913f2e6baa922ce8a20200ae65ec3a9cebe9 feat(batch): support mysql_query for mysql batch ingestion (#19071)
• 9671fa36279e6f2663ea60287b47e7654f66b042 perf(over window): O(1) impl for delta_btree_map::CursorWithDelta::move_next (#19214)
• 03e9c954b0c7e6de061516842370422b3cff35d9 fix: unexpected warning on undefined-field-padding-null error (#19002)
• 127a44b4bd581683e67533bd2ade1d5da3c10179 chore(ci): move fmt and sort checks from build step to check step (#19209)
• 3a5a51d38cdfa8294b904f194ffe865bb30a9589 chore(deps): Bump tower-http from 0.5.2 to 0.6.1 (#19216)

nightly-20241031

compare changes with previous nightly build

• de7b7cd2b66ebe8bacf654f3e488cab2f3c241e8 chore: upgrade clippy utils to a version on the main branch (#19219)
• 6defe4c92f93cac731d09883b7deedcc24cc985e feat(compaction): per table vnode on compactor side (#19059)
• 8457de2cff4d731cf6fcd94ec9ad7605f442c3cf chore: bump spark version (#19200)
• 8a8b36879872d8462c0f17bcbcff516c818c9f8f chore: bump toolchain to nightly-2024-10-11 (#19170)
• 07c7bc357017b383fe6033818151ffc0860c8d2c refactor: remove private link related connection (#18975)

nightly-20241030

compare changes with previous nightly build

• a36c3172230d2aaf5632f60a4a31f0958d11f98f chore(deps): Bump fancy-regex from 0.13.0 to 0.14.0 (#19184)
• a3793c8f82bc8880cd4595bf0c54ad67283280aa chore: update image version to v2.1.0-rc.1 (#19190)
• be73a02e29a37fa720071123eeefa15f735c7d1f fix(optimizer): fix lateral subquery with cte (#19162)
• 18bf666906760907b028389fd3dd9c1b63e5df0b feat(dashboard): list streaming job from streaming_job table & show info (#19134)
• c706e8dab3815bd6c8f2297d49141e5da2065227 fix: include schema for view name in rw_relations show create view (#19180)
• 68b470e4d3c8a70cb8e7653fc7a1dae217417889 refactor: improve assertion message (#19191)
• 0e844fe43bc1602fe05b77b1962a259f8bdba7d5 fix(source): pause source correctly (#19148)
• b6051d84c7e7d5dc4bf3994d1ea95ca93166723f refactor: rename ConnectorSchema/SourceSchema to FormatEncode (#19174)

nightly-20241029

compare changes with previous nightly build

• 98baacd19f521cd75ec7dc9cbdde903e7ef01d12 feat(meta): maintain per database job fragment info in barrier manager (#19062)
• 4c05b7ab0140ca707fed2cfcc4d0c1656b13fe7d fix: reject invalid storage cache capacities configs (#19016)
• 51a70d8696e46a905893272dc7dcc4a5f503f0c3 feat(ci): run e2e tests against postgres/mysql backends on demand (#19152)
• 55d42c865715c808500d7b74797812ad7061c43c feat(dashboard): show vnode count for mv/table/internal-table (#19133)
• 661939a3491fe40807dbb84593e6ef3157b74b91 feat(connector): Add topic to mqtt additional columns (#19017)
• 193e93fd8d9f9dbae717fe6a5b411e7f33382f27 ci: fix scout in docker pipeline (#19164)
• 31a650521dae6bbb83c87c37db7795c60e2a0a0a chore(dashboard): remove unused function (#19158)
• d39ac9c085ec60cc202902322e2e200bcbc5e635 chore(deps): Bump simd-json from 0.13.3 to 0.14.2 (#19118)
• e24e19b9ba7c9f133eec21e3889242e31b48b937 chore(deps): Bump elasticsearch from 8.5.0-alpha.1 to 8.15.0-alpha.1 (#18908)
• d44814ddba49a768c72578c785a8681e70b5e3a6 chore(deps): Bump notify from 6.1.1 to 7.0.0 (#19137)
• b0dad757e8a3de951ef03ce83feffbb3a1b43c48 perf(over window): skip remaining affected rows when rank is not changed (#18950)
• 0935cf2dff746165b03dc5aa15a942f2934ce7a2 refactor(meta): decouple global barrier worker from context with trait (#19034)

nightly-20241028

compare changes with previous nightly build

• 0d013eab1bba8cd86c575057e165586262362636 perf(over window): pass through Updates not affecting window function outputs (#19056)
• 9f12b10b34fd73a7b3b984c7d1fbeb0b721e583f fix(meta): fix vnode count data type when querying system table rw_fragments (#19155)
• c582c48d88565e1c692d47225c63b54a9dd0a6b8 feat(compaction): introduce auto split and merge policy and config (#18806)
• 16107fde0f6ee184546d2510bffc460625f34a14 perf(over window): avoid recompute information about window frames (#19036)
• 0d5ffe77800072da0a674f8c6f2176de7625c573 fix(meta): fix migration query under postgres meta store for #18753 (#19151)
• aea38863c7acf7f00a921319f5f95751ed09ffb1 chore(madsim,deps): use patched sqlx managed by the madsim-rs organization instead of a personal repository (#19130)
• 81703a9d1eea53b0e22ee42af2c20fcf8c25516e fix(ci): fix docker scount notification (#19143)
• e99ad679863f4c5c6fbcb605739b8a9fe1e3a02c feat(stream): merge stream chunks at MergeExecutor (#17968)
• 6263ea63592bcfa159082658969a1a86585ed473 refactor(meta): reorganize global barrier manager field (#18920)
• f0988c41382e100da4cb906efeb0a48a2493a9ec fix(runtime): set default crypto provider (#19136)

nightly-20241027

compare changes with previous nightly build

• a176aced3c047d780adc8f3981a4921faa526255 fix(ci): fix standalone tests startup (#19067)

nightly-20241026

compare changes with previous nightly build

• 32666f2183f49585c69d8690afec796e14e5ab78 refactor(test): unify sr_register usage for avro and protobuf (#19107)

nightly-20241025

compare changes with previous nightly build

• d5d0283650fafef0eb2592c273a1d0a95a04f35c feat(ci): add docker scout to check vulnerabilities in docker pipeline (#19128)
• 70d1d4264cd4e72700a9a1334fcc6bee5df10c52 chore: enable integrity check before offline scaling (#19104)
• ef22d0e032d8d102ae314ea209917bfc1ad5f69b fix(pg-cdc): handle citext data type and other string compatible data types (#19103)
• 58ecec217dab253284beac1419da878928e296e9 refactor: use 1 for vnode count of singletons (#18753)
• fcb6c358d0b96c7fa41b9fdafec9e52f01cc19a1 feat(subscription): support cusor order (#18801)
• a89abcf014f0438e0cea0880534c9521eb7e4a6f feat(mysql-cdc): support mysql source capture multiple databases (#19038)
• 083d66d704182453ee3bc71c66868a6f1f9e246e feat(source): column pruning for parquet file source (#18967)
• ca75c21701f7098e82ad2124d356acb0935fb74f feat(iceberg): support s3 path style for iceberg source (#19111)
• ad0bf17d33860d642190f7f6a8c4319c614e5d2f fix(meta): align job's max parallelism while replacing table (#19052)
• e22462d848d6b04ecab99e6af67016a8bbc38d0f feat(frontend): allow users to confirm if background_ddl is enabled for a materialized view (#19085)
• 7e8bf1c72e19bc8c3d3c25f153597a6507d76fdb chore(deps): Bump tonic from 0.12.2 to 0.12.3 (#19077)
• 6e57ac32deb4dbe0e634c71d7c418fa837667783 chore(deps): Bump org.eclipse.jetty:jetty-http from 12.0.10 to 12.0.12 in /java (#18911)
• abdeca68bd451a2cc230f9d198c38f51617c629d chore(deps): Bump tinyvec from 1.6.0 to 1.8.0 (#18906)
• deb7eb54121166119baa2ab923ac47af32f11dc3 feat(stream): enable background ddl for StreamNow, StreamSource (#19099)

nightly-20241024

compare changes with previous nightly build

• 65ed113a3c52c0fe079c9fa2751fda48d92ead49 feat: expose connection options for sql meta store (#19040)
• 8cf092f55bcaa11c63e06370030a9efe09fc4fed fix(batch): warn when fragment mapping not found (#19105)
• 707a0d2c51afc87c47e0750c5f7a8086dbdeccf3 test(secret): use correct RW_SECRET_STORE_PRIVATE_KEY_HEX length in test (#19092)
• 73a3d9749154177a6b7c49cd5c99f19ef3e13fe9 fix(over window): fix table iter pk prefix to use deduped partition key (#19081)
• bdfd89214171104a8c31e9bf40cb64a2919d9d77 fix: remove redundant consumer_durable_name (#19090)

nightly-20241023

compare changes with previous nightly build

• e6f830b68f42022b8ee1d9e4f68836cddcb3086a fix(dashboard): show creating job edges in bp graph (#19066)
• fd975e025e2f9964970a35afb186a0e2e336493c fix(optimizer): apply join transpose rule (#19060)
• 3a4c4de29d3d1da55df186bc1124b0f221a89f82 fix(batch): use streaming api for postgres_query when fetching rows from upstream (#19064)
• db245612ba9c319e97c7105daf12587e1fc9809c doc(dev-guide): cargo binstall -> cargo install (#19054)
• 5ca0a739671e38e2f582937d53ab4941b2de0586 feat(frontend): support explain format json (#19041)

nightly-20241022

compare changes with previous nightly build

• 87b7164b8029e779c7ccd5f6a475376c491ad0dc feat: Add dashboard Backfill Snapshot Read Throughput(rows) by MV (#19018)
• b299fe9ff8ea2ad56d5e45ba12fa2d74891ee3f2 chore(ci): update notify for mqtt, iceberg tests + increase standalone test timeout (#19042)
• 8865c58b9ad6d543a41f66ecc212e87aea7b017f test: snapshot with option for sink into table (#19051)
• 4745119eb1640efa5cee364290ea44924bb513ad test: add planner test case for distinct on (#19048)
• 9b42fb05de7b04e0cbb5dfe87048b11f2e9179ee test: refactor pg tvf tests into inline tests (#19004)
• f637d0fbe14b1b02bae619774c5dfc745ee3e059 feat: improve progress msg in SHOW JOBS for source backfill (#18925)
• 0790a7312434d8491caede9df925bb666cd891c2 chore(deps): Bump tower from 0.4.13 to 0.5.0 (#18957)

nightly-20241021

compare changes with previous nightly build

• ff61c885fd873de3686111d2db3aaa61b99c0e75 refactor: distinguish between placeholder and compat for vnode count (#18976)
• 3e7fffd9850bd09d68dd87d0e1ca0033bf324430 feat: persist processed rows for SourceBackfill for SHOW JOBS (#18983)
• d17ce39433c8f0726ed9125dad0d3fca49203e07 fix(meta): rename changelog shall check self.from (#19030)
• 9bbf418a8136b26dd3f7e240abf683a192ce7010 refactor(meta): clarify the completeness of internal table catalogs (#18944)
• a04365a6c9cb3c27c44934a71232541a52d2dade refactor(meta): validate time_travel_retention_ms (#18979)
• 43544ac655d9ff0c1e284d456179faa675003fe3 fix(sqlparser): Display changelog with quoted identifiers (#19029)
• 23bdf31b996a790250be77268b7314cf6c994598 feat: add is_shared column to rw_sources (#19027)
• 1ca2ea7597bd3190bdf6495cbf20bf32d1b42b05 feat(over window): add over_window_accessed_entry_count to metric for better observability (#18943)
• 4dec2eaf45d56d1021b9b3371de770952042fe26 chore: add missing system columns in pg_class and pg_attribute (#19011)
• 120747f951fad0edc6bf75582d9a99e9ff2aa6a1 fix(metric): better agg state cache miss ratio metric (#19012)
• 2a71574417e468199550c2bc40829d45543f587e fix(iceberg): fix iceberg source position and all delete (#18932)
• 3c57ef8dd5d26f6eaedf42c169309a0751bcf5e6 feat: enable shared source in session variable by default, and add cluster-level config to disable (#18749)

nightly-20241020

compare changes with previous nightly build

• 96aeb52372543bf1cc5ab814e50210cb3fb1d898 test: use tempfile for rw metadata storage in deterministic tests (#18970)
• 5b93390125f5a78d72ba531df3cfce04b23688a1 chore(deps): unpatch madsim (#19013)
• aed4e6e979fbd15befa30a90c9f267c6d370c94f fix(common): fix conversion between Decimal to arrow Decimal128 (#19007)
• 60e124dba68ad8770e2909553cd5e4de29fa2f69 ci: parallelize e2e_test/source_inline (#18971)

nightly-20241018

compare changes with previous nightly build

• 14f67e9f6c07e03de114bd8b1f3cfb5b66ef8c65 fix(ci): ensure timeout steps also notify (#18962)
• 7f37a8e8621c48aac3c28063a297fee87e3d4e21 feat(frontend): support create database with owner (#19005)
• 73a4bb979f2b11624c46fd64d8d5534eddafc659 fix: changelog does not support fully qualified path (#18785)
• 3b161303f37fdc080756faab759455176eaa0d48 feat(batch): ensure user-facing context is always present for external system errors (#18982)
• 339ebf8119a8ebceae63dfe76f005c9b7793e71e fix(test): fix some flaky e2e source tests (#18989)
• ab526062690ea35b26de6eb0f70b0a9f9fd91e05 feat(meta): record Hummock GC history (#18946)
• 0226bd5933b753b95c79b084be6b92f6bd4992b6 fix(frontend): allow INCLUDE payload without other columns (#19003)
• c0d6af170770ca34635c467ac03f5ac8327c9b21 feat(meta): commit epoch in separate group delta (#18893)
• d6277fb78f287e433ccf8a473b3a30393b57cc9e feat: report sink failure event to event log (#18958)
• 87e2ebfd04b8bf9adfa6da6bf9e9c45fbcf85e33 feat(snapshot-backfill): control log store back pressure in backfill executor (#18798)
• b801234b7219858e9aa59c0463607e3744dcc70f fix(ci): change backfill test config (#18994)
• 20bd924b9a1091392ac5d04af798ca5a6e86fe75 fix(meta): read correct id field (#18988)
• d241a3228a40a508a640511c4f2332280817dfbc doc(dev-guide): minor improve testing guide (#18986)
• 75190a791bef7f50d45de7cbc5aa5f8fb0599010 chore(deps): Bump uuid from 1.10.0 to 1.11.0 (#18985)
• ee97c1d52a0a27cf6684fd082ebc4065407d71e0 fix(ci): add contents write permission (#18981)
• 9e4a6fdacafbf821835ecf76e8b9fa2c1b093b35 revert: we allow single quotation in title again (#18973)
• e1177e59c374111d2c82835be87f4961819d9610 fix(ci): use explicit permissions (#18980)

nightly-20241017

compare changes with previous nightly build

• 79ee46fdfab686486875713e1705d58cf0bd2da0 feat(ci): simplify the cherry-pick workflow (#18978)
• fb1edb60c7a1aaed794aad8fd1d93ad4255e04f8 chore(test): longer streaming_upload_attempt_timeout_ms for backfill test only (#18960)
• 9a6a7f9052d5679165ff57cc01417c742c95351c chore: cleanup v2 naming for sql metastore (#18941)
• c56294f16d30d9bdb0b8d4ea9901dc95319edf1d chore(ci): eliminate the progress output of wget (#18965)
• 5204ea4495ce77e5bbff00997c8155488867b5cc feat(sink): introduce batching strategy for file sink (#18472)
• bf36aad0a839ad1babaff778b376059d18362f86 refactor(compaction): Simplify task checking in group scheduling (#18884)
• 340dba6bea00e9acc475ea2a96a9b8742b7aa565 feat(postgres-cdc): support replicating Postgres schema change (#18760)
• 73fa688369b7b04062e546a2916fc2b646fbfc62 feat: support background DDL for MV on shared source (#18919)
• b55732f77060315da6e3e68fda0f99ed22106981 fix: prevent mqtt enumerator in dead loop if disconnected (#18961)
• 4ec4db1baeec1fcc1982a375d18b04ad3556dd3f chore(deps): Bump rustls-pemfile from 2.1.1 to 2.2.0 (#18956)
• 7920e3df88f453516885d9a7ce076f4afbba7800 feat(mssql-cdc): add ssl support for sqlserver-cdc (#18912)

nightly-20241016

compare changes with previous nightly build

• 360fe8adc5b65fda09ff2a87388b7102326ea190 fix(meta): fix contention (#18945)
• 0154e89cb33cd2e32a8113ce73c90c36ab2cf00a chore: update package version to 2.2.0-alpha (#18934)
• 6d35ac1eadd40ae81e89ef67806fbb11b87979fe refactor(storage): proactively prevent uncommitted SSTs from GC (#18882)
• b089d1ee3c98c15190e73513795e287cef7db2f9 fix(sqlparser): keyword after char shall not be consumed on error (#18921)
• a2b6f4feb4a739b91f8a61e3351b7bd18392bae1 chore: fix overwrite to append new migration when using generate cli (#18939)
• e2a04af13c56a49ac8057a758006b6c0f73d6afd fix(java-binding): use table committed epoch in demo query (#18937)
• b7f9da48d657962f38992c5961cd852ecd7972d5 refactor(source): handle INCLUDE header data type properly (#18821)
• 6e775bd1d0ea175d9ca9ad00a5bb177dadc5267e feat(ci): add cherry-pick job to release-2.1 (#18933)
• 82c124d57a7dd8fbffeb1fd7535c472e98573a32 chore(deps): Bump the aws group with 3 updates (#18905)
• 66021134eaa9ac60266ec38137fcfb55cccedc87 feat(meta): support commit epoch on multiple graphs at once (#18819)
• 959321025e972bff406349787c0702e224ae0128 feat(risedev): support launching system rw in risedev-dev (#18889)
• 04755fdc27aa9c0c44746b73a03dea3714a0526e fix(cdc): fix the filter expression of CdcFilter executor (#18868)
• dd4b1d4fb37ed2ba2ff40309c7234640666f3c56 chore(deps): Bump rustls-pki-types from 1.7.0 to 1.10.0 (#18923)
• d997ebe842a4c9079b5a438a6398f9b9d357fc64 refactor: impl ack and migrate to durable consumer for Nats (#18873)
• 6863a24b86085e9ef6cc87fc4ac8d396602c735d test: add recovery in nexmark q4 sim test (#18577)
• a8b83e38128ed426e5faf2432a4caf36cc8889c5 feat(stream agg): add agg state cache miss ratio metric (#18900)
• 5d914bab1a74eb3361575a57d3142627ae64a8cb chore(deps): Bump opensearch from 2.2.0 to 2.3.0 (#18922)
• 890ec7a429b7809fd72199ecd1ed2f9152f290d9 feat: try add integrity check before offline scale (#18901)
• efbbc7dd7c0a72f0c7341a594eb4960db84f13c5 refactor(test): move some tests from source_legacy to source_inline (#18894)

nightly-20241015

compare changes with previous nightly build

• 2af00e81d826ed4f5198e429b5803bfa3dc29652 fix(storage): purge dropped columns for sql backend (#18914)
• 6463815b8175621c43db257179d1ef6774e43acf feat: deprecate etcd meta store (#18621)
• 4cf3a4842cf64f3206af9584c0c34928abeeaec8 chore(deps): Bump serde_with from 3.8.0 to 3.8.1 (#18910)
• 08fe4115ac5c1429de9b9723ee1a144e095aeceb doc: Create SECURITY.md (#18917)
• 88c66475e17f6227a3776e06a8a7be9127a8c667 feat(log): change opendal and reqsign log level to INFO (#18913)
• e2712901d4991ed8e851f90021290e80516e40dc feat: allow nats additional column offset and payload (#18897)
• df8840b8cc7ffbbe77210bc4a50ae5f6fa273e50 chore(deps): Bump libtest-mimic from 0.7.0 to 0.8.1 (#18909)
• cfc3f84593da8d8805583bc4d003661ff02ba85c chore(deps): Bump tempfile from 3.10.0 to 3.13.0 (#18772)
• adc48b579905be0fefafb0064cdf3a50ffe10631 chore(deps): Bump commons-io:commons-io from 2.11.0 to 2.14.0 in /java (#18791)
• a6f2b88b1bd81e7216fdb85035cc2ad9d0288c95 fix: misused source row count metric (#18902)
• 67514c1cb2944b298a39608bcf3daeca4318bea9 chore(deps): Bump pbjson-build from 0.5.1 to 0.7.0 (#18781)
• 0045fd6f2d3de1338daaf223960e38d8da924ea6 chore(deps): bump typed-builder from 0.18.2 to 0.20.0 (#18850)
• 883bcb1e781e8909c3c9227584c3611a2232ad39 refactor(test): mv scripts/source e2e_test/source_legacy/basic/scripts (#18891)
• f217edde107fbf6369d1bd1088b7e35e6c6ba912 feat: improve observability for subscription/cursor SHOW commands (#18896)

nightly-20241014

compare changes with previous nightly build

• 909b7157978ae5c6fe33d08de955cc4987bc455d refactor(test): rename non-inline-style e2e_test/source to source_legacy (#18890)
• 7dcd1bde52fb0f9ed67a38da454d82b0342e91f4 chore(deps): Bump rust-embed from 8.4.0 to 8.5.0 (#18817)
• f2999416f31c26bb6c823dd4cfc7080d07a0592b test(ci): switch all tests to sql backend (#18099)
• a348b8a76ed4146df117188fa8954784fff0e1df fix: use larger max_cached_recent_versions_number for CI (#18883)

nightly-20241013

compare changes with previous nightly build

• f50535978d243e8158e2647430ba408c7b895008 feat: create connector source worker before the txn write of catalog (#18885)
• 798896fc4eec986193b49a0443fa0452caf41b4f fix(storage): fix sub level id for time travel (#18886)
• 52f233186b73d60f6d150b5da57946c729537711 feat(iceberg): introduce read/write bytes metrics for iceberg and all files sys_catalog (#18841)
• 55723f0bf0e754eda2b2a6d859802e5b657b5d95 chore(sink): dump delta-rs to 0.20.1 (#18840)
• 75161ed8bb22c26aca1bc7505c5f8b42a6b850ec chore(deps): Bump parse-display from 0.9.0 to 0.10.0 (#18880)
• da368b2dab3bfa7d61c89699f6adf9d7c7f6507f chore(deps): Bump sysinfo from 0.31.4 to 0.32.0 (#18879)
• 00fbff45f978e57ced1eb7d70a16666ebeeeac3c fix(mssql-cdc): fix sqlserver-cdc backfill command (#18848)
• 7bdc37e21a27e64cc945d4c7b1b53133fd254ad6 fix: filter out creating associated source catalog in notification snapshot (#18874)

nightly-20241011

compare changes with previous nightly build

• 9ef40ba54a3cfce9f4946440ef78c8020b1b72f5 test: move kafka alter source test to source_inline (#18867)
• 360d55267c390e62ba27063867474a4d05e5a656 fix(udf): upgrade arrow-udf to 0.5 which fixes rust udf compilation (#18862)
• b1ea9e486f4665a64f660ebfa9d079502222ff12 refactor: enable time travel by default (#18854)
• 755edb4c67476d1f12fcfec5bf7216b277c8d01e fix: fix subscription clean during recovery (#18866)
• e9d2e29cc79557c9eb4fb17d42bf41a22cd654fa fix(udf): unlimit external udf message size (#18591)
• da977e921f397391d9025c068e0b0f025405e44c feat(batch): support basic postgres tvf (#18811)
• aaad651dfda2308f96bf96b485b844486dc92665 fix(storage): Fix panic caused by group overlapping ssts optimization (#18861)
• d6dc650a3e1103d6243d95b57100b20fe04874d7 fix(meta): rename --license-key-file to --license-key-path (#18838)
• 7cd2ab071df0d0e394598857d80c8f716a53bd30 fix(ci): fix there is no reactor running in meta recovery tests, when kill-meta enabled (#18852)
• b01906ec6fe3fe0d962cd02196c2c9c25583c13b refactor(meta): reject directly setting license key when it's managed by watching a file (#18823)
• 7fa1dfffd33747036a24253accda11887f9db705 refactor(cache): upgrade foyer to 0.12 and related deps (#18822)
• 442086b3c1d27331972780648ab4ce17546c47ec chore(deps): Bump rust_decimal from 1.35.0 to 1.36.0 (#18773)
• b11e15c1425b2bc9d9db1df492237eb4728d3667 fix(jdbc-sink): fix incorrect data type mapping of Timestamptz (#18843)
• a9bf2e9863b52ec833ca56e6120ad816a3947439 chore(deps): bump async-nats from 0.35.0 to 0.37.0 (#18851)
• 9a71862c8fdb6e64cf23f4ac4c81aac40c5f5c3e chore(cdc): enhance cdc primary key error message (#18829)
• dce14c94fe9bcd6315cd8b8e7db8353ba7bc0feb feat(over window): add compute count metric for general over window executor (#18847)
• 7720bee5f49c021c1064e6d33dd4b62077caceb7 refactor(over window): move build_changes to OverPartition for better modularity (#18846)

nightly-20241010

compare changes with previous nightly build

• b12460d08b5fadef15d43495cf63b6fe21dd4ee8 fix: fix base64 decode on + (#18837)
• 840c37bd089f03f06aabca8e0cf76998c47c7a9d refactor(sink): refactor es and opensearch to support async (#17746)
• f8be7735d21db85d3513dc1604bdae742f5285f6 chore(deps): bump the tonic group with 2 updates (#18682)
• 80dd8afb397fb30f6d7083e8c81d6e0d7c46b97e fix: don't initialize executor_row_count metrics when disabled (#18836)
• 286e25b1716d6a622800fe11665ef87a1a511fa5 feat(storage): support disable auto group scheduling (#18833)
• df6aa8a6e05ae9834b89e2e285ab901c966dc0c4 fix(metric): correctly update storage object metric (#18835)
• f32d01389249afbfcabbf32efe6435c6a26c5766 chore(deps): Bump bytes from 1.6.0 to 1.7.2 (#18827)
• c68aa5e36ed32af92ee3bb6642c68be9f96c6f2b refactor(expr): separate user-facing rw_vnode from the internal one (#18815)
• b4976e0d0bf3c77544a3555c1f42d60f517145fb chore(deps): Bump wasmtime from 22.0.0 to 22.0.1 (#18828)
• e0fc16cd2c12ae0fba0296730c57a4fd47f81ba6 fix(sqlparser): Display multiple source INCLUDE columns correctly (#18813)
• 71001d8444b7176fd2f406eecc65c8e516f0d4e6 refactor(meta): limit max inflight time travel query (#18810)
• 0311d835f7980ab37cb8775b6a4d407ab121eb7a feat: ban ALTER SOURCE for shared source, and refine all ALTER clauses (#18750)
• 786cd632d362afc1befcfc24beda8b5bc4920f7b fix: Remove DISTINCT in recursive CTEs for MySQL (#18825)

nightly-20241009

compare changes with previous nightly build

• b8c3fcb52964ff87d743cb9349e7097f77047c21 fix(ci): bump protoc version (#18824)
• 81780bec1a5ac2b7e20d47c09baeb0e9db99a585 fix(expr): cast untyped start and step when inferring type for generate_series (#18786)
• 2478845fa72964725a38824f5d132cf421863d22 feat(meta): watch and reload license key from file (#18768)
• ee4d9e798bb0d6ff9cc5e8f20e92faa4c54b8ba2 chore(deps): Bump procfs from 0.16.0 to 0.17.0 (#18818)
• 18604b2fc3abf7ad055402bafd6db8b463499700 build(mac): brew install llvm -> lld (#18804)
• c96c91330e2bd757e4eb7d98b3f60e3b3fe2f9a9 fix(batch): fix distributed batch query not properly throwing errors to PG client (#18812)
• 20699d4103e2328e25fb937f432fd3bdfdcc7209 feat(subscription): support blocking cursor (#18675)
• 00965ba60c469407ed57cd165b86ea1171651b02 chore(deps): Bump syn from 1.0.109 to 2.0.66 (#18792)

nightly-20241008

compare changes with previous nightly build

• c359bd9662d45fb90f0957101be20d658cd1fde3 refactor: remove prefix "rw_" in session variables (#18769)
• 419c85397c4b65dcd522e62a94f14ab6df96abae feat(expr): support any children in rw_vnode (#18405)
• c174d919974edc68424304bd733a98856f1d3af7 fix(meta): persist correct vnode count for table catalogs in sql backend (#18759)
• 8d6453d3c2d84d4ca569d6847a99b983ea51b9d7 feat(sink): add json encode for file sink (#18744)
• e2c76182d0da075e8623479bc2dbe220ef23cf67 refactor(storage): add SST retention time sanity check for compaction (#18779)
• b129ec1576665936e39f461f5f0ebbdfd429af3b chore(deps): upgrade sqlx 0.7.3 to 0.7.4 (#18799)
• 8ee00aa5e47f7e139ab79a3e170e2043ceb322fc chore(deps): Bump sqllogictest from 0.21.0 to 0.22.0 (#18793)
• 24a68514ba039535e1afbfa21d2becbff0731462 fix(meta): avoid removing control stream node when failed to send request (#18767)

nightly-20241007

compare changes with previous nightly build

• ba761f2362886ab3b73602a3423b73c97a9f5b26 feat: switch madsim integration and recovery tests to sql backend (#18678)

nightly-20241006

compare changes with previous nightly build

• 51c5e3e0bef991767f48a0c3bf6814426711b5f2 chore(deps): Bump chrono-tz from 0.9.0 to 0.10.0 (#18774)
• 6d126d52d2ab8c5cf1485e4c9a30cc81ad4b81fb chore(deps): Bump sea-orm from 0.12.14 to 0.12.15 (#18775)
• 1d8b5361db3eb99b269bf2a321cab0ad156e307d chore(deps): Bump indexmap from 2.2.6 to 2.6.0 (#18788)

nightly-20241002

compare changes with previous nightly build

• 66890dbf054bd2c9980e4817dac7affe974286d0 fix(meta): fix test (#18783)

nightly-20241001

compare changes with previous nightly build

• ef8c50f5f56c04374cda3c8b8204afc10ee6a88f fix: telemetry event at paywall not reporting (#18770)
• 72ba6d53e47c92c0bf97a49f8e84a4c6b4391cce chore(deps): Bump sysinfo from 0.30.0 to 0.31.4 (#18776)
• a9104146b3965ed837ed12d1c0d98861b21c7f73 chore(deps): Bump async-std from 1.12.0 to 1.13.0 (#18778)
• f0ad2d7e607e2b60b6ee078e1001dcf336952a63 chore(deps): bump the google-cloud group with 2 updates (#18717)

nightly-20240930

compare changes with previous nightly build

• 17ea32610a22d697ccf4499ddd1cc6f90c5b1492 chore(deps): Bump itertools from 0.12.1 to 0.13.0 (#17075)
• e82932f47bf57dbe393b4ec28ba7431b40e8232e feat(streaming): plan asof join (#18683)
• fae0201524761c33c293ddc689c866bf78395020 refactor: deprecate max_committed_epoch of hummock version (#18644)
• 7ff4d98830bacdfdc28beb1f81a25eecaacdc732 refactor(storage): add SST sanity check during commit epoch (#18757)
• 254dce9e05ba129dd2830007f639c2068e8e5ad8 chore: add comment for KafkaSplit's start_offset (#18766)
• 61b745935c7c97988d5f1c38baaccbdedc74a7ec feat: ensure number of tokio worker threads to be at least 4 (#18762)
• f7e506844fa0435b8736bef2e6b426f316b43366 feat(frontend): show job's max parallelism in system tables (#18672)
• 6731c26be9f6be7f49c9ebc84378a5ba46f6fee1 chore(ci): remove gcs source test (#18763)

nightly-20240929

compare changes with previous nightly build

• 83e3c4b4e35ed1dea80cee4ba49cbb4c49d6e267 feat(storage): Introduces a split implementation based on split_key to replace the previous move implementation. (#18594)
• 5e20f2dd3032530a846986aefeaec2ae3a116a97 refactor(meta): persist job-level max parallelism & check when ALTER .. SET PARALLELISM (#18740)
• 1da18155b7fd88f64063c3f7b4b719d7405d4aab chore: unready source worker should not affect other sources. (#18761)
• 4dfc4c961a8c7b6f69c900fafe18d074c01676ae feat: improve sink metrics (#18758)
• e6435e60e558eab607a2ed3902a04903eccf78ae fix: better error message for ./risedev d timeout (#18756)
• d492843560530901dffe6e39b41cda8b8b5c2d37 refactor(storage): remove most usage of max_committed_epoch (#18641)
• 04af708f5bb84e639e5ac074bd6ba290eecf67b7 refactor: unify duplicated alter_*_name methods (#18751)

nightly-20240928

compare changes with previous nightly build

• b4ac5aba12d40df880931821ee68fde45f1d4683 chore(deps): Bump vergen from 8.3.0 to 8.3.2 (#17781)
• dcbd574af0ecccceba520f90facf09820c2df22c chore(deps): Bump org.elasticsearch:elasticsearch from 7.17.19 to 7.17.23 in /java (#17886)
• 939493e4156c60742513ef3a707bbfcf9c9c3da6 chore(deps): Bump serde_default from 0.1.0 to 0.2.0 (#17790)
• edc3c8fb68e0f3ff148c41b83ed113be1df1f069 chore(deps): Bump redis from 0.25.1 to 0.25.4 (#17828)
• f8b2b847048a0b6bf66de814cd97dd0f050f03e6 chore(deps): bump lz4 from 1.25.0 to 1.28.0 (#18742)
• adc30c0f7c3e76c8e1d2a15673b359bb80e8f7f1 feat: add rw_actor_splits (#18746)
• 8e6ebb0c613f41946f0fcde348caf982e4a815b5 refactor: add metrics for time travel and recent versions (#18690)

nightly-20240927

compare changes with previous nightly build

• d8774819c47930d0da158792983188fbeff4eb59 feat(sink): support async for mongodb dynamodb (#17645)
• 1ddf30e8a22ecfdfe793bf8d054a2bd73b32bc32 fix(risedev): fix log check and startup failure for postgres (#18745)
• 2b4e57aa2e84d36bac7489dfb57173331f88e04a refactor(meta): reduce time travel hummock version size (#18699)
• 811fa54cf5e9fd127923e568761914f1a1e9f245 fix(meta): fix query for running_fragment_parallelisms in postgres backend (#18743)
• 056ccfbb6789a27d00bc9914e7c957aa64add32c fix(nats): align the semantics of scan.startup.mode=latest of NATS with others (#18733)
• 83fe9f38ff032cb86c9b31caed4f9fd5a54ab58e fix on conflict syntax (#18735)
• 8f0d5decac74f806046b06f197e879664793cd20 refactor(storage): support pagination for LIST during full GC (#18737)
• e9c216185bb5226d14f484126f5dc2f851b2312c feat(storage): per table try wait epoch (#18622)
• d6f8ca22d9884d2a8367f4eaecda3171b4933c3a fix(nats): align offset and property fields between NATS and others for better ux and dx (#18732)
• 4548a995596b160a168aa9e11386cfd2d2322c89 chore: update image version to 2.0.1 (#18741)
• 39141cc47b0acf5ef81c1d8a5ecf792ffcf36c06 refactor: sink metrics (#18730)

nightly-20240926

compare changes with previous nightly build

• 75080b765e5914c07e906de4eff7f16146aa207d fix(binder): disallow CTE in right table of temporal join (#18738)
• dba5975b25ff11cea2fa4403473ea949c4433f92 feat: user-facing part of variable vnode count (#18515)
• f394288ae4eff029353ef5b3c1ffac105f46313d chore(sink): validate file sink when creating (#18707)
• 647b37ec184491ad0a80cbba1c94cd21c18ffeed chore: change streaming upload timeout to 20s only for backfill test (#18652)
• cb926aff4e36861bb81e1d790c4795c265b6eff6 chore(deps): bump the aws group across 1 directory with 4 updates (#18719)
• 5e2cfac4f0fa8c29c21ac6691c7bf8c0abd45a90 chore(deps): bump pbjson from 0.6.0 to 0.7.0 (#18718)
• 375e9338534daacafe628342c85da882c9d37a72 feat: support information_schema.table_constraints (#18716)
• b3765cea8fb25987d8fbce3377781ae89bbd9697 fix(ci): change GITHUB_TOKEN permission to write-all (#18724)
• e535aa7c743ee603265a18e0d145132bb5b3e11a refactor(kinesis): rename fields of KinesisOffset and KinesisSplit to make everything explicit (#18704)
• 1db23dec67b89ac7f0b4b67ea978d202783d119b feat: support pg_catalog.pg_sequence (#18714)
• fdc57908b8622672f9f4f9ae68f3cf8feb56eb51 fix(pg-cdc): fix compatibility issue of publish_via_partition_root (#18691)
• 45ef6c45bac250e351a1dda1ad366e02ffe98343 feat(sink): add es dynamic route (#18698)
• d1baacf347ae1e7dec11191e6f138a4438445d9a chore(ci): fix the warning due to Node 16 EOL for GitHub Actions. (#18710)

nightly-20240925

compare changes with previous nightly build

• 0679b2fbb5f54e4c9e9daf6f8ab5018da9ebb9fa fix(mysql-cdc): add missing data types when parse schema change event (#18706)
• 2c89c244f0ca12dd3b151444cfed4ca838087eff feat(telemetry): record cloud hosted UUID (#18019)
• 1993bae530d756d64d33b631eebfadd4c1adc473 feat: support information_schema.schemata (#18709)
• 7f39b6a0f3a1b50547259f9bddeca0b12c374d5b fix(kinesis): bump aws-sdk-kinesis to 1.18.0 to fix creation failure (#18702)
• 9aa26bd099ff2e55bd1cb844d65930631b845ed3 feat: support SET TIME ZONE INTERVAL '+00:00' HOUR TO MINUTE; (#18705)
• 9629fddebdaabd49a94e3ac9fb0935995fd3049a fix(ci): fix release static build (#18689)
• a7e60cb7d1998af459bbe30ba52ba2a6ad62cb48 fix(foyer): upgrade foyer to involve some bug fixes (#18693)
• e5498fb3f462a9350b196cbce99485be01488a21 build(deps): bump iceberg-rust to latest main (#18686)
• 0bcebd888f5281e9c356809d0daea77677d9ea18 feat: allow append only table with pk (#18634)
• ce70a51f4fb05ef9b0efc01c645e490ff68b624d fix(agg): fix embedded UDAF as window function (#18632)
• 93601398c5b9526775c28398fa43f377a4530afa fix(ci): add contents write permission cherry pick workflow (#18695)
• dac92a43ecf00220ec456eff6cc55e5c664cdebc fix(scale): Remove the wrong assert for broadcast dispatcher in the scaling post apply stage (#18681)
• 97f3d9c3c5da252c7b1bf66ef0aa0c64c43e2460 chore(deps): bump dashmap from 5.5.3 to 6.1.0 (#18429)
• c7975e0be2c31d9c156a9a2ea5810ddd2357ded2 fix(pg-cdc): check ancestors and descendants for pg partitions (#18648)
• c467dcdc46ab8462a4794165ef64bfe671d98ef1 fix(scale): Fixed propagation issue with low level no shuffle scale in the sql backend (#18676)

nightly-20240924

compare changes with previous nightly build

• 29adeb97370b6a77a4dcc857ac55249016dc3be9 refactor(storage): adapt time travel cache to partial checkpoint (#18646)
• 4401694e4e96eec9e26be7384d26c0460b88e9e1 chore(ci): build v1.10.2 from source for backwards compat tests (#18667)
• de73970705424e11a9bf16950402efc860bd3944 feat(cdc): support default column in auto schema mapping (#18571)
• 75cc17b7178f5a5ffadf5b75fec18c05c99de2f7 fix(ci): correct release check (#18666)
• 5f800b9a167c8850f048145adc9d6af7ea7ad22c feat(streaming): support up to 16-bit vnode count in row id gen (#18529)
• a24d557083748db6d62f7481bbd6cfbbe3f57eef chore(risedev): add env fields in schema of risedev profile (#18665)
• 7b977a2e7d222f4666a698941c0ca845e62f5552 feat: support per-fragment vnode count (#18444)
• fe01e22d41923c150a8e0b7d784d12695a452972 refactor(test): migrate s3 parquet test from s3 to minio (#18662)
• 6eaecbb21e4e3acd11cf5ec158efcf3451457cae fix(ci): build 2.0.0 from source in backwards compat test (#18649)
• 148a5fceca496c06e5e244cf86e55878c697ea1b fix: s3 source main-cron (#18655)
• 477706f999cdac4725fb8be0c4e1c9ec59cc510a fix(test): fix backward compatibility test (#18656)
• e7458205ac18407c89a6f88d497ea36e2ceab7be fix(pg-cdc): fix postgres.publication_exist, found in issues#18628 (#18629)
• 27458d8b038992c0e3f570bac4cfa2887a5a5454 feat(storage): bump opendal to v0.49.0 (#18266)
• d5dbc6ac1cd266140bb0740b3fedc53dbb53b07c fix(rpc): workaround for tonic hanging on large error message (#18639)
• a5ec3eaacf649a42cc3bb496eef73e3be6e0c7cc refactor(rpc): prefer monitored_connect over connect (#18635)

nightly-20240923

compare changes with previous nightly build

• a898dcc8f42001b5a4f4dcd965cd0bde8e67b45a feat(frontend): generate query epoch by committed epoch of involved tables (#18592)
• 5e427d6c318f4489253495d668bfd7dfa27449ca feat(batch): enable batch query metric by default and only collect node level data (#18627)
• f3bd45624227f9eb1b3182f1e8bb7919a71a09b8 refactor(agg): separate semantics of AggKind and AggType (#18640)
• 92317823a2aa4f4342a74528c721f983d2e39aaa fix(meta): fix type conversion (#18638)
• 8fa0468689002dc919c2560e1a264f754d5dc4d9 chore(dashboard): rename relation dependency graph to relation graph (#18612)
• 039af9dd98477abd1e24cf52caeeff56bcd90e9b refactor(meta): remove BarrierInfo and reduce pause and resume log output (#18630)
• 2651278ac2a301fa23229b39eb66432515f33c51 feat(frontend): frontend wait hummock version id for ddl (#18613)
• bfa43091930004d008bcd0b3bd7407549e06c6d7 refactor(object store): OpenDAL write concurrent = 256 (#18590)

nightly-20240922

compare changes with previous nightly build

• bc65ffbf0cf58eadcff87930bc69cb5a8bbcf56d fix(pg-cdc): fix support for partitioned table (#18456)

nightly-20240921

compare changes with previous nightly build

• 98a2d410ab40f58d2bc24b296aee6f644e380229 chore(cdc): update sea-schema version to 0.15 (#18521)

nightly-20240920

compare changes with previous nightly build

• 2b8f6a6be6a283b8cfc0f4b348092d897573cc5a fix: fix flaky simulation integration test cancel ddl (#18626)
• 409bc8d1b088f122ebe2285e5ef2dfff2fd12775 refactor(meta): deprecate pin/unpin snapshot (#18493)
• 6b29ec919341cccd6ce41dcf1280b900a68485bf feat(storage): variable vnode count support (#18415)
• eb2d9f4f773265e0d6ffa0ea1d3f6f27ddcd1d93 chore(deps): bump sqlx from 0.7.2 to 0.8.1 in /integration_tests/feature-store/server (#18617)
• 5652a1f66e651773072a34859f3c1d4b0a3b3527 build(deps): bump arrow-udf to use arrow 52 (#18609)
• 3c94345bd553ec83a911d4945c498b3484194d65 refactor: minor refactor on create/alter source (#18619)
• 3c743902d4922fe47aee472b1dc6ef51d4e67dc8 feat(subscription): add metrics for cursor (#18052)
• af98091c3e9e29535584690896f4cc7e0e67898a refactor(compaction): deprecate safe_epoch (#18491)
• 767dbc156022e964ef66ba7f164c5febc6a17d5f fix: forbid the usage of single quotation in title to avoid failure of cherry-pick (#18614)
• c6db8013319c4f540f9b2c93fc1671472e3757b2 refactor: refactor arrow conversion (#18608)
• 82ac534a92c3b269f5da3d7a4142297beb7e031c feat: upgrade foyer to 0.11.3 (#18313)
• 4bc3b28d7eac7e9c9700b76b41ce0fc5350069c1 refactor(iceberg): extract IcebergCommon config (#18600)
• 5aca9144e5bd937b3427bb9bfdce9f9cea013860 feat(sink): support sink map to protobuf (#18552)
• ff2d937735133e7135d83c0d532fb9daa10141a7 refactor(state table): cleanup StateTable::seal_current_epoch (#18578)
• 4ba84deb9f328c8eaaab56b388a4e409ef77d1fb feat(storage): notify frontend with more hummock version info (#18570)
• 759efe3fda4937881e9ad5d5387cb4bc685813e8 feat(iceberg): support position delete for iceberg source (#18579)
• 57f01de196ff6e8e5f3e6adc6abfe438ba63b5d1 chore: update image version to v2.0.0 (#18574)
• 8b0c5a3e11a40d20a91c42a85b637717f4fea2d3 feat(dashboard): add backpressure to the relation dependency graph (#18280)
• 89f93df1812e7470faa50df522e47c184fb7c6d4 feat(sink): support sink map to avro (#18551)

nightly-20240919

compare changes with previous nightly build

• 97a329350a14d747e0eae31e4e1c86ff3d4d4ea6 feat(storage): introduce merge group metrics and corrects some config (#18576)
• c7b6fff8a9328eedbae145697ab2eac0d1d84b89 fix(catalog): add views' dependencies in rw_depend (#18596)
• 57f828500e6343efc7a7e7bafd080b8cc8a7295f fix: fix message_queue image tag in docker compose (#18602)
• d353d1332795c3c46d80f9b2950169029e6859d8 feat: improve error of building key encoder (#18563)
• 0f1984a58cdb318158c707723057eeded71bcc09 chore(scale): Report an error instead of asserting for unsatisfied conditions in the plan. (#18589)
• ff479f60996f9ac61b0d8b5ddf40370c8b0217f8 refactor(storage): remove legacy delta type GroupMetaChange and GroupTableChange (#18585)

nightly-20240918

compare changes with previous nightly build

• 5b625f81b06217c4350a57b8457006e6b9bfdd8d refactor(stream): make degree_state field in JoinHashMap an Option (#18539)
• 4ec727d5e07370d1d358e4026322893aef73e962 chore(deps): bump next from 14.1.1 to 14.2.12 in /dashboard (#18568)
• b1a38b36e51e5c0b9411666314d756c9e38a5d83 feat(compaction): introduce new task status for serverless compaction (#18575)
• 745fb16fec638178c7a862dfaa2ffd23b076ddaa fix: Fix the panic caused by scaling with a single fragment downstream of no shuffle (#18581)
• da08cd5a377bae3b0394e65a952f9b9f82aa9e18 fix(ci): fix static link for openssl (#18569)
• 25b0cc9362ea931367e245c45d6ebea85804c286 refactor(connector): replace protobuf-native with protox for schema compiling (#18543)
• a3c6e48c67cf666d382c8e3fcb2afc451c8bcdc5 refactor: unify to subscribe mutation via barrier sender (#18255)
• 06d5cdef259ecd67aa0a09d4e362f9ec80436aa8 refactor: maintain subscriptions in local barrier worker (#18516)
• 3ee3b2cb084586a9a39ba0b922473bce7421a02c refactor(storage): deprecate code of range delete (#18525)
• 6ca719f546b44ca04fd2c5a36c4090736e11155b feat(metrics): support filtering metrics by name in Prometheus scrape handler (#18556)
• f19dec52daba2f27d80e7fa5aba8974b7c252bd3 fix(optimizer): correctly derive cardinality for group top-n (#18561)
• 24336eb97df1c8ee9348d627d317daa501a3b178 refactor(state table): replace new_ series with gen_ series (#18566)
• 3b3dda7dc8f6c8c8b248c89eb0eee36336efd9a8 fix(test): fix current_branch for backwards compat test (#18565)
• 2b9280f2333ac857864bf510158f34ee79fa6b51 fix(test): fix version sort and handle case where release branch is current branch (#18562)

nightly-20240917

compare changes with previous nightly build

• 1c04a5ce3ab2cc605abd6cf4367017433b63c7d8 feat(mqtt): Add max_packet_size option to MQTT Source (#18520)
• ee62b43cf779ca0a27c64fd3e4b7deacd90d9478 chore: use Postgres meta backend for risedev d full (#18560)
• 604fdc8d7a679eda3d21c656215cb4b4bf4728aa fix(sink): remove some todo!() with error (#18550)
• 9c6cec19592a16367ab76f5a26c57b730645bd69 fix: revert split assignment check (#18554)
• 11ad34b24ff2ef5a26c9d91ba386b825030807cc fix(metadata v2): existing actor splits were not updated (#18553)
• 91677683efdbefdb04bb2fb18f16d3183d183347 fix(stream): handle 0.0 case of approx percentile, and fix computation of quantile_count (#18546)

nightly-20240916

compare changes with previous nightly build

• 9413b28b5a836bc577a9c50fa8cd223cd137618b test: s3 source with additional column payload (#18496)
• 789d34503b9782d1ee23b4e0f2822d494954a16d fix: dup split assignment in source (#18541)

nightly-20240915

compare changes with previous nightly build

• ac966f1c897544afb00a1e71c14bf01d32866388 refactor(test): refactor some ut for hummock (#18534)
• c80351bf0ed4704d2576d9573fddf3048fdd0e6a refactor: revert "#18324 configure whether to recursively scan file source" (#18517)
• 21345cb4ccf6db18f0b5901cb8da4ad5926d106a fix(ci): fix cassandra ci (#18536)

nightly-20240913

compare changes with previous nightly build

• 72396b2c3bfe00e13bef0653fe68547b93160f8a fix: only notify creating materialized views to FE when using sql backend (#18476)
• 8eed7692b9ee86403d2e62790d7a197947f41f1e feat(source): support ingesting protobuf map (#18512)
• 546c49c9f10b82b8a9d77f7a1b0d4e93f1ec319a fix: make guarded label metrics correctly drop (#18454)
• c2192182dd06cba7935d5f6a3028f0fd461c7bfc refactor(source): move protobuf to codec crate, and refactor tests (#18507)
• 870bcdee2e0e8d35d69134498e045de136d34a1f feat(iceberg): support eq delete merge on read for iceberg (#18448)
• f226e21baf27feef8676b9cdf93044f0c2b9be81 fix(udf): use a dedicated runtime to create udf flight client (#18461)
• bc9c2ad38ca04640714e9105fe40cfe3acdd9918 feat(streaming): introduce streaming AsOf JOIN executor (#18242)
• 9533cd9e11059c3a3cdc0d2f0094d368c67ad2ac feat(cdc): support for verify-ca and verify-full of Postgres SSL (#18015)

nightly-20240912

compare changes with previous nightly build

• b9ac1ac1f606a3ed1c894db2a69de134660a2924 fix(iceberg): fix decimal overflow for iceberg arrow (#18511)
• 6c3b68f3d488c665ac95b5874d18cc55b2c3bd1e chore: make file sink publicly accessible (#18508)
• 36271315bbf262dbf7a375f93713d2c15157ee1f chore(deps): bump send and express in /dashboard (#18489)
• 34a2ad42da6a6dcb45b1260b6cecfce89d1c360c feat(iceberg): support create iceberg table w/ one layer nested type & scan table w/ nested type (#18498)
• 317641f46b2b7af762ba470aa12a43a9f64d1e28 refactor(batch): schedule with streaming mapping if serving is temporarily unavailable (#18505)
• 4afcd4d92b613fdc01a27ad71091683bbdf31a97 refactor(storage): make HummockVersion a generic struct (#18478)
• a9b1de383e0163865c7e93b84e031f34017e8d62 feat: schemaless ingestion for encode json (INCLUDE payload) (#18437)
• c882bdaa8acd73a6d3a20ff4809dda89aead6fb8 refactor(storage): cache recent versions and unify batch query and time travel (#18477)
• 5065bb4d1e4ed81aa38f784ae4acb4b644c3e6a8 fix(sink): fix sink_decouple = false and commit_checkpoint_interval defalut = 1; (#18348)

nightly-20240911

compare changes with previous nightly build

• 23410f09511f3f75cbd2dbe4813fe766081e6656 feat: allow customize pulsar subscription name (#18379)
• 8e7b09918f08e5fc12d4929a9e9c1f25918e1026 fix(source): parse protobuf into expected struct/array (#18419)
• 6038298c9fd2eedb3885e4187b39807ff3ee4c87 refactor: bump toolchain to 2024-07-19 (#18470)
• 084bc6dc4db0d4f6392bd3c156e52cff4fab1880 perf: optimize memory allocation on building object path (#18465)
• 00150a3022ee6a8d9c50ed150580e61e5d372253 fix(iceberg): fix map - arrow convert & fix sink nested types to iceberg (#18463)
• 783350997dd7f01f5f0e3e2dff2441a2dfce158b feat(sink): reimplement sink coordinator worker and support scale (#18467)
• 829bef3330581be251d6057f090b29037de0c853 feat(expr): support map keys/values/entries (#18480)

nightly-20240910

compare changes with previous nightly build

• 596138e0364cc98e02224ea387d6644d9bf9e8cb feat(source): configure whether to recursively scan file source (#18324)
• dc07da010735ac30751df2edbed14591272f6385 feat(meta): support time travel query on a per-table basis (part1) (#18459)
• f970372021e3b65587b5ad61a2f6b8e0f8be6abe fix(metric): include table_change_log when calculating object size (#18452)
• 99d6121aca3104d08b1dff44941584bbfad58b42 feat(compaction): support merge compaction group (#18188)
• 952962c03192cf4540485d38037c26a5afb52e21 refactor: replace RdKafkaStats with Guarded metrics (#18462)
• 267939635bcb74b0be8f08d039786e65f359c4a3 fix: empty string as license key stuck docker compose (#18421)
• 8c7a3641d38df253fa71029c134f09b9ab369d1f refactor(frontend): extract filling fields in fragment graph (#18466)

nightly-20240909

compare changes with previous nightly build

• 29d2e1ef33fad45e8348af9d0de251bbd7ddd194 refactor: track progress minor refactor (#18446)
• 462253e12b6957bf6e20aaaff44335215d8eacfc chore: update image version to 2.0.0-rc.1 (#18460)
• 8d5b62b17b2a98b6bb28d492c9cfe72f9dcd787c feat: variable vnode count support in batch/streaming scheduler (#18407)
• 9a03718accb10e75cc92b0d27c7ec58cdf0b0c64 doc(readme): update architecture image (#18418)

nightly-20240908

compare changes with previous nightly build

• becb896acd442d32eab4c14dccf558a4b346c643 fix(iceberg): fix select empty iceberg table (#18449)

nightly-20240907

compare changes with previous nightly build

• 71753f11223cd0b4d67ba6baf7ac636fa246e7fc fix(meta): correctly update serving parallelism mapping (#18439)
• e2c89f418e8dbe5fd5b078ea8d57288f438d49d0 feat: support scale SourceBackfill executor (#16825)
• 3762877b55bb5187533023b582c0f7f8901f0323 fix(storage): fix commit_epoch new_sst_id_number (#18438)
• 7d3f2856f2466681346c2edb6dfe890e0c41c5e2 refactor: remove drop_actors rpc and simplify barrier worker state (#18354)
• 4accc4a6c641871d73095443875d0f0b1c573d9e feat: introduce cluster limit (#18383)
• b00d750abe8c03d52b0a18af8077ed0f7ad3aa70 fix(storage): fix duplicated sst_id generated by split_sst function (#18431)
• 5311069848eb3d74bb71fa7d3bafa8c7967420d1 fix(sink): set query timeout for jdbc sink to avoid stuck (#18430)
• 6489e9ac8cb656810196a71426ee7dcf3467a2bd feat: track progress for SourceBackfill (blocking DDL) (#18112)
• df7f54f0b35cfe39937ae49bf9b5dab4343d51de fix(meta): fix catalog ref count (#18427)

nightly-20240905

compare changes with previous nightly build

• 79ba86964acf0616693a8534c9eac1a67eb2889d fix(backup): fix restore order (#18423)
• 0134191ca8055543ece575e9f074f4ef3b23ac45 fix(storage): fix correct_commit_ssts with sst table_ids (#18414)
• 1d220eeba6f813673f8efc72b1fcf9e449171cf0 fix(error): fix sinkError and connector error (#18425)
• 8a32a9b21b45c4d91f3235411d35a0a2b3f5102a feat: variable vnode count support in vnode structure (#18381)
• cb29fe0a3221638b5dc62c3ea543b08dd567135d feat: support more SSL related configurations in Kafka connector (#18361)
• f2f58272cc51a118a83c1fce02bd42afeb98c80b fix(storage): fix assertion (#18413)
• 3b98b71f27525ad936820c264198c0f02334d252 fix(source): Protobuf Any as canonical JSON (#18380)
• 670a94f118d61462b720d51d62f164986d23cb23 feat: variable vnode count support in table distribution (#18373)

nightly-20240904

compare changes with previous nightly build

• 0dd06ffa9a34b0cb584393990f8de8a41267718d feat(snapshot-backfill): only receive mutation from barrier worker for snapshot backfill (#18210)
• 9923c3abb8f87a6f643d5f753585e2b31f4de7e0 feat(risedev): support providing env var in yaml config (#18396)
• ee33271dd98cfa84e390656e5b5836f7453b07cc fix: parquet test missing comma (#18397)
• 6402328e7ff51b16691c6b72858b7075250246fe refactor: add some comments for source splits (#18034)
• c4b1dd4555c329a58fc92955b4a7aa9a3d957d89 feat(expr): support jsonb_populate_map (#18378)
• 0a4ccde5c4b8762d825b150a0a73a80afdec6253 fix(sqlparser): display create items with comma properly (#18393)
• a7480e1c6225e6bd32314fec9f5e0a686a9850b9 feat(risedev): attempt to kill existing session when launching risedev-dev (#18370)
• 847610a0e1007207d44dcadd6c61f5373ed4bdb9 refactor: use high watermark to finish backfill faster (#18342)
• e9d77a4d5166f78cb65d6b3a8d6d5d67f56b5b05 feat(streaming): add must_output_per_barrier flag for stream simple agg (#18374)
• c843edcd298be55a0bdc938dfa3c240676fba9c1 feat: support arrow map -> rw map (#18375)

nightly-20240903

compare changes with previous nightly build

• c0ce8a8a6b4d51e463a605fb307fa4b014e0f609 feat(sink): support async for bigquery sink (#17488)
• c519f0dbb13cd66d1b5cb5bfb1325150f0e5d70b feat(telemetry): add telemetry data point when checking license (#18371)
• 51d3c63f702ddf8860f6223a1e9a425e034d3375 fix: DROP DATABASE doesn't clean up the source stream job (in v1) (#18033)
• 5ab2a59ed708f2d704455bbaa5e51e56e58db757 feat(iceberg): support iceberg sink create table (#18362)
• 5d8b1650ecd0d9e34a3eed519a9723752c5ea633 feat(iceberg): support create table for jdbc catalog (#18364)
• f05d549efb5c1c2287047c07750ed210645c8b81 fix(streaming): correctly retrieve initial split assignments from combined mutation for sink-into-table (#18356)
• 124011d94ab7ac44c8a7868688e4b326a10ac0b0 feat(cdc): support constant default value for alter table ADD COLUMN (#18322)
• 9d6515b6a8af4e8935bea76e46aa14aecc174cf7 feat(compaction): support recreate block stream for fast compact iter (#18350)
• 5a314486da5c3f70e71fb13e2398df96ca00ee87 feat(compute): give more batch memory for serving node (#18365)
• 46bce7e4091c09db08714aad1bbee32b6c3f8da4 refactor: add target_offsets to determinine if source backfill finished (#18297)
• ea6674ea31f2f6fd1b526c11405168f39a5131e0 feat(risedev): prompt user with argv[0] instead of hard-coded ./risedev (#18358)

nightly-20240902

compare changes with previous nightly build

• b2eea6b799d31aae8240e41cf3f0e0d29e542255 refactor(prost): optimize some Debug representation (#18211)
• cd0eefd2e5cd83f7d79894693f64340e74fbd7fc fix(compaction): disable fast compact for split sst (#18347)
• f60be139715b8b54e1e2c8d5d9ac8431de29395b fix(meta): correctly update serving vnode mapping after scaling (#18349)
• 602c6adccaf6de63b430bfdfce045653172a015d refactor(metrics): suppress more actor-level metrics (#18318)
• e78832351945c171f5e10f895932463b6f3c4664 fix(ci): fix flaky barrier read e2e transaction test (#18346)
• ef71eefea35c1fbb1ad3669ce3dc7b17f71c4019 chore(docs): Update README.md (#18340)

nightly-20240901

compare changes with previous nightly build

• e41b348f2d7d800b45e4102192200fb1698037ca fix(risedev): fix risedev (#18339)
• 5055cc920aed3f555bcb3ba5a42962555789bd26 refactor(metrics): refine relabelled metrics interfaces (#18331)
• 29db1d95c416ea14c8ebb39ba74857d01c631d34 fix: fix column nullable definition for index column properties (#18334)
• 59b400a92d623798541c0a49dab0647795afaa38 chore: Update README.md (#18226)
• 04ef98d6e76ced350adce9a75862336118ce09d0 fix: allow ENCODE BYTES use generated columns (#18294)
• 70c1146a0ede4c8a5b6a5adcde07b3f0449c2b5e fix: s3 source cannot read incremental files (#18017)

nightly-20240829

compare changes with previous nightly build

• 655ce2a8cdd83f2338b2de23bef1133cf4e1916a fix: Sort by actor_id in CustomFragmentInfo and rebalance_actor_vnode for consistent ordering (#18317)
• edb149317bad534730c09d04e639a622ba19e5fa refactor(metrics): remove actor_id label from back-pressure metrics based on metrics level (#18213)
• a5cbeb716ed8dc58287346e02b31252d784ded37 fix(source): parquet file source use number of rows to determine the end of the file reading (#18149)
• ca99aee25cb4df928dc525e727d170bf1366e12b chore: remove unnecessary debug info (#18309)
• fac2904d4bac6a3c2ad8bd126ace04ec02ebda1b refactor(meta): build actors in inject barrier (#18270)
• a137e302a955eb89def9efd2fedbff28bd3798b3 feat: deprecate current epoch in pinned snapshot (#18230)
• 242dbff19fa9322178db0830e4a125507ecc98e2 feat(sink): use official default message_timeout_ms (#18304)
• 2529c56382ea94ae2ea6a2da21036546f389b29a feat(sink): add es retry_on_conflict and max_task_num (#17867)

nightly-20240828

compare changes with previous nightly build

• befb04f1e433ecbb564a059f34b4299ba4952bed fix: disable sink decouple for trivial sink (#18302)
• 14a46ff1440604d9c6013605b81b4bd305e4b2ba fix(connector): upsert+protobuf source (#18301)
• 40074f69cd9472888a01d1735d9d963227504963 feat(connector): support azblob file source (#18295)
• 0239e22a4187fa424451f09ea9e293f62243ae5b fix(cdc): only allow ADD and DROP in auto schema change (#18245)
• 54134df47b30554313bb91177ed2ad8e4621d441 fix(frontend): add project after simple agg to avoid noop updates (#18292)
• 0e5a46f48f5f99ee32c649493ad9d2d7d023677d feat(connector): add webhdfs sink (#18293)
• 86f403a5af372fb3c3259ea0be7ed1914799eb41 refactor: simplify SourceBackfill's memory state (#18296)
• 53c2b0a99473f7306d0b352572f7058e82e41006 perf(dashboard): only fetch ids and specified fragments for a relation (#18272)
• ee77691f03772ede158a9d5a3e87861b6abf7205 feat(storage): remove shared buffer compaction grouped payload (#18277)
• 9f52981fb7af366074601c9ba12feef5bc76bca5 refactor(storage): reorg writer code (#18291)
• 04b9cb4e61e1a5119ef636a6335d02a178372dcb fix(iceberg): fix path style access (#18259)
• 7998622f78e0ed97fddbf1ebf2634e06a45a52d1 fix(source/nats): add missing default attribute for vectors (#18261)
• 205acb42896abfcbae15094aa4b3ed244e4567f8 chore: remove sqlite file (#18282)
• 7a97076cedfc1018facc892ed1c350327fe6b8db fix(meta): fix list changed log epoch retrun null epoch (#18273)

nightly-20240827

compare changes with previous nightly build

• 63b4d47ea9f54f567983e097e15d8dc9b0384849 fix: should use table_metric for MV throughput (#18265)
• 6bd0e89cf056969fe0f9028815d83786c53dfec6 chore: create an issue in the doc repo if it is a breaking change (#18263)
• 22926d67fff102cd7114b150b8437e3afd4a4231 refactor(iceberg): Separate iceberg source pb from source pb (#18209)
• edaace2f93877f1bc25c523da4d434b869aaf73a fix(batch): fix CTAS pins snapshot too long (#18248)
• 79d98109deee8518740be6a00c3e4f41c29d5f1b feat(storage): Support for repairing the size of a split sst based on table stats (#18053)
• e3a9d37ddad6ed519c0fa48ec457a0ea994c441a feat(agg): support aggregate: prefixed scalar function in streaming agg (#18205)
• 3c31ef9755c0bc9799dd4b65151b095beb148120 fix(storage): Fix the bug that switch table_id breaks block data constraints. (#18235)
• 9932d12bf91c822164a06551fe2b56363d77e99c feat(connector): introduce azblob sink (#18244)
• 860a936beb8ee4fae1576429b5a02a9c765dea6d fix: allow nullable for target table columns for normal sink (#18251)

nightly-20240826

compare changes with previous nightly build

• ce7a5afc0ecc86adbf5fb7c49cb4337d5d4df983 feat: inject pause/resume for creating sink into table (#17651)
• f00c82f3baceec71185b6af752335ccd193dc7c0 feat(cdc): add metrics for auto schema change (#18216)
• 62b7586a5ff788fa793934666e20810e52268354 fix: fix fe internal rpc host addr issues (#18241)
• 4e3b9ff081d59c28668bdca9817930eb3fd013c3 feat(storage): introduce new field sst_size for SstableInfo (#18005)
• b0e50b4e4428e38ba35670a775b25aea2c4f1890 fix(log-store): fix incorrect lag caused by invalid initial read epoch (#18231)
• b36ac57677906e2a1e006c6cb132e6fee8abd224 fix(sink): allow starrocks.host to be an fe follower (#17679)
• 86a15baada5b17bfc8934662f6d0a7c7d85f6ffd fix: make fs list exec singleton (#18199)
• ec61f325a5d07a15a4a2c1b2cd55c3a7f5f38402 feat(suscription): Improving usability of subscription (#18217)
• 07911cfa3d02ed90cb1f2e3a8926fbfd456d6b91 chore: add license to file sink (#18227)
• b2d2de3ff8de41051fdf6c1d9c54c0db9dc5b4ad fix(ci): fix java connector node ci (#18229)
• 7d20fa851b93ccc43c3dfabf4d808ddc145b1888 feat(sink): set default_sink_decouple = true for all sink (#18182)
• 220fded7139658d727fa811c6ae9d051d0696574 feat(source): add NATS source consumer parameters (#17615)
• 2af7a5662626704b9f2cdb55958235eb5331bb4d chore(test): sleep after flaky test (#18218)

nightly-20240825

compare changes with previous nightly build

• 5c1af4c21215f8887133233a5978402019c3d3fa feat(iceberg): use native glue impl for iceberg source (#18106)

nightly-20240824

compare changes with previous nightly build

• d33623accf6e932592f4a945af436c18ced812fd chore(build): add curl & less into docker image (#18223)
• 7009743799259775b80611043cb7d089c25f7743 feat(dashboard): visualize average backpressure rather than spot backpressure (#18219)

nightly-20240823

compare changes with previous nightly build

• 5cae57827c8ca56868b9cc0de0686d6f14e68510 fix(udf): allow aggregate: prefixed sql udf (#18203)
• 321610dd9c51ee107a3bcd4e6b32d731d1943044 refactor: add some comments for MV progress tracking (#18110)
• 958794583771632b208068838191a3516cd6d9cc feat(source): support temporary source (#18174)
• ec6074454e39b3b82ee0684f9e714bca95e5bb0c chore(doc): Update README.md (#18206)
• c83ce8632a2f6b1bba83686ec5053c6bfb1c6117 fix(udf): allow udaf as window function (#18181)
• 2fa80346d429a1c5121f0056e07ecbcc1232477b fix(binder): fix function call binding (#18177)
• 4cef811b975cde923d524553a08f1b42d2028534 ci: install addr2line-rs binary along with flamegraph (#18171)
• d27169721cce2fed6858913017642cf7922f0148 feat: use opendal as the s3 sdk by default (#18011)
• d8c718bfa1e118c1eade3e2fa108371d4eff1c79 refactor(test): kafka sink with protobuf/avro as inline style (#18193)
• 71732b1c4446d216f500267a9e1f2bf426a88519 fix: fix kinesis early exit (#18183)

nightly-20240822

compare changes with previous nightly build

• 9c4984d4ec2d17351a741cab8ac512b39dd2261a refactor(iceberg): use load table v2 to extract iceberg columns (#18196)
• 84b7b9444c794886b40970212c9181545249e948 refactor: make telemetry event a separate crate (#18192)
• 955c6bba246f116d77f334864836a40fb17b5c84 feat(iceberg): add license for iceberg sink with glue (#18187)
• c101a3066a3d3f33b7c216ffdac9dc5d18cf4aba fix(sqlparser): fix subclause order when parsing function calls (#18130)
• 4a5bc4cbf20c16600ba645fa0b849955d09019c2 feat(cdc): add event log and fill table catalog for auto schema change (#18060)
• c37ca316b01526a75d145bc63f76e00c11cfff04 feat(storage): adjust base compaction limitation (#17979)
• dcdf421f8489853adf6e3ede18d53804bbf1ef97 chore: remove whitespace from ci-profiling.md (#18178)
• 477115c6bdb4ae709e5066181b3b15f01b14c5a4 doc: add docs on profiling on osx (#18175)
• fe56ce2d78e4faa700a7bf546dea82d73295006c fix(pgwire): allow configuring tcp keepalive idle time (#18136)
• 7ebe0a2a5e7cd6e5b18de39bd165e94f3956190e feat(sink): introduce file sink in PARQUET format (#17311)
• a4a4583019fa42942bdefaa5f9badc21e50983b1 test(mssql-cdc): enrich tests and small refactor (#18081)
• da46c4dd2be4c69d77f8347fbe18b72d6653cfaf refactor: register actor barrier sender asynchronously (#18104)

nightly-20240821

compare changes with previous nightly build

• dd6515625d980443ae1a71cf6332438cdbd49e86 fix: fix log store truncate offset incorrect check (#18165)
• ff55fcf2bd2e70c0abb67f9d18806c38aaf7982f feat(compactor): compaction iterator support filter key with key_range (#18031)
• fc3f4e10ac11745ba1879c28360394b04a6b39df fix(meta): correctly persist license key system param in SQL backend (#18163)
• d3ba75c45dc174375e4a9394258ad7d9832c5194 fix(meta): fix time travel metadata for partial checkpoint (#18160)
• 723833ac3d7e5be1bd07aa35f7dfe2525902da57 fix: check split assignment before pushing mutation (#18134)
• 5d35cf3686b46298b76d9c9dd29ff62cb5960cf9 refactor(meta): only bail out on server-side error when registering worker node (#18127)
• ac655ea49796f495ee7fc129280b23428a0c38c7 feat: support custom kafka group id prefix (#18115)
• f94ec7ef10dad1c23ba3989ea2ba1059e467e6b2 fix(ci): fix bad indent of cassandra sink test (#18150)
• 4f095a05276b2928ecf1d49c813c27708d25bdb9 chore: patch cargo patch-production profile (#18139)
• ed73a52ccf6ee2caa2bd475efb5414eca40cd8a9 perf(stream): set noop_update_hint when jsonb access exists (#18065)
• 4b32136fe06ec284fb1466fdcadce40ffedef414 refactor(expr): show candicates when function not found (#18078)
• 6b5e364873401b42d75dbe3cba59f4787ebeaae0 ci: fix log size limit failure by increasing limit (#18117)
• 6d8abb7bc1a115d6c386e9edba924ebf916ceb61 fix: flaky udf e2e error ui test (#18132)
• 281a6960bc99b7ab2b9000954e1eec2a2673a62a feat(ci): add fast-docker-build-arm workflow (#18124)

nightly-20240820

compare changes with previous nightly build

• e6e1f485cc0f778d4f67e33a362d67359e4fa736 refactor(grafana): provide finer grain filter for object store op (#18004)
• eb1cae5601e5283e6bf4c2db4aca3a8dd2812fe7 fix: overwrite splite assignment for config change (#18128)
• d469cc353f1a257c7e8db630157f7efa0b56f893 fix(meta): fix cpu core limit check query (#18122)
• 26a57216db4c41a2da7acef02c1d9ceb7a87b5af feat(cdc): auto schema change for mysql cdc (#17876)
• 190875c83c51ceeacee688512e2d59474cd4989a fix: remove overly strict assertion during stream pause/resume (#18131)
• aba3232cae52426211c261d43b456e778185a793 fix(sink): fix sink in to Cassandra failed when using column name containing upper case letter (#17493)
• e383ad60cb3a07c0008b53921816271fd64979a1 fix(connector): file source do not panic when credential is wrong (#17935)
• 747245f12815715f6df6a3b97eafc1267578292d fix: udf error ui (#18118)
• 0595aa1d901b2f99811ba1e7e40161b395ddd6a4 feat(connector): add support for path-style access in Iceberg sink connector (#17747)
• d5e832cbe0339f5deedab3baa18b8f0df8f91772 feat(test): switch meta to sql backend in simluate test (#15612)

nightly-20240819

compare changes with previous nightly build

• 11f0acfe548ff0f317c979e51570041058955b9e feat(iceberg): bump iceberg-rust to v0.3.0 (#18088)
• f98904d6f06117827883f2c9823adb0aa3eb819c feat: support more map functions (#18073)
• a9bfcd27e4e748d476ead2d606037049cd3cff44 fix(meta): adapt time travel for partial checkpoint (#18093)
• da74340e3ed5a1d7cbf1657343612cf74956e610 fix(parser): fix unparse of include columns (#18098)
• 7f80900147bc3da2db31cf386bb25a1b86cbefc0 feat(license): add limit for total cpu cores in the cluster (#18022)
• f1fd63ed1470fb076971df43deb1f85af5027a7a feat(frontend): support single phase approx percentile in batch (#18083)
• 7b97788491a5fb8389c7ad0f849810ad591efc57 fix(schedule): Restrict parallelism that exceed the virtual node limit during automatic scaling. (#18006)
• 09865088ffd0077779665e8d409a3e6fe67d1ee4 refactor(meta): only retry on connection error when registering worker node (#18061)
• f54c7fe03198f5dd0c8a822763c380414290ce7f feat: Optimize the performance of scaling in the SQL backend. (#16722)
• a10e61f8500de49ca669638db82f52209fbc22f6 test(frontend): test two phase approx percentile with group key is banned (#18085)
• 1699b3ed093ba7c516f1ce2da7ad7669cd01f8c5 refactor: reduce periodical logs (#18080)
• d16847d1057990cc1ef4c4283503d39d64a08f75 feat(frontend): bind default value for approx_percentile relative_error (#18082)
• 33dc6fd65c9332e6c2751021652e5766802cf87e chore: remind labeling new features or improvements as experiemental if needed (#18084)
• e36a630dfe0ee60ac9b4293c76f87fc2f035a59b chore: fix RustRover macro expansion (#18077)

nightly-20240818

compare changes with previous nightly build

• 7cb75e565470570fd191fac062e91864d4582c98 feat: support constructor MAP {..} and index operator m[k] for map (#18066)
• d11d3ea1762d300cbb569caa79e40e4dcfb3f2f2 chore: update Cargo package version to 2.1.0-alpha (#18075)
• 83b3647ece55ad04d3b4a565c524f8f415d9a71d fix(stream): fix functional dependencies for row merge (#18072)
• 31e52d4f66745280097e981eb154042cab46296b ci: add cherry-pick 2.0 (#18074)

nightly-20240816

• f5f09a6c15b1291623a1fccc9638035cf19002c0 feat: enable shuffle for snapshot backfill (#18063)