What's Changed

Exciting New Features ✨

• feat: add div0 divnull method by @sundy-li in #14693
• feat(query): grant object visibility check by @TCeason in #14458
• feat: add a new kvapi::Key type: Tenant by @drmingdrmer in #14704
• feat(query): privilege access check need consider if exists by @TCeason in #14700
• feat: add kvapi::Value to define meta-service value behavior by @drmingdrmer in #14708
• feat(geo): support geometry type for create table by @ariesdevil in #14615
• feat(query): support vacuum temporary files by @zhang2014 in #14690
• feat: add parent() method to kvapi::Key to describe the hierarchical structure of meta-service data by @drmingdrmer in #14716
• feat(query): integrate hash join with new filter framework by @Dousir9 in #14689
• feat: add kvapi::ValueType::dependency_keys() to retrieve relation such as table-name-to-table-id and table-id-to-table-meta by @drmingdrmer in #14735
• feat: support multi-statement transactions by @SkyFan2002 in #14562
• feat: DESC[RIBE] [TABLE] by @dracoooooo in #14736
• feat(query): new implementation of analyze table by @sundy-li in #14725
• feat: add Analyze for merge into by @JackTan25 in #14763
• feat(executor): implement time slicing for shared executor scheduling across queries by @dqhl76 in #14770
• feat(sqlsmith): Support generating binary and geometry values by @b41sh in #14758
• feat: get_client_info() returns the server time in milliseconds by @drmingdrmer in #14805
• feat: http handler txn support. by @youngsofun in #14802
• feat(planner): implement explain query in verbose mode by @leiysky in #14801
• feat: add new string view type [part 1] by @ariesdevil in #14662
• feat: upgrade arrow to version 50 by @sundy-li in #14784
• feat(query): introduce arrow-udf-js by @sundy-li in #14799
• feat: support left-related join spilling by @xudong963 in #14828
• feat: impl like selectivity compution by probability predication by @xudong963 in #14846
• feat: add vacuum table result table by @lichuang in #14830
• feat(query): read write inverted index by @b41sh in #14827
• feat: change tracking enabled table support replace into by @zhyass in #14831
• feat: support notification syntax with rpc, and support task error notification integration by @ZhiHanZ in #14845
• feat: support right-related join spilling by @xudong963 in #14853
• feat: add vacuum table dry run result table by @lichuang in #14851
• feat: add transaction and mutli-line SQL script support for task by @ZhiHanZ in #14865
• feat: add vacuum drop table dry run result table by @lichuang in #14870
• feat(query): support variant is_ functions by @b41sh in #14876
• feat(query): support queries queue by @zhang2014 in #14878
• feat(query): create drop inverted index by @b41sh in #14859
• feat(query): show grants support mget table/db name by table/db id. by @TCeason in #14862

Thoughtful Bug Fix 🔧

• fix: do compact first , then do sort for cluster table by @JackTan25 in #14707
• fix(query): filter response session settings with ScopeLevel::Session by @everpcpc in #14702
• fix: fix create or replace table as select bug by @lichuang in #14730
• fix: check if global memory is negative by @xudong963 in #14733
• fix: Don't return empty credentail if load failed by @Xuanwo in #14734
• fix: wrong result for CSV EMPTY_FIELD_AS = STRING. by @youngsofun in #14744
• fix: fix create or replace stage bug by @lichuang in #14745
• fix: force attachment to use EmptyFieldAs::FieldDefault. by @youngsofun in #14767
• fix: fix create or replace stage bug, only remove dir when success by @lichuang in #14766
• fix(planner): use correct alias for column reference in select clause by @leiysky in #14775
• fix(executor): queries pipeline executor schedule incorrectly by @dqhl76 in #14787
• fix: fix hash join spilling hang and add more tests by @xudong963 in #14794
• fix(ast): add missing #[drive(skip)] by @andylokandy in #14823
• fix: Transaction state is not changed correctly when returning error in commit statement by @SkyFan2002 in #14824
• fix(query): today, yesterday, tomorrow need consider tz by @TCeason in #14841
• fix: flaky sqllogic test by @zhyass in #14848
• fix: fix wrong results for left join spilling by @xudong963 in #14860
• fix: allow insert in multiline statment by @andylokandy in #14882
• fix: rewrite ifnull and nvl as coalesce by @PsiACE in #14877
• fix: fix incorrect schema in window expr in subquery with lazy_materialization by @ariesdevil in #14895

Code Refactor 🎉

• refactor: new impl for loading CSV. by @youngsofun in #14645
• refactor: add UdfError::Exist by @drmingdrmer in #14691
• refactor: remove kvapi::Key prefix const by @drmingdrmer in #14701
• refactor: add Humanizer traits for operator display by @leiysky in #14718
• refactor: schema_api::update_mask_policy by @drmingdrmer in #14724
• refactor: rename ShareAccountNameIdent::acount to tenant; remove unused serde derive by @drmingdrmer in #14722
• refactor: support decimal for quantile_cont by @ariesdevil in #14726
• refactor(query): refactor runtime part 4 by @zhang2014 in #14728
• refactor(query): migrate profile to common base by @zhang2014 in #14739
• refactor: refactor hash join spill probe side by @xudong963 in #14743
• refactor(storage): treat the standard stream as append only when only insertion by @zhyass in #14748
• refactor: Add RoleGrantee to provide meta-service key for RoleMgr by @drmingdrmer in #14752
• refactor: upgrade Openraft v0.9.0-alpha.5..v0.9.0-alpha.6 by @drmingdrmer in #14769
• refactor: hash join build spilling by @xudong963 in #14768
• refactor: Add RoleIdent to define kvapi::Key to access roles in meta-service by @drmingdrmer in #14772
• refactor: make IndexNameIdent.tenant a Tenant struct by @drmingdrmer in #14776
• refactor: refactor join spilling settings by @xudong963 in #14781
• refactor: add TenantUserIdent as meta-service key to access user data by @drmingdrmer in #14793
• refactor(executor): tracking memory for plan level by @zhang2014 in #14747
• refactor(ast): rename CreateOption::CreateIfNotExists(false) -> CreateOption::None by @andylokandy in #14795
• refactor(ast): refine crate structure by @andylokandy in #14803
• refactor: upgrade Openraft to v0.9.0-alpha.7 by @drmingdrmer in #14809
• refactor: tenant is not allowed to be empty by @drmingdrmer in #14815
• refactor(ast): add derive-visitor for expr and query by @andylokandy in #14814
• refactor(ast): add derive-visitor for Statement by @andylokandy in #14822
• refactor: remove backward compatibility for data version V0 by @drmingdrmer in #14818
• refactor: use relative expire time ttl to replace expire_at, tolerate query-meta time drift by @drmingdrmer in #14807
• refactor: upgrade openraft v0.9.0-alpha.7 to v0.9.0-alpha.8 by @drmingdrmer in #14832
• refactor: enable feature flag openraft:storage-v2 by @drmingdrmer in #14838
• refactor: use NonEmptyString to access UDFMgr. by @drmingdrmer in #14844
• refactor: use NonEmptyString for creating StageMgr by @drmingdrmer in #14850
• refactor: add StageIdent and StageFileIdent by @drmingdrmer in #14863
• refactor: use KVPbApi to simplify StageMgr by @drmingdrmer in #14871
• refactor: use NonEmptyString to create Settings by @drmingdrmer in #14875
• refactor: enable openraft feature flag generic-snapshot-data by @drmingdrmer in #14881
• refactor: use SettingIdent to build string key to access settings by @drmingdrmer in #14883
• refactor(ast): migrate to derive-visitor by @andylokandy in #14829
• refactor(ast): remove databend-common-functions dependency by @andylokandy in #14893

Build/Testing/CI Infra Changes 🔌

• ci: add tests for transaction. by @youngsofun in #14826
• ci: fix clippy tests. by @youngsofun in #14855

Others 📒

• chore: add plan id for mutations by @JackTan25 in #14685
• chore: install sqlite3 for build-tool by @ariesdevil in #14695
• chore: install libsqlite3 for build-tool by @ariesdevil in #14696
• chore: minor refactor of adjust_bloom_runtime_filter by @xudong963 in #14697
• chore: extract common functions from metactl by @drmingdrmer in #14698
• chore(query): support custom query graceful shutdown timeout by @zhang2014 in #14712
• chore(query): improve new filter execution by @Dousir9 in #14715
• chore: add metrics for compact and recluster by @zhyass in #14717
• chore: static features for async-compression deps by @everpcpc in #14711
• chore: remove stream pruner by @zhyass in #14723
• chore(query): improve aggregate unary function by @sundy-li in #14729
• chore: polish error message. by @youngsofun in #14737
• chore: rename by @JackTan25 in #14740
• chore: refactor RoleMgr by @drmingdrmer in #14741
• chore: build-tool base for both arm64 & amd64 by @everpcpc in #14751
• chore: fix setup protoc aarch64 by @everpcpc in #14754
• chore: fix compile error by @SkyFan2002 in #14753
• chore: change build image to debian/alpine by @everpcpc in #14755
• chore(ci): fix LIBCLANG_PATH for alpine build-tool by @everpcpc in #14764
• chore(query): fix location ver by @sundy-li in #14762
• chore(query): add max_running_query_execute_time for http status by @zhang2014 in #14750
• chore(query): support string eq int by @sundy-li in #14759
• chore: add more transaction test by @SkyFan2002 in #14746
• chore(ci): change build runner with target by @everpcpc in #14761
• chore: fix build for cloud by @everpcpc in #14773
• chore(query): use create_time of process info for http status by @zhang2014 in #14777
• chore: show create table with location by @ariesdevil in #14778
• chore(query): add drop guard for impl drop trait by @zhang2014 in #14783
• chore(ci): fix target for release by @everpcpc in #14785
• chore(query): remove codes of storage/parquet2 by @sundy-li in #14305
• chore: patch z3-sys & add musl build to ci by @everpcpc in #14786
• chore: fix musl build dockerfile by @everpcpc in #14790
• chore: release add musl targets by @everpcpc in #14791
• chore(test): reduce the number of tables in large join test by @Dousir9 in #14797
• chore: add Date Type for runtime filter by @JackTan25 in #14780
• chore: rename update_star_columns by @JackTan25 in #14806
• chore(query): add unit test for mem stat by @zhang2014 in #14812
• chore(ci): include nohup.out in failure artifact by @everpcpc in #14820
• chore: remove unused package patch by @everpcpc in #14821
• chore(ci): fix upload/download sha256sums artifacts by @everpcpc in #14833
• chore(ci): fix release artifacts conflict by @everpcpc in #14834
• chore(ci): fix artifacts for benchmark by @everpcpc in #14837
• chore: tolerate inaccuracy of expire time for test by @drmingdrmer in #14836
• chore(query): modify new_agg_hashtable payload transmission method on singleton and support new_agg_hashtable on cluster by @Freejww in #14798
• chore: Enable stage table distributed by @JackTan25 in #14847
• chore: disable inlist to subquery under cluster by @xudong963 in #14864
• chore(query): enable fast return for cluster hash join by @Dousir9 in #14854
• chore: log txn id for multi stmt transaction by @SkyFan2002 in #14861
• chore: improve tolerance for expire test test_kv_read_v1_on_follower by @drmingdrmer in #14868
• chore: open join spilling by default by @xudong963 in #14843
• chore: increase tolerance for expiration test by @drmingdrmer in #14880

New Contributors

• @dracoooooo made their first contribution in #14736

Full Changelog: v1.2.344...v1.2.371