This release adds many great new Embeddings database features: BB25 normalization, ngram tokenization, database expression indexes and conditional JSON data storage

See below for full details on the new features, improvements and bug fixes.

New Features

• Add Bayesian normalization "bb25" method for keyword scoring (#1023, #1037, Thank you @sigridjineth)
• Add BB25 normalization for sparse encoders (#1042, #1046 Thank you @jaepil)
• Add log-odds conjunction fusion for BB25 hybrid search (#1041 Thank you @jaepil)
• Add keyword ngram tokenizer (#1017, #1038)
• Add support for custom SQL functions with DuckDB (#1047)
• Add deterministic setting to database functions (#1048)
• Add ability to index database expressions (#1049)
• Add store field to columns configuration (#1050)

Improvements

• Cache Agent Template (#1051 Thank you @HemantSudarshan)

Bug Fixes

• Fix: escape single quotes in dynamic column names (#1026 Thank you @RinZ27)
• Fix: cursor error with multithreaded keyword search (#1039)