This release adds support for new ANN backends along with a bonanza of bug fixes from 7 new contributors!

See below for full details on the new features, improvements and bug fixes.

New Features

• Feature request: Support zvec as vector backend (#1040, #1130) @morgan-coded
• Add milvus-lite embedded dense ANN backend (#1149) @morgan-coded
• Allow disabling API routes (#1165)

Improvements

• Add tokenbatch size to HFTrainer pipeline (#1133)
• Only resolve transform functions when env variable set (#1155)
• Ensure explain / batchexplain don't return numpy values (#1157)
• Set minimum milvus-lite version (#1162) @morgan-coded
• Add safeopen option for retrieve task (#1166)

Bug Fixes

• Raise SQLError on unterminated bracket, function and similar clauses (#1125) @Sanjays2402
• Torch 2.13 introduced build errors (#1134)
• Fix FileTask/UrlTask crash on non-string elements (#1135) @chuenchen309
• Fix Graph.filter() crash on ids that don't exist in the graph (#1137) @chuenchen309
• fix: Task.filteredrun crashes/corrupts output for single-action tasks with merge disabled (#1138) @chuenchen309
• Fix Aggregate IndexError on a SQL query that matched nothing across shards (#1139) @chuenchen309
• Fix HTMLToMarkdown crash on a meta description tag with no content attribute (#1140) @chuenchen309
• Fix TemplateTask.match KeyError when a rule key is missing from element (#1141) @chuenchen309
• Fix Terms KeyError on a query with no similar() clause (#1142) @chuenchen309
• Fix Terms.search AttributeError on an empty terms index (#1143) @chuenchen309
• Fix msgpack Accept header crashing streaming API endpoints (#1144) @chuenchen309
• Fix console printing a traceback for a query with no results (#1145) @chuenchen309
• Fix Cluster.shard TypeError on tuple documents and empty-string ids (#1146) @chuenchen309
• Raise SQLError on empty select expressions and a dangling AS (#1147) @chuenchen309
• Fix LIMIT bind-parameter crashing content SQL search (#1148) @chuenchen309
• The authorization header doesn't work on Swagger UI (#1150, #1151) @link89
• Fix Segmentation dropping paragraph/section splits on CRLF line endings (#1152) @lntutor
• Validate resolver target classes (#1153)
• Fix IVFSparse returning unsorted sparse ANN search results (#1154) @winklemad
• Fix build error with Milvus on Windows (#1156)
• HNSW search crashes with RuntimeError when limit exceeds indexed element count (#1158) @AmirF194
• Fix HNSW search RuntimeError when limit exceeds indexed element count (#1159) @AmirF194
• Fix Scoring.delete() crashing on ids that were never indexed (#1163) @AmirF194

This release adds support for the turbovec ANN backend and LiteParse text extraction

See below for full details on the new features, improvements and bug fixes.

New Features

• Add turbovec an ANN backend (#1109)
• Add text extraction with LiteParse (#1118)

Improvements

• Migrate from ggml-py to ggml-python (#1105)
• Revert noisy logging workaround when fixed upstream (#1106)
• Add reindex parameter for API (#1111)
• Use gliner fork to relax transformers version caps (#1112)
• Remove find from the default list of bash tools (#1113)
• Support multiple Dense layers for Late Interaction Pooling (#1117)
• Limit tabular pipeline to local CSV files (#1119)
• Update commonprefix to commonpath (#1120)

Bug Fixes

• fix(tokenizer): respect lowercase=False in ngramtokenize (#1114, thank you @JSap0914)
• FastAPI 0.137+ modified how routers work (#1115)

This release adds LiteRT vector support and Knowledge Distillation training

See below for full details on the new features, improvements and bug fixes.

New Features

• Add support for LiteRT vectors (#1097)
• Add URLRetrieve pipeline (#1100)
• Add a Knowledge Distillation Trainer (#1103)
• Add a Progressive Distillation Example Notebook (#1104)

Improvements

• Reduce noisy logging messages with Transformers v5 (#1102)

This release adds a zero dependency minimal install, LiteRT and Transformers v5 support

See below for full details on the new features, improvements and bug fixes.

New Features

• Zero dependency minimal install (#1089, #1090, #1091, #1092, #1093, #1094)
• Transformers v5 Support (#1083, #1084, #1085, #1086, #1087, #1088)
• Add LiteRT-LM LLM (#1095)

This release is primarily a bug fix, security and compatibility release.

See below for full details on the new features, improvements and bug fixes.

New Features

• Add support for last token pooling (#1072)
• Add support for streaming Labels pipeline (#1075)
• Add safeopen parameter for Textractor (#1077)

Improvements

• Support Authorization tokens with MCP endpoints (#899)
• Update embeddings documentation to mention loading ONNX vector models (#1071)
• Performance Improvement: Use JOIN when JSON fields being used (#1078)
• Limit template scope with Agents (#1080)

Bug Fixes

• Security issue with LiteLLM (#1065)
• Issue with latest version of llama-cpp-python (#1066)
• Issue with latest version of Model2Vec (#1067)
• UnsupportedProtocol: Request URL is missing an 'http://' or 'https://' protocol (#1073)

This release adds the new TxtAI Coding Agent Toolkit

See below for full details on the new features, improvements and bug fixes.

New Features

• Add coding agent toolkit (#1054, #1055, #1056, #1057, #1058, #1059, #1060, #1061)
• Add agent tools example notebook (#1062)

Improvements

• Apply sigmoid to fused BB25 scores (#1053)

Bug Fixes

• Fix translation chunk concatenation when showmodels is enabled (#1052 Thank you @jnMetaCode!)

This release adds many great new Embeddings database features: BB25 normalization, ngram tokenization, database expression indexes and conditional JSON data storage

See below for full details on the new features, improvements and bug fixes.

New Features

• Add Bayesian normalization "bb25" method for keyword scoring (#1023, #1037, Thank you @sigridjineth)
• Add BB25 normalization for sparse encoders (#1042, #1046 Thank you @jaepil)
• Add log-odds conjunction fusion for BB25 hybrid search (#1041 Thank you @jaepil)
• Add keyword ngram tokenizer (#1017, #1038)
• Add support for custom SQL functions with DuckDB (#1047)
• Add deterministic setting to database functions (#1048)
• Add ability to index database expressions (#1049)
• Add store field to columns configuration (#1050)

Improvements

• Cache Agent Template (#1051 Thank you @HemantSudarshan)

Bug Fixes

• Fix: escape single quotes in dynamic column names (#1026 Thank you @RinZ27)
• Fix: cursor error with multithreaded keyword search (#1039)

TxtAI got skills! This release adds a lot of great new agent capabilities: agents.md, skill.md and memory

See below for full details on the new features, improvements and bug fixes.

New Features

• Add support for agents.md (#1014)
• Add support for skill.md (#1015)
• Add support for Agent memory (#1016)
• Add ability to store agent memory by session (#1025)
• Add example demonstrating how to use agents.md files (#1018)
• Add example demonstrating how to use skill.md files (#1019)

Improvements

• Pad to multiples of 8 for both fp16 and bf16 (#1020)
• Add input merging modes for language modeling training (#1021)

Bug Fixes

• Fix issue with latest version of FastAPI (#1022)

This release adds the following new features, improvements and bug fixes.

Note that the Docker build will not be updated since it installs all packages by default;

Bug Fixes

• httpx import at module level causes ModuleNotFoundError for Embeddings-only usage (#1012)
• Add minimal build workflow (#1013)

Adds OpenCode integration, improved instruction prompts support for vectors, additional keyword tokenization methods and more

See below for full details on the new features, improvements and bug fixes.

New Features

• Add OpenCode LLM (#1010)
• Add additional tokenization methods for keyword search (#1006)
• Add example notebook covering OpenCode LLM (#1011)
• Add example notebook showing how to finetune LLMs (#1008)

Improvements

• Add addobject to API endpoints (#959)
• Load instruction prompts for Sentence Transformers models (#1004)

Bug Fixes

• Add KMP_DUPLICATE_OK workaround for Faiss OMP: Error 15 (#1009)
• Fix Baseball Sample Application (#1003)