The infino-python package provides high-performance Python bindings for the Infino engine using PyO3 and maturin It allows Python applications to perform SQL, full-text (BM25), and vector search directly over object storage without requiring a server or daemon infino-python/src/lib.rs4-15
The bindings act as a thin, synchronous wrapper around the Rust core catalog API infino-python/src/lib.rs13-15 Data exchange between Python and Rust is handled via the Arrow C Data Interface, ensuring zero-copy or low-overhead transfers for large datasets infino-python/src/lib.rs9-11
| Python Entity | Rust Core Entity | Role |
|---|---|---|
infino.connect() | infino::connect_with | Entry point to open a catalog at a URI infino-python/src/lib.rs93-140 |
Connection | infino::Catalog | Manages tables and executes cross-table SQL infino-python/src/lib.rs191-256 |
Table | infino::Table | Handle for append-only, snapshot-isolated data operations infino-python/src/lib.rs331-507 |
IndexSpec | infino::IndexSpec | Fluent builder for defining FTS and vector indices infino-python/src/lib.rs145-189 |
The Table.append() method is polymorphic, accepting pyarrow.Table, pyarrow.RecordBatch, pandas.DataFrame, or a list[dict] infino-python/src/lib.rs370-406
Sources: infino-python/src/lib.rs370-406 infino-python/tests/test_smoke.py175-198
infino.connect(uri, **kwargs) initializes the storage backend. It supports S3-compatible stores with explicit credentials and a tiered disk cache infino-python/src/lib.rs93-104
memory:// (ephemeral), ./data (local), s3://bucket/prefix (cloud) infino-python/tests/test_smoke.py51-109cache_dir, cache_budget_bytes, and cold_fetch_mode (e.g., "hybrid_with_prefetch") infino-python/src/lib.rs94-96storage_options as a dictionary for backend-specific config like aws_region infino-python/src/lib.rs110-115Tables are created with an IndexSpec that defines which columns receive full-text or vector indices infino-python/src/lib.rs145-151
bm25_search(column, query, k, mode="or"): Performs ranked lexical search. Returns a pyarrow.Table containing _id and score infino-python/src/lib.rs423-445vector_search(column, query, k, nprobe=None, filter_column=None, filter_query=None): Performs kNN search with optional pre-filtering infino-python/src/lib.rs458-491delete(predicate) / update(predicate, rows): Executes mutations based on SQL-like filters, returning MutationStats infino-python/src/lib.rs345-368The Connection.query_sql() method allows running DataFusion-powered SQL. Infino exposes search capabilities as TVFs for relational composition infino-python/tests/test_smoke.py122-129
| Function | Usage |
|---|---|
bm25_search | SELECT * FROM bm25_search('table', 'col', 'query', 10) |
vector_search | SELECT * FROM vector_search('table', 'col', [0.1, ...], 5) |
Sources: infino-python/src/lib.rs219-256 infino-python/tests/test_smoke.py112-145
Rust InfinoError variants are mapped to standard Python exceptions infino-python/src/lib.rs38-50
| Rust Variant | Python Exception |
|---|---|
NotFound | KeyError |
Schema, Query, Cardinality, AlreadyExists | ValueError |
Io, Backend | RuntimeError |
This diagram illustrates how a Python search call traverses the binding layer into the Rust core.
Sources: infino-python/src/lib.rs423-456 infino-python/tests/test_smoke.py58-62
The package uses maturin for builds and pytest for validation infino-python/tests/test_smoke.py1-9
maturin develop creates a symlink in the environment infino-python/tests/test_smoke.py6pyarrow and duckdb can read Infino superfiles as standard Parquet infino-python/tests/test_parquet_interop.py98-125The package includes a py.typed marker and a _infino.pyi stub file to provide full IDE autocompletion and type checking support infino-python/tests/test_smoke.py31-37
Wheels are built for multiple platforms (Linux x86_64/ARM, macOS Intel/Silicon) using the abi3 stable Python ABI, covering CPython >= 3.9 .github/workflows/publish-python.yml12-16
The repository provides extensive examples in infino-python/examples/:
Sources: .github/workflows/publish-python.yml86-102 infino-python/tests/test_smoke.py31-37 infino-python/examples/README.md1-19
Refresh this wiki