Infino includes a comprehensive in-tree benchmarking infrastructure designed to measure performance across the entire storage stack, from single-artifact Superfile reads to multi-segment Supertable operations in cloud object storage. The system is split between a custom Rust-based benchmark harness and an automated CI performance pipeline.
infino-bench-utils)The benchmark harness is a standalone framework located in the benches/ directory. It avoids external benchmarking libraries to maintain direct control over the measured lifecycle, including deterministic corpus generation, artifact building, and resource sampling benches/README.md7-17 The harness is exposed as a library infino-bench-utils and consumed by the benchmarking binary benches/utils/corpus.rs8-12
The harness evaluates Infino across a matrix of storage tiers and query modalities:
| Tier | Description | Default Scale |
|---|---|---|
| Superfile | Single-artifact, in-memory read path benches/README.md32-33 | 1M Docs benches/utils/corpus.rs147-154 |
| Supertable | Multi-artifact table in object storage with warm/cold paths benches/README.md34-36 | 10M Docs benches/utils/corpus.rs149-154 |
Modalities:
To ensure measurement accuracy, the harness executes each matrix cell in a dedicated child process. This prevents memory residue from previous tests (e.g., allocator-retained pages) from polluting the Resident Set Size (RSS) metrics of subsequent runs benches/README.md23-28 The harness uses block_on_inmem to drive async search kernels on a shared multi-thread tokio runtime benches/utils/corpus.rs66-76
For details on configuration and local execution, see Benchmark Harness (infino-bench-utils).
Sources: benches/README.md1-41 benches/utils/corpus.rs1-160 benches/utils/tiers.rs61-77
Infino utilizes GitHub Actions to enforce performance and correctness gates on every pull request. The infrastructure is designed to provide "production-grade" numbers by running benchmarks on ephemeral cloud hardware.
The Benchmark (Azure) workflow provisions a dedicated Azure VM (e.g., Standard_D8s_v7) to run A/B comparisons between a PR and main .github/workflows/supertable-bench-azure.yml28-32
eastus) to ensure transfers stay on the Azure backbone .github/workflows/supertable-bench-azure.yml3-7sccache with an Azure Blob Storage backend .github/workflows/supertable-bench-azure.yml5-7s3, azure, gcs, and s3s_fs (emulator) backends, selected via INFINO_BENCH_STORE benches/utils/tiers.rs9-13The CI pipeline includes a public-api guard to prevent accidental drift in the crate's surface area .github/workflows/ci.yml49-68 Benchmark results can be published as sticky PR comments or Actions run summaries .github/workflows/supertable-bench-azure.yml115-117
For details on the CI pipeline and cloud resource management, see CI/CD & Performance Infrastructure.
Sources: .github/workflows/supertable-bench-azure.yml1-117 .github/workflows/ci.yml49-68 benches/utils/tiers.rs1-13
Performance metrics are only valid if the results are correct. Infino uses deterministic corpora and "Brute-Force Oracles" to validate search paths during benchmark runs.
The following diagrams illustrate how the benchmark harness relates to core system entities:
Benchmark Execution Flow
Sources: benches/utils/ingest/supertable.rs1-87 benches/utils/corpus.rs24-35
Storage Tier Architecture
Sources: benches/utils/tiers.rs20-28 benches/utils/tiers.rs63-77
The harness supports "Dataset Mode," where a supertable is ingested once to a fixed prefix and then reused for multiple search benchmarks benches/utils/dataset.rs6-12 A dataset.json sidecar tracks the Knobs (doc count, dimension, seeds) to ensure the reader and writer agree on the corpus parameters benches/utils/dataset.rs42-53
Sources: benches/utils/dataset.rs1-71 benches/README.md135-153