Skip to content

Releases: ralforion/orionbelt-semantic-layer

v2.23.1

Choose a tag to compare

@ralfbecher ralfbecher released this 22 Jul 10:20
0be25a2

[2.23.1] - 2026-07-22

Fixed

  • Quarter- and week-grain period-over-period metrics failed on four dialects (#241). A period-over-period query at quarter grain (e.g. Sales QoQ Ratio with a Sales Quarter dimension) crashed or produced invalid SQL because the prior-period date was computed with an unsupported unit: Dremio emitted DATE_ADD(d, INTERVAL '-1' QUARTER) (QUARTER/WEEK are not valid Calcite interval qualifiers — now TIMESTAMPADD, which also fixes week), Postgres emitted INTERVAL '-1 quarter' (invalid interval input — now expressed as months), and Databricks and ClickHouse raised on the unsupported unit (now add_months×3 and native addQuarters). MySQL, BigQuery, Snowflake, and DuckDB were already correct.
  • Dremio miscompiled the previousValue period-over-period comparison (#241). Sales Previous Year failed on Dremio with FUNCTION ERROR: Value <n> for monthOfYear must be in the range [1,12] — its executor read the self-joined decimal column's bytes as the output date. Comparisons that also reference the current-period measure (ratio/difference/percentChange) plan correctly; previousValue now references the base measure in a value-preserving way so the plan holds.
  • Grain-to-date cumulative metrics emitted a hardcoded DATE_TRUNC (#241). MTD Sales / YTD Sales failed on MySQL (FUNCTION DATE_TRUNC does not exist) and used BigQuery's wrong date-part syntax, because the grain-to-date PARTITION BY bypassed the dialect. It now routes through the dialect's time-grain node (MySQL DATE_FORMAT, ClickHouse toStartOfMonth, etc.), the same one time-grain dimensions use.
  • BigQuery rejected any query grouped by a time-grain dimension (#242). render_time_grain emitted DATE_TRUNC(col, 'month') (a quoted string), but BigQuery requires the bare date-part keyword MONTH / ISOWEEK (A valid date part name is required). This affected time-grain dimensions and cumulative metrics.
  • Snowflake could not execute period-over-period metrics (#242). The date-spine CTEs were declared quoted (WITH "date_range") but referenced bare (FROM date_range); Snowflake folds a bare identifier to upper case, so it could not find the quoted lower-case CTE (Object 'DATE_RANGE' does not exist). CTE references are now quoted to match, consistent with the star/CFL path. Separately, the Snowflake spine passed a non-constant row count to GENERATOR(ROWCOUNT => ...) (which requires a constant); it now generates a fixed upper bound and filters to the range.

Added

  • Per-vendor measure/metric execution sweeps (#242, #244). New integration tests execute every measure and metric of the commerce model against each engine — local Postgres/MySQL/ClickHouse/DuckDB testcontainers, live Dremio via pgwire, live BigQuery/Snowflake/Databricks, and a pyspark-gated local Spark run for the Databricks dialect — asserting the compiled SQL actually runs. This catches dialect SQL that compiles cleanly but the engine rejects at run time, the gap that let the fixes above ship undetected. The Databricks integration seed also gains a bulk PUT + read_files path (#243) so it completes within a SQL warehouse's auto-stop window.

Full Changelog: v2.23.0...v2.23.1

v2.23.0

Choose a tag to compare

@ralfbecher ralfbecher released this 21 Jul 11:07
3a48f90

[2.23.0] - 2026-07-21

Changed

  • High-precision DECIMAL values are preserved exactly across every surface (issue #136). Values wider than IEEE-754 double precision (past ~15-16 significant digits) previously lost precision because the executor cast every Decimal to float, so 123456789012345678.90 came back as 123456789012345680.00. The Decimal is now carried end to end. This changes two consumer-visible output shapes: raw REST JSON delivers DECIMAL cells as exact decimal strings (e.g. "123456789012345678.90") instead of rounded floats, a fixed value-independent contract now documented on the columns[].type field; and Arrow Flight advertises governed DECIMAL measures/metrics as DECIMAL(p, s) instead of DOUBLE, so BI tools over JDBC/ODBC see the precise type. The pgwire NUMERIC surface (already exact since #116), the shared result cache, and value-formatted / TSV output keep their shape and are now exact too. Note: the MCP server relays REST query results, so its consumers see the same decimal-string shape (no MCP change required for a passthrough client).

Fixed

  • Arrow Flight catalog probes were rejected under sqlglot 30 (#237). sqlglot 30 renamed the SELECT node's FROM argument key (from to from_), so information_schema / pg_catalog discovery queries fell through the classifier and returned a malformed result instead of the model's table/column listing, breaking BI-tool schema discovery (DBeaver, Tableau) over Flight SQL. The Flight driver test suite was not run by CI, which let this regress unnoticed.
  • Driver package test suites now run in CI (#239). The root testpaths excluded drivers/*, so the Flight and ClickHouse driver suites were never exercised; both had silently rotted (the sqlglot 30 break above, and the ClickHouse tests had drifted from the driver's query_arrow() migration, realigned in #238). Each driver package's suite now runs from its own directory in the test job.

Full Changelog: v2.22.2...v2.23.0

v2.22.2

Choose a tag to compare

@ralfbecher ralfbecher released this 19 Jul 06:44
04410f8

[2.22.2] - 2026-07-19

Fixed

  • OSI converter renamed dimensions to their physical column code on the round trip (osi-orionbelt 0.1.2). The OSI field name is the physical column code, so OBML -> OSI -> OBML renamed every dimension to its code (Order Placed On came back as order_dt) and could then trip the dimension-name collision fallback for names that were unique in the source. The dimension name is now preserved in an obml_dimension_name extension and restored on import, so the space-qualified collision fallback is genuinely foreign-OSI only. A non-string value in that extension is ignored rather than crashing the converter on an unhashable dict key.
  • OSI converter silently dropped multiple dimensions on one column (osi-orionbelt 0.1.2). OBML allows several dimensions over one column (grain variants, role-playing via via); OSI represents one dimension per field, so every dimension past the first was dropped with no warning. Extra dimensions are now preserved in an obml_extra_dimensions extension (with a fidelity warning on export) and rebuilt on import, each carrying its own synonyms and vendor extensions. Malformed extension payloads are filtered, never crash.
  • obsl convert did not surface input schema violations. Both the local and --server convert paths now report OSI/OBML input schema errors (advisory - the conversion still runs), matching the REST convert endpoints, instead of silently coercing invalid input.

Changed

  • label is no longer an authorable property on dimensions, measures, and metrics. It was a vestige of the old list-keyed-by-label format that the resolver silently ignored - the identity comes from the mapping key, exactly as it already did for dataObject and column labels. Authoring label: on one of these now fails schema validation and is rejected by the parser. No shipped model authored it; description and synonyms cover the adjacent needs. Internally the resolved identity field was renamed label -> name to match what it holds.

Full Changelog: v2.22.1...v2.22.2

v2.22.1

Choose a tag to compare

@ralfbecher ralfbecher released this 15 Jul 22:56
97491e8

[2.22.1] - 2026-07-16

Fixed

  • The 2.22.0 UI Docker image could not start. import gradio raised ModuleNotFoundError: No module named 'requests', so orionbelt-semantic-layer-ui:2.22.0 exited immediately on boot and the Cloud Run UI deploy rolled back. Gradio below 5.50 eagerly imports its CLI from __init__, which chains to import requests; requests is not a gradio dependency and nothing in the ui extra provides it. Only the UI image was affected: the API, Flight, PyPI packages, and the running deployment were fine throughout (Cloud Run kept serving the previous UI revision).
  • The ui extra now floors gradio at 5.50 (was >=5.0). The old floor let a Dependabot group update (#206) resolve gradio down to 5.23.1 in order to keep a pydantic bump to 2.13.4, which gradio 5.50.0 caps at <=2.12.3. This restores gradio 5.50.0 and pydantic 2.12.3, the combination 2.21.1 shipped.

Added

  • CI now imports the UI against the dependency set the UI image actually ships (uv sync --no-dev --extra ui). Every existing job installs --all-extras --all-groups, which pulls mkdocs-material and with it requests, so a gradio that cannot import in the shipped venv passed all of CI. Nothing built or ran a UI container before the release tag, and the tag-triggered image build never started the container it published.

Full Changelog: v2.22.0...v2.22.1

v2.22.0

Choose a tag to compare

@ralfbecher ralfbecher released this 15 Jul 22:11
3591137

[2.22.0] - 2026-07-15

Added

  • Python 3.14 support. All three Docker images (API, UI, Flight) now build on python:3.14-slim, and 3.14 joined the CI test matrix alongside 3.12 and 3.13. Reaching it required snowflake-connector-python 4.3.0 -> 4.6.0 and pyarrow 19.0.1 -> 25.0.0: the previously locked versions had no cp314 wheels, so the slim image tried to build them from source and failed.

Changed

  • sqlglot 30. The constraint moved from >=26.0,<27.0 to >=30.0,<31.0. sqlglot 30 renamed the Select args that collide with Python keywords (from -> from_, with -> with_) and made exp.Expr the common base class, so the OBSQL translator (compiler/sql_translator.py) and the pgwire federation-subquery flattening (pgwire/router.py) were updated to match. No behavior change, but installs pinning sqlglot 26 will need to move.
  • Dependency constraints widened: pyarrow >=16.0,<20.0 -> >=16.0,<26.0 (the old cap had no version-specific rationale and blocked the cp314 wheels the 3.14 images need) and structlog >=25.1,<26.0 -> >=25.1,<27.0.

Fixed

  • OSI converter dropped metrics whose SQL referenced physical codes (osi-orionbelt 0.1.1). OBMLtoOSI emits metric SQL against the physical code (e.g. SUM(fact_orders.amount)), but OSItoOBML resolved references by OSI dataset/field name only, so any metric over a data object whose code differed from its display name was dropped on the return trip. References now resolve by both name and physical code, including quoted physical identifiers (a Snowflake/Databricks "net_amount" field expression or a WH.PUBLIC."fact_orders" source table) that previously fell through to a LOSSY unconverted metric.
  • OSI converter silently overwrote colliding dimension names (osi-orionbelt 0.1.1). _extract_dimensions keyed dimensions by bare field name, so the same field name in two datasets (Orders.date, Invoices.date) meant the second silently replaced the first. The later one is now qualified with its data object and a warning is recorded.
  • OSI converter emitted invalid OBML for fields with spaces in the display name (osi-orionbelt 0.1.1). Resolving a physical code to a display name containing a space spliced Orders.Net Amount into the intermediate SQL, which the downstream parsers split on the space, emitting {[Orders].[Net]} Amount with no LOSSY warning. Resolved names that are not bare SQL words are now bracket-quoted, and the dataset.column parsers accept and unquote bracketed identifiers.
  • validate_osi raised AttributeError on malformed input (osi-orionbelt 0.1.1). Its semantic loops now guard against datasets and fields that are not lists of dicts, returning schema errors instead of crashing.

Full Changelog: v2.21.1...v2.22.0

v2.21.1

Choose a tag to compare

@ralfbecher ralfbecher released this 13 Jul 10:02
7c40ac1

[2.21.1] - 2026-07-13

Fixed

  • Arrow Flight SQL result cache was inoperative since 2.20.0. Flight's cache read/write path called a result-codec API (encode/decode/decode_table) that 2.20.0 replaced with the data-only encode_data/decode_data codec plus a columns_json schema sidecar. Both calls raised an error swallowed by a try/except, so every Flight query silently missed the cache and never stored a result. Flight now uses the current codec and shares one cache entry per compiled query with REST and pgwire.
  • Flight cache hits could stream a schema that did not match what FlightInfo advertised. An empty or all-null result was re-typed from row values on write (collapsing typed columns to null), and a Flight hit streamed that schema even though the client was promised the model's types. Flight now preserves the exact Arrow schema on write and casts a cache hit back to the advertised schema, so the streamed schema is stable for any writer (REST, pgwire, Flight) and any result shape.

Changed

  • Cache key and freshness-TTL derivation is now shared across REST, pgwire, and Flight via a single layer-clean service/query_execution.py (resolve_cache_plan), replacing three copies that could drift. Same compiled query, same key and TTL on every surface. Internal refactor, no behavior change.

Full Changelog: v2.21.0...v2.21.1

v2.21.0

Choose a tag to compare

@ralfbecher ralfbecher released this 09 Jul 07:52
04d3a7e

[2.21.0] - 2026-07-07

Added

  • Cross-session content-addressed model cache. Identical OBML loaded into different sessions now compiles once and is shared, keyed by a stable content-derived model_id, instead of each session recompiling its own copy under a random id. In admin-curated (MODEL_FILES) mode the curated model is compiled once for the process and every user session references it. Shared compiled models are refcounted and evicted once no live session references them; loads that fold in extra state (extends, inherits, raw dict, dedup=false) stay private with a random id. Because model_id is part of the result-cache key, identical model plus SQL now shares result-cache entries across sessions too.

Changed

  • model_id is now stable and content-derived for shared models: identical YAML always yields the same id, including after a remove and reload. Ids widened to 16 hex characters.
  • Demo commerce models use synthesized row-count measures. Two redundant declared count measures were dropped and one relabeled override kept, exercising the auto-synthesized counts introduced in 2.19.0.

Fixed

  • Single-model shortcut resolution now scopes to protected sessions. In admin-curated single-model mode, the top-level shortcut endpoints ignore transient user sessions, so a duplicated per-session copy of the curated model no longer triggers a spurious "multiple models loaded across sessions" error.

Full Changelog: v2.20.0...v2.21.0

v2.20.0

Choose a tag to compare

@ralfbecher ralfbecher released this 06 Jul 16:56
fa04b7a

[2.20.0] - 2026-07-06

Added

  • Ontology Graph tab additions. An Export Onto button downloads the model's OBSL RDF ontology (Turtle), synthesized row-count measures now appear in the graph, and the in-graph toolbar gained rotate-left / rotate-right controls plus a more visible PNG export button.
  • Ontology predicates obsl:anchoredTo and obsl:referencesColumn. Grain-anchored measures (auto-synthesized row counts) declare their object grain via obsl:anchoredTo, a new measure source form; expression-based measures declare the columns their formula reads via obsl:referencesColumn (a dependency edge, distinct from the declared-columns[] obsl:sourceColumn). The ontology, SHACL shapes, and spec were updated together, and exported graphs stay valid against obsl.shacl.ttl.

Changed

  • The Ontology Graph is now rendered from the exported OBSL ontology rather than re-derived from the model, so the graph and the RDF export can no longer drift.
  • format=arrow response is now a length-prefixed frame (application/vnd.orionbelt.result+arrow): [u32 json length][JSON envelope][gzip'd Arrow data]. The row data is cached; the metadata envelope (sql, explain, timing, cached, columns) is rebuilt fresh per request. The column schema + row count are stored alongside the cached blob so a raw-arrow hit serves the data verbatim with no decode. JSON and TSV responses are unchanged.

Fixed

  • Cache hits reported the original DB execution time instead of the cache read time. execution_time_ms on a hit is now the cache fetch time on every surface (JSON, TSV, Arrow, pgwire).
  • Empty / all-null raw-field results lost their column types on a cache hit. The result column schema is now cached, so types + formats survive regardless of the data.
  • Expression-based measures showed as orphan nodes in the ontology graph, and synthesized counts were absent; both now connect to their data object.
  • Accept negotiation for format=arrow now recognizes the emitted application/vnd.orionbelt.result+arrow media type in addition to the legacy Arrow-stream token.

Full Changelog: v2.19.0...v2.20.0

v2.19.0

Choose a tag to compare

@ralfbecher ralfbecher released this 05 Jul 16:27
9ea992d

[2.19.0] - 2026-07-05

Added

  • Auto-synthesized row-count measures. Every countable dataObject now yields a grain-anchored row-count measure whose name and label are the same human string (default "Sales Count"), exactly like a declared measure such as "Order Count". Counts are governed named measures: reference "Sales Count" in select.measures, with no ad-hoc COUNT(*) in queries, and a dataObject is never a queryable FROM target. Knobs: per-object countable / countLabel, and model-level exposeCounts / countLabelPattern (whose only token is {object}). A declared measure of the same name overrides synthesis (for example COUNT(DISTINCT sale_id) for a self-fanning model). Counts participate fully in discovery, the BI catalog, metric references, and Artefacts Composability Resolution; their knobs roundtrip through OSI, and the derived measures are computed on read so they never persist to YAML/OSI.

Security

  • Fixed two SQL-injection vectors in the Postgres wire (pgwire) extended-protocol parameter substitution. Numeric-typed text bind parameters were spliced into SQL unvalidated (a value like 0 AND "x" = 'y' became active SQL); they are now strictly parsed per their type OID and re-rendered as canonical numeric literals, and anything that is not a plain number is rejected. The $N placeholder scanner is now SQL-aware and skips line comments, nested block comments, quoted identifiers, and dollar-quoted strings, so a placeholder inside those contexts is no longer substituted (closing a break-out path via newline, */, or $$).

Full Changelog: v2.18.2...v2.19.0

v2.18.2

Choose a tag to compare

@ralfbecher ralfbecher released this 04 Jul 20:59
900bdb0

[2.18.2] - 2026-07-04

Changed

  • Result cache is now format-independent. The cache stores raw, locale-neutral rows keyed on the query alone, so raw JSON, value-formatted JSON, TSV, and Arrow all share one entry; value formatting is applied on delivery instead of being baked into the cache. format=arrow now honors format_values (locale-aware display strings are baked into the IPC blob); raw format=arrow, used by the playground round trip, is unchanged.

Performance

  • Faster cache hits. Hit/miss counters accumulate in memory and flush to DuckDB lazily, removing a per-hit metadata write under the global lock. The payload file read and the gzip + Arrow decode are offloaded to a worker thread so a cache hit never blocks the event loop (REST and oneshot). Raw format=arrow hits are served via byte-passthrough: the stored blob is returned verbatim with no decode and no re-encode.

Full Changelog: v2.18.1...v2.18.2